# Flexible and Explainable Solutions for
Multi-Agent Path Finding Problems

Aysu Bogatarkan Faculty of Engineering and Natural Sciences, Sabanci
University, Istanbul, Turkey<EMAIL_ADDRESS>
###### Abstract

The multi-agent path finding (MAPF) problem is a combinatorial search problem
that aims at finding paths for multiple agents (e.g., robots) in an
environment (e.g., an autonomous warehouse) such that no two agents collide
with each other, and subject to some constraints on the lengths of paths. The
real-world applications of MAPF require flexibility (e.g., solving variations
of MAPF) as well as explainability. In this study, both of these challenges
are addressed and some flexible and explainable solutions for MAPF and its
variants are introduced.

##  1 Introduction

Artificial Intelligence (AI) applications are being used widely among people
with different background and interests. For these applications to be
successful, two of the important features (and challenges) needed by AI
methods are flexibility and explainability. A flexible AI method developed to
solve a problem can accommodate variations of the problem, and thus can be
used to investigate different options for a better understanding. An
explainable AI method can provide answers to queries about the (in)feasibility
and the optimality of solutions. One of the well-studied problems in AI that
necessitates solutions for these two challenges is the multi-agent path
finding (MAPF) problem.

MAPF problem aims to find plans for multiple agents in an environment without
colliding with each other or obstacles, subject to some constraints on the
maximum or the total plan length. Optimal solutions for MAPF can be found by
optimizing the makespan of the whole plan or the total plan length of agents.
These optimization functions can be extended according to the needs of an
application. While single-agent shortest pathfinding can be solved in
polynomial time [7], MAPF with constraints on plan lengths is intractable
[18]. MAPF has been studied in various domains, such as robotics [12], traffic
control [8], video games [20], autonomous aircraft towing vehicles [16], and
autonomous warehouse systems [21].

This study focuses on finding flexible frameworks for MAPF and its variants
and an explainable framework for one of these variants of MAPF. In all of
these solutions, Answer Set Programming (ASP) [14, 17, 13, 5, 6]—a logic
programming paradigm based on answer sets [11, 10] is utilized.

For some real-world applications, being able to solve MAPF problem may not be
enough to address all challenges or the realistic conditions of the
application. For instance, in real-world automated warehouses, the robots’
battery levels change as they travel around and it may be necessary for them
to be charged to complete their tasks. Furthermore, some parts of the
warehouses (for instance with human occupants or tight passages) may require
robots to move slowly to ensure safety. In a warehouse that is not completely
autonomous, some changes may occur during the execution of a plan: existing
agents may leave the environment, or new agents may be included in the team
with new tasks, existing obstacles may be removed from the environment or
moved to some other location in the environment. One aim of our study is to
address these issues with with flexible frameworks in the spirit of
elaboration tolerance [15].

We also investigate the challenge of explainability for one of these problems,
in particular, considering queries about the (in)feasibility and the
optimality of solutions, as well as queries about the observations about these
solutions. Given a solution for a general variation of MAPF, our explainable
framework is able to explain infeasibility or nonoptimality of this solution,
confirm its feasibility and suggest alternatives for the solution, and provide
explanations for some queries.

##  2 Flexible Solutions for MAPF: Current Status

Let us describe our contributions and current status of our research on
flexible frameworks for MAPF.

###  2.1 Multi-Modal Multi-Agent Path Finding

In real-world automated warehouses, the robots’ battery levels change as they
move around, and, in some parts of these warehouses, due to presence of humans
or tight passages, the robots may need to move slowly to ensure safety. Hence,
to be able to handle more realistic autonomous warehouse scenarios, a
mathematical model general enough to handle multi-modal transportation
conditions and multi-objective optimizations is needed. Furthermore, the
computational framework is required to be flexible such that a large set of
variations of MAPF problems can be addressed. Motivated by these challenges,
we mathematically modelled a general version of MAPF (called mMAPF– multi-
modal MAPF with resources) as a rich graph problem and introduced a flexible
method to solve mMAPF declaratively, using ASP [3]. We have implemented our
framework using the ASP solver Clingo [9]. Our method can handle the following
variations of MAPF: multi-objective optimization, waypoints, resource
constraints and multi-modal transportation.

Figure 1 shows an example mMAPF scenario introduced in our earlier studies
[3]. In this example, the warehouse consists of three shelf units denoted as
obstacles (black cells). The charging stations are located at cells 24 and 27
highlighted by yellow, and the corridor where the agents should go slowly,
covers cells 3–8 highlighted by red. In this area, it takes 2 timesteps to
move from one cell to its neighbor. The maximum battery level is set to 10.
Robot A1 starts with an initial battery level of 10, while robot A2 has an
initial battery level of 8. Each robot wants to collect some items on its way
(denoted by stars that match the color of the relevant robot) and to deliver
these items to the opposite corner of the warehouse. Therefore, eventually,
they want to swap their places.

Figure 1: An mMAPF instance. Initially, robot A1 is located at cell 1 and
robot A2 is located at cell 30. The goal of A1 is to deliver some items to
cell 30, while the goal of A2 is to deliver some items to cell 1. Each robot’s
waypoints are shown by stars with the same color as that robot.

A solution for this mMAPF instance is shown by the colored paths in Figure 2.
Note that each robot visits its waypoints on the way to the goal. The
traversals of these paths (i.e. location of each robot in each timestep) are
shown in Table 2.1. The traversal of each slow edge (e.g., A2 moving from cell
6 to 5) takes two timesteps. The battery levels of the robots are also shown
in this table. The battery level decreases at each time step, unless a robot
is at a charging station. The battery level gets to its maximum when robots
decide to charge while at a charging station. For instance, A2’s battery level
increases to 10 when the charging station at cell 27 is visited.

Figure 2: A solution for the mMAPF instance shown in Figure 1. A1 follows the
blue path while A2 follows the green path to reach their goals, ensuring they
collect items at their waypoints.

Time | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
$A1$ Location | 1 | 2 | 3 | transit | 4 | 14 | 24 | 25 | 26 | 27 | 17 | 7 | transit | 8 | 9 | 10 | 20 | 30 | –
$A1$ Battery | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | –
$A2$ Location | 30 | 29 | 28 | 27 | 17 | 7 | transit | 6 | transit | 5 | transit | 4 | 14 | 24 | 23 | 22 | 21 | 11 | 1
$A2$ Battery | 8 | 7 | 6 | 5 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 10 | 9 | 8 | 7 | 6

Details of the mathematical model and the ASP formulation can be found in our
paper, Multi-Modal Multi-Agent Path Finding with Optimal Resource Utilization
[3].

###  2.2 Dynamic Multi-Agent Path Finding

Another flexible framework introduced was an algorithm for MAPF in dynamic
environments. When changes occur in a dynamic environment, such as obstacles
being removed or moved, existing agents leaving the environment or new agents
being added to the team, the aim is to find a new solution for the new team of
agents in the modified environment. We call this problem Dynamic Multi-Agent
Path Finding Problem (D-MAPF) [4].

One of the possible solutions for D-MAPF is replanning: consider a new MAPF
instance defined by the current locations and goal locations of both the
existing and the new agents, and the updated environment, and compute a
solution for this instance. Although replanning finds a solution, if one
exists, it does not re-use the plans of the existing agents and may not be
computationally efficient.

With this motivation, we proposed a novel method to solve D-MAPF, using Answer
Set Programming (ASP) . The main idea (and novelty) of this method is, instead
of replanning for all the agents right away, to revise and augment the
existing MAPF solution: (revise) try to schedule the waiting times of existing
agents as they traverse the rest of their paths, (augment) while computing
paths for the new agents within a given makespan (i.e., the length of the
plan). In this way, the paths for the existing agents can be re-used as part
of the new plan. We have implemented our framework using Python and the ASP
solver Clingo. In our experimental evaluations, we observed that the re-use of
plans as proposed by our method improves the computational efficiency in
timings significantly compared to replanning.

Figures 3–6 present an example (from our earlier studies [4]) to illustrate
the overall idea of our D-MAPF algorithm.

Figure 3: Initially, two agents are at $A1$ and $A2$; their goals are to reach
$A1^{\prime}$ and $A2^{\prime}$.

Figure 4: A solution to MAPF instance described in Fig. 3: $A1$ starts at 1,
moves to 2, 3, 6, and reaches 9; and $A2$ starts at 3, moves to 6, 5, 4,
reaches 7.

Figure 5: While executing the plan shown in Fig. 4, at time step $t=1$,
another agent $A3$ joins the team with goal $A3^{\prime}$. A solution to this
D-MAPF instance is computed, and a path for $A3$ is found, as shown above:
$A1$ is at 2, moves to 3, 6, 9; $A2$ is at 6, moves to 5, 4, 7; $A3$ starts at
9, moves to 6, 5, 2.

Figure 6: While executing the plan shown in Fig. 5, at time step $t=2$,
another agent $A4$ joins the team with goal $A4^{\prime}$ . A solution to this
D-MAPF instance is computed, no solution found makespan $t=4$, so the makespan
is increased by 1. The new solution with makespan $t=5$ is found by scheduling
the waiting times of $A1$, $A2$ and $A3$, and by computing a plan for $A4$, as
shown above: $A1$ is at 3, waits at 3, then moves to 6,9; $A2$ is at 5, waits
at 5, then moves to 4,7; $A3$ is at 6, waits at 6, then moves to 5,2; $A4$
starts at 7, moves to 4, 1, and waits at 1.

The problem definition, ASP formulation, algorithm and experimental
evaluations are described in detail in our paper A Declarative Method for
Dynamic Multi-Agent Path Finding [4].

##  3 Explainable Solutions for MAPF: Current Status

We also investigate the challenge of explainability for mMAPF problems,
considering queries about the (in)feasibility and the optimality of solutions,
along with queries about observations about these solutions. For instance,
suppose that an mMAPF solution is being executed in a warehouse and an
engineer in this warehouse would like to check whether some modifications of
this mMAPF solution would still be feasible or not.

  * •

Explaining infeasibility or nonoptimality. Suppose that the modified solution
is found infeasible, then, an explanation regarding infeasibility of this
modified solution could be “due to collisions with obstacles or other robots”
or “due to low battery-level”. An explanation regarding nonoptimality of the
modified solution could be “because some more time is needed to complete
tasks” or “because some more charging is required”.

  * •

Confirming feasibility and suggesting alternatives. Suppose that the modified
solution is found feasible. Furthermore, a better solution is computed (e.g.,
where the tasks are completed earlier). Then, in addition to confirming that
the plan is feasible, alternative solutions could be provided to the engineer.

In an alternative scenario, suppose that the engineer would like to better
understand the mMAPF solution being executed in the warehouse, and asks
various queries about it. For such queries, it is useful to generate
explanations using counterfactuals.

  * •

Explaining why an agent is waiting too long at a location. Suppose that the
engineer observes that the agent is waiting for a while at some location, and
she wants to know why. An explanation could be that “if the agent does not
wait at that location for a while, it will collide with another robot.”
Alternatively, an explanation could be “actually, there is no need for the
agent to wait there so long, but it needs to follow a different itinerary such
as … to complete tasks on time” or “actually, there is no need for the agent
to wait there so long, but it needs to follow a different itinerary such as …
and will be late a bit.”

Such queries and explanations would help the engineer to better understand the
strengths and weaknesses of the solution being executed, as well as the
limitations of the infrastructure.

With these motivating real life scenarios, we introduced a method to generate
explanations for a variety of queries about mMAPF solutions, using the
expressive formalism and efficient solvers of ASP [2]. Our explainable
framework is implemented using Python and the ASP solver Clingo.

An Example Scenario for a Query About Waiting

|
---

| Time | A1 Location | A2 Location
---|---|---
0 | 11 | 8
1 | 7 | 8
2 | 6 | 7
3 | 5 | 6
4 | - | 2

| Time | A1 Location | A2 Location
---|---|---
0 | 11 | 8
1 | 11 | 7
2 | 7 | 6
3 | 6 | 2
4 | 5 | -

(a) | (b) | (c)
Figure 7: (a) A1 and A2 denote the initial positions of Robots 1 and 2; A1′
and A2′ denote goal locations. Cell 7 is a waypoint for both robots, (b) and
(c) are two optimal plans for this instance.

Consider the mMAPF instance (Scenario 1 of [2]) shown in Figure 7(a) in a
small warehouse, where Robot 1 is initially located at Cell 11 and aims to
reach Cell 5, and Robot 2 is initially located at Cell 8 and aims to reach
Cell 2. Cell 7 is a waypoint for both robots, and the upper bound on makespan
of a plan is 4. For simplicity, suppose that the batteries of the robots are
fully charged initially, and sufficient for execution of any given plan, and
that all edges have the normal mode of transportation.

About Plan 1 described in Figure 7(b), suppose that an engineer asks the
following query:

> “Why does Robot 2 wait at Cell 8 (at any time)?”

Our algorithm checks whether there is an optimal plan where Robot 2 does not
have to wait initially.

Once an alternative solution, Plan 2 described in Figure 7(c), is found, our
algorithm presents the following explanation:

> “Actually, Robot 2 does not have to wait at Cell 8 from time step 0 to 2.
> Here is an alternative optimal plan: Plan 2…”

An Example Scenario for Nonexistence of Solutions Query

Figure 8: Robots 1 and 2 are initially located at the given positions on the
grid; the goal of each robot is the initial location of the other. Black cells
are obstacles.

Consider the instance (Scenario 6 of [2]) in Figure 8(b). Robots 1–2 are
initially located at $1$ and $3$; their goals are at $3$ and $1$,
respectively. Black cells denote obstacles. For simplicity, we assume that
their batteries are initially fully charged and enough to traverse all of
their paths, all edges have normal mode of transportation, and the only
waypoint of each robot is located at its initial position.

There is no solution for this mMAPF instance. Suppose that an engineer wants
to find out the reason and asks the following query:

> “Why does not the instance have a solution?”

The algorithm generates the following explanation:

> “There is no solution because Robot 1 and Robot 2 collide at Cell 8 at time
> step 3.”

The algorithm further generates the following explanation:

> “There is no solution because Robot 2 collides with the obstacle at Cell 2
> at time step 1; this suggests removing this obstacle.”

Details of the algorithm and the ASP formulations, together with more examples
and experimental evaluations are presented in our paper, Explanation
Generation for Multi-Modal Multi-Agent Path Finding with Optimal Resource
Utilization using Answer Set Programming [2].

##  4 Discussions and Future Work

We have introduced flexible and explainable frameworks for MAPF problem,
addressing different challenges. While mMAPF focuses on a flexible framework
considering resource optimization and multi-modality, D-MAPF considers dynamic
changes in the environment. In addition to these frameworks, an explainable
framework for mMAPF is also introduced, generating explanations about
feasibility and optimality about given mMAPF solutions along with some
observations or suggestions, via queries and counterfactual reasoning. For all
these frameworks, we contributed the literature with publications. We have
also developed an educational multi-robot system to apply MAPF with robots
[19].

Currently, we are working on applying these frameworks on some real-world
applications in collaboration with a logistics company. We are also working on
improving our methods introduced for D-MAPF [4] and mMAPF [3], extend their
experimental evaluations and we aim to publish journal articles for our
improved methods.

## Acknowledgments

I would like to thank Prof. Esra Erdem (Sabanci Universitys, Computer Science
and Engineering) and Prof. Volkan Patoglu (Sabanci University, Mechatronics
Engineering) for their supervision in this research, Dr. Orkunt Sabuncu (TED
University, Computer Engineering) and Dr. Alexander Kleiner (Robert Bosch
GmbH, Corporate Research) for useful discussions, and the anonymous reviewers
for their comments and suggestions. This work is supported by Tubitak Grant
118E931.

## References

  * [1]
  * [2] Aysu Bogatarkan & Esra Erdem (2020): _Explanation Generation for Multi-Modal Multi-Agent Path Finding with Optimal Resource Utilization using Answer Set Programming_.  Theory Pract. Log. Program. 20(6), pp. 974–989, 10.1017/S1471068420000320.  Available at https://arxiv.org/abs/2008.03573.
  * [3] Aysu Bogatarkan, Esra Erdem, Alexander Kleiner & Volkan Patoglu (2020): _Multi-modal Multi-agent Path Finding with Optimal Resource Utilization_.  In: Proceedings of 5th International Conference on the Industry 4.0 Model for Advanced Manufacturing, pp. 313–324, 10.1007/978-3-030-46212-3_24.
  * [4] Aysu Bogatarkan, Volkan Patoglu & Esra Erdem (2019): _A Declarative Method for Dynamic Multi-Agent Path Finding_.  In: Proceedings of the 5th Global Conference on Artificial Intelligence, pp. 54–67, 10.29007/cnzw.
  * [5] Gerhard Brewka, Thomas Eiter & Miroslaw Truszczynski (2011): _Answer set programming at a glance_.  ACM Commununications 54(12), pp. 92–103, 10.1145/2043174.2043195.
  * [6] Gerhard Brewka, Thomas Eiter & Miroslaw Truszczynski (2016): _Answer Set Programming: An Introduction to the Special Issue_.  AI Magazine 37(3), pp. 5–6, 10.1609/aimag.v37i3.2669.
  * [7] Edsger W. Dijkstra (1959): _A Note on Two Problems in Connexion with Graphs_.  Numer. Math. 1(1), pp. 269–271, 10.1007/BF01386390.
  * [8] Kurt M. Dresner & Peter Stone (2008): _A Multiagent Approach to Autonomous Intersection Management_.  J. Artif. Intell. Res. (JAIR) 31, pp. 591–695, 10.1613/jair.2502.
  * [9] Martin Gebser, Benjamin Kaufmann, Roland Kaminski, Max Ostrowski, Torsten Schaub & Marius Schneider (2011): _Potassco: The Potsdam Answer Set Solving Collection_.  AI Commun. 24(2), p. 107–124, 10.5555/1971622.1971623.
  * [10] Michael Gelfond & Vladimir Lifschitz (1988): _The stable model semantics for logic programming_.  In: Proceedings of International Logic Programming Conference and Symposium, pp. 1070–1080.
  * [11] Michael Gelfond & Vladimir Lifschitz (1991): _Classical negation in logic programs and disjunctive databases_.  New Generation Computing 9, pp. 365–385, 10.1007/BF03037169.
  * [12] Jae-Yeong Lee & Wonpil Yu (2009): _A coarse-to-fine approach for fast path finding for mobile robots_.  In: Proc. of IROS, pp. 5414–5419, 10.1109/IROS.2009.5354686.
  * [13] Vladimir Lifschitz (2002): _Answer set programming and plan generation_.  Artificial Intelligence 138, pp. 39–54, 10.1016/S0004-3702(02)00186-8.
  * [14] Victor Marek & Mirosław Truszczyński (1999): _Stable models and an alternative logic programming paradigm_.  In: The Logic Programming Paradigm: a 25-Year Perspective, Springer Verlag, pp. 375–398, 10.1007/978-3-642-60085-2_17.
  * [15] John McCarthy (1998): _Elaboration Tolerance_.  In: Proc. of CommonSense.
  * [16] Robert Morris, Corina S. Pasareanu, Kasper Søe Luckow, Waqar Malik, Hang Ma, T. K. Satish Kumar & Sven Koenig (2016): _Planning, Scheduling and Monitoring for Airport Surface Operations_.  In: Planning for Hybrid Systems, Papers from AAAI Workshop.
  * [17] Ilkka Niemelä (1999): _Logic programs with stable model semantics as a constraint programming paradigm_.  Annals of Mathematics and Artificial Intelligence 25, pp. 241–273, 10.1023/A:1018930122475.
  * [18] Daniel Ratner & Manfred K. Warmuth (1986): _Finding a Shortest Solution for the N $\times$ N Extension of the 15-PUZZLE Is Intractable_.  In: Proc. of AAAI, pp. 168–172.
  * [19] Emre Hilmi Songur, Aysu Bogatarkan, Farukhzhon Barotov, Batuhan Yildirim, Gokay Coruhlu, Esra Erdem & Volkan Patoglu (2019): _Cok-etmenli yorunge bulma probleminin lego robotlarinda uygulanmasi: bir egitsel robotik calismasi_.  In: 5\. Türkiye Robotbilim Konferansı (TORK 2019), Ozyegin Universitesi Yayinlari, pp. 172–177.  Available at https://tork2019.ozyegin.edu.tr/sites/default/files/tork2019_r1.pdf.
  * [20] Trevor Scott Standley & Richard E. Korf (2011): _Complete Algorithms for Cooperative Pathfinding Problems_.  In: Proc. of IJCAI, pp. 668–673, 10.5591/978-1-57735-516-8/IJCAI11-118.
  * [21] Peter R. Wurman, Raffaello D’Andrea & Mick Mountz (2008): _Coordinating Hundreds of Cooperative, Autonomous Vehicles in Warehouses_.  AI Magazine 29(1), pp. 9–20, 10.1609/aimag.v29i1.2082.


# A Survey and Comparative Study on Multi-Cloud Architectures: Emerging Issues
and Challenges for Cloud Federation

Deepika Saxena
Department of Computer Applications
National Institute of Technology
Kurukshetra, India
<EMAIL_ADDRESS>
Rishabh Gupta
Department of Computer Applications
National Institute of Technology
Kurukshetra, India
<EMAIL_ADDRESS>
Ashutosh Kumar Singh
Department of Computer Applications
National Institute of Technology
Kurukshetra, India
<EMAIL_ADDRESS>
The authors would like to thank National Institute of Technology Kurukshetra,
India for financially supporting this research work.

###### Abstract

Multi-cloud concept has broaden the world of cloud computing and has become a
buzzword today. The word “Multi-cloud” envisions utilization of services from
multiple heterogeneous cloud providers via a single architecture at customer
(e.g. organization) premises. Though cloud computing has many issues and
offers open research challenges, still the academics and industrial research
has paved a pathway for multi-cloud environment. The concept of multi-cloud is
in maturing phase, and many research projects are in progress to provide a
multi-cloud architecture which is successfully enabled in all the respects
(like easy configuration, security, management etc.). In this paper, concepts,
challenges, requirement and future directions for multi-cloud environment are
discussed. A survey of existing approaches and solutions provided by different
multi-cloud architectures is entailed along with analysis of the pros and cons
of different architectures while comparing the same.

_Keywords_ cloud broker $\cdot$ cloud federation $\cdot$ inter-operable
$\cdot$ multi-cloud.

##  1 Introduction

Cloud computation world is transforming in anticipation and is rightly
contributing in successful technical progress of almost every organization
across the world [1], [2], [3], [4]. From economic point of view, cloud
computing represents a business model for renting software and hardware
resources [5], [6], [7], [8]. Multi-Cloud has extremely attracted the
corporate world. EMA surveyed 260 enterprises and majority of the respondents
i.e., 61% reported using two or more public cloud provider. All of us are
familiar with the fact that the amount of useful information produced by an
organization today in one day is approximately equal to the amount of
information produced in twenty days earlier a couple of decades back [9],
[10], [11], [12], [13]. Single cloud framework is not suitable and has become
insufficient to serve the ever growing requirement [14], [15], [16], [17].
Seeing the current trend of computational growth in organizations, it is quiet
obvious that future is multi-cloud.

###  1.1 Multi-cloud concept

With the advent of cloud, small and big organizations all are progressing
without need to concern about the storage and maintenance of their business
data. They are not required to spend in billions for the same. All the
responsibility is envisaged upon the cloud service providers (CSPs) [18],
[19], [20]. So, cloud computing has become the backbone of modern business
world. Organizations contacts various cloud service providers and consumes the
services by signing Service-Layer Agreement SLA document [21], [22], [23],
[24], [25]. A CSP contacts various resource providers at datacenters in order
to satisfy the demands of the customer [26], [27], [28], [29]. Usually, it is
said that cloud computing provides infinite resources and elastic services
[30], [31]. Practically, we know that resources cannot be infinite at any
datacenter. To raise the elasticity or capacity of cloud service providers and
fulfill the ever growing demand of services, resources from different resource
providers need to collaborate, inter-communicate and work in cooperation and
coordination. So, collaboration of various cloud service providers gives root
to the concept of Multi-Cloud. Multi-cloud simply means an enterprise can take
services from more than one cloud service provider through a common interface
or a single API. In multi-cloud environment, multiple cloud providers
aggregate in a single pool to provide three basic services of cloud:
computation, storage and networking. Computation service allows customers to
instantiate their own computational nodes irrespective of their physical
server node at different datacenters. Storage service enables users to store
infinite data as per they desire without vendor- lock-in. They can store
public data at public cloud and confidential data at private cloud

###  1.2 Vendor lock-in

In Single cloud environment, security of user data is completely in the hands
of the cloud provider and he leverages entire control of user data. This
situation represents “vendor lock-in” drives user data towards potential
security risks. To resolve this issue, multi- cloud approach provides
appropriate solution. Networking provides communication between computational
and storage nodes despite of the networking infrastructure hosted upon
physical connectivity between actual physical servers. Multi-cloud allows user
to organize, transfer, synchronize and manage sharing of files between cloud
storage services like Dropbox, Google Drive, Copy, OneDrive, FTP, WebDav,
MEGA, etc. 26 cloud drives supported so far [20].

###  1.3 Challenges of Multi-Cloud

  * •

API’s differ: Different sets of resources are there, different formats and
encodings, and several simultaneous versions of single cloud exist.

  * •

Abstractions differ: Network architecture differ- VLANs, topology differ,
addressing grounds differ.

  * •

Difference in storage architecture- local/attachable, disks and backups.

  * •

Hypervisor and machine configuration differ etc.

  * •

Mismatch of cyber-laws of various interconnected clouds depending on their
geographical location physically.

  * •

Interoperability of different cloud providers always a major issue for multi-
cloud set-up.

  * •

Difficult to tackle the security challenges related to multi- provider
environment.

  * •

Different cloud provider follows their own management scheme for load
balancing and monetization features.

Interoperability: The biggest challenge in cloud computing is the diversity of
resources, management and rules and regulations of cloud providers, diverse
SLAs, differences in security characteristics at various CSPs etc.
Interoperability means the ability of diverse system to work in cooperation
and serve the common output. Interoperability of multi-cloud has several
heterogeneous dimensions:

  * •

Vertical interoperability Along the cloud stack, interoperability increases as
we move from IaaS to SaaS. To deal with such heterogeneity, standardize
platform and mapping interface layers are required.

  * •

Horizontal interoperability The inter-communication issues lying between
various service providers, technically, physically, logically and legally etc.
while contributing for multi-cloud.

Across the world, thousands of data centers are geographically distributed
which vary in cost of electrical consumption. At various data centers, there
exist heterogeneity in resource usage there logical and physical
infrastructure. Each different CSP follows his own SLA policies, management
rules, security features, cloud service cost etc. In order to deal with the
above challenges, various inter-cloud or federated cloud approaches were
proposed. Many projects are in progress to set-up multi-cloud architecture. In
this paper, number of multi-cloud or federated cloud or inter-cloud approaches
are surveyed.

###  1.4 Organization

The remainder of this paper is as follows: Section1 introduces concept of
multi-cloud and challenges of multi-cloud. Section 2 gives detailed
architectures of various cloud federation available till now. Section 3
provides critical analysis on various architectures discussed in this paper.
Section 4 concludes the paper and highlights the future scope.

##  2 ARCHITECTURAL VIEWS OF MULTI-CLOUD

###  2.1 CHARM

In the paper [32], (Quanlu Zhang et.al,2015) have presented “CHARM: A Cost-
Efficient Multi-Cloud Data Hosting Scheme with High Availability”. Figure 1
shows the basic principle of multi-cloud data hosting. Here the basic
principle of data hosting on multi-cloud is to distribute data across multiple
clouds to gain redundancy and prevent vendor-lock-in. The key component in
this model is “Proxy” which redirects the request from client application and
coordinate data distribution among multiple clouds. Different clouds exhibit
huge difference in terms of their services like pricing policies and work
performance. For instance, Google cloud platform charges more for bandwidth
consumption but Amazon S3 charges more for storage space and Rackspace
provides all web operations free via series of REST ful APIs. In this paper,
in order to reduce monetary cost and guaranteed availability, two redundancy
schemes are combined- replication and erasure coding. The advantage of the
CHARM multi- cloud model is that it provides guidance to the customers to
distribute their data on multiple clouds in cost effective manner. CHARM makes
fine-grained decisions about which storage mode to use and which clouds to
place data in.

Figure 1: Basic principle of multi-cloud data hosting

###  2.2  OPTIMIS Cloud Federation

OPTIMIS cloud federation [33] have presented toolkit regarding cloud service
and cloud provider is presented. This toolkit can optimize the whole service
life cycle of cloud including service construction, deployment, and operation
on basis of aspects such as trust, risk, eco-efficiency and cost. Several
architecture of cloud are also addressed in this paper. More significantly,
two architecture of cloud are discussed here.

  * •

Federated Cloud: In this architecture, service provider (SP) takes services
from a single infrastructure provider (IP) , but, one infrastructure
Provider(IP) can sub-contract (or can lease some services) with another IP ,
in order to raise the capacity, elasticity and fault tolerance of cloud
service.

  * •

Multi-provider hosting: In this architecture, Service Provider (SP), directly
negotiates with more than one infrastructure providers, to deploy services,
monitors their operation and migrates services from misbehaving IPs to
maintain trust relationship.

###  2.3 Multi-Cloud Brokering Architecture

In this type of architecture, a third party broker aggregates services from
multiple infrastructure providers (IPs) and delivers these services to the
actual service provider. The broker act as SP for multiple IPs and it behave
as IP for the SP. This model simplifies the complexities of taking services
from several different IPs. Here, the broker provides single entry point to
many IPs and more fruitful is that the broker can get bulk of discount from
IPs from economic point of view. Moreover, in terms of business policies have
to make long term relation with the broker for excellent growth of his
business. Hence, risk and trust prediction becomes easier with broker in
between IPs and SP.

###  2.4 RESERVOIR MODEL

Rochwerger et al., 2009 [34] have introduced Reservoir model for federated
cloud computing. According to this architecture, different infrastructure
provider (IP) can aggregate to provide flexible and scalable cloud computing
services as shown in figure 2. Here, IPs own the computational resources,
network resources and storage resources in an infinite pool. Service providers
are the agents who offers computational and storage services to the customers
with help of IPs resources at back end. In Reservoir model, IPs and SPs are
clearly separated. IPs operates reservoir sites that actually owns and manages
resources. The reservoir sites collaborates to form a Reservoir Cloud.
Virtualization layer is set up for separation of different Virtual Execution
Environment (VEEs) and the resource usage optimization at each reservoir site.
Service applications of customers are deployed on reservoir cloud using
service manifest, on one or more VEE as per the requirement. Service manifest
will give information regarding capacity of physical infrastructure such as
the number of VEEs that can be hosted on a CPU, structure of service
application in terms of software stack (OS, middleware, application, data and
configuration). Service manifest also indicates Key Performance Indicators
(KPI). KPI measures load parameters, response time, throughput, number of
active sessions and resource allocations i.e. memory, bandwidth, number of
VEEs. To resolve interoperability issues, this model provides VEE management
interface that supports VEEM-to-VEEM communication. The broad limitation of
this model is that it has not mentioned any security issues handling while
collaborating various IPs in reservoir cloud. The practical implementation of
this reservoir model is not given.

Figure 2: Conceptual view of Reservoir Model of cloud federation

###  2.5 Federated Cloud Management (FCM)

In FCM [35], several Infrastructure Providers exclusively dedicate their
computing infrastructure to a single cloud broker. Figure 3 shows FCM
architecture where, various cloud brokers collaborates to provide federated
cloud environment. In FCM, interoperability is resolved by enabling two levels
of brokering.

  * •

User service call is submitted to Generic Meta –Broker Service (GMBS).

  * •

This call is then forwarded to selected Federated Cloud Broker, which further
selects VM from underlying infrastructure provider for actual execution of
service. FCM incorporates high level brokering for cloud federation and on
demand automated service deployments. FCM Repository maintains all the
required information about virtual appliances at cloud broker level.

Figure 3: FCM Architecture

###  2.6 Distributed Management Task Force (DMTF) Architecture

Cloud, & Incubator [36] have presented Distributed Management Task Force
(DMTF) Open Cloud standards Incubator models for enabling interoperability in
multi-cloud environment. According to DMTF, three different models for
supporting federated cloud scenario have been identified. Datacenter includes
Service Providers (SP) and infrastructure provider (IP). The three models
differs in the working style of datacenters as follows:

  * •

Firstly, at the datacenter, service provider can act as cloud broker or an
agent to federate underlying infrastructure provider resources and make the
services available to the overlying customers as shown in figure 4.

Figure 4: First case for cloud federation according to DMTF approach

  * •

2\. In second case, multi-cloud model is presented, where datacenter can
request services from various infrastructure or resource providers, thus they
support different SLA’s parameters. Figure 5 shows this scenario.

Figure 5: Second case for cloud federation according to DMTF approach

  * •

In third case, if a datacenter is served by one SP and hosted by one IP
overflows the available computing capacity limit, then another infrastructure
provider can contribute extra computing capacity for the former IP. But,
Service Provider is unaware of this resource provisioning.

###  2.7 Contrail Architecture

Carlini et al. [37] introduced Open Contrail Architecture for cloud
federation. The goal of Open Contrail project is to minimize the burden on
user and increase efficiency of the cloud platform, by performing horizontal (
resolving interoperability issues in cloud federation) and vertical (providing
cloud federation facility at cloud stack level i.e. IaaS and PaaS services)
integration. This architecture utilizes the resources of various cloud
resource providers, regardless of the heterogeneity of underlying
infrastructure. Complete open source approach is adopted here. It also
supports user authentication, integration of SLA management and application
deployment. Contrail architecture has three layer to support federation of
clouds. Figure 6 shows Open Contrail Federation:

  * •

Interface Layer: This is top most layer, which provides interaction between
user and contrail federation. It consists of REST API, http and Command Line
Interface.

  * •

Core Layer: This is middle layer, which contains various modules to provide
functional requirements like application life-cycle management and non-
functional like security requirements.

  * •

Adapter Layer: This is bottom layer of the architecture, that contains modules
to retrieve information and operate on different cloud resource providers. It
copes with the heterogeneity of underlying infrastructure.

Figure 6: Contrail Cloud Federation

###  2.8 Combinatorial Auction based Collaborative Cloud

Zhang, Li and Zhu, 2015 [38] have presented a Combinatorial Auction- Based
Collaborative Cloud Services Platform in the form of model for cloud
collaboration considering cost of communication and negotiation among clouds
under collaboration. This market model for cloud collaboration includes three
distinct layers as shown in figure 7. First, user layer receives requests from
end-user. Second, Auction-layer matches requests with cloud services provided
by CSP. Third, CSP-Layer forms coalition to improve serving ability to satisfy
demands of the user. For coalition formation and to find suitable partner for
collaboration, Breadth Traversal Algorithm (BTA) and Revised Ant Colony
Algorithm (RACA) are introduced. Here CSPs dynamically collaborate with one
another and submit their group’s bid as a single bid to the auction-layer.
Whenever a request is generated by the user, at user-layer, BTA and RACA
algorithm executes at auction-layer, for the selection of CSP to enable
dynamic collaboration of cloud to fulfill the request of the user. The results
presented with this model are not tested with real-time data so it is somewhat
unacceptable model.

Figure 7: Auction-Based Collaborative Cloud

###  2.9 Layered Multi-Cloud Architecture

Villegas et al. [39] have shown a layered multi-cloud architecture by creating
federation at each service layer as shown in figure 8. At each layer, broker
is present, who follows some pre-defined collaboration terms for successful
inter-cloud federation. They have added federation of cloud model by showing
how it works in delivering Weather Research Forecasting (WRF). They have given
conceptual view of decoupling SaaS, PaaS and IaaS clouds. For such federation,
well defined policies and protocols must be defined. PaaS layer act as
middleware and act as a bridge for vertical integration. It is a bridge
between application and elastic infrastructure resource management.

Figure 8: Cloud federation layered model

###  2.10 mOSAIC Multi-Cloud Architecture

Petcu et al. [40] have introduced Open-source API and Platform for Multiple
Clouds (mOSAIC). This project provides solution for application portability
and interoperability across multiple clouds. It offers a middleware for
deployment of multi-cloud application. mOSAIC also handles the problem of
changing of cloud vendors. Here, application instance runs in mOSAIC component
getting access through dedicated APIs. Communication among its components
takes place through message queues. E.g. AMQP. Cloud Agency of mOSAIC is a
service for deployment of cloud applications. Here Cloud Agency handles
interoperability problem by introducing a uniform interface for accessing
multiple clouds. As shown in figure 9, mOSAIC model has provision for :
-application portability between clouds via set of APIs.
-Support dynamic negotiations with multiple cloud providers using brokering system.
-user-centric SLA.

Figure 9: mOSAI Cloud model

###  2.11  EGI FEDERATED CLOUD

EGI Federated cloud [41] integrates public, private or hybrid cloud in the
form of multi-national cloud system. Each resource center operates cloud
management framework (CMF) according to its own preferences and then joins
Federation and Collaboration services . User can access all services of cloud
federation with a single identity.

###  2.12  CONTAINER TECHNOLOGY

Containers [42] have replaced traditional hypervisor based Virtual machines.
All most every top company like Microsoft, Facebook, Google are using Docker
Swarm or Kubernetes technology. Basically, containers provide easy to deploy
and secure orchestration of specific self-contained virtual infrastructure.
Cloud containers can virtualize single application environment. Containers
create an isolation boundary at application level rather than at server level
.Isolation ensures that if anything goes wrong in a container, then it may not
effect another container deployed on same VM or server. In absence of
container, VM requires complete operating system to be installed, but with
container technology neither requires complete installation of operating
system nor even they requires complete virtual copy of host server’s hardware
resources. Containers only requires software , libraries and basics of an
operating system and cloud containers can be very easily copied and deployed
on another server very easily.The only limitation of containers is their
management. Containers splits the virtualization into smaller chunks, that
creates difficulty in management.[17].

###  2.13  SPINNAKER

Spinnaker [43] is an open source ,multi-cloud delivery platform. It provides
two important features : 1. Cluster management 2. Deployment management.
Cluster management: User can manage resources in the cloud cluster which is a
logical grouping of servers in spinnaker. The logical grouping is based on
user application. Server groups defines the deployable artifacts i.e. VM
image, Docker image, source location.
Deployment management: The key feature of spinnaker are pipelines. Pipelines
consists of sequence of actions (like deploy, resize, disable, manual
judgement etc.) known as stages.

###  2.14 CLOUD FOUNDRY

Cloud Foundry [44] provides a standardized platform to the applications of
customers by decoupling the application from its infrastructure. To enable
this, CF offers two types of VMs: the component VMs as platform infrastructure
and the host VMs that host applications for outside world. Hence, the
organizations can make a business decision on where to deploy workloads i.e.
on premise, in managed infrastructure or in public cloud.

###  2.15 SUPER CLOUD PROJECT

This project (Lacoste et al. [45]) has divided multi-cloud architecture into
three layers: data, compute and network layer. This work focus on user-centric
vision of cloud i.e. virtual private cloud (VPC) just like VPN, inspite of
usual provider-centric cloud. To deal with interoperability issues, a resource
distribution layer i.e. SUPERCLOUD has been introduced. VPC are set up in the
form of U-Clouds consisting of data units, compute unit and network units.
These U-Clouds are VPC set upon top of various cloud service providers. For
security, they have introduced shared data encryption and middleware-based
encryption and decryption of data. Figure 10, shows deployment architecture of
SUPERCLOUD project work.

  * •

Compute Plane: This is topmost plane which applies nested virtualization for
abstracting computational resources of various CSPs. Here, user can run
several virtual execution environment like, containers and virtual machines.
This plane ensures proper execution and protection of each user VMs. These VMs
at compute plane are deployed over user-hypervisor layer, which itself is set-
upon provider’s hypervisor.

  * •

Data Plane: This plane manages all the storage management responsibility and
provides data management APIs for every execution environment on the compute
plane.In order to get access to the cloud’s storage, a user’s compute VM
realizes data management VM as a proxy to get access to the cloud data.

  * •

Network Plane: The network management is a network virtualization platform
based on Software-Defined Network (SDN).Here, the host VM sends request to the
network management VM to get direct access to the storage data. This plane
enables isolation between multiple tenants by providing abstraction for
network topology and traffic routing.

Figure 10: SUPERCLOUD Architecture

###  2.16 Utility-oriented Federation of Cloud Environment

Buyya et al. [46] have proposed inter-cloud framework which supports scaling
of applications across multiple vendor clouds. In this inter-cloud model, as
shown in figure 11, every client in federated environment needs to contact
cloud brokering service that can dynamically establish service contracts with
cloud coordinators through trading functions exposed by cloud Exchange.

Figure 11: Utility based Federated Cloud Model

Cloud Coordinators- They caters the deployment and management responsibility
for domain specific enterprise clouds and their membership to overall
federation driven by market-based trading and negotiation protocols. Also,
sensor infrastructure will monitor the power consumption, heat dissipation,
utilization of computational nodes in virtual cloud environment. Cloud Broker
finds out suitable Cloud service provider through cloud exchange and
negotiates with cloud coordinators for resource allocation. Cloud Exchange
keeps information about cloud’s current usage cost and demand patterns. In the
end, it is concluded that federated cloud service gives better performance
than existing non-federated cloud approaches.

Table 1 entails the summary and comparative analysis of aforementioned multi-
cloud architectures.

Table 1: Comparative Summary of Multi-cloud Architectures Model, Author |  Aim |  Conceptual Approach |  Strength |  Weakness
---|---|---|---|---
CHARM (Zhang et.al, 2015) [32] |  Cost-efficient multi-cloud data hosting scheme with high availability |  “Proxy” redirects the request from client application and coordinate data distribution among multiple clouds. In order to reduce monetary cost and guaranteed availability, two redundancy schemes are combined- replication and erasure coding |  It provides guidance to the customers to distribute their data on multiple clouds in cost effective manner |  Various issues regarding cloud computing are uncovered like, Management of multi- cloud, load-balancing, security etc.
OPTIMIS CLOUD FEDERATION (Ferrer et al., 2012) [33] |  To provide toolkit to optimize whole service lifecycle as a cloud |  Service Provider (SP), directly negotiates with more than one infrastructure providers, to deploy services. a third party broker aggregates services from multiple infrastructure providers (IPs) and delivers these services to the actual service provider |  This model simplifies the complexities of taking services from several different IPs. The broker provides single entry point to many IPs and more fruitful is that the broker can get bulk of discount from IPs from economic point of view |  Only conceptual architecture of several multi-cloud or federated approach architecture are given , but exact practical implementation is not given. How to handle interoperability issues is not focused clearly
DISTRIBUTED MANAGEMENT TASK FORCE (DMTF) ARCHITECTURE (Paper et al., 2009) [36] |  Open Cloud standards Incubator models for enabling interoperability in multi-cloud environment. |  service provider can act as cloud broker to aggregate services from infrastructure provider and make them available to the customers. Different IPs can collaborate to provide services to one service provider. |  They have conceptually provided Incubator models for handling interoperability in multi-cloud environment. |  Service Provider is unaware of this resource provisioning. So, lack of reliability and trust. Security is major concern that is not included here.
RESERVOIR MODEL (Rochwerger et al., 2009) [34] |  Reservoir model for federated cloud computing |  Different infrastructure provider (IP) can aggregate to provide flexible and scalable cloud computing services. Virtualization layer is set up for separation of different Virtual Execution Environment (VEEs) and the resource usage optimization at each reservoir site |  The resource usage optimization at each reservoir site. Interoperability is handled by VEE management interface that supports VEEM-to-VEEM communication |  Security, access feature, management of various multi-cloud architecture are not discussed.The practical implementation of this reservoir model is not given
LAYERED MULTI- CLOUD ARCHITECTURE (Villegas et al., 2012) [39] |  Layered multi- cloud architecture by creating federation at each service layer . |  In layered multi-cloud architecture, at each layer, broker is present, who follows some pre- defined collaboration terms for successful inter-cloud federation. |  They have given conceptual view of decoupling SaaS, PaaS and IaaS clouds. They have identified, that for such federation, well defined policies and protocols must be defined. |  This architecture lacks management and authorized access handling.It is not clear exactly how PaaS can act as a middleware.
Table 1: Comparative Summary continued Model, Author |  Aim |  Conceptual Approach |  Strength |  Weakness
---|---|---|---|---
CONTRAIL ARCHITECTURE (Carlini et al., 2012) [37] |  To minimize the burden on user and increase efficiency of the cloud platform, by performing horizontal (resolving interoperability issues in cloud federation) and vertical (providing cloud federation facility at cloud stack level i.e. IaaS and PaaS services) integration. |  This architecture utilizes the resources of various cloud resource providers, regardless of the heterogeneity of underlying infrastructure. Complete open source approach is adopted here. |  It handles interoperability issues by using open source environment completely. It also supports user authentication, integration of SLA management and application deployment. |  Though this model is quiet efficient, still there is s limitation that how can we sure that each cloud resource provider is trustworthy regarding his deliverables.
COMBINATORIAL AUCTION-BASED COLLABORATIVE CLOUD (Zhang et al., 2015) [38] |  Their aim is to provide model for cloud collaboration considering cost of communication and negotiation among clouds under collaboration. |  First, user layer receives requests from end-user. Second, Auction-layer matches requests with cloud services provided by CSP. Third, CSP- Layer forms coalition to improve serving ability to satisfy demands of the user. For coalition formation and to find suitable partner for collaboration |  This is a market-model for multiple cloud collaboration, which selects the suitable CSP according to demand of the user. |  The results presented with this model are not tested with real-time data so it is somewhat unacceptable model.
mOSAIC MULTI- CLOUD ARCHITECTURE (Petcu et al., 2013) [40] |  This project provides solution for application portability and interoperability across multiple clouds. |  mOSAIC architecture is built from loosely coupled components that enhance the chances for the open-source software prototypes to be adopted. |  mOSAIC handles the problem of changing of cloud vendors. It has application portability between clouds via set of APIs. -Support for dynamic negotiations with multiple cloud providers using brokering system. |  This model lacks security management. Event-driven programming styleof mOSAIC application is complex.
EGI FEDERATED CLOUD [41] |  EGI Federated cloud integrates public, private or hybrid cloud in the form of multi- national cloud system. |  Each resource center operates cloud management framework (CMF) according to its own preferences and then joins Federation and Collaboration services. |  User can access all services of cloud federation with a single identity. |  This federated cloud model lacks trust and unauthorized access issues and other security features are not considered.
Table 1: Comparative Summary continued Model, Author |  Aim |  Conceptual Approach |  Strength |  Weakness
---|---|---|---|---
FEDERATED CLOUD MANAGEMENT (FCM) ARCHITECTURE (Marosi et al., 2011) [35] |  FCM incorporates high level brokering for cloud federation and on demand automated service deployments |  several Infrastructure Providers exclusively dedicate their computing infrastructure to a single cloud broker, which further selects VM from underlying infrastructure provider for actual execution of service |  Conceptually, the model is simple and easy to understand |  Interoperable issues between various cloud brokers in practical scenario is not considered. Lack of management and security constraints
CONTAINER TECHNOLOGY [42] |  containers aims to provide easy to deploy and secure orchestration of specific self- contained virtual infrastructure |  Containers create an isolation boundary at application level rather than at server level |  Containers only requires software , libraries and basics of an operating system and cloud containers can be very easily deployed on another server very easily. |  Containers splits the virtualization into smaller chunks, that creates difficulty in management
SPINNAKER [43] |  Spinnaker is an open source ,multi- cloud delivery platform |  It provides two important features:1.Cluster management 2.Deployment management. |  Multiple heterogeneous cloud services can be availed through single API. |  Management and handling of multiple cloud clusters by connecting through pipelines is a complex. It lacks security management.
Cloud Foundry [44] |  It aims to provide a standardized platform to the applications of customers by decoupling the application from its infrastructure |  CF offers two types of VMs: the component VMs as platform infrastructure and the host VMs that host applications for outside world. |  The organizations can easily make a business decision on where to deploy workloads i.e. on premise, in managed infrastructure |  This model do not handles load balancing, security and data storage handling issues.
SUPER CLOUD [45] |  This work aims to provide self- managed, self- protecting virtual private clouds separately for each user. |  All the CSPs are at bottom layer,supercloudlayer handles all the interoperability issues and provides complete ease of usage in the hands of user. |  Supercloud is self-managed, self-protecting cloud of clouds,providing complete management of multi- clouds in the hands of user. |  Particularly in our view, this project model leaves no concern untouched regarding multi-cloud set- up as such, also this project work is in progress till now. But this project favours users more than the cloud service providers.
UTILITY-ORIENTED FEDERATION OF CLOUD COMPUTING ENVIRONMENTS [46] |  Inter-cloud framework which supports scaling of applications across multiple vendor clouds. |  Every client in federated environment needs to contact cloud brokering service that can dynamically establish service contracts with cloud coordinators through trading functions exposed by cloud Exchange |  This model is efficient with respect to trust as cloud broker selects CSP by negotiating with cloud coordinators. |  Though trust is given priority in this model, still there seems no concern for security management.

After analyzing various models of multi-cloud set-up reveals that most of the
work done in this regard( taking services from multi- cloud) handles only
interoperability issues and lacks concern for trust and security management
while collaborating. Also, much of the research is limited upto theoretical
concept of joining multiple clouds together by having some Broker service
between various infrastructure providers and service providers. Still in our
opinion, SUPERCLOUD project model is most suitable among the various available
multi-cloud architectures till now. This is because, it has covered almost
every concerned feature for enabling successful set- up of multi-cloud.

##  3 Conclusion and Future Directions

Single-cloud services are not suitable to meet the ever growing and variety of
demands of the organization. Hence, more effective way is to employ multiple
clouds to serve an organization. Every data cannot be treated with‘equal
status. As some data may be more confidential than other, and some data will
be completely public for the advertisement. Hence according to the security
requirement , cost and official management of different data, it can be placed
on different appropriate clouds as per the suitability of the organization.
But to take services from multiple clouds is quiet complex as you have to
individually deal with multiple cloud service providers. So, its better if
there would be some unified platform from where can access services of
multiple clouds- Multi-cloud infrastructure. To set-up multi-cloud, their
exists lots of complex issues- like interoperability, monetizing, security
management issues etc. In order to deal with these issues many researchers
from academic as well as industrial background have provided solutions in the
form of federated cloud or multi-cloud architectures. In this paper, we have
presented analytical review of various multi-cloud architectures and also we
have given comparative analysis in the form of pros and cons of each
architecture. Finally, we have concluded that in our view, SUPERCLOUD project
model is at the top among all the existing multi-cloud architecture concepts.
But this project is in progress and it is not fully implemented work.

## References

  * [1] S. Deepika and S. A. Kumar, “A proactive autoscaling and energy-efficient vm allocation framework using online multi-resource neural network for cloud data center,” _Neurocomputing_ , vol. 426, pp. 248–264, 2021.
  * [2] D. Saxena and A. K. Singh, “Communication cost aware resource efficient load balancing (care-lb) framework for cloud datacenter,” _Recent Advances in Computer Science and Communications_ , vol. 12, pp. 1–00, 2020.
  * [3] D. Saxena, K. S. Vaisla, and M. S. Rauthan, “Abstract model of trusted and secure middleware framework for multi-cloud environment,” in _International Conference on Advanced Informatics for Computing Research_. Springer, 2018, pp. 469–479.
  * [4] D. Saxena, I. Gupta, J. Kumar, A. Singh, and W. Xiaoqing, “A secure and multi-objective virtual machine placement framework for cloud data center,” _IEEE Systems Journal_ , 2021.
  * [5] A. K. Singh and D. Saxena, “A cryptography and machine learning based authentication for secure data-sharing in federated cloud services environment,” _Journal of Applied Security Research_ , pp. 1–24, 2021.
  * [6] A. K. Singh, D. Saxena, J. Kumar, and V. Gupta, “A quantum approach towards the adaptive prediction of cloud workloads,” _IEEE Transactions on Parallel and Distributed Systems_ , 2021.
  * [7] A. Singh and J. Kumar, “Secure and energy aware load balancing framework for cloud data centre networks,” _Electronics Letters_ , vol. 55, no. 9, pp. 540–541, 2019.
  * [8] D. Saxena and S. Chauhan, “A review on dynamic fair priority task scheduling algorithm in cloud computing,” _International Journal of Science, Environment and Technology_ , vol. 3, no. 3, pp. 997–1003, 2014.
  * [9] S. Saxena and D. Saxena, “Ewsa: An enriched workflow scheduling algorithm in cloud computing,” in _2015 International Conference on Computing, Communication and Security (ICCCS)_. IEEE, 2015, pp. 1–5.
  * [10] D. Saxena, R. Chauhan, and R. Kait, “Dynamic fair priority optimization task scheduling algorithm in cloud computing: concepts and implementations,” _International Journal of Computer Network and Information Security_ , vol. 8, no. 2, p. 41, 2016.
  * [11] D. Saxena and A. K. Singh, “Auto-adaptive learning-based workload forecasting in dynamic cloud environment,” _International Journal of Computers and Applications_ , pp. 1–11, 2020.
  * [12] D. Saxena and A. Singh, “Security embedded dynamic resource allocation model for cloud data centre,” _Electronics Letters_ , vol. 56, no. 20, pp. 1062–1065, 2020.
  * [13] D. Saxena and A. K. Singh, “Energy aware resource efficient-(EARE) server consolidation framework for cloud datacenter,” in _Advances in communication and computational technology_. Springer, 2021, pp. 1455–1464.
  * [14] J. Kumar, R. Goomer, and A. K. Singh, “Long short term memory recurrent neural network (lstm-rnn) based workload forecasting model for cloud datacenters,” _Procedia Computer Science_ , vol. 125, pp. 676–682, 2018.
  * [15] J. Kumar and A. K. Singh, “Workload prediction in cloud using artificial neural network and adaptive differential evolution,” _Future Generation Computer Systems_ , vol. 81, pp. 41–52, 2018.
  * [16] J. Kumar, D. Saxena, A. K. Singh, and A. Mohan, “Biphase adaptive learning-based neural network model for cloud datacenter workload forecasting,” _Soft Computing_ , pp. 1–18, 2020.
  * [17] J. Kumar, A. K. Singh, and A. Mohan, “Resource-efficient load-balancing framework for cloud data center networks,” _ETRI Journal_ , vol. 43, no. 1, pp. 53–63, 2021.
  * [18] D. Saxena, A. K. Singh, and R. Buyya, “OP-MLB: An online vm prediction based multi-objective load balancing framework for resource management at cloud datacenter,” _IEEE Transactions on Cloud Computing_ , 2021.
  * [19] D. Saxena and A. K. Singh, “OSC-MC: Online secure communication model for cloud environment,” _IEEE Communications Letters_ , 2021.
  * [20] D. Saxena and S. Saxena, “Highly advanced cloudlet scheduling algorithm based on particle swarm optimization,” in _2015 Eighth International Conference on Contemporary Computing (IC3)_. IEEE, 2015, pp. 111–116.
  * [21] S. Chhabra and A. K. Singh, “A secure vm allocation scheme to preserve against co-resident threat,” _International Journal of Web Engineering and Technology_ , vol. 15, no. 1, pp. 96–115, 2020.
  * [22] I. Gupta and A. K. Singh, “A confidentiality preserving data leaker detection model for secure sharing of cloud data using integrated techniques,” in _2019 7th International Conference on Smart Computing & Communications (ICSCC)_. IEEE, 2019, pp. 1–5.
  * [23] R. Gupta, D. Saxena, and A. K. Singh, “Data security and privacy in cloud computing: Concepts and emerging trends,” _arXiv preprint arXiv:2108.09508_ , 2021.
  * [24] D. Varshney, B. Babukhanwala, J. Khan, D. Saxena, and A. kumar Singh, “Machine learning techniques for plant disease detection,” in _2021 5th International Conference on Trends in Electronics and Informatics (ICOEI)_. IEEE, 2021, pp. 1574–1581.
  * [25] A. S. Chauhan, D. Rani, A. Kumar, R. Gupta, and A. K. Singh, “A survey on privacy-preserving outsourced data on cloud with multiple data providers,” in _Proceedings of the International Conference on Innovative Computing & Communications (ICICC)_, 2020.
  * [26] D. Saxena, R. Chauhan, and S. Saxena, “Global journal of engineering science and researches.”
  * [27] K. Kaur, I. Gupta, A. K. Singh _et al._ , “A comparative evaluation of data leakage/loss prevention systems (dlps),” in _Proc. 4th Int. Conf. Computer Science & Information Technology (CS & IT-CSCP)_, 2017, pp. 87–95.
  * [28] S. Chhabra and A. K. Singh, “A probabilistic model for finding an optimal host framework and load distribution in cloud environment,” _Procedia Computer Science_ , vol. 125, pp. 683–690, 2018.
  * [29] R. Patel, V. Choudhary, D. Saxena, and A. K. Singh, “Review of stock prediction using machine learning techniques,” in _2021 5th International Conference on Trends in Electronics and Informatics (ICOEI)_. IEEE, 2021, pp. 840–846.
  * [30] D. Saxena and A. K. Singh, “workload forecasting and resource management models based on machine learning for cloud computing environments,” _arXiv preprint arXiv:2106.15112_ , 2021.
  * [31] J. Kumar, A. K. Singh, and R. Buyya, “Ensemble learning based predictive framework for virtual machine resource request prediction,” _Neurocomputing_ , vol. 397, pp. 20–30, 2020.
  * [32] Q. Zhang, S. Li, Z. Li, Y. Xing, Z. Yang, and Y. Dai, “Charm: A cost-efficient multi-cloud data hosting scheme with high availability,” _IEEE Transactions on Cloud computing_ , vol. 3, no. 3, pp. 372–386, 2015.
  * [33] A. J. Ferrer, F. Hernández, J. Tordsson, E. Elmroth, A. Ali-Eldin, C. Zsigri, R. Sirvent, J. Guitart, R. M. Badia, K. Djemame _et al._ , “Optimis: A holistic approach to cloud service provisioning,” _Future Generation Computer Systems_ , vol. 28, no. 1, pp. 66–77, 2012.
  * [34] A. Galis, E. Elmroth, W. Emmerich, F. Galán, and S. Telefónica, “The reservoir model and architecture for open federated cloud computing,” _IEEE Computer Society Press_ , vol. 20, pp. 115–187, 2009.
  * [35] A. C. Marosi, G. Kecskemeti, A. Kertesz, and P. Kacsuk, “Fcm: an architecture for integrating iaas cloud systems.” IARIA, 2011.
  * [36] I. Clouds, “A white paper from the open cloud standards incubator,” _Distributed Management Task Force, Version_ , vol. 1, 2009.
  * [37] E. Carlini, M. Coppola, P. Dazzi, L. Ricci, and G. Righetti, “Cloud federations in contrail,” in _European Conference on Parallel Processing_. Springer, 2011, pp. 159–168.
  * [38] X. Zhang, B. Li, and J. Zhu, “A combinatorial auction-based collaborative cloud services platform,” _Tsinghua Science and Technology_ , vol. 20, no. 1, pp. 50–61, 2015.
  * [39] D. Villegas, N. Bobroff, I. Rodero, J. Delgado, Y. Liu, A. Devarakonda, L. Fong, S. M. Sadjadi, and M. Parashar, “Cloud federation in a layered service model,” _Journal of Computer and System Sciences_ , vol. 78, no. 5, pp. 1330–1344, 2012.
  * [40] D. Petcu, B. Di Martino, S. Venticinque, M. Rak, T. Máhr, G. E. Lopez, F. Brito, R. Cossu, M. Stopar, S. Šperka _et al._ , “Experiences in building a mosaic of clouds,” _Journal of Cloud Computing: Advances, Systems and Applications_ , vol. 2, no. 1, pp. 1–22, 2013.
  * [41] E. Fernández-del Castillo, D. Scardaci, and Á. L. García, “The egi federated cloud e-infrastructure,” _Procedia Computer Science_ , vol. 68, pp. 196–205, 2015.
  * [42] V. V. Trofimov, V. I. Kiyaev, and S. M. Gazul, “Use of virtualization and container technology for information infrastructure generation,” in _2017 XX IEEE International Conference on Soft Computing and Measurements (SCM)_. IEEE, 2017, pp. 788–791.
  * [43] S. B. Furber, F. Galluppi, S. Temple, and L. A. Plana, “The spinnaker project,” _Proceedings of the IEEE_ , vol. 102, no. 5, pp. 652–665, 2014\.
  * [44] D. Bernstein, “Cloud foundry aims to become the openstack of paas,” _IEEE Cloud Computing_ , vol. 1, no. 2, pp. 57–60, 2014.
  * [45] M. Lacoste, M. Miettinen, N. Neves, F. M. Ramos, M. Vukolic, F. Charmet, R. Yaich, K. Oborzynski, G. Vernekar, and P. Sousa, “User-centric security and dependability in the clouds-of-clouds,” _IEEE Cloud Computing_ , vol. 3, no. 5, pp. 64–75, 2016.
  * [46] R. Buyya, R. Ranjan, and R. N. Calheiros, “Intercloud: Utility-oriented federation of cloud computing environments for scaling of application services,” in _International Conference on Algorithms and Architectures for Parallel Processing_. Springer, 2010, pp. 13–31.