monitoring resource and infrastructure management for containerized
applications in multi-cloud environment. Challenges include network
communication, system image and configuration monitoring, and dynamic resource
management. Solutions involve service mesh implementation, Docker server
deployment, and using Platform-as-a-Service (e.g., Aneka PaaS) for Cloud
Computing.

Security, Compliance, and Trust Monitoring: This category addresses the vital
aspects of monitoring security, compliance, and trust management for
containerized applications in multi-cloud settings. Challenges include
accessing provider host information, continuous security control, and multi-
cloud security compliance. Solutions include ML-based monitoring and security
assurance platforms in the form of a SaaS (e.g.,MUSA), which support multi-
cloud application runtime security control and transparency to enhance user
trust.

Application Optimization and Adaptability Monitoring: The challenges of this
category revolve around monitoring application optimization and adaptability
in multi-cloud environment. Challenges include real-time application
monitoring, microservices optimization, application behavior, download
execution, and design and state monitoring. To address these challenges,
solutions encompass the Dynamic Resource Allocation (DRA) framework,
Microservices-based Cloud Application-level Dynamic Orchestrator (MiCADO)
optimization, Topology and Orchestration Specification for Cloud Applications
(TOSCA) event modeling, FogDocker implementation, and adopting a cloud
modeling framework.

System Complexity and Standardization Monitoring: This category focuses on
addressing system complexity and promoting standardization in monitoring
practices across multi-cloud environment. Challenges include proprietary
monitoring solutions and resource isolation mechanisms in the Linux kernel.
Solutions encompass pervasive monitoring and identifying in-container leakage
channels.

Multi-cloud Coordination and Integration Monitoring: In this category, the
challenges revolve around coordinating and integrating monitoring data across
multiple cloud environments. Challenges include multi-cloud monitoring
challenges, cross-level monitoring improvement, and integrated multi-cloud
management monitoring. Solutions encompass self-healing techniques, business
process monitoring, and integration with tools like Gnocchi.

Takeaway 19: Maintaining consistent and reliable performance monitoring in a
dynamic multi-cloud setup requires employing strategies such as benchmarking,
using Kubernetes for containerized services, and adopting standardized
performance metrics Takeaway 20: Effective monitoring of resource utilization,
network communications, and infrastructure across multiple clouds necessitates
the use of tools such as service mesh for managing service-to-service
communication, container orchestration platforms like Kubernetes, and
dedicated monitoring platforms for comprehensive visibility and control.
Takeaway 21: For effective monitoring of containerized applications in a
multi-cloud environment, it is essential to integrate advanced techniques such
as machine learning for real-time anomaly detection, comprehensive security
assurance platforms for continuous surveillance and threat mitigation, and
specialized privacy-centric solutions to ensure data integrity and compliance.

Table 14. Monitoring challenge and solution framework for containerized applications in multi-cloud environment ID |  Challenge |  Challenge Description |  Solution |  Solution Description
---|---|---|---|---
Category 1: Performance Monitoring Consistency and Variability
S01 |  Performance Monitoring |  Ensuring sustained performance monitoring in dynamically varying cloud environments |  Performance Variation Monitoring |  Monitoring performance fluctuations in containerized applications through benchmarking periods
S02 |  Benchmarking Performance Variation and Cost Efficiency Monitoring |  Efficiently tracking performance disparities and cost-effectiveness for refined monitoring in benchmarking |  SDBO Optimization |  Maximizing user budget utility with the optimization feature in SDBO
S29 |  Website Performance and Availability Monitoring |  Implementing continuous oversight over website availability and performance |  Kubernetes Monitoring |  Utilizing Kubernetes for monitoring performance of containerized services in multi-cloud environment
S38 |  User Experience Variability Monitoring |  Addressing monitoring fluctuations in user experience based on diverse factors |  CSP QoS Metrics Collection |  Collecting Cloud Service Provider (CSP) Quality of Service (QoS) metrics for benchmark data in multi-cloud environment
S43 |  Independent Performance Metrics Monitoring |  Establishing application-independent performance metrics for Round Trip Time (RTT) and payload throughput |  Performance Evaluations |  Conducting performance evaluations and autonomic resource provisioning for multi-cloud and MEC platforms using containerized applications
S50 |  Comprehensive Network Performance Measurement Monitoring |  Ensuring comprehensive performance measurements monitoring and accounting for transmission delays in the network |  Server Selection Scheme |  Applying a server selection scheme to minimize latency in multi-cloud environment based on end-to-end performance measurements
S62 |  Microservice Performance Degradation Monitoring |  Addressing unpredictable performance degradation in containerized microservices |  Tail Latency Prediction |  Implementing a modeling approach to predict end-to-end tail latency in multi-layer cloud data of containerized applications
Category 2: Resource and Infrastructure Management Monitoring
S05 |  Network Communication Monitoring |  Managing service-to-service communications over a network |  Service Mesh Implementation |  Deploying a network service mesh for service-discovery, secure connectivity, and visibility in multi-cloud platforms
S09 |  System Image and Configuration Monitoring |  Monitoring of system images, remote software packages, usage history, and container execution, configuration, and monitoring. |  Docker Server Deployment |  Deploying containerized services using Docker Server API version 1.23 in multi-cloud platforms
S21 |  Vcluster and Framework Management Monitoring |  Monitoring and managing the performance and resource usage of each Vcluster and application framework can be complex when dealing with multiple tenants and different types of applications. |  LXC Implementation |  Using Lightweight Containers (LXC) for efficient resource allocation and scalability in multi-cloud environment
S24 |  Multi-Cloud Resource Utilization Monitoring |  Monitoring container performance and resource usage across multiple clouds |  Prometheus and Grafana Monitoring |  Implementing Prometheus and Grafana for monitoring performance and availability of containerized applications in multi-cloud environment
S51 |  Dynamic Resource Management Monitoring |  Applications should add and release resources dynamically by measuring and monitoring usage |  Aneka Platform |  Utilizing the Aneka platform for monitoring resources and managing dynamic resource acquisition and release in multi-cloud environment
Category 3: Security, Compliance, and Trust Management Monitoring
S04 |  Provider Host Information Access Monitoring |  Overcoming the restriction in accessing provider host information |  ML Monitoring |  Using machine learning to supervise performance of containerized applications and OS usage
S34 |  Continuous Security Control Monitoring |  Strengthening continuous monitoring and control over security |  MUSA Platform |  Applying the MUSA Security Assurance Platform for continuous monitoring and dynamic security enforcement of containerized applications
S52 |  Multi-Cloud Security Compliance Monitoring |  Monitoring security compliance consistently across multiple cloud providers |  MUSA Security Compliance |  Using the MUSA Security Assurance Platform to monitor multi-cloud application compliance and ensure security
S53 |  Service Continuity and Data Privacy Monitoring |  Ensuring data privacy, uninterrupted service, fault tolerance, and availability |  NoMISHAP Implementation |  Implementing the NoMISHAP solution to address monitoring challenges in multi-cloud environment with containerized applications
S83 |  Trust Relationship and Network Contact Monitoring |  Building trust relationships, collecting judgments, and maintaining networks of contacts |  Trust Bootstrapping |  Implementing a trust bootstrapping mechanism that combines endorsement and decision tree classification for containerized applications in multi-cloud platforms
Category 4: Application Optimization and Adaptability Monitoring
S20 |  Adaptive System Design and State Monitoring |  Implementing continuous design with dynamic adaptability; Enhancing system state evaluation through robust monitoring |  Cloud Modelling Framework |  Utilizing a Cloud Modelling Framework for continuous design and dynamic adjustment of multi-cloud containerized applications
S36 |  Partial Download Execution Monitoring |  Ensuring accurate execution of applications for partially downloaded files |  FogDocker Implementation |  Using FogDocker with low runtime overhead to ensure execution correctness of containerized applications
S44 |  Event-Driven Application Behavior Monitoring |  Modeling event-triggered behavior of application components |  TOSCA Event Modeling |  Applying TOSCA modeling for event sources, events, and functions in containerized applications
S68 |  Microservice Optimization Monitoring |  Enhancing surveillance over microservices across multiple cloud environments for optimization |  MiCADO Optimization |  Utilizing MiCADO for runtime monitoring and optimization of cloud applications in multi-cloud environment
S85 |  Real-Time Application Monitoring |  Real-time monitoring of applications for immediate corrective action |  DRA Framework |  Implementing the DRA framework with the Monitoring and Tracking Platform (M4) for containerized applications in multi-cloud platforms
Category 5: System Complexity and Standardization Monitoring
S10 |  Proprietary Monitoring Solution Standardization |  Overcoming the constraints of complex, proprietary monitoring solutions and fostering standardization |  Pervasive Monitoring |  Implementing pervasive monitoring solutions for containerized applications within multi-cloud PaaS
S25 |  Kernel Resource Isolation Mechanism Monitoring |  Addressing the limitations of resource isolation mechanisms in the Linux kernel |  Leakage Defense |  Identifying in-container leakage channels and implementing a two-stage defensive approach in multi-cloud environment
S27 |  Multi-Cloud Application Monitoring Complexity |  Simplifying monitoring processes in multi-cloud applications |  DECIDE DevOps Expansion |  Expanding the DECIDE DevOps framework for native multi-cloud containerized applications
S28 |  Monitoring Agent Coordination |  Enhancing the coordination of monitoring agents for improved system behavior |  Autonomic Management |  Implementing an autonomic management system with monitoring agents, a performance model, and adaptive actions for containerized applications
S33 |  Heterogeneous Fog Resource Management Monitoring |  Efficient management of diverse fog resources |  Kubernetes Utility Architecture |  Implementing a master-worker nodes architecture with Kubernetes Utility (Kubeadm) in multi-cloud environment
S42 |  Application Topology and Dependency Monitoring |  Enhancing coordinated adaptation through thorough understanding of application topologies and dependencies |  MicroCloud TOSCA Library |  Utilizing the MicroCloud TOSCA library for standardized description of topologies in multi-cloud containerized applications
S45 |  Elastic Cloud Service Monitoring Research |  Investigating elastic cloud service monitoring as an open research problem |  JCatascopia Monitoring |  Utilizing JCatascopia for multi-layer, platform-independent monitoring of cloud containerized applications
S46 |  Microservice Monitoring Diversity |  Addressing diverse monitoring requirements and varying metrics quality in microservices |  QoM Definition |  Defining Quality of Monitoring (QoM) and implementing an API for cloud provider capabilities in multi-cloud environment
S49 |  Compute and Network Resource Utilization Monitoring |  Strengthening surveillance and prediction capabilities of compute and network resource utilization |  Time Series Resource Utilization |  Developing algorithms for monitoring and predicting resource utilization of containerized applications using time series data
Category 6: Multi-cloud Coordination and Integration Monitoring
S58 |  Integrated Multi-Cloud Management Monitoring |  Implementing integrated monitoring and management across multi-cloud environment |  Gnocchi Integration |  Integrating with federated gnocchi for consolidated monitoring and management of multi-cloud environment
S74 |  Multi-Cloud Monitoring Challenges |  Addressing monitoring challenges in multi-cloud environment |  Self-Healing Techniques |  Implementing a rule-based algorithm for monitoring and self-healing techniques in multi-cloud environment
S84 |  Cross-Level Monitoring Improvement |  Ensuring precise and adaptable metrics computation, overcoming inadequate cross-level monitoring approaches |  Business Process Monitoring |  Conducting cross-cloud and cross-level business process monitoring and coordinated adaptation of containerized applications

###  3.9. Tools and Frameworks (RQ6)

In response to our RQ6 concerning the tools and frameworks utilized for
developing containerized multi-cloud applications, an in-depth exploration of
the selected studies was conducted. This comprehensive analysis led to the
identification of a total of 160 distinct tools and frameworks (see Table
LABEL:tab:ToolsandFrameworks), all of which are systematically classified into
45 overarching categories, further delineated into 6 subcategories. Notably,
the three most prominent categories emerged from this investigation are cloud
services and management platform, infrastructure management technologies, and
networking tools and framework. Each of these categories encapsulates a range
of tools aimed at facilitating the seamless development, deployment, and
management of multi-cloud applications. In the following we briefly report the
each of the category, shedding light on the tools and frameworks that
collectively form the foundation for containerized multi-cloud application
development.

Cloud Services and Management Platforms: Cloud Services and Management
Platforms: This category of tools facilitates the management and orchestration
of cloud services across multi-cloud environments. It includes a broad range
of platforms covering Infrastructure as a Service (IaaS), Platform as a
Service (PaaS), and Software as a Service (SaaS). These tools are essential
for provisioning virtual resources, deploying applications, and delivering
software across different cloud platforms. Examples include Amazon EC2, Google
Cloud App Engine, and Microsoft Office 365.

Infrastructure Management Technologies: This category of tools facilitates the
management of cloud infrastructure. It encompasses container platforms,
virtualization solutions, Kubernetes services, orchestration platforms, and
tools for configuration management and continuous integration. Key
technologies like Docker, Kubernetes, Puppet, Chef, and Terraform play a vital
role in automating and optimizing the infrastructure underpinning
containerized applications.

Monitoring Tools: This category of tools facilitates the monitoring and
analysis of cloud and container operations. It includes solutions for cloud
monitoring, container monitoring, visualization & alerting, resource
monitoring, and logging & analysis. Tools such as Amazon CloudWatch, cAdvisor,
Prometheus, Grafana, and the ELK stack are instrumental in ensuring the
performance and reliability of cloud services.

Networking Tools: This category of tools facilitates the establishment and
management of network functionalities in a multi-cloud environment. It covers
load balancers, reverse proxies, network services, controllers, paradigms,
types, and identity protocols, along with storage networks and event buses.
Technologies like Software-Defined Networking (SDN), Neutron, and Kafka event
bus are crucial for maintaining robust and efficient network operations.

Development and Programming: This category of tools facilitates the
development and programming aspects in cloud environments. It includes
orchestration tools, various programming languages, integrated development
environments (IDEs), service meshes, APIs, and cloud modeling tools. Examples
are Java, Python, Eclipse IDE, Istio, REST API, and Cloud Modelling Framework
(CLOUDMF), which support the creation and deployment of cloud-native
applications.

Security and Assurance: This category of tools facilitates the security and
compliance of cloud environments. It encompasses vulnerability management,
security platforms & tools, access control & policies, Linux security, and
cryptography models & tools. Technologies like OpenVAS Vulnerability Scanner,
Kubernetes.eBPF, Seccomp-BPF, and Attribute-Based Encryption and Signature are
essential for protecting cloud infrastructure and data against various
security threats.

Table 15. Tools and frameworks for containerized applications in multi-cloud environment Subcategory |  Tools and Frameworks
---|---
Category 1: Cloud Services and Management Platforms
Cloud Management Platforms |  Cloudyn (S65), IBM Brokerage Services (S65), CloudBroker Platform (S69), CloudcheckR (S79), MistIO (S79), ManageIQ (S67, S79), CMP2 (S79), AWS SDK (S71), CloudSME Simulation Platform (CSSP) (S69), MiCADO (S69), WS-PGRADE/gUSE (S69), Vagrant (S08), AWS OpsWorks (S38), Universal Compute Xchange (UCX) (S65), Open vSwitch (OvS) (S72), Liferay portlets (S69), Cloud Service Providers (CSPs) (S71), AWS Data Pipeline (S78), Cloud computing (S25), Amazon AWS (S42, S43), GENI (S38), Google Cloud Platform (S38, S53), Microsoft Azure (S38, S48, S51, S54, S60), AWS EC2 (S51), IBM (S49), Amazon Web Services (S54), Google Cloud (S54), SalesForce (S53), Multi-Cloud (S60), Cloud computing platforms (S74), Multiple independent clouds (S74), Cloud computing (S84)
Cloud Deployment Model |  Public clouds: (S37, S53, S59, S61), Google cloud (S07), Microsoft Azure (S08, S16, S19), AWS (S65, S67, S80, S81), Google’s cloud (S65), Google Compute Engine (S67, S80), Alibaba (S65), Tencent (S65), IBM Watson Health Cloud (S78). Private cloud: OpenStack (S20, S21, S43), VMware vSphere Specialized Cloud Platforms: European Grid Infrastructure Federated Cloud (EGI FC) (S81)
Infrastructure as a Service (IaaS) |  Amazon EC2 (S01, S51), AWS (S02, S65, S67, S80, S81), Azure (S02, S08, S16, S19, S67, S80), Google GCE (S03, S67, S80), Aliyun ECS (S03), Baidu BCE (S03), Tencent CVM (S03), JD VM (S03), VMware vSphere (S08), GoGrid (S51), Windows Azure (S51)
Platform as a Service (PaaS) |  Google Cloud App Engine (S02, S10, S15), Heroku (S08, S53), IBM Bluemix Workflow Service (S17, s53), Amazon Elastic Beanstalk (S20), Cloud Foundry (S24), OpenShift (S24, S53, S55)
Software as a Service (SaaS) |  SaaS (S15), Microsoft Office 365 (S53)
Distributed Processing Frameworks |  Hadoop (S21, S37), Spark (S21, S28), Apache Yarn (S37), Storm (S21), Apache Kafka (S34, S35)
Data Storage |  Apache Cassandra (S28), Firebase real-time database (S35), Redis (S36, S42), MySQL (S42), NoSQL (S85), SQL (S85)
Messaging Systems |  Apache Kafka (S34, S35), Zookeeper (S35)
Category 2: Infrastructure Management Technologies
Container Platforms |  Docker (S08, S13, S16, S17, S18, S25, S27, S28, S29, S30, S31, S32, S35, S55, S59, S62, S74, S77, S79, S80), Smart Docker Benchmarking Orchestrator (SDBO) (S01), Docklet (S21), LXC (S24, S25, S29), Docker container engine (S24), Docker container (S23), Docker CE version 18.01 (S36), Docker Hub (S36)
Virtualization |  Virtual Machines (S05, S22, S23, S46, S72), Amazon EC2 VMs (S18), Docker container virtualization technology (S74), Xen (S19), VMware (S19), Hyper-V (S19), KVM (S19), OpenVZ (S19), Libvirt API (S19)
Kubernetes Services |  Microsoft AKS (S03), Aliyun Kubernetes Engine (S03), Tencent Kubernetes Engine (S03)
Orchestration Platforms |  Kubernetes (S18, S22, S29, S55, S56, S59, S62, S63, S80), Docker Swarm (S18, S22, S39, S59, S80), Mesos (S21, S22, S59, S80), Yarn (S21), Kubeadm (S33), Kubernetes for container orchestration (S62), Apache Mesos (S59, S80), MiCADO (S69)
Configuration Management |  Puppet (S08), Chef (S08, S30), Terraform (S27, S29), Shell scripts (S44), CloudBees (S08) Continuous Integration CloudBees (S08), Maven (S08), NPM (S08), Github (S30), Github (S30)
Container Isolation |  Sandboxie (S24), Warden/Garden (S24), Rocket (S24)
Container Infrastructure |  Linux Container (S16), Docker Containers on Linux Host (S13), Kubernetes cluster with a Docker (S04), Amazon AWS ECS (S18), Container Technology (S23)
Category 3: Infrastructure Management Technologies
Load Balancer |  Load balancer (S23)
Reverse Proxy |  Reverse proxy technology (S23), httpd (S36), nginx (S36)
Ports and Interfaces |  TrunkPort (S55)
Network Services |  Neutron (S55)
Gateways and Tunnels |  Gateways (S64), GRE Tunnel (S72)
Network Controllers |  Software-Defined Networking (SDN) controller (S72)
Identity Protocols |  Host Identity Protocol (HIP) (S77)
Storage Networks |  MCSS (S75), Multi-Cloud storage overlay network (S75)
Network Tools |  OpenADN (S50), OpenDayLight (S50), Kuryr (S55), Flannel (S55)
Event Buses |  Kafka event bus (S34)
Category 4: Development and Programming
Orchestration |  MUSA Deployer tool (S52), MUSA Modeler tool (S52), OSM (S55), ManageIQ (S55)
Programming Languages |  Java (S30, S34, S35, S54, S80, S81), Go programming language (S32, S36), Python (S35, S54, S56, S58, S79), C# (S54), Ruby (S54), PHP (S54)
IDEs and Editors |  Eclipse IDE with tight integration with OPTIMUS (S27), MCSLA editor (S27)
Service Mesh |  Consul (S05), Istio (S05), Istio service mesh (S59)
APIs |  REST API (S56), OpenStack API (S64)
Cloud Modeling |  Cloud Modelling Framework (CLOUDMF) (S20), Cloud Modelling Language (CLOUDML) (S20, S81), CAMEL modeling language (S52, S80, S84)
Category 5: Development and Programming
Vulnerability Management |  OpenVAS Vulnerability Scanner (S30)
Security Platforms and Tools |  MUSA Security Assurance Platform (S34), Firewall Deployment (S30)
Access Control and Policies |  Access control policies (S25)
Linux Security |  Linux top command (S34), Kubernetes.eBPF (S40), Seccomp-BPF (S40), Landlock LSM (S40), Pledge (S40), Security Namespace (S40), Linux Security Modules (LSM) (S40)
Cryptography Models and Tools |  Attribute-Based Encryption and Signature (S75), ABEABS cryptographic model (S75)
Category 6: Monitoring Tools
Cloud Monitoring |  Amazon CloudWatch (S19)
Container Monitoring |  cAdvisor (S22), Prometheus (S22), AlertManager (S22)
Visualization and Alerting |  Grafana (S27), Telegraf (S27)
Resource Monitoring |  Resource monitoring tool (S60), Ceilometer (S56, S58), Gnocchi (S58)
Logging and Analysis |  ELK stack (S74)

Takeaway 22: We identified a wide array of cloud services, including IaaS,
PaaS, and SaaS, offered by major providers like AWS, Azure, and Google Cloud.
Takeaway 23: Technologies like Docker and Kubernetes play a significant role,
enabling efficient containerization and orchestration of applications across
different cloud platforms. Takeaway 24: We identified tools like Puppet, Chef,
and Terraform, underlining the importance of automation in modern software
development for streamlined deployment and management.

##  4\. Discussion

###  4.1. Containers in Multi-Cloud: Roles and Strategies (RQ1)

Containers play a crucial role in the development and operation of modern
software systems within multi-cloud environment. Understanding the various
roles and implementation strategies of containers provides significant
insights for both researchers and practitioners.

Multifaceted Role of Containers in Multi-Cloud Environment: Containers fulfill
diverse roles within multi-cloud environment, such as application deployment,
service hosting, and resource management. They facilitate deployment of
applications across a broad range of systems, from web apps to IoT
applications. Furthermore, containers contribute to enhancing security in
multi-cloud operations. For researchers, this variety of roles signifies a
rich area for inquiry, with opportunities to investigate potential security
vulnerabilities, management complexities, and performance overheads. For
practitioners, understanding these roles can inform efficient and effective
deployment strategies, leading to more robust multi-cloud implementations.

Container Technology Features and their Significance: The unique features of
container technology, such as lightweight virtualization, compatibility with
orchestration tools, and application in edge computing, offer significant
advantages. For researchers, these features present opportunities for further
exploration, including how these characteristics (e.g., lightweight
virtualization) influence container performance and effectiveness in multi-
cloud operations. For practitioners, understanding these features can lead to
better utilization of container technology and more efficient multi-cloud
deployments.

Implementation Strategies and their Real-World Implications: The strategic
implementation of containers significantly affects their functionality in
multi-cloud environment. Strategies related to deployment, orchestration,
security, performance optimization, cloud service management, and data storage
can have significant implications. For researchers, these strategies present
an area for exploration, including investigation into their real-world
implications, benefits, and limitations. For practitioners, a refined
understanding of these strategies can lead to more efficient implementations
and operations.

Containerization’s Impact on IoT and Application Development: The
incorporation of containers into IoT and app development has transform these
fields. Containers offer improved software unit encapsulation, portability,
and efficiency, which are particularly important in the distributed and
resource-constrained nature of IoT. Researchers can further explore potential
challenges in detail such as network complexity, security, and the small
footprint requirement of IoT devices. For practitioners, understanding the
impact of containerization can lead to the development of more robust,
efficient, and secure IoT and app solutions.

###  4.2. Patterns and Strategies (RQ2)

Containerized applications in multi-cloud environment requires an
understanding of prevailing patterns and strategies. In this context, we
discuss the key findings related identified patterns and strategies presented
in Table 9.

Diverse Architectural Patterns and Communication Strategies: Our study found
that Microservice Architecture and Service-Oriented Architecture (SOA) are the
most common used architectural patterns for designing container-based
applications in multi-cloud environment. These results align with the existing
literature that emphasizes the use of MSA and SOA because of their support for
modular development, scalability, and efficient resource use (Balalaie et al.,
2016; Dragoni et al., 2017). The other patterns frequently reported are
Service Mesh and Service Chaining, which are employed as communication and
networking patterns, likely due to their ability to manage complex,
distributed service-to-service communications in a scalable way (Zhamak,
2018). The frequent use of Blue-Green Deployment and Object Store Service
deployment patterns has also been identified in this study, which may stem
from their ability to minimize downtime and ensure high availability—essential
factors in a cloud-based setting (Fowler, 2010). Implications: Practitioners
can enhance the design, scalability, and efficiency of their applications in a
multi-cloud environment by understanding and applying these architectural and
communication patterns. Researchers might explore how these patterns evolve
with the emergence of new technologies and standards.

Importance of Container Management and Multi-Cloud Strategies: Our findings
report the significance of robust container management and multi-cloud
strategies. Techniques like the Linux Container (LXC) project, Docker
Container Images, and Lightweight Virtualization are frequently used,
emphasizing the industry trend towards containerization due to its benefits in
efficiency, scalability, and platform independence (Vaughn, 2020).
Furthermore, the popularity of multi-cloud strategies like Hybrid/Multi-cloud
Approach and Multi-cloud Load Balancing confirms the growing industry move
towards using multiple cloud providers to enhance redundancy, flexibility, and
avoid vendor lock-in (Gillam et al., 2013). Implications: Practitioners should
focus on gaining expertise in container technologies and multi-cloud
strategies to take advantage of their numerous benefits. Meanwhile,
researchers might examine the challenges and best practices associated with
the deployment and management of container technologies across multiple cloud
platforms.

Security, Resilience, and Migration Patterns and Strategies: Our study also
identifies patterns and strategies related to security, resilience, and
efficient migration strategies in a multi-cloud context. Strategies such as
File Encryption, Attribute-Based Encryption, and Secure Data sharing are often
employed to ensure data security, an area of growing concern with the increase
in cloud-based applications (Svantesson and Clarke, 2010). Moreover,
resilience strategies such as Fault-tolerance with Redundant Engines and RAFT
Consensus Algorithm are essential to ensure system reliability in distributed,
multi-cloud environment (Ongaro and Ousterhout, 2014).

Migration between different cloud environments, facilitated by strategies like
Service-oriented Migration and Migration to a Virtualized Container, is
another vital aspect, given the growth in cloud services and the need for
interoperability (Jamshidi et al., 2013). Migrating applications and data
between cloud environments is an important consideration in a multi-cloud
scenario (Jamshidi et al., 2013). Strategies such as image-based migration and
migration to virtualized containers, like Docker, are commonly used (Vaughn,
2020). These techniques facilitate portability and provide consistent
environments across different cloud platforms. Additionally, they have
significant implications for scaling and managing applications across
different cloud service providers. Implications: These findings suggest that
security, resilience, and efficient migration should be key areas of focus for
practitioners operating in multi-cloud environment. Researchers may also look
into novel techniques for enhancing cloud security, improving system
resilience, and easing the process of cloud migration.

###  4.3. Quality Attributes and Tactics (RQ3)

The exploration and implementation of quality attributes and associated
tactics are critical to the successful deployment of containerized
applications in multi-cloud environment, shaping performance, security, and
compatibility characteristics. In the following, we will discuss the key
findings of our study, providing insights into the QAs and tactics
specifically tailored for containerized applications operating within multi-
cloud environment.

Performance Optimization in Multi-Cloud Environment: Our study identifies the
role of machine learning, container technology, and location-aware service
brokering in enhancing the performance and efficiency of containerized
applications within multi-cloud environment. Hashem et al. previously
suggested similar efficiency gains from machine learning and container
technology, validating the interpretive value of our results (Hashem et al.,
2016). Specifically, the application of machine learning aligns with a broader
trend towards intelligent cloud resource management, a domain that Buyya et
al. identified as significant in cloud computing research (Buyya et al.,
2020). Meanwhile, location-aware service brokering emerges as a novel and
impactful strategy, stressing the role of geographical considerations in
further optimizing containerized applications’ performance. Implications: For
researchers, the results of our study offer opportunities to explore the
integration of AI techniques with container-based cloud architectures further,
especially the application of location-aware service brokering in multi-cloud
environment. For practitioners, the results indicate that a adaptable approach
involving AI, container technology, and geographical considerations can lead
to significant performance enhancements in multi-cloud environment.

Security Measures in Multi-Cloud Deployments: The results of our study
indicates the multi-faceted approach to ensuring security in multi-cloud
deployments. It highlights the combination of encryption, network security
protocols, machine learning, and user-specified security measures, while also
encouraging deployment across different cloud providers. This finding
complements Singh et al.’s argument for a comprehensive approach to cloud
security, and aligns with the recent concept of distributed cloud security
(Singh et al., 2016). Our results also report the importance of user-specified
security measures, which are in agreement with the user-centric security model
proposed by Shin and Dong-Hee (Shin, 2013). Implications: From a research
perspective, the interact between different security measures and their effect
on overall system security in multi-cloud environment warrants further
investigation. For practitioners, these results point out the importance of
employing a comprehensive security approach in multi-cloud deployments,
including encryption, network security, machine learning, and user-specific
security measures.

Ensuring Compatibility Across Different Cloud Providers: The results of our
study suggests strategies to ensure compatibility in multi-cloud environment
through standardization of interfaces, componentization, and interoperability
approaches. Emphasizing lightweight communication protocols and
interoperability between clouds, the results related to compatibility mirrors
Mell and Grance’s principles of service-oriented architecture, reinforcing the
need for effective communication across different cloud platforms (Mell et
al., 2011). Petcu et al. have also previously stressed the importance of
standardization and componentization for ensuring compatibility (Petcu et al.,
2018). Implications: From a research standpoint, the dynamic between
lightweight communication protocols, interface standardization, and overall
system compatibility in multi-cloud deployments offers fertile ground for
future work. Practitioners, meanwhile, can adopt these approaches to ensure
seamless communication and interoperability across different cloud providers.

These findings not only confirm existing literature results but also offer
valuable insights into the practical application of machine learning,
container technology, and location-aware services in multi-cloud environment.
Future research could further explore the intersection of these quality
attributes to develop more robust, efficient, and secure multi-cloud
applications by using containerise technologies.

###  4.4. Automation Challenges and Solution Framework (RQ4 - RQ5)

The Automation Challenge-Solution Framework for Containerized Applications in
multi-cloud environment (see Table LABEL:tab:Automation-solutions) suggests
that multi-cloud management for containerized application is a complex task
encompassing several dimensions. These include multi-cloud automation,
deployment and scaling, resource management, data and application migration,
testing and bench marking, standardization and interoperability, application
and service management, and runtime and service discovery. This indicates that
addressing one challenge may potentially impact the other areas, necessitating
a holistic approach to tackle these issues. In the following we discuss the
some of the key observations about this framework.

Multi-dimensional Challenges: The Automation Challenge-Solution Framework for
Containerized Applications in multi-cloud environment suggests that multi-
cloud management is a complex task encompassing several dimensions. These
include multi-cloud automation, deployment and scaling, resource management,
data and application migration, testing and bench marking, standardization and
interoperability, application and service management, and runtime and service
discovery. This indicates that addressing one challenge may potentially impact
the other areas, necessitating a holistic approach to tackle these issues. The
multi-dimensionality of the challenges implies the need for an integrated
approach that considers the interdependencies among these dimensions. It
emphasizes the need for solutions that cater to this complexity rather than
address individual challenges in isolation. Implications: For researchers,
these findings underscore the need to develop multi-faceted, holistic
solutions that can address the various interconnected challenges in managing
multi-cloud environment. For practitioners, this implies the importance of
strategic planning and implementing solutions that address the multiple facets
of multi-cloud management while considering their potential impact on one
another.

Emergence of AI/ML Solutions: The framework highlights the increasing
utilization of AI and ML in providing intelligent solutions to manage
containerized applications in multi-cloud environment. This suggests the
growing maturity and applicability of these technologies in complex scenarios,
a significant step towards achieving efficient multi-cloud management.
Previous studies (e.g., (de Laat et al., 2020), (Chen et al., 2021), (Gill et
al., 2022)) have recognized the potential of AI and ML in resolving complex
cloud management issues, which is in line with the findings from the
framework. This further validates the significant role AI and ML have started
to play in this domain. The increasing adoption of AI/ML in multi-cloud
environment signifies an essential evolution in the field. This stresses the
need for further research and development in these technologies to exploit
their full potential in resolving multi-cloud management complexities.
Implications: The growing use of AI/ML solutions implies that researchers need
to focus on improving these technologies and adapting them to specific multi-
cloud management challenges. For practitioners, this points towards the
increasing necessity to invest in AI/ML capabilities and integrate them into
their multi-cloud management strategies.

Importance of Standardization and Interoperability: The framework underscores
the crucial role of standardization and interoperability in multi-cloud
environment, facilitating seamless integration and management across different
cloud platforms. Recent literature validates the importance of standardization
and interoperability in multi-cloud environment, emphasizing their role in
enhancing the efficiency of cloud services (Alonso et al., 2023a). This agrees
with our results, further confirming the critical need for these factors in
the domain. The need for standardization and interoperability reiterates the
necessity for a unified framework or protocol across different cloud
environments. It suggests that attention to these aspects can lead to more
seamless, efficient, and secure management of multi-cloud services.
Implications: For researchers, the results stress the need to investigate
methods to achieve better standardization and interoperability in multi-cloud
environment. For practitioners, the findings suggest the need to adopt
standards and focus on interoperability while selecting and implementing cloud
services, ensuring more efficient management across different platforms.

###  4.5. Security Challenges and Solutions Framework (RQ4 - RQ5)

The framework proposed in our study provides a comprehensive and systematic
approach to addressing the complex security issues associated with
containerized applications in multi-cloud environment. In the following, we
discuss the key takeaway from security challenges and solution framwrork
presented in Section 3.5 and Table LABEL:tab:Security-challenges-solutions.

Emphasize on Container-Specific Security Mechanisms: Security solutions
specifically designed for containerized applications are critical to managing
threats in multi-cloud environment. The selected studies suggest a variety of
methods, including secure container orchestration, container isolation, and
advanced deployment strategies that make use of deep reinforcement learning.
This aligns with the existing literature (e.g., (Zhong et al., 2022), (Li et
al., 2023) (Combe et al., 2016)) which highlights the unique security
considerations brought forth by containerization. Containers, being lighter
than traditional virtual machines and offering process-level isolation, have
revolutionized the way applications are packaged and run. However, they also
introduce new security challenges that need to be addressed by using
container-specific security mechanisms. Containerization, while a powerful
tool for application deployment, necessitates its own suite of security
strategies. Implementing these will be key to leveraging the benefits of
containerization in a secure manner. Implications: For researchers, this
highlights an area of potential study: the development of advanced container-
specific security strategies, including those that use advanced techniques
such as machine learning. For practitioners, this suggests the need to be
well-versed in the specifics of container security. Leveraging features of
containerization platforms, such as Docker’s isolation features, or
incorporating tools specifically designed for container security will be
crucial.

Implementing Multi-Layered Security and Compliance Measures: Implementing
multi-layered security measures that encompass data protection, access
control, and communication security, along with compliance with legal and
regulatory requirements, is of utmost importance. Solutions range from data
encryption, role-based access control, secure container orchestration,
adherence to regulations like GDPR (Regulation, 2016). This is consistent with
current cybersecurity frameworks, such as the NIST cybersecurity framework (of
Standards and Technology, 2018), which emphasize a multi-layered approach to
security. Furthermore, several studies point to the criticality of data
security, user access control, and secure communication in cloud environments
(Austin, 2018) (Singh et al., 2016). In a multi-cloud, containerized
environment, it is not enough to secure data; access and communication
channels must also be secured, and all actions must comply with legal and
regulatory requirements. Implications: For practitioners is to design and
implement a comprehensive security strategy that encompasses multiple layers
of protection. Compliance with legal and industry standards should be an
integral part of this strategy. For researchers, this underlines the need for
studies that address the integration of various layers of security and the
development of comprehensive security frameworks.

Ensuring Security through Standardization and Interoperability: Ensuring
security through standardization and interoperability is highlighted as a
critical step. Secure and standardized interfaces, such as OpenStack APIs, can
be used to enhance security and performance across multiple cloud providers.
This finding is aligned with existing literature, which emphasizes the role of
standardization in achieving security and interoperability in multi-cloud
environment (Buyya et al., 2010) (Petcu, 2011). By implementing standardized
interfaces and practices, organizations can ensure a level of interoperability
and security across cloud environments. Implications: For practitioners, this
means choosing platforms and tools that support standardized interfaces and
practices, or working to implement those standards within their own
environments. Researchers, on the other hand, could focus on the development
of new standards or the improvement of existing ones to better address
security challenges in multi-cloud, containerized environments.

###  4.6. Deployment Challenges and Solutions Framework (RQ4 - RQ5)

Table LABEL:tab:monitoring-solutions offers a comprehensive overview of
challenges and their corresponding solutions encountered during the deployment
of containerized applications in multi-cloud environment. In the following, we
provide a discussion on key findings pertaining to the challenges and
solutions related to the deployment of containerized applications in multi-
cloud environment.

Standardization and Automation in Deployment: Standardization and automation
stand out as foundational pillars for achieving streamlined and uniform
deployments in multi-cloud landscapes. As Raj et al. (Raj et al., 2018)
highlighted, embracing automated deployment pipelines paired with standardized
toolsets can significantly diminish complexity and mitigate human-induced
errors within multi-cloud frameworks. The challenges, captured under
Performance Testing Deployment, Deployment Validation, and Cloud Management
Platform Evaluation Deployment in the table, elucidate the nuanced
complexities associated with deployments spanning multiple cloud platforms.
Solutions such as Standardized Benchmarking and Automation, Testing Process
Management System, and Standardized Output Formats and Evaluation Criteria
emphasize the imperative nature of adopting uniform procedures and harnessing
automated solutions. Multi-cloud deployments, inherently intricate, benefit
profoundly from standardization, bringing forth predictability and coherence,
while automation paves the way for diminishing human-driven inconsistencies
and bolstering deployment speeds. Implications: For researchers, the quest for
optimized frameworks championing standardization and automation within multi-
cloud environment presents a promising research trajectory. For practitioners,
adeptness with standardized methodologies and tools, combined with proficiency
in automation implementation, becomes crucial in navigating the multi-cloud
management maze efficiently

Security and Compliance in multi-cloud environment: Security remains paramount
in cloud deployments. Fernandes et al. underscore that the labyrinth of
diverse platforms and differing standards in multi-cloud configurations
amplifies security complexities (Fernandes et al., 2014). Challenges
articulated as Security Deployment in Multi-container Environment and
Malicious Service Management Deployment in Table LABEL:tab:Deployment-
solutions accentuate the primacy of security considerations. Propounded
solutions like Cross-Container Isolation and Collusion-Resilient Trust
Aggregation Technique spotlight the industry’s shift towards advanced and
meticulous security protocols within multi-cloud paradigms. The ever-evolving
landscape of multi-cloud deployments demands security solutions that are not
only rigorous but also malleable and formidable. To attain comprehensive
security across diverse platforms, strategies must encapsulate both depth of
understanding and breadth of application. Implications: For the academic
community, pioneering avant-garde security solutions apt for multi-cloud
environment offers a fertile domain of inquiry. Meanwhile, practitioners
should enshrine principles of data integrity, trust assurance, and
vulnerability mitigation at the heart of their multi-cloud deployment
blueprints.

Integration, Monitoring, and Infrastructure Management: As highlighted by
Ferrer et al. (Ferrer et al., 2016) the ability to operate seamlessly across
disparate cloud providers and maintain flexible infrastructures is paramount
in harnessing the full potential of multi-cloud deployments. The presented
challenges, including Network Latency Deployment and Vendor Lock-in Prevention
Deployment, underscore the indispensability of agility and cross-platform
compatibility. Proposed solutions such as VM Selection for Composite
Applications and Interoperability Layer Above Cloud Infrastructure accentuate
the importance of crafting adaptable infrastructure and formulating strategies
to mitigate vendor entrenchment. In the context of multi-cloud environment,
flexibility and interoperability transcend mere advantageous attributes; they
become pivotal determinants shaping adaptability, scalability, and the
sustained success of deployment blueprints. Implications: For the academic
community, delving into novel approaches that champion infrastructure
pliability and seamless operation across diverse cloud platforms emerges as a
prospective research avenue. For industry professionals, anchoring strategies
around flexibility and interoperability not only optimizes current deployments
but also fortifies them against future technological evolutions and shifting
business dynamics.

###  4.7. Monitoring Challenges and Solutions (RQ4 - RQ5)

Table LABEL:tab:monitoring-solutions presents a comprehensive view of the
Monitoring Challenge-Solution Framework for Containerized Applications in
multi-cloud environment. It covers a wide range of monitoring challenges
within the intricate multi-cloud setting. In the following, we are providing
discussion on the key findings.

Enhancing Performance Stability through Multi-Cloud Monitoring: In the context
of multi-cloud environment, ensuring consistent and reliable performance
monitoring for containerized applications can be challenging due to the
dynamic nature of cloud resources. Existing studies (e.g., (Zhong et al.,
2022) (Mungoli, 2023)) highlighted the impact of varying resource availability
on the performance of containerized applications across multiple clouds. They
emphasized the need for performance benchmarking to track fluctuations and
maintain a consistent user experience. The solutions presented in our
framework, such as benchmarking performance variation and utilizing Kubernetes
for monitoring, align with existing studies suggestion for benchmarking. Our
proposed framework expands on this by incorporating Kubernetes for improved
multi-cloud performance monitoring. Our framework also suggest reliable
performance monitoring requires a combination of standardized benchmarking
techniques and cloud-specific tools like Kubernetes. This approach can
effectively address performance variations and ensure a consistent user
experience across diverse cloud environments. Implications: For practitioners,
adopt benchmarking practices to track performance fluctuations, and leverage
Kubernetes for unified multi-cloud performance monitoring. For researchers,
explore further refinements in benchmarking methodologies and investigate the
impact of dynamic multi-cloud environment on performance consistency.

Streamlining Resource and Network Management across Multi-Clouds: Managing
network communications, system images, and resource utilization in a multi-
cloud setting poses challenges due to the diversity of cloud platforms.
Sedghpour et al. discussed the complexities of managing network communications
in multi-cloud environment. They emphasized the need for standardized
approaches like service mesh to enhance network visibility and control
(Sedghpour and Townend, 2022). Solutions in our proposed framework, such as
using Docker Server for containerized services and Prometheus/Grafana for
resource monitoring, align with Sedghpour et al.’s (Sedghpour and Townend,
2022) suggestion for standardized approaches. Additionally, our framework
extends these solutions to address broader resource management challenges.
Employing standardized tools like Docker Server, along with platforms like
Prometheus and Grafana, can help streamline resource management and network
communication monitoring across diverse cloud platforms. Implications: For
practitioners, implement Docker Server and utilize platforms like Prometheus
and Grafana for efficient resource monitoring and management across multi-
cloud environment. For researchers, investigate further integration
possibilities for standardized resource management tools and explore the
impact of such tools on overall system performance and reliability.

Strengthening Security, Compliance, and Trust: Ensuring data privacy,
continuous security control, and compliance in multi-cloud containerized
applications is essential but challenging due to the distributed nature of
cloud resources. Existing studies (e.g., (Alonso et al., 2023a) (Raj et al.,
2018)) discussed the challenges of ensuring security and compliance across
multi-cloud environment. They highlighted the importance of continuous
monitoring and enforcement mechanisms to address security and compliance gaps.
Solutions presented in our proposed framework, such as using machine learning
for performance supervision and employing security assurance platforms, are
aligned with existing studies that emphasis on continuous monitoring and
enforcement for security and compliance. Implementing advanced techniques like
machine learning for monitoring and adopting security assurance platforms can
significantly enhance security, compliance, and trust management in multi-
cloud containerized applications. Implications: For practitioners, integrate
machine learning-based monitoring and adopt security assurance platforms to
ensure continuous security control and compliance across diverse cloud
environments. For researchers, explore the scalability and effectiveness of
machine learning-driven monitoring techniques and investigate novel methods
for enforcing security and compliance in multi-cloud setups.

###  4.8. Tools and Frameworks (RQ6)

This study report 160 distinct tools and frameworks for container based
applications in multi-cloud environment, all of which are systematically
classified into 6 categories, further delineated into 46 subcategories. In the
following we briefly discuss some of the key findings about identified tools
and frameworks.

Diverse Cloud Services by Leading Providers: Our study reveals a diverse array
of cloud services, including IaaS, PaaS, and SaaS offerings from major
providers like AWS, Azure, and Google Cloud, emphasizing the extensive options
available to developers (see Table LABEL:tab:ToolsandFrameworks). This
underscores the significant range of choices available to developers.
Noteworthy cloud service providers highlighted in our study include Amazon
EC2, Google Cloud App Engine, and Azure, further underscoring this finding. A
pivotal insight gleaned from our research underscores the paramount importance
of harmonizing cloud service models with specific application prerequisites,
thereby facilitating efficient allocation of resources and optimization of
costs. Implications: For practitioners, our outcomes underscore the critical
necessity of meticulously evaluating and selecting the most fitting service
model based on the unique demands of a given project. This strategic approach
ensures the best alignment between technology and goals. Moreover, the
implications for researchers are promising: an avenue emerges for in-depth
exploration of trade-offs and the identification of optimal practices while
navigating the intricate array of cloud service models. This exploration holds
potential for enhancing both the efficiency and effectiveness of cloud
solutions.

Docker and Kubernetes: Pioneers of Cross-Cloud Containerization: Our
investigation spotlights the profound significance of Docker and Kubernetes in
streamlining the containerization and orchestration of applications across a
spectrum of cloud platforms. Docker adeptly encapsulates applications and
their dependencies within self-contained containers, while Kubernetes takes
center stage as the orchestration juggernaut, automating deployment, scaling,
and management of these containers. These technologies receive direct
validation through table references, serving as prime exemplars. Our research
yields a compelling revelation: Docker and Kubernetes hold pivotal roles in
ensuring consistent application deployment and seamless scalability,
effectively simplifying the complexities of multi-cloud environment.
Implications: Practitioners stand to gain significantly by adopting Docker and
Kubernetes, as this potent combination bolsters efficiency and cultivates
heightened portability. Meanwhile, researchers have a promising avenue to
explore deeper facets, including security, performance optimization, and the
intricate domains of advanced use cases for Docker and Kubernetes within
multi-cloud landscapes. This exploration promises valuable insights that can
finely calibrate the potential of these cutting-edge technologies.

The Rise of DevOps: Embracing Automation Tools: Our exploration spotlights the
important role of DevOps practices and tools such as Puppet, Chef, and
Terraform within contemporary software development landscapes. This emphasis
revolves around the power of automation in facilitating streamlined deployment
and management processes. These tools, by enabling consistent provisioning and
deployment of infrastructure, effectively curtail errors and ensure the
replicability of processes. The table substantiates this notion through
references to Puppet, Chef, and Terraform. Our study unearths a pivotal
insight: DevOps practices wield the ability to augment the software
development life cycle by automating tasks, ushering in a realm of efficient
and dependable deployment. Implications: Practitioners are advised to
seamlessly integrate DevOps principles into their workflows by harnessing the
prowess of these tools. Furthermore, a realm of exploration opens for
researchers, inviting deeper analyses into the evolutionary trajectory of
DevOps practices and their profound impact on elevating software quality and
hastening development velocities.

##  5\. Threats to Validity

This systematic mapping study is susceptible to various threats that could
influence its outcomes. To address these potential threats, we adhered to the
established guidelines for conducting SMSs and SLRs as outlined in (Keele et
al., 2007) and (Petersen et al., 2008b). We further categorized and analyzed
the validity threats to our study based on the four distinct types of validity
threats mentioned in (Zhou et al., 2016) and (Wohlin et al., 2012). In the
subsequent sections, we delve into the specific validity threats that pertain
to the different phases of this SMS.

###  5.1. Internal Validity

Internal validity refers to the factors that could affect the analysis of the
data extracted from the selected studies. The threats to internal validity
could happen in the following steps of this SMS:

  * •

Study Search: The potential to overlook studies during the search process
requires careful measures. To mitigate this risk, we used a combination of
primary and snowballing search methodologies, as detailed in Section 2.2.1. To
increase the collection of primary studies during the initial search phase, we
also took additional measures to address search-related issues. Specifically,
we improved our search strings through pilot searches before applying them to
the databases. This process helped to create a more effective search strategy.

  * •

Study Selection: We have outlined the study screening and selection process in
Table 3. To ensure objectivity and eliminate personal bias in study selection,
we implemented a two-phase approach: (i) initial study screening, and (ii)
qualitative evaluation of the shortlisted studies. During this procedure, the
lead two authors conducted the study screening based on the criteria
explicitly detailed in Section 3. To assess the objectivity of the screening
process and the level of agreement between the two authors, we applied Cohen’s
Kappa (Cohen, 1968). In cases where the lead authors did not agree, the second
and third authors independently reviewed the disputed studies to reach a
consensus. All the researchers involved in this study have extensive knowledge
and research experience in containerization applications within a multi-cloud
environment.

  * •

Data Extraction: Bias from researchers during data extraction can pose a
significant challenge in both SMSs and SLRs. To address this potential issue,
we developed a standardized data extraction form (see Table 5) to ensure
consistent data retrieval. Initially, the first and fourth authors extracted
the data. If any uncertainties arose regarding the extracted data,
comprehensive discussions were convened among all authors. Following the
recommendations in (Wohlin et al., 2012), a subset of the extracted data was
cross-verified by the second and third authors.

  * •

Bias on Themes Classification: Misclassification of data and primary studies
may result in biases stemming from subjective interpretation. To minimize this
risk, we adhered to the thematic analysis guidelines established by Braun et
al. (Braun and Clarke, 2006), and implemented a six-step process for thematic
classification, as detailed in Section 2.3.

  * •

Data Synthesis: We employed both qualitative and quantitative approaches to
assess the gathered data. Potential biases in data synthesis could influence
the interpretation of our findings. To address this concern, we used open
coding and constant comparison techniques from Grounded Theory (Glaser and
Strauss, 1967) to analyze the qualitative data extracted from the selected
studies.

###  5.2. External Validity

Concerns about external validity relate to the applicability and
generalizability of study findings. Our research provides a thorough
examination of containerization in multi-cloud environments, with findings,
analyses, and conclusions specifically tailored to this domain. To ensure
robust external validity, we developed a study protocol that outlines our
research methodology. The literature review spanned a decade, from January
2013 to March 2023, and included peer-reviewed articles from eight preeminent
databases in the fields of software engineering and computer science, which
are listed in Table 2. While the review is anchored in academic research,
which may not encapsulate unpublished industry practices, the systematic
approach and comprehensive timeframe of our analysis make the insights
valuable for both academia and practitioners. To complement and enrich our
findings, future work could include an industrial study (e.g., blogs, white
papers) that would provide a broader view of the application of
containerization in practice

###  5.3. Construct Validity

Construct validity concerns to the accuracy of the operational measures used
for data collection in a study. The primary constructs of this study revolve
around two concepts: “containerization” and “multi-cloud environment”.
Utilizing imprecise or incomplete search terms, or employing unsuitable search
strategies, can lead to potential pitfalls such as overlooking pertinent
papers or including numerous irrelevant ones during the search phase, and
omitting relevant papers during the selection phase. To counter these risks,
we implemented the following measures: (i) we initiated a pilot search to
verify the relevance and comprehensiveness of our search terms; and (ii) we
searched paper on eight databases renowned for computer science and software
engineering research. Additionally, we customized search string according to
each database syntax.

###  5.4. Conclusion Validity

Threats to conclusion validity relate to factors, such as data inaccuracies,
that can impede drawing accurate conclusions. To mitigate these threats, we
adhered to best practices, including the search protocol, pilot search, and
pilot data selection, as recommended by Kitchenham et al. (Keele et al., 2007)
and Petersen et al. (Petersen et al., 2008b). Moreover, to further ensure the
validity of our conclusions, the authors engaged in multiple brainstorming
sessions to collaboratively interpret the results and finalize the
conclusions.

##  6\. Related Work

This section reviews the most relevant existing research in terms of secondary
studies such as literature surveys, state-of-the-art analysis, systematic
reviews, and mapping studies that consolidate published literature on
containerization in multi-cloud environments. The review focuses on (i)
existing challenges, proposed solutions, and emerging trends detailed in
Section 6.1 and (ii) container-based deployment of multi-cloud systems in
Section 6.2.

###  6.1. Challenges, Solutions, and Performance of Multi-cloud Containers

In recent years, several literature surveys have been published to analyze
state-of-the-art on the applications (Pahl et al., 2017), tools and
technologies (Casalicchio and Iannucci, 2020), potential and limitations
(Bentaleb et al., 2022), as well as emerging trends (Watada et al., 2019) of
container-based solutions for cloud computing systems, discussed below.

####  6.1.1. Challenges, Applications, Tools, and Emerging Trends

Containers in multi-cloud environments provide a standardized and portable way
to package, deploy, and run applications, ensuring seamless deployment and
management across diverse cloud platforms (Alonso et al., 2023b) (Saxena et
al., 2021). One of the earliest SLR-based studies by Pahl et al. (Pahl et al.,
2017) on containerized clouds reviewed a total of 46 studies to identify,
taxonomically classify and systematically compare the existing research on
containers and their application in the context of cloud-based systems. The
SLR classified and compared the selected research studies using a conceptual
framework to highlight existing trends and needs for future research. SLR
results indicate container orchestration, microservice delivery, continuous
development and deployment as emerging trends of research on containerized
clouds. The SLR in (Alonso et al., 2023b) reviewed a total of 88 studies
(published from 2011 - 2021) on multi-cloud systems. Similar survey studies
such as (Saxena et al., 2021) concepts, challenges, requirements and future
directions for multi-cloud environments are discussed. A survey of existing
approaches and solutions provided by different multi-cloud architectures is
entailed along with analysis of the pros and cons of different architectures
while comparing the same (Imran et al., 2020).

####  6.1.2. Performance and Resource Utilization

In a study by Bentaleb et al. (Bentaleb et al., 2022), the authors follow the
taxonomical classification from (Pahl et al., 2017) to categorize container-
based technologies for cloud systems. The study argues for the need of
performance metrics to objectively define and evaluate quality attributes such
as resource virtualization, service elasticity, orchestration, and multi-
tenancy in containerized cloud systems. The study also highlights the needs
for future research in terms of best practices, i.e., patterns and tactics
(enabling reuse) and tools (supporting automation) for container-based cloud
deployment. In the context of performance, the studies (Casalicchio and
Iannucci, 2020) (Watada et al., 2019) report literature review and
experimental analysis of factors that influence performance of container-based
cloud systems. Specifically, Casalicchio et al. (Casalicchio and Iannucci,
2020) reviewed a total of 97 research studies investigating performance
evaluation and run-time adaptation of container-based solution. The study
highlighted several unsolved challenges such as I/O throughput optimization,
performance prediction, and multi-layer monitoring performance bottlenecks.
Moreover, Watada et al. (Watada et al., 2019) conducted an experimental study
to compare the performance of VMs, containers and uni kernels in terms of and
technological maturity using standard benchmarks and observed containers to
optimise performance of container-based. The performance of containerized
clouds is also determined by resource utilization, which includes but is not
limited to CPU utilization, memory footprints, energy consumption, network
bandwidth, and execution time.

Kapil et al. (Netaji and Bhole, 2021) reviewed 64 research papers to gain
insight into resource allocation, management, and scheduling. Furthermore,
limitations of existing resource allocation algorithms are discussed,
indicating the need to investigate algorithms or techniques of performance
optimization of containers for cloud systems. The study by Maenhaut et al.
(Maenhaut et al., 2020) provided an overview of the current state of the art
regarding resource management within the broad sense of cloud computing,
complementary to existing surveys in the literature. The study investigated
how research is adapting to the recent evolution within cloud solutions,
including container technologies.

###  6.2. Container-based Cloud Orchestration, Deployment, and Security

This section reviews the most relevant research on the container-based
deployment with a focus on orchestration and security issues of cloud-based
systems, detailed below.

####  6.2.1. Orchestration and Deployment

Naweiluo et al. (Zhou et al., 2022) explored containerization in High
Performance Computing (HPC) environments, contrasting it with cloud computing.
The results of this study indicate that containers enhance application
deployment efficiency, but face challenges in HPC due to high security levels
and the need for extensive libraries and packages that affect cloud
portability, often resulting in vendor lock-ins. The study also provides a
survey and taxonomy on containerization and orchestration efforts in HPC,
pointing out differences with cloud environments and identifying future
research and engineering potentials. A systematic mapping study by Naylor et
al.(Bachiega et al., 2018) explores virtualization in container-based cloud
deployments. The mapping study, based on a comprehensive review of major
databases, identifies a significant research gap in the performance evaluation
of containers, underscoring the need for further investigation on cloud
deployment. Carlos et al. (Awada, 2018) present an optimization strategy for
deploying microservices in multi-cloud environments focusing on minimizing
cloud service costs, network latency, and startup time for new microservices.
Their approach uses a Non-dominated Sorting Genetic Algorithm II (NSGA-II)
compared to a Greedy First-Fit algorithm, demonstrating a 300% improvement
with NSGA-II. This highlights its effectiveness in container and VM
orchestration, offering a significant enhancement in deployment of multi-cloud
solution in containers. Emiliano et al. (Paladi et al., 2018) survey container
orchestration, proposing a reference architecture for autonomic orchestrators,
and identifying research challenges in the field. Their work emphasizes the
importance of container technologies in cloud environments and the need for
advanced orchestration solutions to manage complex multi-container
applications effectively. Uchechukwu Awada (Guerrero et al., 2018a) reviews
container orchestration tools and platforms, comparing the architectures,
components, and capabilities of several Container Service Platforms (CSPs) and
orchestration tools like Amazon ECS, Kubernetes, Docker Swarm, and Mesos. The
study offers insights into the current state of container orchestration and
suggests future research directions, serving as a guide for developers and
organizations.

Koustabh et al. (Dolui and Kiraly, 2018) explore multi-container deployment on
IoT gateways to meet the stringent latency requirements of advanced IoT
applications. Through their study within the AGILE project, they highlight
containerization’s role in overcoming the diversity and resource constraints
of IoT gateways, showcasing containerization’s advantages for IoT gateway
performance optimization. Their research underscores the potential of
containerized environments in enhancing application compatibility,
portability, and efficient deployment across diverse hardware architectures.
Matteo et al. (Nardelli et al., 2018) introduce the Adaptive Container
Deployment (ACD) model to optimize containerized application deployment in
geo-distributed environments, focusing on IoT and fog computing resources.
ACD, formulated as an Integer Linear Programming problem, aims to improve
application performance by leveraging containers’ horizontal and vertical
elasticity. The study evaluates ACD’s effectiveness against greedy heuristics,
highlighting the need for advanced orchestration solutions to exploit emerging
computing environments’ characteristics efficiently.

####  6.2.2. Security of Containerized Clouds

Nicolae et al. (Casalicchio, 2019) analyze security challenges in cloud
orchestration for multi-cloud deployments, proposing a security-enabled
orchestration framework. Their research identifies potential attack scenarios
and security enforcement mechanisms, aiming to enhance security guarantees for
cloud operators in a multi-cloud setting. Mohammad et al. (Bokhari et al.,
2018) offer a detailed analysis of cloud computing, defining its essential
characteristics, architecture, service models (SaaS, PaaS, IaaS), and
deployment models. They discuss the security requirements for public and
private clouds across different service models, aiming to provide researchers
with a comprehensive understanding of cloud computing’s potential and security
challenges. Sari et al. (Sultan et al., 2019) survey the landscape of
container security, addressing challenges and solutions across four
generalized use cases within the host-container threat landscape. Their
analysis covers both software-based solutions utilizing Linux kernel features
and hardware-based solutions for enhanced security. The study aims to clarify
container security requirements and encourage further research in addressing
potential vulnerabilities and attacks. Hendrik et al. (Graupner et al., 2015a)
tackle privacy, security, and trust issues in cloud computing through a multi-
cloud architecture perspective. They propose a novel technique for enhancing
security and unifying access control mechanisms across cloud providers,
addressing the challenges of inadequate cross-provider APIs and non-unified
access control. Rohan et al. (Gupta et al., 2016) propose a security-enhancing
methodology for cloud data storage using container clustering and Docker
instances for on-demand encryption. This approach aims to secure data while
optimizing resource usage and cost, highlighting the benefits of container
technology in improving cloud computing security and efficiency.

###  6.3. Conclusive Summary

Current survey-based studies and systematic reviews, discussed above (e.g.,
(Casalicchio and Iannucci, 2020) (Pahl et al., 2017)), aim to consolidate the
latest findings in published research concerning container-based solutions for
cloud computing systems. These studies investigated various aspects of multi-
cloud, including exploring applications, tools, and technologies, assessing
potential and limitations, and identifying emerging trends.

Our study distinguishes itself from prior research in several ways. First, our
work systematically categorizes container utilization in multi-cloud
environments into coherent categories and themes. Some of the key themes are
‘Scalability and High Availability’, ‘Performance and Optimization’, ‘Security
and Privacy’, and ‘Multi-Cloud Container Monitoring and Adaptation’. We
identified and classified seventy-four patterns and strategies for
containerization, which existing studies have not provided. Furthermore, we
explored the QAs and associated tactics for containerization in multi-cloud
environment. This contribution offers practitioners actionable insights into
tailoring their containerization strategies to meet desired quality standards.
Lastly, our study introduces four distinct frameworks consisting of challenges
and solutions related to security, automation, deployment, and monitoring
within the context of containerization in multi-cloud environment.

##  7\. Conclusions

This SMS presents the current state of research regarding containerization in
multi-cloud environment, focusing on aims of selected studies, the roles of
containers, containers implementation strategies, architectural patterns and
strategies, quality attributes and corresponding tactics. Additionally, this
SMS also explores deployment, monitoring, security challenges, accompanied by
their respective solutions, tools, and frameworks used to implement
containerized applications in multi-cloud environment. From an initial pool of
extensive search results, 86 relevant studies are were finally chosen to
answer the research questions presented in Table 1. The key findings of this
SMS are listed below.

  * •

The findings of this SMS reveal various insights regarding SMS demographics,
publishers, publication types, authors’ affiliations, and research themes. The
yearly distribution reached its peak in 2016, signifying a shifting level of
interest over time. The field of publishing is largely dominated by IEEE
(72.09%), with conferences emerging as the primary avenue for publication.
Notably, academia (93.02%) stands out as the leading affiliation among
authors. The prominent research themes encompass Orchestration and Management,
Scalability and High Availability, Performance and Optimization, and Security
and Privacy. These themes underscore crucial aspects such as container
management, scalability, performance, and security within the context of
multi-cloud environment.

  * •

This SMS identifies and classifies a total of 74 patterns and strategies
across 10 subcategories and 4 categories for container-based applications in
multi-cloud environment. These patterns and strategies encompass a wide range
of aspects such as cloud architecture models, communication and networking,
deployment, service models, cloud management, container management, edge
computing, IoT strategies, security, resilience, fault-tolerance, and cloud
migration. These findings are significant for the application and dvancement
of containerization-based applications in multi-cloud environment, providing
valuable insights into optimizing architecture, communication, management,
security, and migration strategies.

  * •

This SMS identifies and classifies QAs and tactics from selected studies
regarding containerized applications in multi-cloud environment. A total of
ten QAs and their related terms are identified. Along with these QAs, a total
of 47 tactics are identified to enhance the QAs. Notable takeaways include the
utilization of machine learning and location-aware service brokering to
enhance the performance and efficiency of container-based applications in
multi-cloud environment.

  * •

This SMS presents a comprehensive security challenge-solution framework for
containerized applications in multi-cloud environment. This framework offers
valuable insights and actionable strategies for practitioners and researchers
to enhance the security of containerized applications, address complex multi-
cloud security concerns, and ensure compliance with various regulatory
requirements, thereby enabling the development of more robust and secure
containerized applications in multi-cloud environment.

  * •

This SMS presents a framework for the automation challenge-solution framework
for containerized applications in multi-cloud environment. The automation
challenge-solution framework offers a comprehensive guide for practitioners
and researchers to address the complex challenges associated with automating
various aspects of deploying, managing, scaling, testing, migrating, and
ensuring interoperability of containerized applications in multi-cloud
environment. By offering specific solutions for each challenge, this framework
aids in the development of more efficient, secure, and adaptable automation
processes, empowering practitioners to streamline their operations and
enabling researchers to advance the field of multi-cloud containerization.

  * •

The third framework presented in this SMS is the deployment challenge-solution
framework for containerized applications in multi-cloud environment. This
framework provides valuable insights and practical guidance to practitioners
and researchers alike, aiding practitioners in navigating the complexities of
multi-cloud deployment through specific solutions to challenges, while also
serving as a comprehensive resource for researchers to delve into various
aspects of containerized application deployment, orchestration, security,
scalability, and more, thus contributing to the advancement of knowledge and
innovation in the field of multi-cloud deployment.

  * •

Fourth framework presented in this SMS is monitoring challenge-solution
framework for containerized applications in multi-cloud environment. This
framework serves as a comprehensive guide for practitioners, offering specific
solutions to challenges related to monitoring performance, resource
management, security, optimization, system complexity, and multi-cloud
coordination, facilitating their understanding of complex monitoring scenarios
and providing actionable insights. Furthermore, researchers can benefit from
this framework by gaining a deeper insight into the nuances of monitoring
containerized applications in multi-cloud environment, and it can serve as a
foundation for further research and innovation in the field of multi-cloud
monitoring.

The findings of this SMS will benefit researchers who are interested in
understanding the state of research on containerized applications in multi-
cloud environment and conducting further investigations to address the open
research issues highlighted in Section 4. Additionally, the insights gained
from this SMS will support knowledge transfer to practitioners by providing
insights into the challenges, solutions, and methods for monitoring, securing,
and optimizing containerized applications in multi-cloud environment. We
emphasize the importance for practitioners to develop targeted solutions that
effectively tackle monitoring, security, and performance degradation concerns
within multi-cloud environment deployment. As a future endeavor, we intend to
enhance our SMS by conducting industrial case studies with companies, thereby
obtaining practical insights and perspectives from practitioners on the
effectiveness and applicability of our proposed frameworks in real-world
scenarios. This approach will allow us to bridge the gap between research and
practical implementations concerning containerized applications in multi-cloud
environment and contribute to the advancement of both academic and industry
understanding in this domain.

## Acknowledgments

This work is partially sponsored by the National Natural Science Foundation of
China (NSFC) with Grant No. 62172311.

##  8\. Appendices

List of selected studies for containerization in multi-cloud environments |  |  |  |
---|---|---|---|---
Study ID |  Title |  Venue | Year | Reference
Table 15 continued from previous page
Study ID |  Title |  Venue | Year | Reference
Continued on next page
S01  | A component-based adaptation approach for multi-cloud applications | IEEE Conference on Computer Communications (INFOCOM) Workshops | 2014 | (Almeida et al., 2014)
S02 |  A Cost-Efficient Multi-cloud Orchestrator for Benchmarking Containerized Web-Applications |  International Conference on Web Information Systems Engineering (WISE) | 2019 | (Jha et al., 2019)
S03 |  A framework to support multi-cloud collaboration |  IEEE World Congress on Services (SERVICES) | 2020 | (Hua et al., 2020)
S04 |  A Machine Learning Model for Detection of Docker-based APP Overbooking on Kubernetes |  International Conference on Communications (ICC) | 2021 | (Ramos et al., 2021)
S05 |  A multi-cloud service mesh approach applied to Internet of Things |  Annual Conference of Industrial Electronics Society (IECON) | 2022 | (Gattobigio et al., 2022)
S06 |  A Secure Container Placement Strategy Using Deep Reinforcement Learning in Cloud |  International Conference on Computer Supported Cooperative Work in Design (CSCWD) | 2022 | (Deng et al., 2022)
S07 |  A Service Driver Based Application Execution and Development Method in Multi-cloud Context |  International Conference on Data Science and Computer Application (ICDSCA) | 2021 | (Zhang et al., 2021)
S08 |  An adaptable framework to deploy complex applications onto multi-cloud platforms |  International Conference on Computing Communication Technologies - Research, Innovation, and Vision for Future (RIVF) | 2015 | (Pham et al., 2015)
S09 |  An agile container-based approach to TaaS |  International Congress Future Telecommunications: Infrastructure and Sustainability (FITCE) | 2016 | (Verdugo et al., 2017)
S10 |  An Efficient and Reliable Multi-Cloud Provider Monitoring Solution |  Conference on Global Communications (GLOBECOM) | 2022 | (Sabbioni et al., 2020)
S11 |  An Efficient Multi-Keyword Search Scheme over Encrypted Data in Multi-Cloud Environment |  International Conference on Smart Cloud (SmartCloud) | 2015 | (He et al., 2022)
S12 |  An Empirical Investigation of a Fault Tolerant Containerized Application Deployment |  International Conference on Informatics (ICI) | 2015 | (Bisht and Kaur, 2022)
S13 |  Applying Computational Intelligence for enhancing the dependability of multi-cloud systems using Docker Swarm |  Symposium Series on Computational Intelligence (SSCI) | 2017 | (Naik, 2016a)
S14 |  Automatically Design Heuristics for Multi-Objective Location-Aware Service Brokering in Multi-Cloud |  International Conference on Services Computing (SCC) | 2020 | (Chen et al., 2022)
S15 |  Autonomic fine-grained migration and replication of component-based applications across multi-clouds |  National Foundation for Science and Technology Development Conference on Information and Computer Science (NICS) | 2022 | (Pham and Pham, 2015)
S16 |  Band-Area Resource Management Platform and Accelerated Particle Swarm Optimization Alg. for Container Deployment |  IEEE Access | 2016 | (Ouyang et al., 2022)
S17 |  Building a Multi-tenant Cloud Service from Legacy Code with Docker Containers |  International Conference on Cloud Engineering (IC2E) | 2022 | (Slominski et al., 2015)
S18 |  Building cloud native application - Analysis for multi-component application deployment |  International Conference on Computer Communication and Informatics (ICCCI) | 2021 | (Pai and Kumar, 2021)
S19 |  CLAMS: Cross-layer Multi-cloud Application Monitoring-as-a-Service Framework |  International Conference on Services Computing (SCC) | 2014 | (Alhamazani et al., 2014)
S20 |  Cloud MF: Applying MDE to Tame the Complexity of Managing Multi-Cloud Applications |  International Conference on Utility and Cloud Computing (UCC) | 2014 | (Ferry et al., 2014)
S21 |  Cluster as a Service: A Container Based Cluster Sharing Approach with Multi-user Support |  IEEE Symposium on Service-Oriented System Engineering (SOSE) | 2016 | (Cui et al., 2016)
S22 |  Container Deployment Strategy for Edge Networking |  Workshop on Middleware for Edge Clouds and Cloudlets (MECC) | 2019 | (Wong et al., 2019)
S23 |  Container-Based Performance Isolation for Multi-Tenant SaaS Applications in Micro-Service Architecture |  Journal of Physics: Conference Series | 2020 | (Wang et al., 2020)
S24 |  Containerization and the PaaS Cloud |  IEEE Cloud Computing | 2015 | (Pahl, 2015)
S25 |  ContainerLeaks: Emerging Security Threats of Information Leakages in Container Clouds |  International Conference on Dependable Systems and Networks (DSN) | 2017 | (Gao et al., 2017)
S26 |  ContMEC: An Architecture of Multi-access Edge Computing for Offloading Container-Based Mobile Applications |  IEEE International Conference on Communications (ICC) | 2022 | (Watanabe et al., 2022)
S27 |  DECIDE: An Extended DevOps Framework for Multi-Cloud Applications |  International Conference on Cloud and Big Data Computing (ICCBDC) | 2022 | (Alonso et al., 2019)
S28 |  Delivering Elastic Containerized Cloud Applications to Enable DevOps |  International Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) | 2017 | (Barna et al., 2017)
S29 |  Deployment of Multi-Tier Application on Cloud and Continuous Monitoring using Kubernetes |  International Conference on Electrical, Electronics, Communication, Computer Technologies and Optimization Techniques (ICEECCOT) | 2021 | (Gupta et al., 2021)
S30 |  Deployment-Time Multi-Cloud Application Security |  International Conference on Smart Computing (SMARTCOMP) | 2017 | (Sheridan et al., 2017)
S31 |  DEVSML 3.0 Stack: Rapid deployment of DEVS Farm in distributed cloud environment using microservices and containers |  Symposium on Theory of Modeling and Simulation (TMS/DEVS) | 2017 | (Mittal and Risco-Martín, 2017)
S32 |  Disaster Recovery and Business Continuity for Database Services in Multi-Cloud |  International Conference on Computer Applications & Information Security (ICCAIS) | 2018 | (Al-shammari and Alwan, 2018)
S33 |  Dynamic On-Demand Fog Formation Offering On-the-Fly IoT Service Deployment |  IEEE Transactions on Network and Service Management | 2020 | (Sami and Mourad, 2020)
S34 |  Dynamic security assurance in multi-cloud DevOps |  IEEE Conference on Communications and Network Security (CNS) | 2017 | (Rios et al., 2017)
S35 |  Engineering and Experimentally Benchmarking a Container-based Edge Computing System |  International Conference on Communications (ICC) | 2020 | (Carpio et al., 2020)
S36 |  FogDocker: Start Container Now, Fetch Image Later |  International Conference on Utility and Cloud Computing (UCC) | 2019 | (Lorenzo et al., 2019)
S37 |  Importance of Application-Level Resource Management in Multi-Cloud Deployments |  International Conference on Cloud Engineering (IC2E) | 2019 | (Dimitrijevic et al., 2019)
S38 |  Knowledge-Engineered Multi-Cloud Resource Brokering for Application Workflow Optimization |  IEEE Transactions on Network and Service Management | 2022 | (Pandey et al., 2022)
S39 |  Label-affinity-Scheduler: Considering Business Requirements in Container Scheduling for Multi-Cloud and Multi-Tenant |  Brazilian Symposium on Computing System Engineering (SBESC) | 2022 | (Altran et al., 2022)
S40 |  Leveraging kernel security mechanisms to improve container security: A survey |  International Conference on Availability, Reliability and Security (ARES) | 2019 | (Bélair et al., 2019)
S41 |  Location-Aware and Budget-Constrained Service Deployment for Composite Applications in Multi-Cloud Environment |  IEEE Transactions on Parallel and Distributed Systems | 2020 | (Shi et al., 2020)
S42 |  MicroCloud: A Container-Based Solution for Efficient Resource Management in the Cloud |  International Conference on Smart Cloud (SmartCloud) | 2016 | (Baresi et al., 2016)
S43 |  Mobile Edge as Part of the Multi-Cloud Ecosystem: A Performance Study |  Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP) | 2019 | (Dreibholz et al., 2019)
S44 |  Modeling and Automated Deployment of Serverless Applications Using TOSCA |  Conference on Service-Oriented Computing and Applications (SOCA) | 2018 | (Wurster et al., 2018)
S45 |  Monitoring Elastically Adaptive Multi-Cloud Services |  IEEE Transactions on Cloud Computing | 2018 | (Trihinas et al., 2015)
S46 |  Monitoring-Aware Optimal Deployment for Applications Based on Microservices |  IEEE Transactions on Services Computing | 2021 | (Fadda et al., 2019)
S47 |  MSSF: A Step towards User-Friendly Multi-cloud Data Dispersal |  International Conference on Cloud Computing (CLOUD) | 2014 | (Libardi et al., 2014)
S48 |  Multi-agent QoS-aware autonomic resource provisioning framework for elastic BPM in containerized multi-cloud environment |  Journal of Ambient Intelligence and Humanized Computing | 2022 | (Saif et al., 2023)
S49 |  Multicloud- auto scale with prediction and delta correction algorithm |  International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT) | 2019 | (Harwalkar et al., 2019b)
50 |  Multi-cloud Distribution of Virtual Functions and Dynamic Service Deployment: Open ADN Perspective |  IEEE International Conference on Cloud Engineering (IC2E) | 2015 | (Bhamare et al., 2015)
S51 |  Multi-cloud resource provisioning with Aneka: A unified and integrated utilization of Microsoft Azure and Amazon EC2 |  International Conference on Computing and Network Communications (CoCoNet) | 2015 | (Buyya and Barreto, 2015)
S52 |  MUSA Deployer: Deployment of Multi-cloud Applications |  IEEE International Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE) | 2017 | (Casola et al., 2017)
S53 |  NoMISHAP: A Novel Middleware Support for High Availability in Multicloud PaaS |  IEEE Cloud Computing | 2017 | (Acquaviva et al., 2017)
S54 |  Object storage in the cloud and multi-cloud: State of the art and the research challenges |  IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR) | 2018 | (Bucur et al., 2018)
S55 |  On the Deployment of an Open-Source, 5G-Aware Evaluation Testbed |  International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud) | 2018 | (Bolivar et al., 2018)
S56 |  OpenStack Federation in Experimentation Multi-cloud Testbeds |  IEEE International Conference on Cloud Computing Technology and Science (CloudCom) | 2013 | (del Castillo et al., 2013)
S57 |  Orchestrated Multi-Cloud Application Deployment in OpenStack with TOSCA |  International Conference on Smart Computing (SMARTCOMP) | 2017 | (Tricomi et al., 2017)
S58 |  Orchestration Based Hybrid or Multi Clouds and Interoperability Standardization |  IEEE International Conference on Cloud Computing in Emerging Markets (CCEM) | 2018 | (Sitaram et al., 2018)
S59 |  Performance Analysis of Zero-Trust multi-cloud |  IEEE International Conference on Cloud Computing (CLOUD) | 2021 | (Rodigari et al., 2021)
S60 |  Policies Based Container Migration Using Cross-Cloud Management Platform |  International Conference on Information and Automation for Sustainability (ICIAfS) | 2018 | (Janarthanan et al., 2018)
S61 |  Policy-based deployment in a hybrid and multi-cloud environment |  International Conference on Cloud Computing and Services Science (CLOSER) | 2019 | (Di Modica et al., 2019)
S62 |  Predicting the End-to-End Tail Latency of Containerized Microservices in the Cloud |  International Conference on Cloud Engineering (IC2E) | 2019 | (Rahman and Lama, 2019)
S63 |  Preventing vendor lock-ins via an interoperable multi-cloud deployment approach |  International Conference for Internet Technology and Secured Transactions (ICITST) | 2017 | (Pellegrini et al., 2017)
S64 |  Private STaaS with OpenStack Cinder Volumes for Hybrid/Multi-cloud |  International Conference on Advances in Computing and Communication Engineering (ICACCE) | 2019 | (Harwalkar et al., 2019a)
S65 |  Real-World Experience with a Multicloud Exchange |  IEEE Cloud Computing | 2017 | (Zeck and Bouroudjian, 2017)
S66 |  Reliability Management for Blockchain-Based Decentralized Multi-Cloud |  International Symposium on Cluster Computing and the Grid (CCGRID) | 2020 | (Aral et al., 2020)
S67 |  Reliability-Aware Cost-Efficient Scientific Workflows Scheduling Strategy on Multi-Cloud Systems |  IEEE Transactions on Cloud Computing | 2022 | (Tang, 2021)
S68 |  Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications |  The Journal of Supercomputing | 2018 | (Guerrero et al., 2018b)
S69 |  Scalable multi-cloud platform to support industry and scientific applications |  International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) | 2018 | (Kiss, 2018)
S70 |  Search-based Methods for Multi-Cloud Configuration |  International Conference on Cloud Computing (CLOUD) | 2022 | (Lazuka et al., 2022)
S71 |  Secure access control for multi-cloud resources |  IEEE Conference on ALocal Computer Networks Workshops (LCN Workshops) | 2015 | (Graupner et al., 2015b)
S72 |  Secure and Dependable Multi-Cloud Network Virtualization |  International Workshop on Security and Dependability of Multi-Domain Infrastructures | 2017 | (Alaluna et al., 2017)
S73 |  Secure and Parallel Expressive Search over Encrypted Data with Access Control in Multi-CloudIoT |  Cloudification of the Internet of Things (CIoT) | 2018 | (Guechi and Maamri, [n. d.])
S74 |  Self-healing cloud services in private multi-clouds |  International Conference on High Performance Computing & Simulation (HPCS) | 2018 | (Mfula and Nurminen, 2018)
S75 |  SpyStorage: A Highly Reliable Multi-Cloud Storage with Secure and Anonymous Data Sharing |  International Conference on Networking, Architecture, and Storage (NAS) | 2017 | (Shen et al., 2017)
S76 |  Support Services for Applications Execution in Multi-clouds Environments |  International Conference on Autonomic Computing (ICAC) | 2016 | (Pop et al., 2016)
S77 |  SynAPTIC: Secure and persistent connectivity for containers |  International Symposium on Cluster, Cloud and Grid Computing (CCGRID) | 2017 | (Ranjbar et al., 2017)
S78 |  System Restore in a Multi-cloud Data Pipeline Platform |  International Conference on Dependable Systems and Networks – Industry Track | 2019 | (Wang et al., 2019)
S79 |  Systematic and Recomputable Comparison of Multi-cloud Management Platforms |  International Conference on Cloud Computing Technology and Science (CloudCom) | 2018 | (Serhiienko and Spillner, 2018)
S80 |  Towards a lightweight multi-cloud DSL for elastic and transferable cloud-native applications |  International Conference on Cloud Computing and Services Science (CLOSER) | 2018 | (Quint and Kratzke, 2018)
S81 |  Towards Formal-Based Semantic Interoperability in Multi-Clouds: The FCLOUDS Framework |  IEEE International Conference on Cloud Computing (CLOUD) | 2017 | (Challita et al., 2017)
S82 |  Towards Model-Driven Provisioning, Deployment, Monitoring, and Adaptation of Multi-cloud Systems |  IEEE International Conference on Cloud Computing (CLOUD) | 2013 | (Ferry et al., 2013)
S83 |  Towards Trustworthy Multi-Cloud Services Communities: A Trust-Based Hedonic Coalitional Game |  IEEE Transactions on Services Computing | 2018 | (Wahab et al., 2016)
S84 |  Multi-cloud provisioning of business processes |  Journal of Cloud Computing | 2019 | (Kritikos et al., 2019)
S85 |  Evaluating the DevOps Reference Architecture for Multi-cloud IoT-Applications |  SN Computer Science | 2021 | (Bou Ghantous and Gill, 2021)
S86 |  Hybrid IT and Multi Cloud an Emerging Trend and Improved Performance in Cloud Computing |  SN Computer Science | 2020 | (Gundu et al., 2020)

## References

  * (1)
  * Acquaviva et al. (2017) Luca Acquaviva, Paolo Bellavista, Filippo Bosi, Antonio Corradi, Luca Foschini, Stefano Monti, and Andrea Sabbioni. 2017.  NoMISHAP: A Novel Middleware Support for High Availability in Multicloud PaaS.  _IEEE Cloud Computing_ 4, 4 (2017), 60–72.
  * Al-shammari and Alwan (2018) Mohammad Matar Al-shammari and Ali Amer Alwan. 2018.  Disaster recovery and business continuity for database services in multi-cloud. In _Proceedings of the 1st International Conference on Computer Applications & Information Security (ICCAIS)_. IEEE, 1–8.
  * Alaluna et al. (2017) Max Alaluna, Eric Vial, Nuno Neves, and Fernando MV Ramos. 2017\.  Secure and dependable multi-cloud network virtualization. In _Proceedings of the 1st International Workshop on Security and Dependability of Multi-Domain Infrastructures_. 1–6.
  * Alhamazani et al. (2014) Khalid Alhamazani, Rajiv Ranjan, Karan Mitra, Prem Prakash Jayaraman, Zhiqiang Huang, Lizhe Wang, and Fethi Rabhi. 2014.  Clams: Cross-layer multi-cloud application monitoring-as-a-service framework. In _Proceedings of the 11th International Conference on Services Computing_. IEEE, 283–290.
  * Almeida et al. (2014) André Almeida, Everton Cavalcante, Thais Batista, Nelio Cacho, and Frederico Lopes. 2014.  A component-based adaptation approach for multi-cloud applications. In _Proceedings of the 7th International Conference on Computer Communications Workshops (INFOCOM WKSHPS)_. IEEE, 49–54.
  * Alonso et al. (2023a) Juncal Alonso, Leire Orue-Echevarria, Valentina Casola, Ana Isabel Torre, Maider Huarte, Eneko Osaba, and Jesus L Lobo. 2023a.  Understanding the challenges and novel architectural models of multi-cloud native applications–a systematic literature review.  _Journal of Cloud Computing_ 12, 1 (2023), 1–34.
  * Alonso et al. (2023b) Juncal Alonso, Leire Orue-Echevarria, Valentina Casola, Ana Isabel Torre, Maider Huarte, Eneko Osaba, and Jesus L Lobo. 2023b.  Understanding the challenges and novel architectural models of multi-cloud native applications–a systematic literature review.  _Journal of Cloud Computing_ 12, 1 (2023), 1–34.
  * Alonso et al. (2019) Juncal Alonso, Kyriakos Stefanidis, Leire Orue-Echevarria, Lorenzo Blasi, Michael Walker, Marisa Escalante, María José López, and Simon Dutkowski. 2019\.  DECIDE: an extended devops framework for multi-cloud applications. In _Proceedings of the 3rd International Conference on Cloud and Big Data Computing (ICCBDC)_. 43–48.
  * Altran et al. (2022) Luiz Fernando Altran, Guilherme Galante, and Marcio Seiji Oyamada. 2022.  Label-affinity-Scheduler: Considering Business Requirements in Container Scheduling for Multi-Cloud and Multi-Tenant Environments. In _Proceedings of the 12th Brazilian Symposium on Computing Systems Engineering (SBESC)_. IEEE, 1–8.
  * Aral et al. (2020) Atakan Aral, Rafael Brundo Uriarte, Anthony Simonet-Boulogne, and Ivona Brandic. 2020.  Reliability management for blockchain-based decentralized multi-cloud. In _Proceedings of the 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID)_. IEEE, 21–30.
  * Austin (2018) Greg Austin. 2018\.  _Cybersecurity in China: The next wave_.  Springer.
  * Awada (2018) Uchechukwu Awada. 2018\.  Application-Container Orchestration Tools and Platform-as-a-Service Clouds: A Survey.  _International Journal of Advanced Computer Science and Applications_ (2018).
  * Baby and Vysala (2015) Kiran Baby and Anupriya Vysala. 2015.  Multicloud architecture for augmenting security in clouds. In _Proceedings of the 1st global conference on communication technologies (GCCT)_. IEEE, 474–478.
  * Bachiega et al. (2018) Naylor G Bachiega, Paulo SL Souza, Sarita M Bruschi, and Simone Do RS De Souza. 2018.  Container-based performance evaluation: A survey and challenges. In _Proceedings of the 6th IEEE International Conference on Cloud Engineering (IC2E)_. IEEE, 398–403.
  * Balalaie et al. (2016) Armin Balalaie, Abbas Heydarnoori, and Pooyan Jamshidi. 2016\.  Microservices architecture enables devops: Migration to a cloud-native architecture.  _IEEE Software_ 33, 3 (2016), 42–52.
  * Baresi et al. (2016) Luciano Baresi, Sam Guinea, Giovanni Quattrocchi, and Damian A Tamburri. 2016.  Microcloud: A container-based solution for efficient resource management in the cloud. In _Proceedings of the 1st International Conference on Smart Cloud (SmartCloud)_. IEEE, 218–223.
  * Barna et al. (2017) Cornel Barna, Hamzeh Khazaei, Marios Fokaefs, and Marin Litoiu. 2017.  Delivering elastic containerized cloud applications to enable DevOps. In _Proceedings of the 12th IEEE/ACM International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS)_. IEEE, 65–75.
  * Bélair et al. (2019) Maxime Bélair, Sylvie Laniepce, and Jean-Marc Menaud. 2019\.  Leveraging kernel security mechanisms to improve container security: a survey. In _Proceedings of the 14th international conference on availability, reliability and security_. 1–6.
  * Bentaleb et al. (2022) Ouafa Bentaleb, Adam SZ Belloum, Abderrazak Sebaa, and Aouaouche El-Maouhab. 2022.  Containerization technologies: Taxonomies, applications and challenges.  _The Journal of Supercomputing_ 78, 1 (2022), 1144–1181.
  * Bhamare et al. (2015) Deval Bhamare, Raj Jain, Mohammed Samaka, Gabor Vaszkun, and Aiman Erbad. 2015.  Multi-cloud distribution of virtual functions and dynamic service deployment: Open ADN perspective. In _Proceedings of the 3rd IEEE International Conference on Cloud Engineering_. IEEE, 299–304.
  * Bisht and Kaur (2022) Sankalp Singh Bisht and Parmeet Kaur. 2022.  An Empirical Investigation of a Fault Tolerant Containerized Application Deployment. In _Proceedings of the 1st International Conference on Informatics (ICI)_. IEEE, 171–175.
  * Bokhari et al. (2018) Mohammad Ubaidullah Bokhari, Qahtan Makki, and Yahya Kord Tamandani. 2018.  A survey on cloud computing. In _Proceedings of CSI 2015 on Big Data Analytics (CSI)_. Springer, 149–164.
  * Bolivar et al. (2018) Luis Tomas Bolivar, Christos Tselios, Daniel Mellado Area, and George Tsolis. 2018.  On the deployment of an open-source, 5G-aware evaluation testbed. In _Proceedings of the 6th IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (MobileCloud)_. IEEE, 51–58.
  * Bordeleau et al. (2020) Francis Bordeleau, Benoit Combemale, Romina Eramo, Mark van den Brand, and Manuel Wimmer. 2020\.  Towards model-driven digital twin engineering: Current opportunities and future challenges. In _Proceedings of the 1st International Conference on Systems Modelling and Management (ICSMM)_ , Vol. 1. Springer International Publishing, 43–54.
  * Bou Ghantous and Gill (2021) Georges Bou Ghantous and Asif Qumer Gill. 2021.  Evaluating the DevOps Reference Architecture for Multi-cloud IoT-Applications.  _SN Computer Science_ 2 (2021), 1–35.
  * Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006.  Using thematic analysis in psychology.  _Qualitative research in psychology_ 3, 2 (2006), 77–101.
  * Bucur et al. (2018) Vlad Bucur, Catalin Dehelean, and Liviu Miclea. 2018\.  Object storage in the cloud and multi-cloud: State of the art and the research challenges. In _Proceedings of the 7th International Conference on Automation, Quality and Testing, Robotics (AQTR)_. IEEE, 1–6.
  * Buyya and Barreto (2015) Rajkumar Buyya and Diana Barreto. 2015.  Multi-cloud resource provisioning with Aneka: A unified and integrated utilisation of microsoft azure and amazon EC2 instances. In _Proceedings of the 1st International Conference on Computing and Network Communications (CoCoNet)_. IEEE, 216–229.
  * Buyya et al. (2010) Rajkumar Buyya, Rajiv Ranjan, and Rodrigo N Calheiros. 2010\.  Intercloud: Utility-oriented federation of cloud computing environments for scaling of application services. In _Processing of the 10th International Conference on Algorithms and Architectures for Parallel_. Springer, 13–31.
  * Buyya et al. (2020) Rajkumar Buyya, Satish Narayana Srirama, Giuliano Casale, Rodrigo Calheiros, Yogesh Simmhan, Blesson Varghese, Erol Gelenbe, Bahman Javadi, Luis Miguel Vaquero, Marco AS Netto, et al. 2020\.  A manifesto for future generation cloud computing: Research directions for the next decade.  _ACM Computing Surveys (CSUR)_ 51, 5 (2020), 1–38.
  * Carpio et al. (2020) Francisco Carpio, Marta Delgado, and Admela Jukan. 2020\.  Engineering and experimentally benchmarking a container-based edge computing system. In _Proceedings of the 33rd International Conference on Communications (ICC)_. IEEE, 1–6.
  * Casalicchio (2019) Emiliano Casalicchio. 2019\.  Container orchestration: A survey.  _Systems Modeling: Methodologies and Tools_ (2019), 221–235.
  * Casalicchio and Iannucci (2020) Emiliano Casalicchio and Stefano Iannucci. 2020.  The state-of-the-art in container technologies: Application, orchestration and security.  _Concurrency and Computation: Practice and Experience_ 32, 17 (2020), e5668.
  * Casola et al. (2017) Valentina Casola, Alessandra De Benedictis, Massimiliano Rak, Umberto Villano, Erkuden Rios, Angel Rego, and Giancarlo Capone. 2017\.  MUSA deployer: Deployment of multi-cloud applications. In _Proceedings of the 26th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE)_. IEEE, 107–112.
  * Challita et al. (2017) Stéphanie Challita, Fawaz Paraiso, and Philippe Merle. 2017\.  Towards formal-based semantic interoperability in multi-clouds: the fclouds framework. In _Proceedings of the 10th International Conference on Cloud Computing (CLOUD)_. IEEE, 710–713.
  * Chen et al. (2021) Jacky Chen, Chee Peng Lim, Kim Hua Tan, Kannan Govindan, and Ajay Kumar. 2021.  Artificial intelligence-based human-centric decision support framework: an application to predictive maintenance in asset management under pandemic environments.  _Annals of Operations Research_ (2021), 1–24.
  * Chen et al. (2022) Yuheng Chen, Tao Shi, Hui Ma, and Gang Chen. 2022\.  Automatically design heuristics for multi-objective location-aware service brokering in multi-cloud. In _Proceedings of the 19th International Conference on Services Computing (SCC)_. IEEE, 206–214.
  * Cohen (1968) Jacob Cohen. 1968\.  Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit.  _Psychological bulletin_ 70, 4 (1968), 213\.
  * Combe et al. (2016) T. Combe, A. Martin, and R. Di Pietro. 2016.  To Docker or Not to Docker: A Security Perspective.  _IEEE Cloud Computing_ 3, 5 (2016), 54–62.
  * Cui et al. (2016) Wei Cui, Hanglong Zhan, Bao Li, Hu Wang, and Donggang Cao. 2016.  Cluster as a service: A container based cluster sharing approach with multi-user support. In _Proceedings of the 2016 Symposium on Service-Oriented System Engineering (SOSE)_. IEEE, 111–118.
  * de Laat et al. (2020) Maarten de Laat, Srecko Joksimovic, and Dirk Ifenthaler. 2020\.  Artificial intelligence, real-time feedback and workplace learning analytics to support in situ complex problem-solving: A commentary.  _The International Journal of Information and Learning Technology_ 37, 5 (2020), 267–277.
  * Debroy et al. (2018) Vidroha Debroy, Senecca Miller, and Lance Brimble. 2018\.  Building lean continuous integration and delivery pipelines by applying devops principles: a case study at varidesk. In _Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_. 851–856.
  * del Castillo et al. (2013) Juan Angel Lorenzo del Castillo, Kate Mallichan, and Yahya Al-Hazmi. 2013\.  Openstack federation in experimentation multi-cloud testbeds. In _Proceedings of the 5th International Conference on Cloud Computing Technology and Science_ , Vol. 2. IEEE, 51–56.
  * Deng et al. (2022) Qiqing Deng, Xinrui Tan, Jing Yang, Chao Zheng, Liming Wang, and Zhen Xu. 2022\.  A secure container placement strategy using deep reinforcement learning in cloud. In _Proceedings of the 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD)_. IEEE, 1299–1304.
  * Di Modica et al. (2019) Giuseppe Di Modica, Orazio Tomarchio, Hao Wei, Joaquin Salvachua Rodriguez, et al. 2019\.  Policy-based Deployment in a Hybrid and Multicloud Environment.. In _CLOSER_. 388–395.
  * Dimitrijevic et al. (2019) Zoran Dimitrijevic, Cetin Sahin, Christian Tinnefeld, and Jozsef Patvarczki. 2019.  Importance of Application-level resource management in Multi-cloud deployments. In _Proceedings of the 7th International Conference on Cloud Engineering (IC2E)_. IEEE, 139–144.
  * Docker (2021) Docker. 2021.  What is a Container?  https://docs.docker.com/get-started/overview/#what-is-a-container
  * Docker (2023) Docker. 2023.  Use Containers to Build, Share, and Run Your Applications.  https://www.docker.com/resources/what-container/.  Accessed: 2023-08-20.
  * Dolui and Kiraly (2018) Koustabh Dolui and Csaba Kiraly. 2018.  Towards multi-container deployment on IoT gateways. In _Proceedings of the 34th Global Communications Conference (GlobeCom)_. IEEE, 1–7.
  * Dragoni et al. (2017) Nicola Dragoni, Saverio Giallorenzo, Alberto Lluch-Lafuente, Manuel Mazzara, Fabrizio Montesi, Ruslan Mustafin, and Larisa Safina. 2017\.  _Microservices: yesterday, today, and tomorrow_.  Springer, 195–216.
  * Dreibholz et al. (2019) Thomas Dreibholz, Somnath Mazumdar, Feroz Zahid, Amir Taherkordi, and Ernst Gunnar Gran. 2019.  Mobile edge as part of the multi-cloud ecosystem: a performance study. In _Proceedings of the 27th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP)_. IEEE, 59–66.
  * Fadda et al. (2019) Edoardo Fadda, Pierluigi Plebani, and Monica Vitali. 2019\.  Monitoring-aware optimal deployment for applications based on microservices.  _IEEE Transactions on Services Computing_ 14, 6 (2019), 1849–1863.
  * Fernandes et al. (2014) Diogo AB Fernandes, Liliana FB Soares, João V Gomes, Mário M Freire, and Pedro RM Inácio. 2014.  Security issues in cloud environments: a survey.  _International journal of information security_ 13 (2014), 113–170.
  * Ferrer et al. (2016) Ana Juan Ferrer, David García Pérez, and Román Sosa González. 2016\.  Multi-cloud platform-as-a-service model, functionalities and approaches.  _Procedia Computer Science_ 97 (2016), 63–72.
  * Ferry et al. (2013) Nicolas Ferry, Alessandro Rossini, Franck Chauvel, Brice Morin, and Arnor Solberg. 2013.  Towards model-driven provisioning, deployment, monitoring, and adaptation of multi-cloud systems. In _Proceedings of the 6th International Conference on Cloud Computing (ICCC)_. IEEE, 887–894.
  * Ferry et al. (2014) Nicolas Ferry, Hui Song, Alessandro Rossini, Franck Chauvel, and Arnor Solberg. 2014.  CloudMF: applying MDE to tame the complexity of managing multi-cloud applications. In _Proceedings of the 7th International Conference on Utility and Cloud Computing_. IEEE, 269–277.
  * Fowler (2010) Martin Fowler. 2010\.  Continuous Delivery.  https://martinfowler.com/bliki/BlueGreenDeployment.html.
  * Gao et al. (2017) Xing Gao, Zhongshu Gu, Mehmet Kayaalp, Dimitrios Pendarakis, and Haining Wang. 2017.  Containerleaks: Emerging security threats of information leakages in container clouds. In _Proceedings of the 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)_. IEEE, 237–248.
  * Gattobigio et al. (2022) Luca Gattobigio, Steffen Thielemans, Priscilla Benedetti, Gianluca Reali, An Braeken, and Kris Steenhaut. 2022.  A multi-cloud service mesh approach applied to Internet of Things. In _Proceedings of the 48th Annual Conference of the IEEE Industrial Electronics Society (IECON)_. IEEE, 1–6.
  * Gill et al. (2022) Sukhpal Singh Gill, Minxian Xu, Carlo Ottaviani, Panos Patros, Rami Bahsoon, Arash Shaghaghi, Muhammed Golec, Vlado Stankovski, Huaming Wu, Ajith Abraham, et al. 2022\.  AI for next generation computing: Emerging trends and future directions.  _Internet of Things_ 19 (2022), 100514.
  * Gillam et al. (2013) Lee Gillam, Bin Li, and John O’Loughlin. 2013.  Benchmarking cloud performance for service level agreement parameters.  _International Journal of Cloud Computing_ 2, 1 (2013), 3–23.
  * Glaser and Strauss (1967) Barney G. Glaser and Anselm L. Strauss. 1967.  _The Discovery of Grounded Theory: Strategies for Qualitative Research_.  Aldine Publishing Company, Chicago.
  * Graupner et al. (2015a) Hendrik Graupner, Kennedy Torkura, Philipp Berger, Christoph Meinel, and Maxim Schnjakin. 2015a.  Secure access control for multi-cloud resources. In _Proceedings of the 40th Local Computer Networks Conference Workshops (LCN Workshops)_. IEEE, 722–729.
  * Graupner et al. (2015b) Hendrik Graupner, Kennedy Torkura, Philipp Berger, Christoph Meinel, and Maxim Schnjakin. 2015b.  Secure access control for multi-cloud resources. In _Proceedings of the 40th Local Computer Networks Conference Workshops (LCN Workshops)_. IEEE, 722–729.
  * Guechi and Maamri ([n. d.]) Farida Ali Guechi and Ramdane Maamri. [n. d.].  Secure and Parallel Expressive Search over Encrypted Data with Access Control in Multi-CloudIoT. In _Proceedings of the 3rd Cloudification of the Internet of Things (CIoT)_. 1–8.
  * Guerrero et al. (2018a) Carlos Guerrero, Isaac Lera, and Carlos Juiz. 2018a.  Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications.  _The Journal of Supercomputing_ 74, 7 (2018), 2956–2983.
  * Guerrero et al. (2018b) Carlos Guerrero, Isaac Lera, and Carlos Juiz. 2018b.  Resource optimization of container orchestration: a case study in multi-cloud microservices-based applications.  _The Journal of Supercomputing_ 74, 7 (2018), 2956–2983.
  * Gundu et al. (2020) Srinivasa Rao Gundu, Charan Arur Panem, and Anuradha Thimmapuram. 2020.  Hybrid IT and multi cloud an emerging trend and improved performance in cloud computing.  _SN Computer Science_ 1, 5 (2020), 256.
  * Gupta et al. (2021) Manu Gupta, Konte Sanjana, Kontham Akhilesh, and Mandepudi Nobel Chowdary. 2021.  Deployment of Multi-Tier Application on Cloud and Continuous Monitoring using Kubernetes. In _Proceedings of the 5th International Conference on Electrical, Electronics, Communication, Computer Technologies and Optimization Techniques (ICEECCOT)_. IEEE, 602–607.
  * Gupta et al. (2016) Rohan Raj Gupta, Gaurav Mishra, Subham Katara, Arpit Agarwal, Mrinal Kanti Sarkar, Rupayan Das, and Sanjay Kumar. 2016.  Data storage security in cloud computing using container clustering. In _Proceedings of the 7th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON)_. IEEE, 1–7.
  * Harwalkar et al. (2019a) Sudheendra Harwalkar, Dinkar Sitaram, Shivangi Jadon, Dhanaraj V Kidiyoor, and Ornella D’souza. 2019a.  Private STaaS with OpenStack Cinder Volumes for Hybrid/Multi-cloud. In _Proceedings of the 4th International Conference on Advances in Computing and Communication Engineering (ICACCE)_. IEEE, 1–6.
  * Harwalkar et al. (2019b) Sudheendra Harwalkar, Dinkar Sitaram, Dhanaraj V Kidiyoor, ML Milan, Ornella D’souza, Radhika Agarwal, and Yamini Agarwal. 2019b.  Multicloud-auto scale with prediction and delta correction algorithm. In _Proceedings of the 2nd International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT)_ , Vol. 1. IEEE, 227–233.
  * Hashem et al. (2016) Ibrahim Abaker Targio Hashem, Ibrar Yaqoob, Nor Badrul Anuar, Salimah Mokhtar, Abdullah Gani, and Samee Ullah Khan. 2016.  The rise of “big data” on cloud computing: Review and open research issues.  _Information Systems_ 47 (2016), 98–115.
  * He et al. (2022) Heng He, Jiaqi Liu, Jinguang Gu, and Feng Gao. 2022\.  An Efficient Multi-Keyword Search Scheme over Encrypted Data in Multi-Cloud Environment. In _2022 IEEE 7th International Conference on Smart Cloud (SmartCloud)_. IEEE, 59–67.
  * Helali and Omri (2021) Leila Helali and Mohamed Nazih Omri. 2021.  A survey of data center consolidation in cloud computing systems.  _Computer Science Review_ 39 (2021), 100366.
  * Hua et al. (2020) Lei Hua, Ting Tang, Heng Wu, WU Yuewen, LIU He, XU Yuanjia, and Wenbo Zhang. 2020\.  A framework to support multi-cloud collaboration. In _Proceedigs of the 13th World Congress on Services (SERVICES)_. IEEE, 110–116.
  * Imran et al. (2020) Hamza Ali Imran, Usama Latif, Ataul Aziz Ikram, Maryam Ehsan, Ahmed Jamal Ikram, Waleed Ahmad Khan, and Saad Wazir. 2020.  Multi-cloud: a comprehensive review. In _Proceedings of the 23rd International Multitopic Conference (INMIC)_. IEEE, 1–5.
  * Jamshidi et al. (2013) Pooyan Jamshidi, Aakash Ahmad, and Claus Pahl. 2013.  Cloud migration research: a systematic review.  _IEEE transactions on cloud computing_ 1, 2 (2013), 142–157.
  * Janarthanan et al. (2018) Kugathasan Janarthanan, PRLC Peramune, AT Ranaweera, Theviyanthan Krishnamohan, Lakmal Rupasinghe, Kalpa Kalhara Sampath, and Chethana Liyanapathirana. 2018.  Policies based container migration using cross-cloud management platform. In _Proccedings of the 8th International Conference on Information and Automation for Sustainability (ICIAfS)_. IEEE, 1–6.
  * Jha et al. (2019) Devki Nandan Jha, Zhenyu Wen, Yinhao Li, Michael Nee, Maciej Koutny, and Rajiv Ranjan. 2019\.  A cost-efficient multi-cloud orchestrator for benchmarking containerized web-applications. In _Proceedings of the 20th International Conference on Web Information Systems Engineering (WISE)_. Springer, 407–423.
  * Keele et al. (2007) Staffs Keele et al. 2007\.  _Guidelines for performing systematic literature reviews in software engineering_.  Technical Report. Citeseer.
  * Kiss (2018) Tamas Kiss. 2018\.  Scalable multi-cloud platform to support industry and scientific applications. In _Proceedings of the 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO)_. IEEE, 0150–0154.
  * Kritikos et al. (2019) Kyriakos Kritikos, Chrysostomos Zeginis, Joaquin Iranzo, Roman Sosa Gonzalez, Daniel Seybold, Frank Griesinger, and Jörg Domaschka. 2019.  Multi-cloud provisioning of business processes.  _Journal of Cloud Computing_ 8, 1 (2019), 1–29.
  * Lazuka et al. (2022) Malgorzata Lazuka, Thomas Parnell, Andreea Anghel, and Haralampos Pozidis. 2022.  Search-based methods for multi-cloud configuration. In _Proceedings of the 15th International Conference on Cloud Computing (CLOUD)_. IEEE, 438–448.
  * Li et al. (2023) Yuanbo Li, Hongchao Hu, Wenyan Liu, and Xiaohan Yang. 2023\.  An Optimal Active Defensive Security Framework for the Container-Based Cloud with Deep Reinforcement Learning.  _Electronics_ 12, 7 (2023), 1598.
  * Libardi et al. (2014) Rafael Mira De Oliveira Libardi, Marcos Vinicius Naves Bedo, Stephan Reiff-Marganiec, and Julio Cezar Estrella. 2014\.  MSSF: A step towards user-friendly multi-cloud data dispersal. In _Proceedings of the 7th International Conference on Cloud Computing_. IEEE, 952–953.
  * Lorenzo et al. (2019) C Lorenzo, P Guillaume, and P Bellavista. 2019.  FogDocker: Start Container Now Fetch Image Later. In _Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing (UCC)_.
  * Maenhaut et al. (2020) Pieter-Jan Maenhaut, Bruno Volckaert, Veerle Ongenae, and Filip De Turck. 2020.  Resource management in a containerized cloud: Status and challenges.  _Journal of Network and Systems Management_ 28 (2020), 197–246.
  * Mell et al. (2011) Peter Mell, Tim Grance, et al. 2011\.  The NIST definition of cloud computing.  (2011).
  * Merkel et al. (2014) Dirk Merkel et al. 2014\.  Docker: lightweight linux containers for consistent development and deployment.  _Linux j_ 239, 2 (2014), 2.
  * Mfula and Nurminen (2018) Harrison Mfula and Jukka K Nurminen. 2018.  Self-healing cloud services in private multi-clouds. In _Proceedings of the 10th International Conference on High Performance Computing & Simulation (HPCS)_. IEEE, 165–170.
  * Mihai et al. (2022) Stefan Mihai, Mahnoor Yaqoob, Dang V. Hung, William Davis, Praveer Towakel, Mohsin Raza, Mehmet Karamanoglu, and et al. 2022\.  Digital twins: A survey on enabling technologies, challenges, trends and future prospects.  _IEEE Communications Surveys & Tutorials_ (2022).
  * Mittal and Risco-Martín (2017) Saurabh Mittal and José L Risco-Martín. 2017\.  DEVSML 3.0 stack: rapid deployment of DEVS farm in distributed cloud environment using microservices and containers. In _Proceedings of the 2017 Symposium on Theory of Modeling & Simulation_. 1–12.
  * Mungoli (2023) Neelesh Mungoli. 2023\.  Scalable, Distributed AI Frameworks: Leveraging Cloud Computing for Enhanced Deep Learning Performance and Efficiency.  _arXiv preprint arXiv:2304.13738_ (2023).
  * Naik (2016a) Nitin Naik. 2016a.  Applying computational intelligence for enhancing the dependability of multi-cloud systems using docker swarm. In _Proceedings of the 2nd Symposium Series on Computational Intelligence (SSCI)_. IEEE, 1–7.
  * Naik (2016b) Nitin Naik. 2016b.  Building a virtual system of systems using docker swarm in multiple clouds. In _Proceedings of the 2nd International Symposium on Systems Engineering (ISSE)_. IEEE, 1–3.
  * Nardelli et al. (2018) Matteo Nardelli, Valeria Cardellini, and Emiliano Casalicchio. 2018\.  Multi-Level Elastic Deployment of Containerized Applications in Geo-Distributed Environments. In _Proceedings of the 6th IEEE International Conference on Future Internet of Things and Cloud (FiCloud)_. IEEE, 1–8.
  * Netaji and Bhole (2021) Vhatkar Kapil Netaji and Girish P Bhole. 2021.  A comprehensive survey on container resource allocation approaches in cloud computing: State-of-the-art and research challenges. In _Web Intelligence_ , Vol. 19. IOS Press, 295–316.
  * of Standards and Technology (2018) National Institute of Standards and Technology. 2018.  _NIST Cyber Security Framework Version 1.1_.  Technical Report.
  * Ongaro and Ousterhout (2014) Diego Ongaro and John Ousterhout. 2014.  In search of an understandable consensus algorithm. In _Proceedings of the 2014 USENIX Annual Technical Conference (USENIX ATC)_. USENIX, 305–319.
  * Ouyang et al. (2022) Mingxue Ouyang, Jianqing Xi, Weihua Bai, and Keqin Li. 2022\.  Band-area resource management platform and accelerated particle swarm optimization algorithm for container deployment in Internet-of-Things cloud.  _IEEE Access_ 10 (2022), 86844–86863.
  * Pahl (2015) Claus Pahl. 2015\.  Containerization and the paas cloud.  _IEEE Cloud Computing_ 2, 3 (2015), 24–31.
  * Pahl et al. (2017) Claus Pahl, Antonio Brogi, Jacopo Soldani, and Pooyan Jamshidi. 2017.  Cloud container technologies: a state-of-the-art review.  _IEEE Transactions on Cloud Computing_ 7, 3 (2017), 677–692.
  * Pai and Kumar (2021) Pradeep Pai and CRS Kumar. 2021.  Building cloud native application—analysis for multi-component application deployment. In _Proceedings of the 10th International Conference on Computer Communication and Informatics (ICCCI)_. IEEE, 1–6.
  * Paladi et al. (2018) Nicolae Paladi, Antonis Michalas, and Hai-Van Dang. 2018\.  Towards secure cloud orchestration for multi-cloud deployments. In _Proceedings of the 5th Workshop on CrossCloud Infrastructures & Platforms (CrossCloud)_. ACM, 1–6.
  * Pandey et al. (2022) Ashish Pandey, Prasad Calyam, Zhen Lyu, Songjie Wang, Dmitrii Chemodanov, and Trupti Joshi. 2022\.  Knowledge-Engineered Multi-Cloud Resource Brokering for Application Workflow Optimization.  _IEEE Transactions on Network and Service Management_ (2022).
  * Pellegrini et al. (2017) Roland Pellegrini, Patrick Rottmann, and Georg Strieder. 2017\.  Preventing vendor lock-ins via an interoperable multi-cloud deployment approach. In _Proceedings of the 12th International Conference for Internet Technology and Secured Transactions (ICITST)_. IEEE, 382–387.
  * Petcu (2011) Dana Petcu. 2011\.  Portability and interoperability between clouds: challenges and case study. In _Proceedings of the 4th European Conference on Towards a Service-Based Internet (ServiceWave)_. Springer, 62–74.