# Architecture Decisions in Quantum Software Systems: An Empirical Study on
Stack Exchange and GitHub

Mst Shamima Aktar<EMAIL_ADDRESS>School of Computer Science, Wuhan
University, 430072 Wuhan, China  Peng Liang<EMAIL_ADDRESS>Muhammad Waseem
<EMAIL_ADDRESS>Faculty of Information Technology, University of
Jyväskylä, FI-40014 Jyväskylä, Finland  Amjed Tahir<EMAIL_ADDRESS>School of Mathematical and Computational Sciences, Massey University, 4442
Palmerston North, New Zealand
Aakash Ahmad<EMAIL_ADDRESS>School of Computing and
Communications, Lancaster University Leipzig, 04109 Leipzig, Germany  Beiqi
Zhang<EMAIL_ADDRESS>Zengyang Li<EMAIL_ADDRESS>School of
Computer Science, Central China Normal University, Wuhan, China

###### Abstract

Context: Quantum computing provides a new dimension in computation, utilizing
the principles of quantum mechanics to potentially solve complex problems that
are currently intractable for classical computers. However, little research
has been conducted about the architecture decisions made in quantum software
development, which have a significant influence on the functionality,
performance, scalability, and reliability of these systems.

Objective: The study aims to empirically investigate and analyze architecture
decisions made during the development of quantum software systems, identifying
prevalent challenges and limitations by using the posts and issues from Stack
Exchange and GitHub.

Method: We used a qualitative approach to analyze the obtained data from Stack
Exchange Sites and GitHub projects - two prominent platforms in the software
development community. Specifically, we collected data from 151 issues (from
47 GitHub projects) and 43 posts (from three Stack Exchange sites) related to
architecture decisions in quantum software development.

Results: The results show that in quantum software development (1)
architecture decisions are articulated in six linguistic patterns, the most
common of which are Solution Proposal and Information Giving, (2) the two
major categories of architectural decisions are Implementation Decision and
Technology Decision, (3) Quantum Programming Framework is the most common
application domain among the sixteen application domains identified, (4)
Maintainability is the most frequently considered quality attribute, and (5)
Design Issue and Performance Issue are the major limitations and challenges
that practitioners face when making architecture decisions in quantum software
development.

Conclusions: Our results show that the limitations and challenges encountered
in architecture decision-making during the development of quantum software
systems are strongly linked to the particular features (e.g., quantum
entanglement, superposition, and decoherence) of those systems. These issues
mostly pertain to technical aspects and need appropriate measures to address
them effectively.

###### keywords:

Architecture Decision, Quantum Software System, Stack Exchange, GitHub,
Empirical Study

††journal: Information and Software Technology

##  1 Introduction

Quantum computing is a rapidly developing technology that is a combination of
physics, chemistry, mathematics, computer science, and information theory to
revolutionize computation and problem-solving capabilities [1]. Quantum
computing has largely been motivated by academic and industrial interest to
change the scenery of information technology [2]. Integrating quantum software
and programming with quantum hardware is fundamental to quantum computing
research [3]. This integration makes it possible to develop and execute
quantum algorithms designed to utilize qubits’ power and solve complex
problems more efficiently than classical algorithms [4]. Therefore, central to
the potency of quantum computing are quantum algorithms, including Shor’s
large numbers factoring algorithm [5] and Grover’s quantum search algorithm
[6]. Quantum algorithms are used across various domains, such as cryptography,
search and optimization processes, and solving linear equation systems [7][8].
Quantum programming entails developing algorithms specifically for quantum
computers and reducing the execution costs of quantum algorithms on future
devices [9]. Quantum programs are written using domain-specific programming
languages such as Q# and Silq [10], and quantum software is important to fully
use the capabilities of quantum computing systems [11].

In the domain of software, quantum computing is ready to begin renovations
“golden age”, changing the landscape of software engineering [12]. Over the
last few years, the development of quantum programming languages, algorithms,
libraries, and tools has received much attention [13]. Different companies
(e.g., Qiskit111https://qiskit.org/ from IBM,
Cirq222https://quantumai.google/cirq from Google Research,
Q#333https://learn.microsoft.com/en-us/azure/quantum from Microsoft, and
C++444https://www.intel.com/content/www/us/en/developer/tools/quantum-
sdk/overview.html from Intel) and cloud providers (e.g., Amazon Web Services,
Alibaba Cloud, and Azure Quantum) have already invested in tools as well as
programming languages that allow developers, researchers, and users to solve
real-world problems (e.g., risk assessment and fraud detection in banking,
drug discovery, protein folding, supply chain optimization, energy
distribution, and advertising scheduling) [14]. Quantum computing has an
impact on all software life cycle processes and techniques, and quantum
software encounters multifaceted challenges [15], such as making existing
software quantum-safe [16], engineering challenges [17], and architecture
challenges of quantum computing systems [18].

Software architecture is the high-level design and organization of a software
system, including its structural components, interactions, and behavior [19].
Software architecture for quantum software systems structures quantum-
intensive systems, utilizing Qubits and Qugates as the essential components
and connectors for the software architecture design and implementation [18].
The architecture of a software system is designed through a decision-making
process, where the architecture is viewed as a composition of a set of
explicit design decisions [20]. Decision-making significantly influences the
success or failure of a software system, as a high-quality architecture
decision-making process can lead to desirable results, while a poor-quality
architecture and suboptimal architecture decisions more likely lead to
undesirable outcomes [21] and even architectural technical debt [22]. Making
architecture decisions during the development process of quantum software
systems is complex, requiring a comparative structure to analyze and select
among various tools, services, and techniques based on specific use cases and
algorithms [23]. Architecture decisions are increasingly influenced by
practitioner feedback, focusing on hybrid quantum-classical patterns and
service orientation to efficiently address emerging computational challenges
in quantum software [24]. Architecture decisions also affect many aspects of a
software system, including its quality (e.g., maintainability, testability,
and reliability) of quantum software development [25]. Recent workshops, such
as the Quantum Software Architecture Workshop (QSA), highlight the growing
importance of architectural considerations in the domain of quantum computing
[26]. Despite the essential importance of architecture decisions in the
development of quantum software systems, architecture decisions are unexplored
in the area of quantum software development [25]. Software architects
encounter various limitations and challenges and should carefully consider
overcoming architecture challenges in a quantum software system [27]. To this
end, this study aims to analyze the architecture decisions as well as the
challenges in architecture decision-making associated with quantum software
development from discussions made between developers in GitHub issues and
Stack Exchange posts.

To achieve the goal of this study (see Section 3.1), we conducted an empirical
study to investigate architectural decisions made in quantum software
development. We collected data from two sources: GitHub issues (of open source
quantum software projects) and Stack Exchange Q&A sites (Stack Overflow,
Quantum Computing, and Computer Science). More specifically, we collected
1,760 closed issues (from 164 GitHub projects) and 1093 posts (from the Stack
Exchange sites). We manually filtered out irrelevant issues and posts, and we
finally got 151 related issues (from 47 quantum software projects) and 43
posts from Stack Exchange. Furthermore, we manually extracted data from the
151 related issues and 43 posts, and analyzed the extracted data using a
predefined classification [28] and the Constant Comparison method [29] for
answering the research questions.

Our study results show that (1)Solution Proposal and Information Giving are
the most frequently used linguistic patterns, (2) Implementation Decision and
Technology Decision are the two main types of architecture decisions, (3)
Quantum Programming Framework is the most common application domain
identified, (4) Maintainability is the dominant quality attribute considered,
and (5) Design Issue and Performance Issue are the main limitations and
challenges when making architecture decisions in quantum software development.

The contributions of this work: (1) we identified the linguistic patterns
practitioners utilized when making architecture decisions in developing
quantum software systems; (2) we investigated several facets of architecture
decisions in quantum software development, such as the categories of decisions
made, the application domains, and the quality attributes considered; (3) we
present and discuss the challenges and limitations developers face while
designing quantum software systems; (4) we provide guidelines to help
practitioners design quantum software systems with informed architectural
decisions; and (5) we constructed a dataset [30] that collects the
architecture decisions made in quantum software development.

The rest of this paper is structured as follows: Section 2 presents the
background of the study. Section 3 describes the research methodology. Section
4 provides the study results, followed by a discussion of these results in
Section 5. Section 6 clarifies the validity threats. Section 7 discusses the
related work. Finally, Section 8 concludes the work with future research
directions.

##  2 Background

In this section, we introduce the background concepts used in this study,
including quantum computing, quantum software engineering, and architecture
decisions. The overview of the background is shown in Fig. 1.

###  2.1 Quantum computing

Quantum computing is a model of computation that focuses on designing and
developing computer-based systems utilizing the principles of quantum
mechanics (e.g., superposition and entanglement) to process information. In
the early 1980s, physicist Richard Feynman proposed the notion of simulating
quantum systems using quantum mechanical processes [31]. Over a couple of
decades, quantum computing has transitioned from theoretical concepts to
practical implementations, driven by exponentially increasing computational
potential [32]. This transformation has motivated research into quantum
algorithms, error correction, and quantum simulation [33]. Quantum computing
plays an important role in several fields of computer science most notably
natural language processing [34], machine learning [35], and cyber security
[36]. To realize practical quantum applications, it is essential to have
quantum software tools for various design tasks across multiple abstraction
levels, along with practical benchmarks to empirically evaluate and compare
these tools against the current state of the art [37]. However, the
development of practical quantum systems remains difficult due to the lack of
reliability requirements of quantum computing algorithms, physical machines
foreseen, and hardware limitations [38]. Quantum Computing as a Service
(QCaaS) by establishing a reference architecture, demonstrates its practical
application through microservices in the quantum-classic divide and validates
its effectiveness and reusability via insights from professionals in the field
of quantum software engineering.

Figure 1:  a) Overview of Quantum Computing Systems, b) Mining Software
Repositories [Social (coding) Platforms]

###  2.2 Quantum software engineering

Quantum Software Engineering (QSE) represents a transformative approach
combining the principles of quantum mechanics with the established practices
of software engineering, aiming to design, develop, validate, and evolve
software systems and applications fit for the quantum era [24]. QSE is an
approaching genre of software engineering that exceeds the limitations of
traditional computing [39]. The life cycle in QSE is comprehensive,
encompassing quantum requirements engineering, design, implementation,
testing, and maintenance, requiring collaborative efforts from both scientific
and industry to propel real-world applicability [40]. The intersection of
Software Engineering and Quantum Computing Engineering underscores the
necessity for a tailored software development life cycle, which accentuates
the importance of requirements engineering, particularly in the quantum
domain, where it facilitates communication among stakeholders and helps in
comprehending complex application domains [41]. QSE is a growing field of
study and quantum software architectures (QSA) is a sub-domain of study, both
are understudied and face research challenges [27]. Quantum Software
Engineering (QSE) seeks to deal with the evolving challenges of quantum
software development, requiring particular architecture decisions within the
decision-making process to overcome these challenges [42].

###  2.3 Architecture decision

Software architecture is a blueprint that guides the development process,
ensuring the system’s quality attributes (e.g., maintainability, performance,
scalability) and aligning technical decisions with business goals [43]. In
many contexts, including the Internet of Things (IoT) [44], machine learning-
based systems [45], and microservices [46], empirical studies and systematic
mappings emphasize the importance of architecture decisions. The importance of
architecture decisions in quantum software systems is also highlighted by a
systematic review on software architecture for quantum computing systems [18].
The documentation, communication, and evaluation of these crucial decisions
are made easier by existing models and tools [47]. As new architectural styles
and paradigms arise, these decisions help developers manage common challenges
in architecture design [20]. The relevance of architecture decisions even in
cutting-edge fields is reflected in practical perspectives on agile quantum
software development [48]. When software systems become more complex,
understanding, clearly expressing, and making effective architecture decisions
are essential for achieving software quality and meeting stakeholder needs.

##  3 Research design

###  3.1 Research goal and research questions

The goal of this study is formulated using a Goal-Question-Metric approach
[49]: We aim to analyze architecture decisions made during the development of
quantum software for the purpose of investigating their decision description
and categorization, application domains covered, quality attributes
considered, and the limitations and challenges from the point of view of
practitioners in the context of architecture-related posts and GitHub issues
of quantum software systems. In order to perform this empirical study, we
adhered to the recommendations for conducting empirical studies by Easterbrook
et al. [50]. Following the defined goal, we provide five Research Questions
(RQs), their rationale (see Table 1), and an overview of the research process
(see Fig. 2) in the subsections below.

Table 1: Research questions and their rationale Research Question |  Rationale
---|---
RQ1. How do developers express architecture decisions made in quantum software development? |  This RQ aims to provide the linguistic patterns used by developers to express architecture decisions in the development of quantum software systems. Developers express architecture decisions in quantum software systems through various linguistic patterns to explain their ideas and communicate with their peers. The answer to this RQ can help developers identify architecture decision-related content in quantum systems development.
RQ2. What types of architecture decisions are made in quantum software development? |  This RQ intends to categorize the architecture decisions made in quantum software system. Developers make a number of architecture decisions while developing quantum software systems, and these decisions cover various aspects of architecture design. The answer to this RQ can offer insights into the specific categories of architectural decisions being made in quantum software systems.
RQ3. What are the application domains of quantum software in which architecture decisions are made? |  This RQ aims to analyze the application domains of quantum software systems in which architecture decisions are made. Quantum technologies are widely employed in developing software systems across many application domains. The answer to this RQ may provide guidance for making architectural decisions in particular domains.
RQ4. What quality attributes are considered when developers make architecture decisions in quantum software development? |  This RQ aims to provide the quality attributes that developers focus on when making architecture decisions in quantum software development. Quantum software systems have dedicated concerns and the answer to this RQ can provide a comprehensive set of quality concerns highlighted in quantum software development.
RQ5. What are the limitations and challenges of making architecture decisions in quantum software development? |  This RQ aims to investigate the limitations and challenges developers may encounter when making architecture decisions for quantum software systems with related quantum components. Making architecture decisions in the creation of quantum software still has a number of limitations and challenges. The answer to this RQ can provide the directions for future exploration.

###  3.2 Data collection

We collected and analyzed data from 47 GitHub555https://github.com projects
and three Stack Exchange666https://stackexchange.com/sites sites. GitHub hosts
a large number of open-source projects and offers access to rich data from
those projects, including version history and issue tracking data [51]. Stack
Exchange sites contain a number of technical Q&A sites that cover various
topics and domains, which include Stack Overflow777https://stackoverflow.com/,
one of the most active Q&A sites that developers use from all over the world
[52]. Quantum Computing Stack
Exchange888https://quantumcomputing.stackexchange.com/ site is a dedicated
platform for topics related to quantum computing. Computer Science Stack
Exchange999https://cs.stackexchange.com/ is specifically tailored to the field
of computer science questions. Many researchers have used GitHub and Stack
Exchange sites for quantum software system research. For example, Li et al.
[53] explored the challenges related to Quantum Software Engineering (QSE) by
analyzing data from GitHub and Stack Exchange sites. Their research focused on
understanding the broader challenges perceived by developers in the area of
QSE. Similarly, we decided to use both GitHub and Stack Exchange sites in our
research. However, our focus is to analyze architecture decisions in quantum
software systems.

1) Data Collection from GitHub Projects

We followed a two-step process to collect quantum projects hosted on GitHub.
First, we identified a set of quantum projects from GitHub. Then, we collect
the issues from these projects.

In the first step, we performed a pilot search with several terms before we
decided on the most appropriate terms for selecting relevant quantum projects
on GitHub, namely “quantum” and “quantum computing”. The search encompassed
content within the projects (e.g., name, topics, descriptions, and README
files). We found 74,432 projects by using the keyword “quantum”. In contrast,
using the keyword “quantum computing” resulted in significantly fewer projects
(12,401), although with many irrelevant results such as
books101010https://github.com/JackHidary/quantumcomputingbook and other
learning resources111111https://github.com/desireevl/awesome-quantum-
computing. Notably, “quantum computing” is also a subdomain of the keyword
“quantum”. During the search process by using the keyword “quantum”, we also
discovered that the retrieved projects contained the keyword “quantum
computing”. Therefore, we decided to use the keyword “quantum”. To mine those
projects, we used GitHub REST
API121212https://docs.github.com/en/rest?apiVersion=2022-11-28. We applied the
following selection criteria: 1) projects that contain the word “quantum”
(i.e., including the name, description, topics, and README file); 2) the
number of stars $\geq$ 50; and 3) the number of forks is $\geq$ 15 to minimize
the possibility of including student assignments following the idea of prior
studies [54]. We identified 1,226 projects that meet the above selection
criteria. Later, we used the following criteria to filter out our search
results [55]. We manually checked the repository (e.g., description, topics,
and README file) and excluded those that pertained solely to quantum
documentation or study materials or tutorials, not written in the English
language, and included the term “quantum” which was not relevant to quantum
software systems (e.g.,
FirefoxColor131313https://github.com/mozilla/FirefoxColor). In this step, 885
projects were removed, and 341 projects remained.

In the second step, we initially carried out a pilot search using various
keywords that are the same as the architecture search terms “architect*”
(i.e., “architect”, “architecture”, “architectural”, “architecting”,
“architected”) and “design” to search the title and body of the closed issues.
We found issues that include the keywords (“architect”, “architecture”,
“architectural”, “architecting”, “architected”, and “design”). Therefore, we
decided to use those keywords and searched the title and body of the closed
issues of the 341 quantum projects. After searching with the search terms, we
found 1,760 closed issues containing the keywords, and those issues came from
164 quantum software projects. We ended up the data collection process from
GitHub with 164 quantum software systems and 1,760 closed issues.

Figure 2: Overview of the research process Table 2: GitHub Projects, Number of Retrieved Issues, and Number of Related Issues # |  GitHub Project |  Number of Issues |  Number of Retrieved Issues |  Number of Related Issues |  Number of Stars
---|---|---|---|---|---
GP1 |  Qiskit |  3345 |  165 |  19 |  3800
GP2 |  ARTIQ |  1389 |  135 |  12 |  372
GP3 |  Cirq |  1842 |  110 |  9 |  3900
GP4 |  deepchem |  1206 |  93 |  16 |  4600
GP5 |  Qiskit Metal |  379 |  91 |  6 |  250
GP6 |  PennyLane |  783 |  49 |  7 |  1900
GP7 |  liboqs |  533 |  42 |  5 |  1400
GP8 |  qmcpack |  958 |  40 |  3 |  245
GP9 |  Covalent |  652 |  40 |  3 |  500
GP12 |  Qiskit Nature |  361 |  33 |  6 |  239
GP10 |  psi4 |  887 |  32 |  5 |  839
GP11 |  Mitiq |  654 |  32 |  4 |  286
GP13 |  cp2k |  491 |  30 |  2 |  661
GP14 |  RMG-Py |  198 |  27 |  5 |  329
GP16 |  netket |  320 |  26 |  1 |  450
GP17 |  qiskit-metapackage |  528 |  25 |  2 |  3100
GP18 |  Qrack |  141 |  25 |  1 |  154
GP15 |  QCoDeS |  410 |  24 |  4 |  266
GP19 |  tket |  140 |  22 |  1 |  197
GP20 |  QuTiP |  718 |  20 |  3 |  1500
GP22 |  QuantumLibraries |  137 |  19 |  1 |  526
GP21 |  PyRPL |  268 |  18 |  2 |  114
GP23 |  TensorFlow Quantum |  198 |  15 |  1 |  3800
GP24 |  qiskit-aqua |  418 |  15 |  1 |  564
GP25 |  OpenSSL |  185 |  15 |  3 |  172
GP26 |  qiskit-ibm-runtime |  369 |  15 |  1 |  95
GP27 |  OpenQL |  283 |  13 |  2 |  82
GP28 |  qiskit-ibmq-provider |  357 |  10 |  1 |  228
GP29 |  DFTK.jl |  174 |  10 |  1 |  356
GP30 |  PQClean |  160 |  9 |  3 |  423
GP32 |  quantum-core |  217 |  7 |  3 |  2200
GP33 |  OpenQASM |  128 |  7 |  2 |  1000
GP31 |  ecosystem |  155 |  6 |  1 |  70
GP34 |  QRL |  197 |  6 |  2 |  396
GP35 |  qsim |  126 |  5 |  1 |  361
GP36 |  qsharp-runtime |  129 |  5 |  1 |  380
GP37 |  cuda-quantum |  108 |  4 |  1 |  222
GP38 |  circuit-knitting-toolbox |  54 |  3 |  1 |  51
GP39 |  deepqmc |  31 |  2 |  1 |  300
GP40 |  Amazon Braket Python SDK |  51 |  2 |  1 |  254
GP41 |  QUICK |  93 |  2 |  1 |  177
GP42 |  qpic |  6 |  2 |  1 |  136
GP43 |  SIRIUS |  85 |  2 |  1 |  103
GP44 |  QCEngine |  67 |  2 |  1 |  141
GP45 |  qiskit-dynamics |  63 |  1 |  1 |  78
GP46 |  dwave-hybrid |  64 |  1 |  1 |  76
GP47 |  dwave-cloud-client |  165 |  1 |  1 |  54
Total |  |  20223 |  1258 |  151 |

2) Data Collection from Stack Exchange Sites

To identify relevant posts in the selected Stack Exchange sites, we first
conducted a pilot search to construct the search terms. The search terms were
then applied to all parts of the posts, including the title, the question
body, and the answers. Before we decided on the most suitable terms for
capturing posts relevant to architecture of quantum software systems, we first
performed a pilot search with several terms, namely “quantum architect” (i.e.,
“quantum architect”, “quantum architecture”, “quantum architectural”, and
“quantum architecting”) and “quantum design” within all three Stack Exchange
sites. We found posts that include the keywords (“quantum architect*” and
“quantum design”) using the original search terms. Second, Stack Exchange
supports the use of wildcard (*) searches to broaden the search results. We
defined “quantum architect*” and “quantum design” as the initial search terms
related to architecture decisions in quantum software. Table 3 shows the
search terms that were utilized, and the number of posts that were obtained
from all three sites. We retrieved a total of 1093 posts. Note that a post may
contain multiple sets of keywords, which means that there could be duplicate
items in the set of URLs of these posts. After removing the duplicated posts,
we finally got 975 unique posts. More specifically, 324 from Stack Overflow,
604 from Quantum Computing, and 47 from Computer Science Stack Exchange.

Table 3: Stack Exchange Name, Search Term, Retrieved Posts, and Related Posts  # |  Stack Exchange Name |  Search Terms |  Number of Retrieved Posts |  Number of Related Posts
---|---|---|---|---
SEN1 |  Stack Overflow |  “quantum architect*” |  141 |  11
SEN2 |  Stack Overflow |  “quantum design” |  223 |  18
SEN3 |  Quantum Computing |  “quantum architect*” |  344 |  7
SEN4 |  Quantum Computing |  “quantum design” |  328 |  4
SEN5 |  Computer Science |  “quantum architect*” |  21 |  2
SEN6 |  Computer Science |  “quantum design” |  36 |  1
Total |  |  |  1093 |  43

###  3.3 Data filtration

In our analysis of GitHub and Sack Exchange sites, we found many issues and
posts containing the terms “architecture” and “design” not only in the context
of software architecture and design but also other contexts, including
hardware (e.g., CPU
architecture141414https://github.com/m-labs/artiq/issues/1757, GPU
architecture151515https://github.com/Qiskit/qiskit/issues/325, hardware
design161616https://github.com/qiskit-community/qiskit-metal/issues/814, and
hardware
architecture171717https://quantumcomputing.stackexchange.com/questions/16221),
when describing their concerns in the GitHub issues and Stack Exchange posts.
We also found that both Stack Overflow and Computer Science Stack Exchange
featured numerous posts with the term “quantum” not only in the context of a
quantum software system but also in other contexts (e.g., Firefox
Quantum181818https://stackoverflow.com/questions/36007119, quantum
leap191919https://cs.stackexchange.com/questions/135237). Therefore, we need
to filter the retrieved 1,760 closed issues from 164 quantum software systems
and 975 posts (324 from Stack Overflow, 604 from Quantum Computing, and 47
from Computer Science Stack Exchange) and exclude those posts that are not
related to software architecture decisions in quantum software systems. To do
so, we performed context analysis and applied our defined inclusion and
exclusion criteria (see Table 4) to accurately filter out irrelevant issues
and posts.

Table 4: Inclusion and exclusion criteria to manually identify Stack Exchange
posts and GitHub issues related to software architecture decisions in quantum
software systems Inclusion criteria
---
I1. We include a post or issue if it has to do with both software architecture
decisions and quantum software systems.
Exclusion criteria
E1. We exclude a post or issue if it is only related to quantum software
systems but not related to software architecture decisions.
E2. We exclude a post or issue if it is only related to software architecture
decisions but not related to quantum software systems.

Before the formal filtering (manual inspection), we conducted a pilot
filtering adhering to a set of inclusion and exclusion criteria (see Table 4)
to identify GitHub issues and Stack Exchange posts related to architecture
decisions in quantum software systems. Specifically, the pilot filtering
process is composed of the following steps: (1) The first author randomly
selected 10 issues from all the issues containing the keywords retrieved from
the selected GitHub projects and 10 posts from the search results of Stack
Exchange search terms; (2) The first and third authors manually checked
independently whether the issues and posts should be included by following our
defined criteria (see Table 4); (3) Data labeled by the two authors were
compared, and the level of agreement between the two authors was calculated
using the Cohen’s Kappa coefficient [56]; (4) For any posts and issues that
the two authors disagreed with, the two authors discussed them till an
agreement was reached. The Cohen’s Kappa coefficient before the discussion and
resolution of the disagreements for the pilot data filtering was 0.80 for
GitHub issues and 0.76 for Stack Exchange posts, both higher than 0.7. After
resolving those disagreements through discussions, the two authors reached a
100% agreement, indicating a high degree of consistency between the two
authors.

After the pilot data filtering, the first author carried on the formal data
filtering with all the issues retrieved from GitHub projects and all the posts
retrieved from Stack Exchange sites based on the inclusion and exclusion
criteria. As it was easy to distinguish whether the posts and issues matched
the criteria, the first author completed this step alone. There were only 9
issues and posts that the first author was unsure about, which were then
discussed with the third author. After manually filtering all the candidates,
we finally selected 151 related closed issues from 47 quantum software systems
that related to architecture decisions in quantum software systems. These
projects and their corresponding retrieved and related issues are detailed in
Table 2. Similarly, we retrieved 1,093 Stack Exchange posts and identified 43
Stack Exchange posts that discussed architecture decisions in quantum software
systems. This includes 29 posts from Stack Overflow, 11 posts from Quantum
Computing, and 3 posts from Computer Science sites, as detailed in Table 3.

###  3.4 Data extraction and analysis

1) Data Extraction

We conducted the data extraction process by identifying the relevant
information to be extracted from the 151 GitHub issues and 43 Stack Exchange
to answer our defined RQs (see Table 1). Table 5 outlines the data items
extracted from the related posts and issues with the RQs addressed using the
data items. Following this, we conducted both pilot and formal data
extraction.

Before the formal data extraction, the first author conducted a pilot data
extraction with the second author. The first author randomly selected 10
GitHub issues and 10 Stack Exchange posts. The first two authors independently
extracted relevant information (i.e., Decision description, Decision type,
Application domain, Quality attribute considered, and Limitation and
challenge) as detailed in Table 5. In the event that disagreements arose, a
third author was invited to discuss with the two authors and reached a 100%
agreement. After the pilot data extraction, the first author extracted the
data from the filtered posts and issues independently. During this process,
the first author conferred with the second author to ensure the accuracy of
the extracted data regarding the uncertain portions.

2) Data Analysis

To analyze our extracted data, we employed a predefined classification [28]
and the Constant Comparison method [29]. Linguistic patterns refer to
grammatical rules that enable individuals to communicate effectively in a
shared language [57]. Drawing inspiration from the study of Sorbo et al. [28],
which categorized six linguistic patterns from development emails, we adapted
this classification to categorize linguistic patterns relevant to expressing
architectural decisions, to answer RQ1. For the rest RQs (RQ2, RQ3, RQ4, and
RQ5), we conducted a qualitative analysis using the Constant Comparison method
[29]. This method involves a rigorous examination of different aspects of the
data, including codes and categories, to unveil differences and similarities
in the data [58].

We carried out a pilot data analysis for each RQ before the formal data
analysis. The pilot data analysis encompassed several steps, as follows: (1)
The first and second authors independently labeled the content of filtered
posts and issues, cross-referencing them with the corresponding data items in
Table 5; (2) The first author reviewed and verified the labeling results from
the second author to ensure the accuracy of data extraction; (3) The first
author aggregated all codes into higher-level concepts, subsequently
converting them into distinct categories; (4) To ensure the rigor of our
analysis, the other five authors (the third, fourth, fifth, sixth, and seventh
authors) checked and validated the results from the pilot data analysis. The
disagreements were resolved through meetings, utilizing a negotiated agreement
method [59] to enhance the reliability of the pilot data analysis outcomes. In
the next step, The first author carried on with the formal data analysis and
followed similar steps used during the pilot data analysis. To confirm the
rigor of our analysis, all the authors conducted a thorough review of the
analysis outcomes, resolving any discrepancies through collaborative
discussions with the first author. A summary of the data analysis approaches
is provided in Table 5. We have made all the data labeling and analysis
results available in our replication package [30].

Table 5: Data items to be extracted with their description, analysis approaches, and relevant RQs # |  Data item |  Description |  Data analysis approach |  RQs
---|---|---|---|---
D1 |  Decision description |  Developers-based description of architecture decisions made in the development of quantum software systems |  Predefined classification [28] |  RQ1
D2 |  Decision type |  A classification of architecture decisions taken during the development of quantum software systems |  Constant comparison |  RQ2
D3 |  Application domain |  Application domains in which architecture decisions are made throughout the development of quantum software systems |  Constant comparison |  RQ3
D4 |  Quality attribute considered |  Considered quality attributes when developers of quantum software systems make architecture decisions |  Constant comparison |  RQ4
D5 |  Limitation and challenge |  The obstacles and difficulties posed by quantum software development when making architecture decisions |  Constant comparison |  RQ5

##  4 Results

In this section, we present the results of the five RQs as outlined in Table
1. We first describe our methodology for analyzing instances drawn from GitHub
issues and Stack Exchange discussions, and we then present the study results
of each RQ. The results of the five RQs are visualized in Fig. 3.

Figure 3: Linguistic Patterns, Architecture Decisions, Application Domains,
Quality Attributes, and Limitations&Challenges (results of RQ1 to RQ5)

###  4.1 RQ1. How do developers express architecture decisions made in quantum
software development?

Our approach to answering RQ1 involves multiple stages. First, we collected
691 instances of linguistic patterns (552 from GitHub and 139 from Stack
Exchange sites) of how practitioners articulate architecture decisions during
the development of quantum software systems. One GitHub issue and one Stack
Exchange post may identify multiple linguistic patterns because, within a
single issue or post, practitioners might articulate their architecture
decisions in various ways, using different phrases, terminologies, or
contexts. To classify these decision expressions according to their intended
objectives, we utilized a predefined classification scheme based on the
linguistic patterns proposed by Sorbo et al. [28]. Ultimately, we categorized
the extracted architecture decisions expressions into six types of linguistic
patterns. Fig. 3a presents the six linguistic patterns and Table 6 contains
these linguistic patterns as well as their examples and respective
percentages.

Our analysis revealed that Solution Proposal and Information Giving were the
two most common linguistic patterns, accounting for 38.6% and 29.6% of the
instances, respectively. In contrast, Opinion Asking and Problem Discovery
received relatively less attention from practitioners when conveying
architecture decisions within the context of quantum software systems. A
detailed overview of each of those categories is provided below:

Table 6: Linguistic patterns of architecture decisions in quantum software systems Linguistic Pattern |  Example |  Count |  %
---|---|---|---
Solution Proposal |  One solution (to do it generically) that would involve a somewhat major architectural change is to implement a generic interface to asymmetric encryption then implement KEX in terms of that for the algorithms it applies to. (liboqs issue #50) |  268 |  38.4%
Information Giving |  Currently the hybrid Python/C++ architecture of Psi4 is in an odd spot where Psi4 itself is a C++ program that calls an input file as a Python executable. This circular process will be removed and Psi4 itself will become a Python library capable of being imported just like any other module. (psi4 issue #468) |  206 |  29.8%
Information Seeking |  After we have finished the decomposition and local Covalent has a microservice architecture, what should happen when a user does covalent start? (Covalent issue #186) |  130 |  18.7%
Problem Discovery |  These two software packages are very problematic: Conda is very poorly architected/designed. (ARTIQ issue #1471) |  47 |  6.8%
Feature Request |  Architectures supported in liboqs master branch should be updated (liboqs issue #751) |  39 |  5.6%
Opinion Asking |  Please feel free to suggest changes or alternate designs. (deepchem issue #2059) |  5 |  0.7%

(1) Solution Proposal: Our analysis shows that Solution Proposal is the most
frequently used linguistic pattern. Those solutions are offered as pertinent
information or assistance to fellow developers. This category often
encompasses suggestions, recommendations, and best practices tailored to
support and guide peers within the community. In our analysis, developers
mentioned Solution Proposal in about 38.6% of the linguistic patterns we
collected.

(2) Information Giving: This category relates to developers sharing insights,
facts, or knowledge about architecture decisions. Developers might elucidate
the nuances of a quantum algorithm, describe the interaction between classical
and quantum components, or explain design choices made to optimize quantum
coherence. This category embodies the proactive dissemination of architectural
knowledge, ensuring that team members are aligned and informed about the
architecture decisions made. In our analysis, developers mentioned Information
Giving in about 29.6% of the linguistic patterns we collected.

(3) Information Seeking: Information Seeking revolves around developers
looking to enhance product quality or gain clarity on specific functionalities
within quantum software systems. This might manifest as questions, requests
for clarification, or discussions aiming to understand architecture decisions
better. In our analysis, developers mentioned Information Seeking in about
18.7% of the linguistic patterns we collected.

(4) Problem Discovery: In Problem Discovery linguistic pattern, developers
identify, share, and discuss new problems, bugs, or issues they encounter in
quantum software systems. It highlights the initial stages of problem-solving,
where issues are first recognized and made known to the community. In our
analysis, developers mentioned Problem Discovery in about 6.8% of the
linguistic patterns we collected.

(5) Feature Request: When making decisions about the architecture design of
quantum software systems, developers may ask for new features or enhancements
to already existing libraries and software tools. These may also include
recommendations to improve the functionality, usability, or flexibility of the
systems. In our analysis, developers mentioned Feature Request in about 5.6%
of the linguistic patterns we collected.

(6) Opinion Asking: Opinion Asking involves developers seeking the views,
thoughts, or sentiments of peers concerning architecture decisions in quantum
software systems. It may range from inquiring about the practicality of a
proposed solution to gauging how the community feels about specific
architectural decisions. In our analysis, developers mentioned Opinion Asking
in about 0.7% of the linguistic patterns we collected.

Key Findings of RQ1 Finding 1: Developers represent architecture decisions
with six categories of linguistic patterns, among which Solution Proposal
(38.6%) and Information Giving (29.6%) are the two most frequently employed
categories in quantum software development.

###  4.2 RQ2. What types of architecture decisions are made in quantum
software development?

To answer RQ2, we conducted a comprehensive review of the content pertinent to
architecture decisions in the context of quantum software systems, as gleaned
from the accumulated 283 instances of architecture decisions (228 from GitHub
and 55 from Stack Exchange sites). Note that a single post or issue often
contains multiple architecture decisions because developers often explore
various aspects of a problem, seeking information for comprehensive solutions
or gathering diverse opinions. Additionally, the same architecture decision
can be discussed multiple times within a post or issue due to its significance
or as a common challenge faced by developers, leading to repetitive
exploration for clarity or confirmation. Therefore, if a post or issue
mentions multiple architecture decisions, each decision was counted. However,
repeated mentions of the same architecture decision within a single post or
issue were only counted once. We used the Constant Comparison method [29] to
categorize the architecture decisions pertinent to the development of quantum
software systems. Fig. 3b presents twelve types of architecture decisions and
Table 7 provides an overview these types of architecture decisions made in
quantum software development, along with their respective quantities and
percentages.

As shown in Table 7, the two common types of decisions are Implementation
Decision (25.4%) and Technology Decision (24.4%). The remaining categories of
architecture decisions account for a small percentage of the architecture
decisions (less than 15%). We provide a detailed description of each of those
categories below:

Table 7: Types of architecture decisions in quantum software development and their counts & percentages Decision Type |  Example |  Count |  %
---|---|---|---
Implementation Decision |  We designed and implemented an architectural framework that aims to standardize the interaction of an existing infrastructure with IBM Quantum, leveraging the most used open source technologies. (ecosystem issue #266) |  72 |  25.4%
Technology Decision |  Also, since the technology of partial measurement/non-demolition measurement is under development (I believe in some quantum architectures), may I expect to use this SDK to change when the technology is mature? (amazon-braket-sdk-python issue #158) |  68 |  24.4%
Component Decision |  If we want to be strict about the architecture, the log forwarding component should be a controller. (artiq issue #691) |  39 |  13.8%
Tool Decision |  Unfortunately this is something coming from Amber build system. It compiles cuda source files for all possible architectures that a given toolkit supports. (QUICK issue #218) |  27 |  9.5%
Data Decision |  This seems like part of the discussion about long term architecture, harmonizing dc.data with tf.data and TensorGraph layers with Keras layers. (deepchem issue #1036) |  21 |  7.4%
Architecture Pattern Decision |  If we are willing to provide extensibility (which sounds reasonable in a pluggable ecosystem), we should shift to a more suitable pattern such as the observer or pub-sub pattern (…).Again, the pub/sub or obeserver patterns are examples of decoupled architectures. (Qiskit #825) |  16 |  5.3%
Integration Decision |  I will try integrating quantum layers into existing CNN architectures (QC post #33215) |  14 |  4.9%
Decomposition Decision |  After we have finished the decomposition and local Covalent has a microservice architecture, what should happen when a user does covalent start? (covalent issue #186) |  9 |  3.2%
Behavior Decision |  I would propose not keeping the old behavior, simply to make the refactor easier and avoid code complexity (as well as design decisions). (PennyLane issue #226) |  9 |  3.2%
Service Decision |  The sync method in covalent/_results_manager/results_manager.py needs to be moved and updated to be compatible with the new covalent services architecture. (Covalent issue #370) |  5 |  1.8%
Concurrency Decision |  The architecture implemented at the moment is showing some issues that are non trivial to fix (…).and then carefully design how we want to do concurrency and parallelism. (Qcodes issue #272) |  2 |  0.7%
Deployment Decision |  QPIC is designed to be compatible from Python 2 up to the latest Python 3 (…). Is already fixed in this repo but not deployed to PyPI. Could you update the package in PyPI? (qpic issue #12) |  1 |  0.4%

(1) Implementation Decision: Developers need to make informed decisions about
implementing quantum algorithms and operations within their systems. These
decisions often involve intricate technical considerations, such as selecting
suitable programming languages and quantum computing platforms and optimizing
code for quantum hardware. Implementation Decisions are found in 25.4% of the
architecture decisions we collected.

(2) Technology Decision: Developers frequently find themselves at a
crossroads, needing to decide which particular quantum technologies should be
used due to the rapidly developing aspect of quantum technologies. These
decisions have far-reaching implications for the overall system’s efficiency
and functionality, as selecting the right technology can significantly impact
the system’s performance. We found that Technology Decisions represent 24.4%
of the architecture decisions we collected.

(3) Component Decision: Quantum software systems, like other software systems,
benefit from modular and well-structured designs. Developers often need to
decide how to organize and design system components, considering factors like
quantum modules, quantum layers, and the integration of quantum and classical
components. Component Decision is integral to achieving scalability,
maintainability, and overall system reliability. We identified 32 instances
(13.8%) of Component Decision in our collected architecture decisions.

(4) Tool Decision: Quantum systems utilize a myriad of tools tailored for
quantum-specific tasks. In the process of developing quantum software,
developers commonly have to make an essential decision: choosing the right
toolset or frameworks. This decision deeply influences the system’s overall
performance, fidelity in simulation, ease of debugging, and efficient resource
orchestration. Tool Decisions were found in 9.5% of the architecture decisions
we collected.

(5) Data Decision: Data Decision refers to how quantum data structures and
quantum bits (qubits) are managed, accessed, and manipulated. Given the
inherent differences between classical and quantum data, determining the right
strategies for quantum data storage, retrieval, and processing is essential.
This type of decisions encompass selecting appropriate quantum algorithms,
choosing between quantum states, and handling superposition and entanglement.
Data Decisions were discovered at 7.4% of the architecture decisions we
collected.

(6) Architecture Pattern Decision: Architecture Pattern Decision refers to the
high-level organizational structure and modular design choices developers
adopt when designing quantum software or integrating quantum components within
larger systems. Just as classical systems might leverage patterns like layers
or service orientation, quantum systems require patterns optimized for quantum
mechanics. These could include modular quantum algorithm design, pipeline
structures for quantum operations, or layered approaches separating quantum
and classical components. These patterns shape the workflow, scalability, and
maintainability of quantum systems. In a review of architecture decisions made
in quantum software systems, Architecture Pattern Decisions were discovered in
5.3% of the architecture decisions we collected.

(7) Integration Decision: Integration Decision refers to decisions made
concerning how various software components, modules, or subsystems communicate
and work together within quantum software systems. These decisions can involve
the selection of quantum libraries, ensuring compatibility between classical
and quantum sub-systems, or determining how quantum algorithms interface with
other software functionalities. Integration Decisions were discovered at 4.9%
of the architecture decisions we collected.

(8) Decomposition Decision: Decomposition Decision pertains to the choices
made when breaking down complex quantum algorithms or processes into simpler
and more manageable components. Quantum algorithms, given their intricate
nature, often need to be decomposed into a series of quantum gates or
subroutines. The decision on how to decompose not only affects the efficiency
of the algorithm but also its compatibility with specific quantum hardware.
Such decisions may also involve deciding the granularity of modules or
components, ensuring that the architecture remains modular, scalable, and
maintainable. Decomposition Decisions were identified in 3.2% of the
architecture decisions we collected.

(9) Behavior Decision: Behavior Decision revolves around determining how
quantum components or modules will interact and behave over time. Given that
quantum software systems are inherently probabilistic and are susceptible to
decoherence and external interference, Behavior Decisions often involve trade-
offs between performance, accuracy, and fault tolerance. Behavior Decisions
were discovered at 3.2% of the architecture decisions we collected.

(10) Service Decision: Service Decision pertains to the orchestration of
quantum computational tasks within software frameworks. This involves
decisions related to offering quantum functionalities as modular services or
APIs. Considerations of Service Decision include the encapsulation of specific
quantum algorithms, the design of interfaces for quantum service interaction,
and the efficient management of quantum data structures. The focus is on the
structural organization of quantum services within the software, ensuring
modularity, scalability, and robustness while maintaining the peculiarities of
quantum computation. Service Decisions were identified in 1.8% of the
architecture decisions we collected.

(11) Concurrency Decision: Concurrency Decision is concerned with managing
multiple quantum operations or processes within software frameworks.
Concurrency Decisions must consider the potential simultaneous execution of
quantum tasks. This ensures that the quantum software efficiently utilizes
available quantum resources while ensuring the integrity and fidelity of
quantum computations. Concurrency Decision were discovered at 0.7% of the
architecture decisions we collected.

(12) Deployment Decision: Deployment Decision encapsulates strategies and
choices regarding how and where quantum computing applications will be
executed and utilized. Further, deployment decisions extend to defining how
quantum solutions are made available to end-users or other systems, ensuring
that the deployment aligns with the computational, security, and performance
requirements of quantum applications. Deployment Decisions were found in 0.4%
of the architecture decisions we collected.

Key Findings of RQ2 Finding 2: Implementation Decision and Technology Decision
are the most common decision types when making architecture decisions in
quantum software development. Specifically, Implementation Decisions account
for 25.4%, Technology Decision account for 24.4% of the architecture decisions
we collected.

###  4.3 RQ3. What are the application domains of quantum software in which
architecture decisions are made?

To answer RQ3, we analyzed the application domains where architecture
decisions are made in the context of the development of quantum software
systems. The finding of application domains is presented in Fig. 3c and
summarized in Table 8, which contains 194 instances of application domains in
total (151 from GitHub issues and 43 from Stack Exchange sites). During the
data analysis process, we identified one application domain from each post and
issue because our focus was on the primary application domain in which an
architecture decision was made in quantum software systems, and most of the
time the same domain was mentioned multiple times within a post or an issue,
and we rarely found one post or issue that mentioned multiple domains where
architecture decisions were made.

We identified 16 application domains, in which Quantum Programming Framework
is dominant, accounting for more than 25% of the application domains. Software
Development Tool represents more than 16% of the application domains. The
remaining application domains account for less than 15% of the application
domains. A detailed description of the top nine application domains is
provided below.

Table 8: The domains of quantum software systems in which architecture decisions are made Domain |  Count |  % |  Domain |  Count |  %
---|---|---|---|---|---
Quantum Programming Framework |  49 |  25.3% |  Real-Time System |  7 |  3.6%
Software Development Tool |  32 |  16.5% |  Distributed System |  3 |  1.5%
Scientific Software |  27 |  13.9% |  Embedded System |  2 |  1.0%
Optimization and Simulation |  22 |  11.3% |  Blockchain Technology |  2 |  1.0%
Data Processing |  15 |  7.7% |  Desktop Application |  2 |  1.0%
Quantum Cryptography |  15 |  7.7% |  Web Application |  1 |  0.5%
Deep Learning Framework |  8 |  4.1% |  Mobile Application |  1 |  0.5%
Quantum Compiler |  7 |  3.6% |  Game |  1 |  0.5%

(1) Quantum Programming Framework: The most common domain is revealed to be
the Quantum Programming Framework. This domain holds substantial importance in
quantum software systems, as it forms the foundation for the development of
quantum algorithms and applications. It enables programmers to harness the
power of quantum computation by providing the necessary tools, libraries, and
abstractions, thus facilitating the creation of quantum software solutions.
Quantum Programming Framework was found in 25.3% of the application domains we
collected.

(2) Software Development Tool: Tools are pivotal for designing and building
quantum software systems. These tools, ranging from Integrated Development
Environments (IDEs) to debugging platforms, ensure efficient code development,
testing, and deployment. In a quantum context, these tools are specialized to
handle quantum-specific challenges, like qubit manipulation and entanglement.
Software Development Tool was found at 16.5% of the application domains we
collected.

(3) Scientific Software: Scientific Software in quantum systems deals with
applications tailored for research and complex computations. These tools are
designed to understand and utilize quantum phenomena, aiding scientists in
areas like quantum physics, chemistry, and material science. The architectural
decision here focuses on precision, scalability, and accuracy. Scientific
Software was found at 13.9% of the application domains we collected.

(4) Optimization and Simulation: optimization problems are central to quantum
software systems, aiming to model and improve real-world quantum scenarios. By
simulating quantum environments, developers can test algorithms or explore
quantum behavior, making decisions about system robustness, efficiency, and
accuracy. Optimization and Simulation instances were found at 11.3% of the
application domains we collected.

(5) Data Processing: Data Processing emphasizes handling and analyzing quantum
data, which can be vastly different from classical data. Architecture
decisions here revolve around quantum data structures, efficient algorithms
for quantum data manipulation, and ensuring data integrity. Data Processing
was identified at 7.7% of the application domains we collected.

(6) Quantum Cryptography: This domain is primarily concerned with utilizing
the principles of quantum mechanics to secure communication against any
potential interception. It involves techniques like Quantum Key Distribution
(QKD), where cryptographic keys are securely distributed, and eavesdropping
can be detected. The main architectural decisions in quantum cryptography
revolve around ensuring the security of data during transfer. It is about
securing communication channels in a way that is theoretically immune to any
computational hacks, including those using quantum computers. Quantum
Cryptography was found at 7.7% of the application domains we collected.

(7) Deep Learning Framework: Deep Learning Framework can help facilitate the
fusion of quantum computing and neural networks. Such frameworks allow quantum
systems to process vast amounts of data rapidly, making decisions about
quantum neural architectures, training methods, and data-feeding mechanisms.
Deep Learning Framework was found at 4.1% of the application domains we
collected.

(8) Quantum Compiler: Those specialized compilers translate high-level quantum
programming languages into machine-level instructions for quantum processors.
Decisions in this domain concern optimization techniques, error corrections,
and ensuring the efficient execution of quantum operations. Quantum Compiler
was found at 3.6% of the application domains we collected.

(9) Real-Time System: Real-Time Systems are essential for precision quantum
experiments. For instance, ARTIQ offers high-resolution controls and data
acquisition for quantum information setups (e.g., satellite, clock
applications). The system prioritizes nanosecond timing accuracy and user-
friendly interfaces, ensuring swift experiment execution and result
visualization. Real-Time System was found at 3.6% of the application domains
we collected.

Key Findings of RQ3 Finding 3: Our analysis of application domains of quantum
software in which architecture decisions are made classified a total of 194
instances into 16 categories. Quantum Programming Framework is dominant in
these categories, accounting for 25.3% of the application domains we
collected.

###  4.4 RQ4. What quality attributes are considered when developers make
architecture decisions in quantum software development?

To answer RQ4, we extracted data concerning quality attributes. We collected
239 instances of quality attributes (201 from GitHub Issues and 38 from Stack
Exchange sites). We discovered that a single post or issue mentioned multiple
quality attributes when discussing architecture decisions because quantum
software systems, being inherently complex, require developers to consider
several quality attributes simultaneously to ensure robust and effective
architecture decisions. If we identified the same type of quality attributes
within the same post or issue, we considered them only once. Subsequently, we
categorized these instances into sixteen types of quality attributes, which
are presented in Fig. 3d and detailed in Table 9.

Among these quality attributes, Maintainability appears as the most frequently
considered by practitioners, constituting 19.2% of the total instances.
Furthermore, Performance and Compatibility exhibit similar proportions, both
exceeding 16.3%. Additionally, other quality attributes fall below the 10%
mark, signifying that these quality attributes are the least considered by
practitioners in their architecture decision-making of quantum software
development. A detailed description of the top seven quality attributes is
provided below.

Table 9: The quality attributes considered when making architecture decisions in quantum software development and their counts & percentages Quality Attribute |  Example |  Count |  %
---|---|---|---
Maintainability |  It would mean a more confused internal architecture, which would hurt us in maintainability in the long term. (Qiskit issue #1852) |  46 |  19.2%
Performance |  First the current design is intrinsically bad performance for hot loops because we have to look up the state (even if it doesn’t change). (DFTK.jl issue #313) |  39 |  16.3%
Compatibility |  The “my foundational program can do many things, how do I box its capabilities where the QCArchive architecture views it as separate pieces”. (QCEngine issue #197) |  39 |  16.3%
Complexity |  In order to keep Psi4 up to date with best C++11 practices, reduce complexity (…). Currently the hybrid Python/C++ architecture of Psi4 is in an odd spot where Psi4 itself is a C++ program that calls an input file as a Python executable. (psi4 issue #468) |  23 |  9.6%
Usability |  D-Wave Hybrid is a (Python) framework designed for creating classical-quantum hybrid workflows with an emphasis on ease of use. (dwave-hybrid issue #245) |  17 |  7.1%
Consistency |  Ensure the API is as consistent as possible (both between PyTorch based models. (deepchem issue #2059) |  12 |  5.0%
Flexibility |  I think RMG should be designed to not require changes to the core functionality, but have various inherited classes that allow for flexibility and robustness. (RMG-Py issue #1136) |  12 |  5.0%
Reliability |  MSYS also has straightforward benefits for users, such as speed and reliability. (ARTIQ issue #1471) |  11 |  4.6%
Ease of Implementation |  I think we all agree that caching is a good idea; the question is just how easy it is to implement in the current architecture. (ARTIQ issue #1542) |  10 |  4.2%
Security |  I think it is a very important project for the future of our security: being able to work with these new algorithms. (OpenSSL issue #295) |  10 |  4.2%
Scalability |  Using db will solve the scalability issue and will be able to handle a large number of request. (QRL issue #1545) |  5 |  2.1%
Modularity |  We have a good modularity in the compiler, specifically with respect to the distinctions above. (cuda-quantum issue #85) |  5 |  2.1%
Portability |  dc.data is one of the hackier parts of DeepChem and has caused us a good number of scaling and portability headaches. (deepchem issue #1039) |  4 |  1.7%
Interoperability |  We also get better interoperability with the broader python ML ecosystem. (deepchem issue #/1036) |  3 |  1.3%
Cost and Effort |  Lots of money has been poured already into Conda (Continuum Analytics, NumFocus) and VS (Microsoft) with such a shoddy final result, so those tools seem like a dead-end to me. (ARTIQ issue #1471) |  2 |  0.8%
Availability |  Availability of QML tools such as optimizers and different encoding schemes“ and ”availability of simulators and circuit visualizers. (QC post #33215) |  1 |  0.4%

(1) Maintainability: Maintainability refers to the ease with which quantum
software systems can be modified to correct faults, improve performance, or
adapt to evolving requirements. This quality attribute is crucial given the
rapid evolution of quantum technologies, where software must be structured in
a way that accommodates frequent updates and enhancements. Achieving high
maintainability requires clear, modular architecture and comprehensive
documentation to facilitate future updates, testing, and debugging in the
inherently complex quantum computing landscape. In our dataset, developers
mentioned Maintainability quality attribute in about 19.2% of the quality
attributes we collected.

(2) Performance: Performance plays a pivotal role as it directly influences
the computational power and efficiency of quantum algorithms. Quantum
computers are designed to excel in specific computational tasks, and
optimizing performance ensures that these systems can execute complex quantum
algorithms with minimal execution time, making them viable for practical
applications such as cryptography, optimization, and material science
simulations. In our dataset, developers mentioned Performance quality
attribute in about 16.3% of the quality attributes we collected.

(3) Compatibility: Compatibility ensures that quantum software systems can
interact with classical systems, enabling hybrid quantum-classical computing
approaches. This quality attribute is crucial for practical quantum adoption,
as quantum computers are often used in conjunction with classical systems for
solving complex problems. In our dataset, developers mentioned Compatibility
quality attribute in about 16.3% of the quality attributes we collected.

(4) Complexity: Complexity in quantum software systems refers to the intricacy
and difficulty in ensuring scalability, efficiency, and robust quantum
software design. Balancing these factors demands a profound understanding of
both quantum mechanics and software structures, emphasizing the need for
advanced tools and methodologies to navigate the technology landscape. In our
dataset, developers mentioned Complexity quality attribute in about 9.6% of
the quality attributes we collected.

5) Usability: Usability is a key consideration in quantum software systems to
facilitate the effective utilization of quantum resources by developers and
end-users. User-friendly interfaces, tools, and documentation are essential
for enabling practitioners to harness the power of quantum computing without
requiring an in-depth understanding of quantum mechanics. In our dataset,
developers mentioned Usability quality attribute in about 7.1% of the quality
attributes we collected.

(6) Consistency: Consistency plays an important role in quantum software
systems, ensuring that quantum computations produce reliable and reproducible
results. Achieving consistent outcomes is essential in quantum computing, as
it guarantees the correctness of quantum algorithms and maintains data
integrity throughout complex quantum operations. Quantum systems must exhibit
a high degree of internal and external consistency to meet the rigorous
demands of quantum computing tasks. In our dataset, developers mentioned
Consistency quality attribute in about 5.0% of the quality attributes we
collected.

(7) Flexibility: Flexibility in quantum software systems ensures that the
systems can adapt to varied computational tasks, accommodating different
quantum algorithms and protocols. It is vital for catering to a broad spectrum
of quantum applications. In our dataset, developers mentioned Flexibility
quality attribute in about 5.0% of the quality attributes we collected.

Key Findings of RQ4 Finding 4: We identified 16 categories of quality
attributes that are considered by developers when making architectural
decisions in quantum software systems, with Maintainability (19.2%),
Performance (16.3%), and Compatibility (16.3%) being the most considered
attributes by developers.

###  4.5 RQ5. What are the limitations and challenges of making architecture
decisions in quantum software development?

To answer RQ5, we collected and analyzed a total of 205 instances of
limitations and challenges (170 from GitHub issues and 35 from Stack Exchange
posts). During this process, we found that one GitHub issue and Stack Exchange
post may discuss multiple limitations and challenges related to architecture
decision-making in quantum software development. If we found the same
limitations and challenges within the same post or issue, we counted them only
once. We finally identified fifteen types of limitations and challenges that
developers encounter when making architectural decisions in quantum software
development, as presented in Fig. 3e and detailed in Table 10.

We identified two main types of limitations and challenges: Design Issues and
Performance Issues which practitioners encountered in 25.6% and 12.1% of the
limitations and challenges we collected. Other types were mentioned in less
than 10% of the limitations and challenges we collected. A detailed
description of the top eight limitations and challenges is provided below.

Table 10: Limitations and challenges of making architecture decisions in quantum software development, their counts & percentages Limitations & Challenges |  Example |  Count |  %
---|---|---|---
Design Issue |  A CurveManager SoftwareModule replacing pyinstruments would definitely fit very nicely in this modular architecture. In this case, there are a lot of design mistakes that we could avoid with our experience of pyinstruments. (PyRPL issue #84) |  53 |  25.6%
Performance Issue |  The architecture implemented at the moment is showing some issues that are non trivial to fix: pickling, monitoring, performance, in general seems to hard to grasp for the users intermittent test failures due to syncing issues. (QCoDeS issue #272) |  25 |  12.1%
Code Issue |  Just reproducing the first issue with code that is fully executable: (..). I think the issue here is purely to do with labelling. When specifying subsystem_list=[1], DynamicsBackend is only aware that it has a system labelled by index 1 (and it will internally automatically build some measurement definitions based on this labelling). (qiskit-dynamics issue #235) |  16 |  7.7%
Implementation Issue |  Currently the API for KEX makes it a bit difficult to implement a KEM transform like Fujisaki-Okamoto generically. One solution (to do it generically) that would involve a somewhat major architectural change is to implement a generic interface. (liboqs issue #50) |  16 |  7.7%
Compatibility Issue |  At the same time, we need to plan a transition that doesn’t entirely break backwards compatibility. (deepchem issue #1039) |  15 |  7.2%
Complexity Issue |  While this architecture is fine for the first ARTIQ systems that were rather small and simple and even for single-crate Metlino/Sayma systems, it shows limitations on more complex ones. (ARTIQ issue #778) |  15 |  7.2%
Data Issue |  Problem is my data consists of 2 arrays consisting of Double numbers representing the real and imaginary parts of complex numbers. (SO post #55763054) |  12 |  5.8%
Standardization Issue |  But until/unless NIST gives a standardized version of that, we plan to stick with just the KEM formulation. (qmcpack issue #649) |  12 |  5.8%
Maintenance Issue |  Supporting it would mean a more confused internal architecture, which would hurt us in maintainability in the long term; (Qiskit issue #1852) |  8 |  3.9%
Bugs |  Bugs: there are a variety of bugs in the current compiler which have been known about for a long time. (ARTIQ issue #1542) |  7 |  3.4%
Security Issue |  The pseudorandom number generator used by random.random(), Mersenne Twister, is not suitable for cryptography or information security. (SO post #63609099) |  7 |  3.4%
High Error Rates |  The build systems has also only seen compilation on half a dozen architectures and setups, you may encounter errors in the build process. (psi4 #468) |  7 |  3.4%
Usability Issue |  To be honest the BackendConfiguration class is not the most ergonomic to use. (QC post #28416) |  6 |  2.9%
Integration Issue |  Integration with Keras: Although they’re very similar, TensorGraph layers have certain features that it isn’t obvious to me how to reproduce with Keras layers. (deepchem issue #1038) |  5 |  2.4%
Lack of Effort |  A new interface would provide the most benefit here, but it would also require the most effort, both in development and long-term maintenance. (qsim issue #293) |  3 |  1.4%

(1) Design Issue: Design Issues are key challenges that developers encounter
most when making architecture decisions in quantum software development. These
challenges arise when developing quantum algorithms, optimizing quantum
circuits, and seamlessly integrating them with components. Decision-making on
design issues is particularly challenging due to the evolving standards and
tools in quantum computing, which require learning and adaptation. To address
Design Issues, it is important for developers to first learn about quantum
principles and use quantum algorithms. Developers also need to consider the
specific hardware and software stack being used. In our dataset, developers
discussed Design Issues in about 25.6% of the limitations and challenges we
collected.

(2) Performance Issue: Performance Issues in quantum software arise when
applications do not fully leverage quantum speedups, often due to inefficient
algorithms or sub-optimal resource utilization. Developers face these
challenges when making architectural decisions because identifying and using
quantum advantages requires a deep understanding of both quantum mechanics and
the problem domain. To prevent performance Issues, developers need to focus on
algorithm optimization specific to quantum processing, ensuring that quantum
resources are effectively utilized to offer a true computational advantage. In
our dataset, developers discussed Performance Issues in about 12.1% of the
limitations and challenges we collected.

(3) Code Issue: Code Issues in quantum software involve problems like
unreadable code, lack of proper documentation, and non-reusability, impeding
the functionality or efficiency of applications. This type of issues are
prevalent due to the developing stage of quantum programming languages and the
complex nature of quantum computations. To mitigate code issues, developers
should adhere to best coding practices, including thorough documentation, code
reviews, and leveraging classical coding standards where applicable. In our
dataset, developers discussed Code Issues in about 7.7% of the limitations and
challenges we collected.

(4) Implementation Issue: Implementation Issues in quantum software
development primarily stem from the intricate nature of quantum algorithms and
the complexities inherent to nascent technologies, as illustrated by
challenges in integrating quantum algorithms with existing classical computing
systems. Specifically, developers often encounter difficulties when trying to
generalize functions or leverage advanced features due to hardware-software
co-optimization, and practical and robust frameworks that can seamlessly
integrate with various quantum algorithms and protocols. These challenges are
magnified by the high complexity of quantum computations, nuances of post-
quantum cryptography, and the infancy of quantum programming languages. To
prevent these issues, developers should focus on fostering adaptive software
designs, ensuring rigorous testing and validation of newer features, deepening
collaboration between quantum software communities, and enhancing the
flexibility and documentation of quantum development tools and APIs. In our
dataset, developers discussed Implementation Issues in about 7.7% of the
limitations and challenges we collected.

(5) Compatibility Issue: Compatibility Issues in quantum software development
are primarily related to the challenges of creating compatible interfaces
between quantum and classical systems. These difficulties are compounded by
the absence of universally accepted quantum programming frameworks and the
significant obstacles in integrating quantum technology with existing IT
infrastructures. Together, these factors pose substantial challenges to the
advancement and application of quantum computing in current technological
environments. This incompatibility extends to difficulties in data formats,
communication protocols, and system interoperations, which underscore an
essential need for enhanced functionalities across different layers of the
software stack as well as improved interfaces between quantum and classical
computing environments. In our dataset, developers discussed Compatibility
Issues in about 7.2% of the limitations and challenges we collected.

(6) Complexity Issue: Quantum software inherently presents Complexity Issues
due to the multi-faceted nature of quantum mechanics. Developers wrestle with
this type of issues when making architecture decisions, as understanding and
managing the intricacies of superposition and entanglement can be daunting.
This complexity makes designing efficient quantum algorithms challenging. To
alleviate Complexity Issues, a deep dive into quantum principles, alongside
leveraging abstraction layers and tools that simplify quantum processes, is
indispensable. In our dataset, developers discussed Complexity Issues in about
7.2% of the limitations and challenges we collected.

(7) Data Issue: Data Issues refer to challenges in data encoding, storage, and
retrieval using quantum bits in quantum computing. Developers encounter this
type of issues when making architecture decisions because the intricate
process of encoding classical information into qubits is influenced by the
nature and encoding type of data, which can significantly impact the the
performance of algorithms. To counteract Data Issues, developers should choose
the appropriate qubit data encoding patterns, understand quantum data
operations, and utilize appropriate quantum data structures. In our dataset,
developers discussed Data Issues in about 5.8% of the limitations and
challenges we collected.

(8) Standardization Issue: Standardization Issues affect quantum software
development, as the field suffers from a lack of standardized tools and
frameworks. Encouraging collaborative efforts towards standardizing quantum
computing protocols and interfaces, and contributing to community-driven
quantum software libraries can help establish much-needed uniformity. In our
dataset, developers discussed Standardization Issues in about 5.8% of the
limitations and challenges we collected.

Key Findings of RQ5 Finding 5: Design Issues (25.6%) and Performance Issues
(12.1%) are the most common challenges in architecture decision-making in the
development of quantum software systems. Other notable issues include Code
Issues (7.7%)Implementation Issues (7.7%), Compatibility Issues (7.2%), and
Complexity Issues (7.2%).

##  5 Discussion

In this section, we revisit the findings of this study by interpreting the
results and discussing their implications.

###  5.1 Interpretation of results

####  5.1.1 RQ1: How do developers express architecture decisions made in
quantum software development?

Table 6 presents the linguistic patterns identified in the architecture
decisions made during the development of quantum software systems. These
patterns serve as a means to understand the diverse intentions behind decision
expressions, shedding light on the reasoning process that drive the architects
and developers of quantum software systems. Among these linguistic patterns,
two arise as particularly dominant: Solution Proposal and Information Giving.
The frequent utilization of these two patterns underscores their significance
in the context of quantum technology driven architecture decisions. Solution
Proposal, as the name suggests, plays a pivotal role in suggesting potential
solutions as architecture decisions. Given the multifaceted nature of quantum
technologies, any given architectural challenge may have a multitude of viable
solutions. In such cases, practitioners lean on the Solution Proposal pattern
to put forth various potential solutions, thereby enriching the decision-
making process. Information Giving, on the other hand, plays a critical role
in the landscape of quantum software development. When architects and
developers grapple with complex quantum software development challenges, they
often need to convey pertinent information to elucidate the intricacies of
their decision-making. This is where Information Giving comes into play,
enabling professionals to share the necessary context and insights required
for effective architecture decision-making in the quantum computing domain. In
essence, the quantum technology landscape is marked by its dynamism and
diversity, which inherently gives rise to a multitude of potential solutions
for architectural problems. Hence, Solution Proposal acts as a common
linguistic pattern for quantum practitioners to explore and propose
alternative pathways, and Information Giving provides an essential bridge for
communication and understanding of technical details, facilitating the
exchange of vital information necessary for sound architecture decision-making
within the intricate field of quantum software development.

####  5.1.2 RQ2. What types of architecture decisions are made in quantum
software development?

Our investigation indicates the presence of twelve types of architecture
decisions in the context of quantum software development, as shown in Table 7.
Among them, Implementation Decision is one of the most commonly encountered
decisions, which emphasize the significance of carefully defining
implementation details in quantum software systems. Technology Decision is
another common architecture decisions, which illustrates the critical
importance of selecting proper quantum technologies while developing quantum
software systems. Furthermore, Component Decision appears as a recurring
option, emphasizing the requirement for developers to create modular and
layered architectures, a consideration shared by diverse system kinds.
Notably, Tool Decision is crucial for choosing optimal quantum tools and other
tools that streamline the development process. Data Decisions are also
noteworthy, illustrating the importance of data storage and processing in the
context of quantum software systems, given the significant and dynamic nature
of data created during quantum computing tasks. Architecture Pattern Decision
guides the selection of a suitable architectural pattern, ensuring the
specific quality attributes of quantum systems, e.g., maintainability and
performance. Integration Decision focuses on melding quantum components with
traditional systems for smooth data and process interactions. Other
architecture decisions within quantum systems encompass Decomposition
Decision, Behavior Decision, Service Decision, Concurrency Decision, and
Deployment Decision. Collectively, these decisions underscore the multifaceted
considerations that archiects and developers need to address while designing
quantum software systems.

####  5.1.3 RQ3. What are the application domains of quantum software in which
architecture decisions are made?

As illustrated in Table 8, architecture decisions in quantum software
development are made in a total of sixteen application domains, in which
Quantum Programming Framework is dominant. The prominence of Quantum
Programming Framework is attributed to the increasing importance and adoption
of quantum computing in solving complex problems beyond the capability of
classical computers. A substantial portion of quantum software necessitates
the inclusion of specific quantum-related modules, such as quantum gates or
quantum circuits. With the rapidly growing interest in quantum computing, it
is logical that Quantum Programming Framework stands out as the primary domain
where architecture decisions are made in the context of quantum software
development. When architects and developers make decisions in systems that
utilize quantum technologies, they most frequently grapple with Technology
Decisions. This trend can be attributed to the integral role that quantum-
specific frameworks play in building quantum software systems. These
frameworks often set the foundation for implementing quantum algorithms,
managing quantum states, or interfacing with quantum hardware. The interaction
between quantum-related modules and traditional computing components within
these frameworks consequently gives rise to a myriad of architecture design
issues centered around the frameworks themselves. In addition to Quantum
Programming Framework, other significant domains in quantum software systems
include Software Development Tool tailored for quantum computing, Scientific
Software harnessing the power of quantum mechanics, and Optimization and
Simulation solutions that leverage principles of quantum mechanics. Quantum
phenomena also play a pivotal role in domains such as Quantum Cryptography,
offering unmatched security features. Likewise, the synergy between quantum
mechanics and artificial intelligence is evident in the Deep Learning
Framework domain, where quantum-enhanced neural networks are under
exploration. Quantum technology is causing significant changes (e.g.,
optimized quantum algorithms, enhanced computational power, and increased data
processing capabilities) in domains like Quantum Compiler, Real-Time System,
Distributed System, and Embedded System. Moreover, Blockchain Technology is
witnessing innovative approaches influenced by quantum mechanics, ensuring
robust and tamper-proof systems. Traditional Desktop Applications, Web
Applications, and Mobile Applications are also experiencing a transformative
phase, with quantum algorithms enhancing their efficiency and capabilities.
Finally, Games in quantum software systems are still in its nascent stages,
but hold the promise of offering incomparable gaming experiences, tapping into
the probabilistic nature of quantum mechanics.

####  5.1.4 RQ4. What quality attributes are considered when developers make
architecture decisions in quantum software development?

As shown in Table 9, we identified sixteen types of quality attributes
considered by architects when they make architecture decisions in quantum
software development. In our study, two prominent quality attributes,
Maintainability and Performance were mostly discussed in quantum software
development. Maintainability was mentioned 46 times, constituting over 19% of
the identified quality attributes, while Performance was found 39 times,
accounting for approximately 16% of the total quality attributes.
Maintainability involves ensuring that the system remains easy to manage and
evolves over time. Maintainability plays an essential role in quantum software
systems since it ensures that quantum software systems can evolve as
technology advances or requirements change. As one developer proposed “I think
the node should be kept minimal and specific for a lot of good architectural
reasons. (…) More non-core features, more on/off settings => more complexity,
less maintainability, higher risk” (QRL issue #1545). Performance is another
vital quality attribute for quantum software systems, since Performance is
crucial for long-term viability and adaptability. Performance is primarily
associated with time behavior, resource utilization, and system capacity.
Practitioners in quantum software development are particularly concerned about
factors such as training and computing time, as well as the speed of data
processing. These aspects are crucial for optimizing the performance of
quantum software systems, and suggestions include the adoption of better-
distributed architectures to enhance these performance aspects [60], as
mentioned in a GitHub issue “First the current design is intrinsically bad
performance for hot loops because we have to look up the state (even if it
doesn’t change)” (DFTK.jl issue #313).

####  5.1.5 RQ5. What are the limitations and challenges of making
architecture decisions in quantum software development?

As shown in Table 10, developers encounter various types of limitations and
challenges during architecture decision-making of quantum software
development, in which Performance Issues and Design Issues are the main types.
Design Issues frequently arise in quantum software development due to e.g.,
complex quantum technology, quantum software development tools, and quantum
programming frameworks. Quantum software not only requires a redesign of
applications, but also mandates a reconceptualization of the tools and
languages used for quantum software development. Furthermore, from a coding
perspective, there are hurdles to overcome to maintain clarity and scalability
in the quantum context, as highlighted in an issue that states, “This issue is
meant to highlight some design issues from the coding point of view that will
make life hard for current and future coders of RMG-Py” (RMG-Py issue #521).
Performance Issues, on the other hand, are tightly intertwined with the
intrinsic characteristics of quantum systems. Given the delicate nature of
quantum states and the probabilistic outcomes they produce, ensuring
consistent and efficient performance is a challenge. For instance, a developer
noted: “The current design is an intrinsically bad performance for hot loops
because we have to look up the state (even if it doesn’t change)” (DFTK.jl
issue #313). This example touches upon the nuances of state management in
quantum software systems, where even seemingly minor operations can entail
significant overheads. In another context, as quantum algorithms are
integrated into larger systems, unpredictable performance hiccups can occur,
as evidenced by the feedback from one developer that “But the performance got
slightly worse (higher variance) or the model crashed due to NaNs” (deepqmc
issue #25).

###  5.2 Implications

####  5.2.1 Linguistic patterns

Feature Request, Problem Discovery, and Opinion Asking can be considered by
architects and developers in describing their architecture decisions in
quantum software development: Quantum software development is a cutting-edge
field in modern computing, which brings many uncertainties and challenges.
Practitioners use six types of linguistic patterns to communicate with each
other when making architecture decisions in quantum software development.
Among these linguistic patterns, Feature Request, Problem Discovery, and
Opinion Asking were found to be less frequently used for expressing
architecture decisions during the development of quantum software. Feature
Request typically outlines a new functionality or enhancement being added to
the system. The given example, “Pull requests implementing new features:
Datalayer type creation, conversion and dispatch #1338 implementing the data-
layer creation, conversion and dispatcher routines” (QUTIP issues #1278),
specifies a technical enhancement, indicating that pull requests have been
made for new data layer functionalities, which are crucial for the data
handling capabilities of software systems. Furthermore, Problem Discovery is
about identifying and describing issues or limitations within the current
architecture. The example “SIKE not properly disabled on unsupported
architectures” (liboqs issue #1128) highlights a specific issue where a
cryptography algorithm SIKE is not being correctly disabled in environments
where SIKE is not supported, potentially leading to errors or security
vulnerabilities. Lastly, Opinion Asking is a linguistic pattern where
developers seek feedback or alternative perspectives on an architecture
decision, promoting collaborative decision making. The example “Please feel
free to suggest changes or alternate designs” (deepchem issue #2059) opens the
floor for community input on the project, which can lead to innovative
solutions and shared ownership of the project’s evolution. With Opinion
Asking, practitioners can gather a wealth of insights, refine their approach,
and avoid potential pitfalls. While the utilization of these three linguistic
patterns remains under-explored, they hold immense potential in aiding
practitioners during their decision-making processes. We recommend the
architects and developers to use these three linguistic patterns to express
architecture decision during architecture decision-making of quantum software
development.

####  5.2.2 Architecture decisions

Making Implementation Decision and Technology Decisions becomes challenging
for practitioners in quantum software systems due to the integration of
quantum-related components: Making Implementation Decisions and Technology
Decisions in quantum computing systems arises as a daunting effort for
practitioners, primarily stemming from the integration of quantum-related
components. This challenge is accentuated by the nascent nature of quantum
technology and the associated intricacies. Implementation Decision stands as a
cornerstone of building any system, quantum software systems or otherwise. For
quantum software systems, the architecture decision-making process becomes
intricately complex. Practitioners often find themselves at a crossroad,
deliberating on the most suitable means to incorporate quantum-related
components. For instance, determining the feasibility of introducing a
feature, like caching, within a given architecture poses questions related to
effort, compatibility, and user comprehension, as evidenced in this example
“The C++ API of LLVM is complicated and has poor forward compatibility (…). I
think we all agree that caching is a good idea; the question is just how easy
it is to implement in the current architecture (in terms of implementation
effort, as well as ease of understanding for users)” (ARTIQ issue #1542). The
maneuverability of quantum elements, such as qubits, further introduces a
dimension of complexity. As delineated in the example “In addition there are
architectures in which moving a qubit around is a useful concept and one good
way to implement this is to consider move swaps operations as a separate type
of operation” (Cirq issue #3240), the movement of a qubit can necessitate new
operations. Immediate needs, like the one outlined in (OpenQL issue #220) “The
shuttle operation is already introduced in cQASM 2.0. But I would like to have
it already implemented, since I need it for my architecture. Also adding the
operation would not brake any version”, further strain practitioners to
provide a timely solution without jeopardizing existing system versions.
Technology Decision, on the other hand, delves into the selection and
integration of appropriate technologies that cater to the needs of quantum
software systems. The rapid evolution of quantum technologies and tools
bewilders many practitioners. A case in point is the quandary expressed in
(ARTIQ issue #1370) “What’s the latest in Python LLVM bindings anyway is there
one that supports the (C++) IRBuilder interface?”, where the binding specifics
for a language (Python) with a compiler (LLVM) is in question. The evolving
landscape of quantum programming frameworks, as seen in (Qiskit issue #542)
“QISKit into different components or perhaps we can foresee adding other
things rather than (but related to) the SDK”, adds another layer of decision-
making complexity. Furthermore, understanding and utilizing back-end
configurations and interfaces, like the one highlighted in (QC post #28416)
“As for how to use the BackendConfiguration.from_dict() constructor method,
it’s basically designed to take in the JSON parsed output from the IBM quantum
api response payload”, demands a deep dive into often convoluted documentation
and protocols. Practitioners were faced with significant challenges when
making Implementation Decisions and Technology Decisions during the
development of quantum software, particularly due to the integration of
quantum-related components and the intricate nature of quantum technology,
which often leads to critical issues and decisions regarding software
architecture and selecting suitable quantum technology, and consequently we
suggest that practitioners should careful consider Implementation Decisions
and Technology Decisions in quantum software development.

####  5.2.3 Application domains

Practitioners should focus on seamless integration of quantum programming
frameworks, the maintenance of module design, tool and interface compatibility
when making architecture decisions in developing quantum software systems:
Quantum Programming Framework appears as a major domain where important design
considerations are essential in the development of quantum software systems.
When engaging with Quantum Programming Framework, developers should prioritize
the following tasks: ensuring the compatibility of new modules, refining
integration procedures, maintaining a cohesive framework structure, and
staying adaptive to the rapidly evolving nature of quantum computing
paradigms. These focal points will play a crucial role in the successful
deployment and scalability of quantum software systems while ensuring the
robustness, operational efficiency, and versatility of quantum programming
frameworks (see the interpretation of RQ3 results in Section 5.1.3). Given the
popularity and importance of Quantum Programming Frameworks in quantum
software development, there is an imperative need for practitioners to accord
heightened attention to several aspects. Specifically, practitioners should
turn their attention more toward module design when navigating architecture
decisions in this burgeoning field. It is crucial that Quantum Programming
Frameworks and their related quantum software systems are well-integrated.
This coherent integration ensures seamless operations, from compilation to
execution, reflected in our data such as “While the actual operation requested
is technically possible without going to a DAG, it doesn’t fit into the
compilation tooling architecture of Terra” (Qiskit issue #1852). Another
pivotal area deserving increased attention is implementation of Quantum
Programming Frameworks. As quantum computing progresses, so does the need for
Quantum Programming Framework to evolve, aligning with new requirements and
tools. Proposals like “This proposes an implementation of the new framework of
the pulse drawer” (Qiskit issue #4643) underscore the ongoing evolution of
Quantum Programming Frameworks, demanding practitioners to remain agile and
receptive to such transformative changes. Furthermore, maintaining consistency
between the evolving features of Quantum Programming Frameworks and their
overarching design philosophies is crucial. For instance, “I think that’s
consistent with the rest of Cirq’s design” (Cirq issue #1060) reinforces the
importance of a cohesive and consistent framework (Cirq) design, ensuring
smooth interfacing and operability of quantum software systems. Lastly, the
continuous evolution of Quantum Programming Frameworks is exemplified by the
focused efforts to enhance their capabilities, particularly in terms of
control-flow mechanisms. As evidenced by statements like “This issue shall
serve to keep track of ideas, discussions, and updates regarding the design
and implementation of better constructs to describe control-flow feedback in
both cQASM and the OpenQL API Interface” (OpenQL issue #384), it is clear that
developers are actively working on refining the Quantum Programming Framework
cQASM. This example from cQASM and OpenQL highlights the ongoing efforts of
maintaining control-flow feedback in Quantum Programming Frameworks, ensuring
that Quantum Programming Frameworks not only meet current computational
demands but are also well-equipped for the intricacies of future quantum
computing challenges. Therefore, we recommend that practitioners pay more
focus on the seamless integration of Quantum Programming Frameworks and the
maintenance of module design, tool and interface compatibility, as these
elements are fundamental to the architecture of quantum software systems.

####  5.2.4 Quality attributes

The quality attributes considered when making architecture decisions in
quantum software development offer valuable insights for both researchers and
practitioners: The quality attributes delve into the crucial aspects of
software architecture in quantum software development. Understanding these
quality attributes is essential for developing efficient and maintainable
quantum software systems, guiding both the research agenda and practical
approaches in quantum software development. Therefore, there is a need to
provide guidelines for both researcher and practitioners. Researchers can
explore the ways in which quality attributes are considered when formulating
architectural decisions in quantum software development. Researchers can also
use this knowledge to focus on the identified quality attributes (e.g.,
Maintainability, Performance, and Compatibility), tailoring their theoretical
frameworks and empirical studies to address the specific challenges and needs
in quantum software architecture. Recognition of the importance of quality
attributes in quantum software development can be directed towards
constructing methodologies and tools that enhance these quality attributes in
quantum software systems. This focus is crucial, given the inherent
complexities of quantum software and the need for integrating quality
attributes in quantum software with classical software paradigms.
Practitioners can utilize our findings of quality attributes to make well-
informed architecture decisions during the development of quantum software.
The emphasis on Maintainability, Performance, and Compatibility as key quality
attributes indicates the areas where practitioners should focus their efforts.
This knowledge assists in creating more robust and efficient quantum software
systems, directly impacting the practical application of quantum technologies.
Furthermore, understanding these quality attributes can aid in making valid
architecture decisions, ensuring that quantum software not only meets
technological demands but is also maintainable and compatible to future
advancements and requirements.

####  5.2.5 Limitations and challenges

Practitioners should be aware of the limitations and challenges that are
critical for making suitable architecture decisions in quantum software
development: During architecture decision-making in the development of quantum
software, practitioner face an array of limitations and challenges (e.g.,
Design Issues). As an instance, “The authors do lay out some of the challenges
faced when designing quantum computing design tools and programming languages”
(CS post #29032), marks the exploration of practitioners into the field of
quantum software, focusing on the technical and conceptual challenges in
designing effective quantum programming languages and design tools. Poor
design decisions in quantum software can lead to complex, inefficient, or
unreliable outcomes. For instance, “VibrationalOp hard-codes a rather odd and
confusing case here because of a poor design choice in the
VibrationalIntegrals taken here” (qiskit-nature #1032), indicates how poor
design decisions in the early stages can significantly impact the clarity and
functionality of the resulting code or system. To address these challenges and
limitations, practitioners need a high learning curve to study on design
decisions and to choose the suitable technology. For instance, one developer
mentioned, “We can resolve this issue with the conclusion ‘By Design’”
(qsharp-runtime issue #587), marks the significance of thoughtful design
decisions in overcoming challenges during the development of quantum software.
On the other hand, if practitioners are unaware of these limitations and
challenges or and unable to address them appropriately, the limitations may
exacerbate during the development of quantum systems, potentially leading to a
failed quantum software system. As one developer mentioned, “The absence of a
design document enumerating realistic use cases hides the design goals and
blurs the definition of done” (Qiskit issue #825), indicates the critical need
for comprehensive design documentation to avoid misalignment of implemented
quantum software systems. The importance of making informed architecture
decisions during the development of quantum software cannot be ignored;
practitioners awareness and properly handling of the inherent limitations and
challenges not only made up the way for successful outcomes but also prevent
them from the failures of quantum software systems.

##  6 Threats to validity

###  6.1 Construct validity

Construct validity refers to whether our measurement tools accurately assess
the intended properties [61]. In this study, we encountered two potential
challenges to construct validity: search term selection and manual analysis.
Search term selection: The retrieval of related issues from GitHub and related
posts from Stack Exchange sites (including platforms such as Stack Overflow,
Quantum Computing Stack Exchange, and Computer Science Stack Exchange) relies
on keyword-based searches. When it came to selecting search keywords for
finding GitHub issues and Stack Exchange posts, we faced challenges related to
the construct validity of our data collection. One significant concern was
whether the keywords we initially chose would capture all the relevant issues
and posts or if some valuable information might be missed. To address this
potential threat, we took a proactive approach by conducting pilot searches
using alternative search terms before using the original terms discussed in
Section 3.2. This iterative process allowed us to refine our keyword choices,
ensuring that we cast a wider net and minimize the risk of incomplete data,
thus safeguarding the construct validity of our study. Manual data handling:
The stages of data handling in our study, which involve manual processes and
thus risk subjectivity and bias, are as follows: (1) Labeling: To minimize
bias in this stage, we implemented a two-step approach. Initially, we
performed a pilot labeling on both GitHub issues and Stack Exchange posts.
This was done prior to the formal labeling to identify potential biases and
inconsistencies. Furthermore, we established the inclusion and exclusion
criteria to objectively decide which posts or issues should be included in our
study; (2) Extraction: The pilot data extraction process was a collaborative
effort primarily between the first and second authors. To ensure accuracy and
consistency, the first author conducted a thorough recheck of the extraction
results. This step was crucial to maintain the integrity of the data
extraction process. If any discrepancies were found, the third author
participated in discussions with the first two authors to help reach a
consensus; (3) Analysis: To mitigate the threat of data analysis, we used two
qualitative techniques (predefined classification and the Constant Comparison
method) to analyze the extracted data and answer the RQs. Moreover, we
attempted to minimize this threat by performing a pilot data analysis before
the formal formal data analysis. The first and second authors independently
categorized content from filtered posts and issues, cross-referencing the
filtered posts and issues with the corresponding data items in Table 5. The
first author then consolidated these codes into higher-level concepts and
categories, while the other authors reviewed and validated the pilot data
analysis results, resolving disagreements through the negotiated agreement
method [59] to enhance reliability.

###  6.2 External validity

External validity encompasses the extent to which the outcomes of a study can
be applied and verified in diverse settings, thereby assessing the
generalization of the findings. One primary factor influencing external
validity is the selection of data sources. In our effort to mitigate this
potential limitation, we deliberately opted for two widely recognized
developer communities, namely GitHub and Stack Exchange as the primary sources
for extracting architecture decisions in the realm of quantum software
development. It is important to note that GitHub and Stack Exchange have
consistently served as pivotal resources in empirical software engineering
investigations. Previous research endeavors (e.g., [62] and [63]) have
demonstrated that insights obtained from GitHub and Stack Exchange data enjoy
a high degree of validation and resonance among practitioners. This strong
alignment between research findings and real-world experiences bolsters our
confidence in the representatives of these platforms for studying architecture
decisions in quantum software development.

###  6.3 Reliability

Reliability, in the context of our research methodology, pertains to the
degree of consistency exhibited by the chosen approach in yielding results. In
order to mitigate the potential threat to reliability, a preliminary data
labeling exercise was carried out independently by two authors on a sample
consisting of 10 GitHub issues and 10 Stack Exchange posts. The assessment of
inter-rater reliability was conducted through the calculation of Cohen’s Kappa
coefficient, yielding values of 0.80 for GitHub issues and 0.76 for Stack
Exchange posts. Both of these coefficients exceed 0.7, indicating a
satisfactory level of agreement between the participating authors. To further
enhance the reliability of the process encompassing data labeling, data
extraction, and data analysis, any disparities or inconsistencies that arose
were resolved through constructive discussions involving the first author and
other authors. To foster transparency and enable validation of our study
findings, we have made the entire dataset, along with the corresponding data
labeling outcomes for GitHub issues and Stack Exchange posts, readily
available [30]. This provision ensures that other researchers can scrutinize
our work and verify the consistency and reliability of our data handling and
analysis procedures.

##  7 Related work

###  7.1 Quantum software development

In recent years, quantum software development has gained significant attention
in the software engineering community, reflecting the growing importance of
quantum computing. Khan et al. [18] conducted a systematic review to examine
the software architecture for quantum computing systems. Their work provides a
comprehensive overview of existing architectures of quantum software systems,
highlighting the advancements and challenges in this domain. Truger et al.
[64] explored the intersection of warm-starting and quantum computing through
a systematic mapping study, shedding light on how quantum computing can
benefit from optimizing initialization procedures. Haghparast et al. [65] took
a developer’s perspective to identify the challenges in quantum software
engineering and mapped these challenges to a proposed workflow model, offering
insights into the practical issues developers face in this field. Yue et al.
[27] examined the challenges and opportunities in quantum software
architecture, highlighting key considerations in designing quantum computing
software. Felderer et al. [66] explored software engineering challenges in
quantum computing through the First Working Seminar on Quantum Software
Engineering (WSQSE 2022), providing valuable insights into this evolving
domain. Finally, Zhao et al. [67] made a contribution to the field of quantum
programming by introducing Bugs4Q, which is a novel benchmark suite comprising
forty-two real and manually validated Qiskit bugs extracted from prominent
platforms such as GitHub, StackOverflow, and Stack Exchange. These bugs are
supplemented with test cases that facilitate reproducing erroneous behaviors,
and these bug scenarios are invaluable for systematically evaluating debugging
and testing methodologies for quantum programs. These papers collectively
contribute to the understanding of quantum software development, addressing
various aspects such as software architecture, optimization, workflow models,
challenges, and benchmarking in quantum computing systems.

###  7.2 Decisions in software engineering

Decisions are either explicitly presented in knowledge management tools or
implied in various textual artifacts. Extensive research has been conducted to
understand and analyze decisions and decision-making in software engineering.
Li et al. [68] conducted an exploratory study on the Hibernate developer
mailing list, analyzing 9,006 posts to uncover insights into decision
expressions, classification, rationale, approaches, related software
artifacts, and the trend of decision-making over time. Their findings revealed
that decisions in open-source software development are typically expressed as
Information Giving, Solution Proposal, and Feature Request, with the main
categories being Design Decision and Requirement Decision. In a separate
study, Bi et al. [69] investigated how developers use architecture and design
patterns concerning quality attributes and design contexts, revealing
previously unknown relationships among these elements. Zhang et al. [70]
conducted an empirical study on architecture decisions in AI-based systems
development. They analyzed the data from Stack Overflow and GitHub, revealing
that architecture decisions in AI are expressed through various linguistic
patterns, with Solution Proposal and Information Giving being common. Their
study identified Technology Decision, Component Decision, and Data Decision as
primary decision types, with a focus on Performance as the key quality
attribute. It also highlighted Design Issues and Data Issues as significant
challenges in the architecture decision-making of AI-based systems. Waseem et
al. [46] introduced decision models aimed at assisting microservices
practitioners in selecting appropriate patterns and strategies for
Microservices Architecture (MSA). These models cover four key MSA design
areas, including microservices application decomposition, security,
communication, and service discovery. To create these models, the authors
conducted an extensive literature review and evaluated their effectiveness
through interviews with 24 practitioners, and the results show that these
decision models were found to be valuable tools for guiding microservices
patterns and strategy selection. Shahin et al. [71] explored the visualization
of architectural design decisions, focusing on the use of the Compendium tool
to improve understandability and communication in architectural design.
Lastly, Liu et al. [72] provided insights into the decision-making of students
in requirements engineering course projects, emphasizing the importance of
understanding the characteristics of decision-making during various stages of
requirements engineering. These works collectively contribute to our
understanding of decision-making in software engineering, encompassing diverse
facets such as decision expression, architecture patterns, visualization, and
decision-making in the software development process.

###  7.3 Architecture decisions in quantum software development

Khan et al. [18] conducted a systematic review to delve into the architectural
aspects of quantum software development. Their study provides insights into
how architectural processes, modeling notations, design patterns, tool
support, and challenging factors impact quantum software architecture. The
authors highlighted the transformation of quantum bits (Qubits) into
architectural components, offering a fresh perspective on architectural
decisions in quantum computing systems. Sodhi et al. [25] investigated Quantum
Computing Platforms (QCPs) from a software engineering perspective. They
proposed a QCPs architecture and a programming model, and evaluated the impact
of QCPs characteristics on Quality Attributes (QAs) and Software Development
Life Cycle (SDLC) activities. Their findings suggest that QCPs are best suited
for specialized applications like scientific computing, with some QAs, such as
maintainability and reliability, being negatively affected by certain QCPs
characteristics (e.g., limitations in qubit state transmission, hard
dependencies on quantum algorithms, and restrictions on copying and deletion
of qubit states). Furthermore, Khan et al. [39] researched on agile practices
in quantum software development delved into the challenges and applicability
of agile practices in this domain. While their work primarily focuses on the
adoption of agile practices, it also indirectly touches on limitations and
challenges in quantum software development. Their study demonstrated that
agile software development practices have the potential to address some of the
challenges inherent in quantum software development, but it also highlighted
new obstacles that impede their effective incorporation. This research offers
valuable insights into the landscape of challenges faced when making
architecture decisions in quantum software development.

Distinguished from the aforementioned research, our investigation delved into
a comprehensive analysis of architectural decision-making within quantum
software development. Specifically, our study encompassed an examination of
the expressions and classifications of architecture decisions, the diverse
application domains in which these decisions manifest, the critical quality
attributes that factor into decision-making, as well as the formidable
limitations and challenges encountered during this architectural process. To
conduct this exploration, we conducted data mining across Stack Exchange and
GitHub, amassing valuable insights into the intricacies of quantum software
development’s architectural landscape.

##  8 Conclusions

Quantum software architecture refers to the design and organization blueprint
for quantum software systems that leverage the principles of quantum computing
to solve specific problems or tasks. It focuses on addressing the challenges
and limitations inherent to quantum computation, providing a structured
approach to building applications that leverage the power of quantum
technologies. Architecture decisions are essential for architects and
developers to make informed design choices during the development of quantum
software systems, and once a system is developed, it can be challenging and
costly to change at the architecture level. This research aimed to explore the
architecture decisions within the context of quantum software development.
Specifically, we conducted an empirical study on GitHub issues and Stack
Exchange posts to explore architecture decisions in quantum software
development from practitioners’ perspectives. We used a keyword-based search
to collect data from GitHub and Stack Exchange, and finally got 151 GitHub
issues and 43 Stack Exchange posts which include 29 Stack Overflow posts, 11
Quantum Computing Stack Exchange posts, and 3 Computer Science Stack Exchange
posts. We manually extracted architecture decisions related to quantum
software systems to investigate decision expressions, decision types, involved
application domains, quality attributes considered, and limitations and
challenges encountered in architecture decision-making of quantum software
development. The main results are that:

  * 1.

In quantum software development, architecture decisions are expressed through
six distinct linguistic patterns, with Solution Proposal and Information
Giving being the most prevalent. Solution Proposal constitutes 38.6%, while
Information Giving accounts for 29.6% of the linguistic patterns used.
Conversely, Opinion Asking and Problem Discovery are relatively less used in
the linguistic patterns we collected.

  * 2.

Architecture decisions primarily fall into two major categories:
Implementation Decision and Technology Decision. Specifically, Implementation
Decision represents 25.4%, and Technology Decision accounts for 24.4% of the
of the architecture decisions we collected.

  * 3.

The Quantum Programming Framework stands out as the most common application
domain among the sixteen domains identified, representing 25.3% of the quality
attributes we collected.

  * 4.

Two important quality attributes considered in architectural decision-making
of quantum software systems are Maintainability (19.2%) and Performance
(16.3%) of the quality attributes we collected.

  * 5.

Practitioners in quantum software development face significant limitations and
challenges. Design Issues are prominent, accounting for 25.6%, while
Performance Issues are a notable concern at 12.1% of the limitations and
challenges we collected.

The findings of this study offer valuable insights to researchers and
practitioners. By identifying common linguistic patterns, major decision
categories, prevalent application domains, and critical quality attributes,
this research equips professionals in quantum software systems with a deeper
understanding of the made architectural decisions. Additionally, it highlights
common challenges of making architecture decisions in quantum software
development, such as design and performance issues, providing researchers and
practitioners with the awareness needed to address these issues.

In the next step: (1) We intend to conduct a comparative study of architecture
decisions on GitHub, Stack Exchange, and other platforms like developer
mailing lists, aiming to reveal insights into the current focus of
architecture decision-making in quautum software systems, including its
advantages and deficiencies. (2) To provide a comprehensive perspective, we
will validate and extend the proposed taxonomy of architecture decisions using
a practitioner survey or interview, collecting insights from professionals
actively engaged in quantum software development. (3) Additionally Our
overarching goal is to address the limitations and challenges identified in
this study, contributing to a better architecture decision-making in quantum
software development.

## Acknowledgements

This work has been partially sponsored by the National Natural Science
Foundation of China (NSFC) under Grant No. 62172311, the Special Fund of Hubei
Luojia Laboratory, and the Natural Science Foundation of Hubei Province of
China under Grant No. 2021CFB577. The authors would also like to acknowledge
the financial support from the China Scholarship Council.

## References

  * [1] S. T. Marella, H. S. K. Parisa, Introduction to Quantum Computing, IntechOpen, 2020\.
  * [2] A. Bhasin, M. Tripathi, Quantum computing at an inflection point: Are we ready for a new paradigm, IEEE Transactions on Engineering Management 70 (7) (2023) 2546–2557.
  * [3] A. J. McCaskey, E. F. Dumitrescu, D. Liakh, M. Chen, W.-C. Feng, T. S. Humble, A language and hardware independent approach to quantum–classical computing, SoftwareX 7 (2018) 245–254.
  * [4] J. Preskill, Quantum computing in the nisq era and beyond, Quantum 2 (2018) 79.
  * [5] D. Willsch, M. Willsch, F. Jin, H. De Raedt, K. Michielsen, Large-scale simulation of shor’s quantum factoring algorithm (2023), arXiv preprint arXiv:2308.05047.
  * [6] A. Mandviwalla, K. Ohshiro, B. Ji, Implementing grover’s algorithm on the ibm quantum computers, in: Proceedings of the 6th IEEE International Conference on Big Data (BigData), IEEE, Seattle, WA, USA, 2018, pp. 2531–2537.
  * [7] A. Montanaro, Quantum algorithms: an overview, npj Quantum Information 2 (1) (2016) 1–8.
  * [8] A. W. Harrow, A. Hassidim, S. Lloyd, Quantum algorithm for linear systems of equations, Physical Review Letters 103 (15) (2009) 150502.
  * [9] B. Heim, M. Soeken, S. Marshall, C. Granade, M. Roetteler, A. Geller, M. Troyer, K. Svore, Quantum programming languages, Nature Reviews Physics 2 (12) (2020) 709–722.
  * [10] S. M. Sundaram, T. R. Murgod, Quantum software engineering and technology, in: Technology Road Mapping for Quantum Computing and Engineering, IGI Global, 2022, pp. 102–116.
  * [11] J. Zhao, Quantum software engineering: Landscapes and horizons (2020), arXiv preprint arXiv:2007.07047.
  * [12] M. Piattini, G. Peterssen, R. Pérez-Castillo, Quantum computing: A new software engineering golden age, ACM SIGSOFT Software Engineering Notes 45 (3) (2021) 12–14.
  * [13] M. A. Serrano, J. A. Cruz-Lemus, R. Perez-Castillo, M. Piattini, Quantum software components and platforms: Overview and quality assessment, ACM Computing Surveys 55 (8) (2022) 1–31.
  * [14] B. Kommadi, Quantum computing solutions: Solving real-world problems using quantum computing and algorithms, Springer, 2020.
  * [15] A. García de la Barrera, I. García-Rodríguez de Guzmán, M. Polo, M. Piattini, Quantum software testing: State of the art, Journal of Software: Evolution and Process 35 (4) (2023) e2419.
  * [16] L. Zhang, A. Miranskyy, W. Rjaibi, G. Stager, M. Gray, J. Peck, Making existing software quantum safe: A case study on IBM Db2, Information and Software Technology 161 (2023) 107249.
  * [17] C. G. Almudever, L. Lao, X. Fu, N. Khammassi, I. Ashraf, D. Iorga, S. Varsamopoulos, C. Eichler, A. Wallraff, L. Geck, A. Kruth, J. Knoch, H. Bluhm, K. Bertels, The engineering challenges in quantum computing, in: Proceedings of the 20th Design, Automation, and Test in Europe (DATE), IEEE, Lausanne, Switzerland, 2017, pp. 836–845.
  * [18] A. A. Khan, A. Ahmad, M. Waseem, P. Liang, M. Fahmideh, T. Mikkonen, P. Abrahamsson, Software architecture for quantum computing systems - a systematic review, Journal of Systems and Software 201 (2023) 111682.
  * [19] L. Bass, P. Clements, R. Kazman, Software Architecture in Practice, 3rd Edition, Addison-Wesley Professional, 2012.
  * [20] A. Jansen, J. Bosch, Software architecture as a set of architectural design decisions, in: Proceedings of the 5th Working IEEE/IFIP Conference on Software Architecture (WICSA), IEEE, Pittsburgh, Pennsylvania, USA, 2005, pp. 109–120.
  * [21] I. Malavolta, H. Muccini, S. R. V, Enhancing architecture design decisions evolution with group decision making principles, in: Proceedings of the 6th International Workshop on Software Engineering for Resilient Systems (SERENE), Springer, Budapest, Hungary, 2014, pp. 9–23.