The integral of the loss is convergent thus up to extraction $L(\beta_{t})$
converges to 0. Since $\beta_{t}$ is bounded a second extraction ensures the
converges of the iterates. Thus there is a subsequence $\beta_{\psi(t)}$ such
that it converges to $\beta_{\infty}$ and $L(\beta_{\infty})=0$ thus it is an
interpolator. And we have

$\begin{split}\phi_{\alpha_{\infty}}(\beta_{\infty})-D_{t}&=\phi_{\alpha_{\infty}}(\beta_{\infty})-\phi_{\alpha_{t}}(\beta_{t})-\langle\nabla\phi_{\alpha_{t}}(\beta_{t}),\beta_{t}-\beta_{\infty}\rangle\\\
&\geq\phi_{\alpha_{t}}(\beta_{\infty})-\phi_{\alpha_{t}}(\beta_{t})-\langle\nabla\phi_{\alpha_{t}}(\beta_{t}),\beta_{t}-\beta_{\infty}\rangle\\\
&=D_{\phi_{\alpha_{t}}}(\beta_{\infty},\beta)\\\ &\geq 0\end{split}$

Because $\alpha\mapsto\phi_{\alpha}(\beta)$ is decreasing and
$\alpha_{t}\geq\alpha_{\infty}$. Note that
$\phi_{\alpha_{\infty}}(\beta_{\infty})-D_{\psi(t)}\to 0$ by convergence of
$\beta_{\psi(t)}$ to $\beta_{\infty}$. Thus by convergence of $D$ we have that
$\phi_{\alpha_{\infty}}(\beta_{\infty})-D_{t}\to 0$ and in turn
$D_{\phi_{\alpha_{t}}}(\beta_{\infty},\beta)\to 0$. Recall that

$\nabla^{2}\phi_{\alpha}(\beta)=\frac{1}{4}\text{diag}\left(\frac{1}{\sqrt{\beta^{2}+4\alpha^{2}}}\right)\geq\frac{1}{4\sqrt{\|\beta\|^{2}+4\max\alpha_{i}^{2}}}Id$

thus $\phi_{\alpha_{t}}$ is strongly convex of with a parameter that does not
depend on $t$ by decreasingness of $\alpha_{t}$ and boundedness of
$\beta_{t}$. Thus we have that
$D_{\phi_{\alpha_{t}}}(\beta_{\infty},\beta)\geq\mu/2\|\beta_{t}-\beta_{\infty}\|^{2}$
which concludes the convergence. ∎

####  B.1.4 Proof of Proposition 3

In the last section we have shown that the iterates converge toward an
interpolator. However that interpolator is not a minimizer of the potential
$\phi_{\alpha_{\infty}}$ as shown in proposition 2. However by strong
convexity of $\phi_{\alpha}$ we have a control of the distance between
$\beta_{\infty}$ and the minimizer of $\phi_{\alpha_{\infty}}$ which we denote
by $\beta^{\ast}_{\alpha_{\infty}}$

###### Proof.

Let $\phi(\beta)=\phi_{\alpha_{\infty}}(\beta)+\iota_{X\beta=Y}$ where $\iota$
is the indicator function equal to $0$ if the condition is satisfied and
$+\infty$ otherwise. Note that $\phi$ is still strongly convex. Indeed, we can
show that $\phi-\mu\|.\|^{2}$ is convex: let $\beta,\nu\in\mathbb{R}^{d}$,
$\lambda\in(0,1)$. $C=\\{\beta\text{ s.t. }X\beta=Y\\}$ is a convex set as an
affine subspace of $\mathbb{R}^{d}$. Therefore:

$\lambda\beta+(1-\lambda)\nu\notin C\implies\beta\notin C\text{ or }\nu\notin
C$ (40)
$\iota_{X(\lambda\beta+(1-\lambda)\nu)=Y}=\infty\implies\iota_{X\beta=Y}=\infty\text{
or }\iota_{X\nu=Y}=\infty$ (41)

We also have that $\phi_{\alpha_{\infty}}-\mu\|.\|^{2}$ is convex on a bounded
set which contains $\beta_{\infty}$ and $\beta_{\alpha_{\infty}}^{\ast}$.
Finally $\phi-\mu\|.\|^{2}$ is convex, i.e. $\phi$ is $\mu-$strongly convex on
a bounded set.

We have $\beta_{\infty}$ solution of

$\min\phi(\beta)-\langle r_{\infty},\beta\rangle$ (42)

And $\beta_{\alpha_{\infty}}^{\ast}$ is the solution to

$\min\phi(\beta)$ (43)

Since $\phi$ is strongly convex we have that
$f=\phi-\mu\|\cdot-\beta^{\ast}\|^{2}$ is convex. Notice that $0\in\partial
f(\beta_{\infty})-r_{\infty}$ thus

$\phi(\beta^{\ast})-\langle
r_{\infty},\beta_{0}\rangle-\mu\|\beta^{\ast}-\beta_{\infty}\|^{2}\geq\phi(\beta_{\infty})-\langle
r_{\infty},\beta_{\infty}\rangle$ (44)

We thus observe that:

$\phi(\beta^{\ast})-\langle
r_{\infty},\beta^{\ast}\rangle\geq\phi(\beta_{\infty})-\langle
r_{\infty},\beta_{\infty}\rangle+\mu\|\beta^{\ast}-\beta_{\infty}\|^{2}$ (45)

Thus by optimality of $\beta^{\ast}$
($\phi(\beta^{\ast})-\phi(\beta_{\infty})\leq 0$)

$\begin{split}\langle
r_{\infty},\beta_{\infty}-\beta^{\ast}\rangle&\geq\phi(\beta^{\ast})-\langle
r_{\infty},\beta^{\ast}\rangle-(\phi(\beta_{\infty})-\langle
r_{\infty},\beta_{\infty}\rangle)\\\
&\geq\mu\|\beta^{\ast}-\beta_{\infty}\|^{2}\end{split}$ (46)

Finally using Cauchy-Schwarz inequality we have

$\boxed{\|r_{\infty}\|\geq\mu\|\beta^{\ast}-\beta_{\infty}\|}$ (47)

∎

###  B.2 Additional Numerical Experiments

##### Reminder

We adopt the same set-up as Pesme et al. (2021) for sparse regression. We
select parameters n = 40 and d = 100, and then create a sparse model
$\beta^{*}_{l_{0}}$ with the constraint that its $l_{0}$ norm is equal to 5.
We generate the features $x_{i}$ from a normal distribution with mean 0 and
identity covariance matrix $N(0,I)$, and compute the labels as
$y_{i}=x_{i}^{T}\beta^{*}_{l_{0}}$. We always use the same step size of
$\gamma=1/(1.3||\bar{X}^{T}\bar{X}||_{2})$. Notice that
$||\beta_{t}-\beta^{*}_{l_{0}}||^{2}_{2}$ is the validation loss in the
experiments.

##### Introduction

As outlined in Section 3.3 and prooved in Section B.1 in a more general
setting, our investigation reveals that Noisy-SGD yields an identical solution
to GD, albeit (1) operating on an altered potential $\phi_{\alpha_{\infty}}$,
and (2) starting from an effective non-zero initialization denoted as
$\tilde{\beta}_{0}$. In contrast, the sole parameter affected by SGD is
$\alpha$. In the context of Noisy-SGD, it is noteworthy that $\alpha_{\infty}$
diminishes with increasing $\sigma$. This signifies that as more noise is
introduced, the effective $\alpha$ becomes smaller. If this did not influence
the effective initialization, it would directly imply an “improved” implicit
bias, in the case of the sparse regression under study.

##### Trade off

To be more precise, if the effective initialization $\tilde{\beta_{0}}$
remained unaltered, the noisy process would ultimately converge to
$\beta^{\star}_{\alpha_{\infty}}$, which is the solution that minimizes
$\phi_{\alpha_{\infty}}$. We already know that this solution is sparser than
the one obtained via SGD because $\alpha_{\infty}$ is smaller (as referenced
in Equation 21). As demonstrated in Proposition 3, the presence of the new
effective initialization $\tilde{\beta_{0}}$ does introduce some perturbation
to the solution. However, the degree of its influence directly depends on the
level of noise introduced. This ensures that $\beta_{\infty}$, which is the
solution obtained through Noisy-SGD, remains in close proximity to
$\beta^{\star}_{\alpha_{\infty}}$. This scenario implies the existence of a
tradeoff. Increasing the value of $\sigma$ will:

  1. 1.

Augment the sparsity of $\beta^{\star}_{\alpha_{\infty}}$ as
$\beta^{\star}_{\alpha_{\infty}}$ gets closer to the (sparse) ground truth
$\beta^{\star}_{l_{0}}$

  2. 2.

Expand the gap between $\beta_{\infty}$ and $\beta^{\star}_{\alpha_{\infty}}$,
which could potentially diminish the impact of the first outcome if
$\beta_{\infty}$ gets far from $\beta^{\star}_{l_{0}}$

In summary, if the influence of 1. surpasses the effect of 2., Noisy-SGD would
enhance the implicit bias. Conversely, if 2. outweighs 1., this enhancement
may not necessarily occur. To empirically quantify the trade-off, we measure
the gap between $\beta^{\star}_{\alpha_{\infty}}$ and $\beta_{\infty}$ across
varying values of $\alpha$ and $\sigma$. Our approach involves (a) obtaining
$\beta_{\infty}$ through Noisy-SGD, (b) approximating $\alpha_{\infty}$
numerically based on the loss logs, and (c) subsequently executing GD with
$\alpha=\alpha_{\infty}$ to obtain $\beta^{\star}_{\alpha_{\infty}}$. In
Figure 4, our findings underscore that, for $\alpha=0.1$, the distance between
$\beta^{\star}_{\alpha_{\infty}}$ and $\beta_{\infty}$ seem to indeed increase
with growing values of $\sigma$, validating the second point (2.). Moreover,
as revealed in Figure 3, when using $\alpha=0.1$, Noisy-SGD consistently
converges to sparser solutions compared to standard SGD. This trend remains
consistent across various noise levels, as illustrated in Figure 5.
Specifically, as we elevate the magnitude of $\sigma$, the solutions become
progressively closer to the sparse interpolator (ground truth). In this
context, the influence of the new initialization $\tilde{\beta_{0}}$ appears
to be of secondary importance, with therefore point 1. consistently holding
more significance than point 2.

Figure 5:  Diagonal Linear Network from $\alpha=0.1$: Implicit Bias of SGD and
Noisy SGD different values of $\sigma$ in Equation 18, from $\alpha=0.1$.
Shaded areas represent one standard variation over 5 runs, and plain lines
represent the average values. Starting from this initialization, the bigger
$\sigma$ is, the closer the solution obtained with Noisy-SGD is to the sparse
solution $\beta^{*}_{l_{0}}$.

Nonetheless, Figure 4 reveals that the magnitude of the gap between
$\beta^{\star}_{\alpha_{\infty}}$ and $\beta_{\infty}$ is smaller or
comparable to the gap between $\beta_{\infty}$ and $\beta^{\star}_{l_{0}}$ in
Figure 3. This observation implies that Noisy-SGD may be advantageous only in
this specific scenario, as the impact of 2. is small. However, a question
arises: if the initial $\alpha$ were even smaller, would 1. still dominate 2.?

Figure 6 provides insight into this inquiry. Even with an initial $\alpha$
reduced by a factor of ten, we observe that the introduction of noise
continues to exert a positive influence on the sparsity of the solution.
Furthermore, when examining the gap between $\beta^{\star}_{\alpha_{\infty}}$
and $\beta_{\infty}$ in Figure 7, we find that, similarly to the case where
$\alpha=0.1$, it remains of a comparable order of magnitude to the gap between
$\beta_{\infty}$ and $\beta^{\star}_{l_{0}}$ observed in Figure 6. This
observation helps to elucidate why the advantages of Noisy-SGD persist under
these conditions: the distance between $\beta^{\star}_{\alpha_{\infty}}$ and
$\beta_{\infty}$ empirically also depends (inversely) on $\alpha$, which
mitigates the impact of 2. over 1.

Figure 6:  Diagonal Linear Network from $\alpha=0.01$: Implicit Bias of SGD
and Noisy SGD different values of $\sigma$ in Equation 18, from small
initialization $\alpha=0.01$. Shaded areas represent one standard variation
over 5 runs, and plain lines represent the average values. Starting from this
initialization, the bigger $\sigma$ is, the closer the solution obtained with
Noisy-SGD is to the sparse solution $\beta^{*}_{l_{0}}$.  Figure 7:  DLN from
$\alpha=0.1$: Distance between $\beta^{*}_{\alpha_{\infty}}$, the solution
that minimizes $\phi_{\alpha_{\infty}}$ —obtained by GD from
$\alpha_{\infty}$— and $\beta_{\infty}$, the one obtained by Noisy-SGD from
$\alpha=0.01$ (see Proposition 3). Shaded areas represent one standard
deviation over 5 runs. Similar to the case of $\alpha=0.1$ (see Figure 4), the
distance is of the same order of magnitude than the distance between
$\beta_{\infty}$ and the sparse solution $\beta_{l_{0}}$ (see Figure 6). It
explains why why the implicit bias is enhanced by Gaussian noise in this case.

##### Conclusion:

The effect of Noisy-SGD on the solution’s sparsity, as discussed in Section
3.3, remains significant even when initiated with smaller values of $\alpha$,
which corresponds to cases where the solutions obtained from GD and SGD
already exhibit some enhanced degree of sparsity. Therefore, the practical
implications of the theoretical trade off appears to be modest, and the
reinforcement of the implicit bias from Noisy-SGD prevails: the introduction
of Gaussian noise enhances or at least maintains the implicit bias of SGD in a
robust way.


# Building Trust in AI-Driven Decision Making for Cyber-Physical Systems
(CPS): A Comprehensive Review
††thanks: This research was supported by the Science Foundation Ireland and
the Department of Agriculture, Food and Marine on behalf of the Government of
Ireland Vistamilk research center under grant 16/RC/3835 and by part by
Science Foundation Ireland (SFI), co-funded under the European Regional
Development Fund under Grant Number 13/RC/2077-P2.

1st Rahul Umesh Mhapsekar  Walton Institute, (PAS)
South East Technological University
Waterford, Ireland
<EMAIL_ADDRESS>2nd Muhammad Iftikhar Umrani  Walton
Institute, (PAS)
South East Technological University
Waterford, Ireland
<EMAIL_ADDRESS>3rd Malik Faizan  Walton
Institute, (PAS)
South East Technological University
Waterford, Ireland
<EMAIL_ADDRESS>4th Omer Ali  Walton Institute, (ENL)
South East Technology University (SETU)
Waterford, Ireland
<EMAIL_ADDRESS>5th Lizy Abraham  Walton Institute, (ENL)
South East Technology University (SETU)
Waterford, Ireland
<EMAIL_ADDRESS>
###### Abstract

Recent advancements in technology have led to the emergence of Cyber-Physical
Systems (CPS), which seamlessly integrate the cyber and physical domains in
various sectors such as agriculture, autonomous systems, and healthcare. This
integration presents opportunities for enhanced efficiency and automation
through the utilization of artificial intelligence (AI) and machine learning
(ML). However, the complexity of CPS brings forth challenges related to
transparency, bias, and trust in AI-enabled decision-making processes. This
research explores the significance of AI and ML in enabling CPS in these
domains and addresses the challenges associated with interpreting and trusting
AI systems within CPS. Specifically, the role of explainable AI (XAI) in
enhancing trustworthiness and reliability in AI-enabled decision-making
processes is discussed. Key challenges such as transparency, security, and
privacy are identified, along with the necessity of building trust through
transparency, accountability, and ethical considerations.

###### Index Terms:

Artificial Intelligence, Cyber-Physical Systems, XAI, trustworthy AI.

##  I Introduction

Advancements in various fields such as agriculture, autonomous systems,
healthcare, and transportation have made the interaction between the cyber and
physical domains more complex. This complexity has given rise to Cyber-
Physical Systems (CPS), which efficiently integrate the cyber and physical
domains. CPS entails a closely interlinked communication network where
embedded computing devices, smart controllers, physical environments, and
humans interact systematically [1]. Artificial Intelligence (AI) and Machine
Learning (ML) are crucial in enabling systems to quickly react and make real-
time decisions based on new information. AI and algorithmic decision-making
have become ubiquitous in modern life, greatly enhancing efficiency and
complexity through access to extensive data, sophisticated algorithms, and
powerful computing capabilities. However, interpreting these systems has
become increasingly challenging due to their complex logic, potentially
leading to difficulty in assessing their reliability. Moreover, their
limitations, biases, and ethical concerns can render them fragile and unfair
[2].

Agriculture, being one of the oldest and most vital industries globally, faces
significant challenges due to the rapid growth of the global population. This
surge in population demands more food production and job opportunities.
However, traditional farming methods are proving inadequate to meet these
escalating needs. Consequently, there is a push towards adopting automated
techniques to bridge this gap. Emerging technologies like the Internet of
Things (IoT), CPS, Big Data analytics, AI, and ML are revolutionizing
agriculture [3].

Similarly, recent technological advances have increased the demand for
autonomous systems such as road vehicles, Unmanned Aerial Vehicles (UAVs), and
robots across various civilian and military applications. Traditionally,
operations of these systems are controlled by a human operator, also called
human autonomy, where decisions are made by humans[4]. On the other hand,
integration of AI into these systems not only increases the level of autonomy
but also enables them to become more situational aware and make independent
decisions, thereby reducing human intervention in task execution [5].
Moreover, the decision-making capabilities of these systems, such as
prediction and classification, heavily depend on the quality of environmental
data acquired through sensors. Therefore, intelligent algorithms can help
improve the perception capabilities of these systems and compensate for
uncertainties. Recently, various research efforts have been considered to
develop resilient and reliable autonomous systems that can be deployed in
complex and dynamic environments [4].

AI has significant potential in healthcare, which incorporates enhancing
health through prevention, diagnosis, and management of disease, injury, and
various physiological and mental health issues. Medical professionals and
associated healthcare sectors govern public health services across primary,
secondary, and tertiary care levels [6]. Accessibility to healthcare services
is determined by varying social and economic factors as well as differing
healthcare policies across nations, leading to challenges due to economic
constraints, workforce shortages, and logistical issues. These challenges can
compromise the effectiveness of disease diagnosis and subsequent treatment,
thus impacting overall health and increasing the risk of mortality. To
mitigate these challenges, the adoption of technologies such as sensor fusion
in conjunction with AI, ML and data analytics is proposed. Such integrations
aim to facilitate more accessible healthcare services and support in the
prediction, prognosis, and diagnosis of diseases, ultimately contributing to
enhanced healthcare quality and patient well-being [7].

Integration of AI is necessary to achieve sustainable and efficient CPS,
making it crucial to develop more robust and reliant algorithms. This requires
seamless integration with classical systems for improved real-time analytics.
However, AI systems are sensitive to data, bias and often opaque in their
decision-making capabilities, thus termed as ”black-box” systems. Some of
these factors directly challenge trust, governance, and associated risk
factors in CPS domains, thus further hindering their adoption [8]. Explainable
AI (XAI), an emerging subset of AI, accounts for these trustworthiness aspects
of AI. This article provides several use-cases where the integration of XAI in
CPS domains may provide enhanced explainability of AI actions to increase
trust [9, 10].

##  II Related Works

This review aims to develop trust in AI-enabled decisions for CPS across
various use cases, such as Agriculture, Autonomous Systems, and Healthcare.
Some reviews discuss advancements in AI for CPS and the associated challenges.
Babajide et al. in [11] discussed that the integration of AI and ML with IoT
and CPS is essential for their advancement, ensuring efficient operations and
dependable systems. Challenges such as data fusion, security, and privacy
persist, requiring innovative solutions such as blockchain, and edge
computing. Data fusion challenges include handling diverse modalities, proper
calibration for accurate fusion, addressing trivial data, and synchronizing
operational timing for real-time applications. Security remains a significant
concern due to the vast number of interconnected devices, necessitating robust
AI and ML systems to counter adversarial attacks and protect consumer data.
Similarly, Jiyeong et al. [12] has discussed the integration of AI with
Industrial Cyber-Physical Systems (ICPS) for optimization of the process. It
proposes the concept of AI-augmented ICPS (AICPS) and discusses its
components, interactions, and design considerations. The advancements in
industrial AI technologies are examined for their applications in AICPS,
highlighting benefits such as real-time monitoring and management. Similarly,
utilizing Smart Agriculture (SA) as a CPS can increase the efficiency of farms
by gathering real-time data on weather conditions, crop health, and soil
quality. Various use cases such as irrigation and solar energy systems have
benefited from SA integration which uses sensor and actuator networks that
enable conservation of energy and maintain optimal conditions for plant growth
[13].

From the perspective of a short review briefly discussing the the role of
advanced computing technologies for enhancing the capabilities of autonomous
systems has been presented by Chen et al. in[14]. This study delves into the
state-of-the-art of autonomous systems and explores their integration with
federated learning, IoT, and big data. It also discusses the evolution and
adaptation of ML algorithms addressing uncertainties, and improving
communication and control capabilities using IoT and big data for real-time
data processing in autonomous systems. The authors of [15] investigate the
utilization of potential computational technologies in designing trustworthy
AI systems. It introduces six important dimensions to design such systems:
safety and robustness, fairness, explainability, privacy, and accountability.
In addition to that, it examines the key technologies and their applications
within each dimension across real world AI systems. Moreover, the work
underscores the significance of mitigating bias issues within AI algorithms to
ensure fairness in practical AI applications. It also offers a comprehensive
overview of various mitigation techniques aimed at counteracting biases in AI
systems.

The fusion of technological advancements in Wireless Sensor Networks (WSN),
medical sensors, and cloud computing (CC) presents CPS as a significant option
for various healthcare applications, including patient services within
hospitals and at home [16]. CC enables efficient management and analysis of
larger dataset related to the healthcare information collected by CPS, often
characterized by large volumes, complex structures, and diverse
characteristics. Milenkovic et al. in [17] proposed alternative data
management solutions utilizing NoSQL databases to improve the processing,
storage, indexing, and analysis of healthcare information, mitigating the
limitations of traditional relational databases. The integration of smart
devices with advanced sensing and network capabilities, such as wearable
technology and lab-on-chip meters, contributing towards smart healthcare [18].
In particular, wireless body area networks (WBANs) are recognized as a leading
technology that provides seamless healthcare services and facilitates
continuous health monitoring. These networks form a subset of medical CPS
capable of functioning in complex environments like hospitals, addressing
communication challenges in various medical scenarios [17]. While many studies
center on the technological landscape of AI and its integration with CPS in
various domains, there is a gap in addressing the trust in AI system, thus
served as the primary motivation for this article. This review makes the
following contributions:

  * •

A comprehensive account of AI-enabled CPS applications

  * •

Identification of the need for trust in AI-enabled applications and their
decision-making process

  * •

The need to integrate XAI for establishing trust, transparency and granularity
in AI-decision making

##  III AI for CPS domains

This section discusses the applications and benefits of using AI in the use
cases considered in this study. Fig. 1 shows the applications of AI in Smart
Farming, Autonomous systems and Healthcare which is the focus of this
research.

Figure 1: Applications of AI in proposed use cases

### Use of AI and CPS in Agriculture:

AI plays a pivotal role in agricultural management across various domains
[19]. In soil management, it utilizes techniques like management-oriented
modeling and artificial neural networks for predictive performance on soil
parameters. Similarly, in pest and weed management, AI aids in the accurate
monitoring and identification of infected plants through techniques such as
digital image analysis and sensor ML. Disease management benefits from AI by
enabling early detection through image analysis of crops and animals. Crop
management is revolutionized with AI recommendations tailored to crop choices
and resource allocation. Water-use optimization is enhanced through AI-managed
irrigation systems, integrating real-time data for efficient resource
utilization [19]. Precision Farming is another application area where CPS and
AI can be leveraged to improve the results. It integrates computational
intelligence into complex systems, such as precision agriculture, using
cooperative sensor swarms and layered architectures. Researchers have proposed
CPS frameworks for precision spraying, monitoring, and decision-making in
agriculture, using WSN and robots to enhance efficiency and involve human
operators for improved performance [20].

Similarly, it has been used in the dairy industry for several applications
such as disease detection in cows (clinical mastitis, ketosis, metritis, etc).
CPS has significantly contributed towards smart dairy farming. Farm Management
System (FMS) is one of the applications that serves as the key element of
dairy farm optimization, integrating vital functions like herd management,
milk production monitoring, and financial administration into a centralized
software solution. This system not only streamlines operations but also
enhances decision-making through data analysis[22]. Deep Learning (DL) can
improve Milk Quality Analysis (MQA) for milk adulteration detection [23].
However, there has been a lack of DL implementation in MQA due to lack of
trust in the DL predictions. While AI offers numerous advantages in
agriculture, addressing security and privacy concerns is crucial to build
trust in the use of CPS and AI within the agricultural sector.

### Role of AI in Autonomous Systems:

AI holds significant promise for enhancing the reliability and safety of
autonomous systems across various applications, such as smart cities, remote
sensing, rescue operations, transportation, and others [24]. In the domain of
UAV-based precision agriculture, AI techniques such as data fusion and DL play
a crucial role for improving the overall accuracy of crop yield prediction
[25]. Similarly, DL aids in accurately categorizing the variety of seasonal
crops that are cultivated within specific fields [26]. This information is
valuable for various tasks, like agriculture insurance, predicting market
trends, land leasing decisions, and supply chain management. Moreover, unique
framework has been developed for AI-enabled package delivery drones to avoid
failures in package delivery missions [27]. It uses a neural network
regression model to monitor the battery consumption of drones in a real-time
environment. To minimize infrastructure damage in flood-affected areas,
Convolutional Neural Networks (CNNs) algorithm was deployed in UAVs to detect
flood-affected zones [28]. In this scenario, the UAVs captured aerial imagery,
which was later used to extract various flood-related features, including
damaged infrastructure. Additionally, AI techniques can bring intelligence in
industrial robots to improve their operational and cognitive capabilities. For
example, AI techniques were used to design an autonomous product-quality
evaluation system. This system monitors the product quality and distinguishes
between defective and non-defective products without human intervention [29].
Similarly, these techniques have been used to find anomalies in robotic data,
predict potential failures, reducing downtime, and maintenance costs [30]. In
smart cities, the inclusion of AI enables the development of intelligent
transportation systems. This targeted deployment has the potential to detect
real-time traffic, ensuring safety for public and autonomous vehicles [31].
Furthermore, CPS ensures human safety in human-robot collaborative operations
[32]. This system utilizes sensors to continuously monitor the environment and
trigger an alarm if the safety distance between humans and robots is below the
threshold. Although AI and CPS have promising capabilities to design
intelligent autonomous systems, but the unpredictable behavior of AI systems
could significantly deteriorate the performance of these systems.

### Role of AI and CPS in Healthcare:

After the COVID-19 pandemic, a significant increase has been observed in the
adoption of AI technologies for healthcare. CPS plays a vital role in
healthcare beyond hospitals, extending to assisted living and elderly care.
Its application in healthcare is generally categorized into two areas. In
Assisted application, health monitoring is conducted without limiting the
daily activities of an individual. Real-time physiological data is acquired
through Photoplethysmography (PPG) or biosensors, providing medical guidance
to patients. This opens up opportunities to remotely support and assist many
individuals, particularly the elderly [33]. In controlled application, a rapid
and extensive level of support is offered, similar to the facilitates in
hospitals and intensive care units. This rapid and extensive level of support
involves integrating data from multiple sources, which include body sensors,
bedside monitors, and clinical observations. The information data is combined
as a closed-loop network with human intervention for decision-making. The
combination of these two healthcare application areas transforms the
healthcare system into one complex and vast safety-critical CPS. It also
offers many benefits like improved physiological status, patient safety, drug
research and development, and medical flow [34], [35].

##  IV Challenges of using CPS and AI in Agriculture, Autonomous Systems and
Healthcare

Several challenges have been identified based on the applications and benefits
associated with CPS and AI integration in the above-mentioned use cases. This
section explores the challenges in integrating AI and CPS for the applications
used in this study. Fig 2. shows the benefits and challenges associated to
understand the tradeoffs for using CPS and AI in the specific application
areas.

Figure 2: Summary of the studies used in the review

#### Challenges associated with CPS and AI in Agriculture

Transparency is crucial in AI, revealing how a system makes decisions. In
agriculture, Artificial Neural Networks (ANNs) are common for their
adaptability in remote sensing, despite opacity and high costs as complexity
rises. Balancing transparency, computational efficiency, and performance is
vital in selecting ML techniques for agricultural applications, notably in
remote sensing and precision agriculture. AI biases from skewed data or
excluding diverse farming practices pose challenges, causing injustices and
disadvantages to small-scale farmers. Opacity in ML systems raises trust and
responsibility concerns, with farmers facing dilemmas over contradictory AI
recommendations [36]. Cybersecurity is critical in modern agriculture,
especially concerning system security and data privacy. With the rise in
interconnected computers and electronic devices in agricultural production
facilities, assessing cybersecurity risks is necessary. This involves
considering all components and data communication, alongside detailing use
cases and stakeholder roles for a comprehensive cybersecurity assessment [37].
CPS integration also introduces vulnerabilities to hacking and cyber-attacks,
similar to other technological domains. Instances of ransomware attacks in
various industries highlight the potential for significant damage,
particularly as agricultural infrastructure becomes more electronically
reliant. Moreover, the concentration of ownership in the agricultural sector
increases susceptibility to cyber threats, with potential ramifications
extending to national food security and social order [36]. Resolving these
issues is vital for using AI in agriculture for benefits, as reluctance may
hinder progress.

#### Challenges associated with AI in autonomous systems

Similarly, extensive research has been carried out over the years to
investigate the potential challenges involved in designing, developing, and
deploying the robust and reliable autonomous system [4, 38, 39]. These studies
have identified both technical and non-technical challenges, such as security
issues, sensor noise and bias, sensor fusion techniques, sensor data quality,
fault tolerance, real-time data processing, data integration, trust
management, and system-human collaboration. Addressing these challenges is
imperative for the development of robust, trustworthy, and reliable autonomous
systems.

#### Challenges associated with CPS for healthcare

The challenges associated with CPS in healthcare are complex and critical for
optimizing performance and the safety of patient care. Representation of
medical device systems in CPS models integrates complex and advanced treatment
algorithms that interact with the physical system components, notably the
patient[34]. Their growing complexity and connectivity, pose security, safety
and dependability challenges for CPS-based medical devices. The
confidentiality and integrity of patient data are essential to maintain
privacy and security in healthcare. Moreover, unauthorized access or misuse of
such data can lead to reputational damage, loss of privacy, mental distress,
and subsequent health deterioration [40]. Similarly, CPS keeps incorporating
up-to-date versions of IoT [41] by integrating WSN [16], machine-to-machine
(M2M) communication, radio frequency identification, pervasive computing
technology, network communication tools, and novel control models [35]. These
CPS applications benefit significantly from the inclusion of intelligent
devices and wireless networks that facilitate sophisticated services informed
by data from the physical world [17]. A primary concern for handling large-
scale healthcare data involves establishing a robust distributed storage
architecture that can process and analyze large volumes of data [42]. These
limitations can delay complex query processing, reducing data transmission
loads and enabling context-sensitive predictive modeling [35]. Moreover, there
is a demand for high-performance CPS systems which include IoT systems,
incorporating immediate system feedback loops and ensuring interoperability
amongst medical devices [41],[42].

##  V Building Trust in AI and CPS for applications

Fig. 3 shows the technological landscape of XAI integration for building trust
in AI and CPS applications.

Figure 3: The technological landscape of explainable AI integration with CPS.

#### Building Trust for IoT applications in Agriculture

Risks associated with AI in agriculture, such as bias, lack of transparency,
and vulnerability to hacking, require attention from designers and
policymakers. Solutions to mitigate these risks include ensuring diverse
perspectives in AI training data, developing explainable AI, and implementing
robust cybersecurity measures. Additionally, governments may encourage local
data collection, incorporate indigenous agricultural knowledge, and establish
regulations to address privacy, surveillance, and data ownership concerns
[36].

A deeper understanding of AI, ethics, and stakeholder interests, along with
evidence-based approaches, is crucial for navigating the complexities of AI in
agriculture. Despite the potential of intelligent automation in the
agriculture domain, the risks of inheriting trust deficiency may overshadow
its applicability. First, the lack of data ownership and privacy may deeply
affect the farming community as the AI-based models may require insights from
a broader region which would eventually deprive the trade originality for
farmers. Next, a mere bias in the algorithm may cause additional usage of
resources, resulting in additional costs depending on the region. This bias
would eventually translate into varied produce pricing thus further
overburdening the community. Furthermore, the adoption of AI specifically in
rural communities may pose additional challenges; cost, learning curve, and
adaptability to name a few. Therefore, a deeper insight into AI decision-
making capabilities and the overall impact must be presented to the community
for adoption. In addition to that, a more curated use case must be presented
for better understanding, thus further advocating the need for XAI in the
smart agriculture domain. In regard to the adoption of these technologies, the
overwhelming dependency on these systems may marginalize autonomy in the
process, thus further diminishing the required trade skills. Finally, the
long-term implications, such as biodiversity, use of technology, land usage
changes, and continuous integration and dependency must be thoroughly
addressed. Therefore, in light of these challenges, the integration of XAI in
Smart Agriculture domain may promise to establish the trust and governance
required.

#### Building Trust in Autonomous systems

The integration of AI models enhances the operational capabilities of
autonomous systems, but it also brings significant trust challenges.
Reliability and robustness of these system depends on data quality, trust in
AI models, decision-making processes, and responses to uncertainties [43].

The AI-powered autonomous systems calls for high degree of trust. As these
systems are becoming increasingly integrated into our modern lives, ranging
from medical diagnosis to self-driving cars and even high-investment high-
frequency trading; a trust deficit can bring dire consequences. It underpins
the end-user as well as confidence in the operation, reliability, and
capability of these systems. It is crucial to understand that not the trust
factor but the ability to understand the model behavior and its decisions can
enable long-term fully integrated trustworthy AI systems. Imagine the absence
of trust in every day use-cases around us. For instance, a self-driving
vehicle that fails to detect a pedestrian crossing the road; an inaccurate
algorithm providing untrustworthy medical diagnosis, and even an AI algorithm
that fails to accurately invest in new emerging stock could potentially end up
causing billions in damages, and in worst case loss of lives. XAI can provide
deeper insights into AI behavior, its feature selection, ability to adapt to
future problems, and most importantly the rationale behind its decisions.

#### CPS Trust building for healthcare

CPS malfunctions lead to significant economic consequences and disrupt the
functionality of associated systems. These systems require physical
infrastructure and complex networking models. Because of diverse healthcare
applications, CPS are becoming more flexible, reliable, and complex.
Enhancement to CPS architecture offers significant improvement in
adaptability, scalability, usability, security, and safety. These advancements
mark a significant transition from conventional embedded systems. CPS must
establish trust in healthcare by providing transparent status and readings,
ensuring real-time performance.

Amongst all other domains, the application of AI in healthcare requires a
great degree of trust, privacy, security and strict governance policies. The
underlying insights of AI working are even more critical in this domain as
compared to other sectors. Modern healthcare systems heavily rely on the use
of AI models ranging the entire spectrum from diagnosis to prognosis including
initial diagnosis, treatment planning, insurance cover management, treatment,
and patient care. In this regard, trust in these systems cannot merely be
treated as a matter of convenience but in fact a matter of life and death. For
instance, consider an AI system that can classify a tumor with a very high
degree of accuracy but ruling out its malignant or benign nature could
potentially result in delay of treatment, and in worst cases the complete
absence of any treatment resulting in life-threatening scenarios.
Additionally, modern AI systems often advise on patient re-admissions or even
the probability of complications during surgery. If the decision-making
capabilities are opaque and are not validated, healthcare providers may
completely ignore to act upon them thus missing early intervention
possibilities. XAI in this regard may play a crucial role for identifying,
demystifying and addressing the shortcomings of this ”black-box” approach for
AI models. With XAI integrated into the AI systems, physicians and clinical
aid can better understand how a model reaches a certain conclusion, and
whether the high-accuracy figures actually relate to medical knowledge and are
free from bias and error. This will ensure collaborative communication between
humans and future AI models to gradually build trust and confidence in AI-
based decisions to complement rather than supplant human knowledge, expertise,
and judgment.

##  VI Conclusion

The paper explores the integration of Artificial Intelligence in Cyber-
Physical Systems, focusing on agriculture, autonomous systems, and healthcare
domains. In agriculture, use cases include soil management, pest and weed
management, precision farming, and smart dairy. For autonomous systems, use
cases cover UAVs, autonomous vehicles, and robotics. In healthcare, use cases
involve real-time health monitoring, resource utilization, and drug
manufacturing. The study aims to identify challenges such as security and data
privacy across these applications and discusses tradeoffs between advantages
and implementation challenges. Furthermore, it underscores the importance of
cyber security, transparency, fairness, and trustworthiness in AI-based
decision-making and reviews the role of Explainable AI (XAI) in addressing
these challenges and enhancing trust and reliability in AI-based decisions.

## References

  * [1] Sakhnini, J., Karimipour, H. (2020). AI and Security of Cyber Physical Systems: Opportunities and Challenges. In: Karimipour, H., Srikantha, P., Farag, H., Wei-Kocsis, J. (eds) Security of Cyber-Physical Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-45541-5-1
  * [2] Kaur D, Uslu S, Rittichier KJ, Durresi A. Trustworthy artificial intelligence: a review. ACM computing surveys (CSUR). 2022 Jan 18;55(2):1-38.
  * [3] Kirtan Jha, Aalap Doshi, Poojan Patel, Manan Shah, ”A comprehensive review on automation in agriculture using artificial intelligence”, Artificial Intelligence in Agriculture, Volume 2, 2019, Pages 1-12, ISSN 2589-7217, https://doi.org/10.1016/j.aiia.2019.05.004.
  * [4] Vachtsevanos, G., Lee, B., Oh, S., & Balchanos, M. (2018). Resilient design and operation of cyber physical systems with emphasis on unmanned autonomous systems. Journal of Intelligent & Robotic Systems, 91, 59-83.
  * [5] Ma, Y., Wang, Z., Yang, H., & Yang, L. (2020). Artificial intelligence applications in the development of autonomous vehicles: A survey. IEEE/CAA Journal of Automatica Sinica, 7(2), 315-329.
  * [6] Obermeyer Z, Emanuel EJ. Predicting the future – Big data, machine learning, and clinical medicine. N Engl J Med 2016;375:1216-9
  * [7] Institute of Medicine. 1993. Access to Health Care in America. Washington, DC: The National Academies Press. https://doi.org/10.17226/2009.
  * [8] Gawde, Shreyas, et al. ”Explainable Predictive Maintenance of Rotating Machines Using LIME, SHAP, PDP, ICE.” IEEE Access 12 (2024): 29345-29361.
  * [9] Petkovic, Dragutin. ”It is Not “Accuracy vs. Explainability”—We Need Both for Trustworthy AI Systems.” IEEE Transactions on Technology and Society 4.1 (2023): 46-53.
  * [10] Zolanvari, Maede, et al. ”Trust xai: Model-agnostic explanations for ai with a case study on iiot security.” IEEE internet of things journal 10.4 (2021): 2967-2978.
  * [11] B. A. Salau, A. Rawal and D. B. Rawat, ”Recent Advances in Artificial Intelligence for Wireless Internet of Things and Cyber–Physical Systems: A Comprehensive Survey,” in IEEE Internet of Things Journal, vol. 9, no. 15, pp. 12916-12930, 1 Aug.1, 2022, doi: 10.1109/JIOT.2022.3170449.
  * [12] J. Chae, S. Lee, J. Jang, S. Hong and K. -J. Park, ”A Survey and Perspective on Industrial Cyber-Physical Systems (ICPS): From ICPS to AI-Augmented ICPS,” in IEEE Transactions on Industrial Cyber-Physical Systems, vol. 1, pp. 257-272, 2023, doi: 10.1109/TICPS.2023.3323600.
  * [13] B. Et-taibi, M. R. Abid, I. Boumhidi and D. Benhaddou, ”Smart Agriculture as a Cyber Physical System: A Real-World Deployment,” 2020 Fourth International Conference On Intelligent Computing in Data Sciences (ICDS), Fez, Morocco, 2020, pp. 1-7, doi: 10.1109/ICDS50568.2020.9268734.
  * [14] Chen, J., Abbod, M., & Shieh, J. S. (2019). Integrations between autonomous systems and modern computing techniques: a mini review. Sensors, 19(18), 3897.
  * [15] Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., … & Tang, J. (2022). Trustworthy ai: A computational perspective. ACM Transactions on Intelligent Systems and Technology, 14(1), 1-59.
  * [16] A. Milenkovic, C. Otto, and E. Jovanov, “Wireless sensor networks for personal health monitoring: issues and an implementation,” Computer Communications, vol. 29, no. 13-14, pp. 2521–2533, 2006.
  * [17] J. Wan, D. Zhang, S. Zhao, L. T. Yang, and J. Lloret, “Context-aware vehicular cyber-physical systems with cloud support: Architecture, challenges, and solutions,” IEEE Commun. Mag., vol. 52, no. 8, pp. 106–113, Aug. 2014.
  * [18] Yilmaz, T., Foster, R., and Hao, Y., Detecting vital signs with wearable wireless sensors. Sensors 10(12):10837–10862, 2010.
  * [19] Ryan, M., Isakhanyan, G., & Tekinerdogan, B. (2023). An interdisciplinary approach to artificial intelligence in agriculture. NJAS: Impact in Agricultural and Life Sciences, 95(1). https://doi.org/10.1080/27685241.2023.2168568
  * [20] Guo, P., Dusadeerungsikul, P. O., & Nof, S. Y. (2018). Agricultural cyber physical system collaboration for greenhouse stress management. Computers and electronics in agriculture, 150, 439-454.
  * [21] albert De Vries, Nikolay Bliznyuk, Pablo Pinedo, ”Invited Review: Examples and opportunities for artificial intelligence (AI) in dairy farms*”, Applied Animal Science, Volume 39, Issue 1, 2023, Pages 14-22,ISSN 2590-2865.
  * [22] Sharad Agarwal, Awais Rashid, and Joseph Gardiner. 2022. Old MacDonald had a smart farm: Building a testbed to study cybersecurity in smart dairy farming. In Proceedings of the 15th Workshop on Cyber Security Experimentation and Test (CSET ’22). Association for Computing Machinery, New York, NY, USA, 1–9. https://doi.org/10.1145/3546096.3546097
  * [23] R. U. Mhapsekar, N. O’Shea, S. Davy and L. Abraham, ”Hybrid Blended Deep Learning Approach for Milk Quality Analysis,” in IEEE Transactions on Emerging Topics in Computational Intelligence, doi: 10.1109/TETCI.2024.3369331.
  * [24] Zhang, T., Li, Q., Zhang, C. S., Liang, H. W., Li, P., Wang, T. M., … & Wu, C. (2017). Current trends in the development of intelligent unmanned autonomous systems. Frontiers of information technology & electronic engineering, 18, 68-85.
  * [25] Maimaitijiang, M., Sagan, V., Sidike, P., Hartling, S., Esposito, F., & Fritschi, F. B. (2020). Soybean yield prediction from UAV using multimodal data fusion and deep learning. Remote sensing of environment, 237, 111599.
  * [26] Cai, Y., Guan, K., Peng, J., Wang, S., Seifert, C., Wardlow, B., & Li, Z. (2018). A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach. Remote sensing of environment, 210, 35-47.
  * [27] Urban, M., Aniculaesei, A., & Rausch, A. (2023, September). AI-Driven Runtime Monitoring of Energy Consumption in Autonomous Delivery Drones. In Intelligent Systems Conference (pp. 267-283). Cham: Springer Nature Switzerland.
  * [28] Munawar, H. S., Ullah, F., Qayyum, S., Khan, S. I., & Mojtahedi, M. (2021). UAVs in disaster management: Application of integrated aerial imagery and convolutional neural network for flood detection. Sustainability, 13(14), 7547.
  * [29] Sundaram, S., & Zeid, A. (2023). Artificial intelligence-based smart quality inspection for manufacturing. Micromachines, 14(3), 570.
  * [30] Pookkuttath, S., Rajesh Elara, M., Sivanantham, V., & Ramalingam, B. (2021). AI-enabled predictive maintenance framework for autonomous mobile cleaning robots. Sensors, 22(1), 13.
  * [31] Gangwani, D., & Gangwani, P. (2021). Applications of machine learning and artificial intelligence in intelligent transportation system: A review. Applications of Artificial Intelligence and Machine Learning: Select Proceedings of ICAAAIML 2020, 203-216.
  * [32] Nikolakis, N., Maratos, V., & Makris, S. (2019). A cyber physical system (CPS) approach for safe human-robot collaboration in a shared workplace. Robotics and Computer-Integrated Manufacturing, 56, 233-243.
  * [33] T. Taleb, D. Bottazzi, M. Guizani, and H. Nait-Charif, “Angelah: a framework for assisting elders at home,” IEEE Journal on Selected Areas in Communications, vol. 27, no. 4, pp. 480–494, 2009
  * [34] I. Lee, O. Sokolsky, S. Chen et al., “Challenges and research directions in medical cyber-physical systems,” Proceedings of the IEEE, vol. 100, no. 1, pp. 75–90, 2012.
  * [35] A. Banerjee, S. K. S. Gupta, G. Fainekos, and G. Varsamopoulos, “Towards modeling and analysis of cyber-physical medical systems,” in Proceedings of the 4th International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL ’11), pp. 154–158, October 2011
  * [36] Sparrow, R., Howard, M., & Degeling, C. (2021). Managing the risks of artificial intelligence in agriculture. NJAS: Impact in Agricultural and Life Sciences, 93(1), 172-196.
  * [37] Kristen E, Kloibhofer R, Díaz VH, Castillejo P. Security Assessment of Agriculture IoT (AIoT) Applications. Applied Sciences. 2021; 11(13):5841. https://doi.org/10.3390/app11135841
  * [38] Yazdanpanah, V., Gerding, E. H., Stein, S., Dastani, M., Jonker, C. M., Norman, T. J., & Ramchurn, S. D. (2023). Reasoning about responsibility in autonomous systems: challenges and opportunities. AI & SOCIETY, 38(4), 1453-1464.
  * [39] Hagos, D. H., & Rawat, D. B. (2022). Recent advances in artificial intelligence and tactical autonomy: Current status, challenges, and perspectives. Sensors, 22(24), 9916.
  * [40] Kocabas, O., Soyata, T., and Aktas, M. K., Emerging security mechanisms for medical cyber physical systems. IEEE/ACM Trans. Comput Biol. Bioinform. 13(3):401–416, 2016
  * [41] Gubbi, J., Buyya, R., Marusic, S., and Palaniswami, M., Internet ofThings (IoT): Avision, architectural elements, and future directions. Futur. Gener. Comput. Syst. 29(7):1645–1660, 2013
  * [42] F. Xia, L. Ma, J. Dong, and Y. Sun, “Network QoS management in cyber-physical systems,” in Proceedings of the International Conference on Embedded Software and Systems Symposia (ICESS ’08), pp. 302–307, July 2008.
  * [43] Martínez-Fernández, S., Franch, X., Jedlitschka, A., Oriol, M., & Trendowicz, A. (2021, May). Developing and operating artificial intelligence models in trustworthy autonomous systems. In International Conference on Research Challenges in Information Science (pp. 221-229). Cham: Springer International Publishing.