# Pretrain, Prompt, and Transfer: Evolving Digital Twins for Time-to-Event
Analysis in Cyber-physical Systems

Qinghua Xu
Department of Engineering Complex Software Systems
Simula Research Laboratory
Oslo, Norway
<EMAIL_ADDRESS>
&Tao Yue
Department of Engineering Complex Software Systems
Simula Research Laboratory
Oslo, Norway
<EMAIL_ADDRESS>
&Shaukat Ali
Department of Engineering Complex Software Systems
Simula Research Laboratory
Oslo, Norway
<EMAIL_ADDRESS>
&Maite Arratibel
Orona Group
Hernani, Basque Country, Spain
<EMAIL_ADDRESS>

###### Abstract

Cyber-Physical Systems (CPSs), e.g., elevator systems and autonomous driving
systems, are progressively permeating our everyday lives. To ensure their
safety, various analyses need to be conducted, such as anomaly detection and
time-to-event analysis (the focus of this paper). Recently, it has been widely
accepted that digital Twins (DTs) can serve as an efficient method to aid in
the development, maintenance, and safe and secure operation of CPSs. However,
CPSs frequently evolve, e.g., with new or updated functionalities, which
demand their corresponding DTs be co-evolved, i.e., in synchronization with
the CPSs. To that end, we propose a novel method, named PPT, utilizing an
uncertainty-aware transfer learning for DT evolution. Specifically, we first
pretrain PPT with a pretraining dataset to acquire generic knowledge about the
CPSs, followed by adapting it to a specific CPS with the help of prompt
tuning. Results highlight that PPT is effective in time-to-event analysis in
both elevator and ADSs case studies, on average, outperforming a baseline
method by 7.31 and 12.58 in terms of Huber loss, respectively. The experiment
results also affirm the effectiveness of transfer learning, prompt tuning and
uncertainty quantification in terms of reducing Huber loss by at least 21.32,
3.14 and 4.08, respectively, in both case studies.

##  1 Introduction

Cyber-Physical Systems (CPSs) serve as essential elements in actualizing the
vision of Industry 4.0 [1]. Unlike conventional physical systems, a typical
CPS incorporates a cyber component, linking physical systems through a
network. This combination of cyber and physical systems enables more
intelligent and adept industrial applications, especially in crucial
infrastructures such as transportation systems. However, the increasing
complexity, heterogeneity, and constantly evolving nature of CPSs, brought
about by introducing a rich array of functionalities, opens them up to
significant threats and challenges. This often renders existing security and
safety techniques ineffective, emphasizing the need to devise novel techniques
to ensure the dependability of various CPS tasks.

Among these tasks, time-to-event (TTE) analysis [2, 3], also known as survival
analysis, is of great importance, as CPSs are characterized by the interaction
of computational and physical processes, often facing uncertainty, and the
reliability of the systems is of paramount importance. TTE analysis allows for
modeling and predicting the time until certain events occur, such as
predicting the passenger waiting time in an elevator system and predicting
time-to-collision in aADS. TTE analysis can also help to understand and
quantify the reliability and operational resilience of the systems under
different conditions or in response to different threats. Therefore, applying
TTE analysis in CPSs can facilitate CPS operators, and other relevant
stakeholders, to take timely preventive measures, optimize resource
allocation, etc., so to make the systems safer and more efficient.

Digital Twins (DTs) have gained substantial attention in recent years by
performing safety and security tasks such as anomaly detection. Early works
[4, 5] rely heavily on rule-based models and domain expertise to construct
DTs, whereas data-driven DT construction is receiving increasing interest [6],
due to the success of applying machine learning in software engineering. The
efficacy of a DT function hinges on its synchronization with the real CPS,
which inspires researchers and practitioners to create a DT that faithfully
simulates the CPS. However, the continuous evolution of the CPS, e.g., due to
hardware or software updates, demands the evolution of its corresponding DT.
One straightforward solution is to train a new DT from scratch with data
collected from the updated CPS. However, data from the updated CPS is not
always guaranteed, such as in the case of a newly deployed elevator producing
limited data that is insufficient for deep learning training.

To combat the data scarcity, in our prior work, we proposed RISE-DT [7], an
uncertainty-aware transfer learning method to evolve DT for industrial
elevators. RISE-DT aims to transfer knowledge from the DT constructed for the
source elevator system to a target (new) elevator system. Concretely, RISE-DT
first employs uncertainty quantification (UQ) to select the most uncertain
samples, which are the most informative samples as well since they tend to be
close to the decision boundary. We then train a source DT and a target DT with
these samples. The transfer learning process minimizes the conditional and
marginal losses between the representations in the source and target DTs,
allowing knowledge to be transferred across the domains.

In this paper, we propose PPT to extend RISE-DT. Our key contributions are
three-fold. First, we improve the performance of the RISE-DT by employing
prompt tuning in PPT. Prompt tuning has emerged as an effective method for
tuning pretrained models, especially for large language models, to downstream
tasks [8]. Second, comparing with RISE-DT, we study two more UQ methods,
namely Bayesian and ensemble methods, to select the most suitable one for TTE
analysis. Third, we newly introduce an autonomous driving system (ADS) dataset
in our empirical study to demonstrate the generalizability of PPT. Hence, we
evaluate the application of PPT in two domains: elevator systems (vertical
transportation) and ADS (horizontal transportation). Experiment results show
that PPT is effective in TTE analysis in both elevator and ADS case studies,
averagely outperforming the baseline by 7.31 and 12.58, respectively, in terms
of Huber loss. We also dissect the individual contribution of each
subcomponent in PPT and find that prompt tuning, UQ, and transfer learning are
effective and efficient.

The rest of the paper is as below. We present the background and definitions
in Section 2. Section 3 delineates the architecture details of PPT. In
Sections 4 and 1, we show the design of our experiment and present the
results. Section 6 presents the related work and Section 7 concludes the
paper.

##  2 Background and Definitions

###  2.1 CPS evolution

Industry 4.0 [1] has advanced the digital transformation of the manufacturing
sector, mainly via CPSs. A typical CPS architecture has physical and cyber
elements, the symbiotic relationship facilitated through a feedback loop,
incorporating sensors, actuators, communication networks, and computational
units. The advances in developing these components, particularly computational
units, have spurred the widespread adoption of CPSs in our daily lives.

CPS evolution is often triggered by internal changes, such as upgrading old or
introducing new CPS functionalities. CPS behaviors are also closely
intertwined with their operating environment. Such an operating environment
can be very dynamic and uncertain, e.g., the driving environment of autonomous
vehicles, which subsequently influences their decision-making at runtime.
Therefore, we posit that a CPS should be studied along with its operating
environment. Formally, we define a subject system $\Sigma$ comprising the CPS
$\Psi$ and its environment $\Phi$ below:

$\Sigma:\Psi\rightleftharpoons\Phi$ (1)

Correspondingly, the evolution of the CPS is, thus, defined in Equation 2,
where $\Sigma_{S}$ and $\Sigma_{T}$ are the source and the target system of
the evolution, and $\Delta\Psi_{S}$ and $\Delta\Phi_{S}$ represent changes in
the CPS and the environment, respectively.

$\Sigma_{S}\xrightarrow[\Delta\Psi_{S}]{\Delta\Phi_{S}}\Sigma_{T}$ (2)

###  2.2 TTE analysis in industrial elevators and ADSs

TTE analysis tasks, in general, can be described as a 4-tuple:
$\langle\Sigma,\mathcal{D},\mathcal{E},\tau\rangle$, where
$\Sigma,\mathcal{D},\mathcal{E},\tau$ represent the subject system, dataset,
events of interest, and time interval. TTE analysis analyses dataset
$\mathcal{E}$ collected from $\Sigma$ to predict time interval $\tau$ after
which event $\mathcal{E}$ will occur (Equation 3).

$f:\mathcal{D}\mapsto{\tau_{\mathcal{E}}}$ (3)

Industrial elevators are vertical transportation systems for buildings and are
essential for modern urban lives. The cyber aspect of an elevator, such as the
control algorithm, is encapsulated in the elevator software, while the
physical components, including elements like motors, brakes, and cables,
facilitate the movement of the elevator. Typically, each building has multiple
elevators deployed and controlled by dedicated controllers, which are
connected to a traffic master with a dispatcher (i.e., software) scheduling
elevator operations to optimize the Quality of Services (QoS) to deliver. A
common TTE analysis is about predicting the waiting time of each passenger
based on information such as arrival floor, destination floor, weight, and
capacity. We formally define the elevator TTE analysis task as in Definition
2.1.

######  Definition 2.1 (Industrial Elevator TTE Analysis)

$\displaystyle\Sigma^{E}$ $\displaystyle\mapsto{\text{An elevator system and
its environment}}$ (4) $\displaystyle\mathcal{D}^{E}$
$\displaystyle\mapsto{\text{A sequence of passenger information}}$
$\displaystyle\mathcal{E}^{E}$ $\displaystyle\mapsto{\text{Passengers arrive
at their destinations}}$ $\displaystyle\tau^{E}$
$\displaystyle\mapsto{\text{Estimated arrival time}}$

Elevator dispatchers vary between buildings, and their usage patterns depend
on traffic factors such as building type, time of day, and day of the week
(known as traffic template). Thus, both elevator dispatchers and their
environment evolve, which affects TTE analysis’s performance. We define the
evolution directions in Definition 2.2, where $\Delta\Psi^{E}_{S}$ represents
CPS changes, that is, dispatcher version changes, and $\Delta\Phi^{E}_{S}$
denotes environment changes, i.e., traffic template changes.

######  Definition 2.2 (Industrial Elevator Evolution)

$\Sigma^{E}_{S}\xrightarrow[\Delta\Psi^{E}_{S}]{\Delta\Phi^{E}_{S}}\Sigma^{E}_{T}$
(5)

ADSs, as another type of CPSs, are equipped with various sensors, such as
optical and thermographic cameras, radar, lidar, and GPS. Their cyber part is
mainly responsible for planning and controlling the vehicles’ behaviour.
Specifically, an ADS relies on sensors to perceive its environment (e.g., road
conditions, weather conditions, and other vehicles), which are then utilized
to guide the ADS’s decision-making, e.g., determining an appropriate
navigation path and formulating strategies to manage traffic controls (e.g.,
stop signs) and obstacles. TTE analysis such as predicting the time to a
potential collision (known as time-to-collision) can facilitate the ADS to
make well-informed decisions, which is defined as in Definition 2.3.

######  Definition 2.3 (ADS TTE Analysis)

$\displaystyle\Sigma^{A}$ $\displaystyle\mapsto{\text{An ADS and its running
environment}}$ (6) $\displaystyle\mathcal{D}^{A}$
$\displaystyle\mapsto{\text{A sequence of vehicle and environment
properties}}$ $\displaystyle\mathcal{E}^{A}$
$\displaystyle\mapsto{\text{Vehicle collisions}}$ $\displaystyle\tau^{A}$
$\displaystyle\mapsto{\text{Estimated collision time}}$

ADS behaviours differ under varying driving conditions, including weather
conditions and behaviours of nearby vehicles. In this work, we concern the
evolution of ADSs under different driving conditions as depicted in Definition
2.4, an area widely studied in ADS testing [9, 10].

######  Definition 2.4 (ADS Evolution)

$\Sigma_{S}^{A}\xrightarrow{\Delta\Phi^{A}_{S}}\Sigma_{T}^{A}$ (7)

###  2.3 Digital Twin

El Saddik [11] has defined DT as a digital replica of a physical entity. Yue
et al. [12] extended this definition and proposed a DT conceptual model (see
Figure 1). In the CPS context, a CPS (e.g., an elevator system or an ADS) is
considered the physical twin. A typical DT comprises two key components: a
Digital Twin Model (DTM) and a Digital Twin Capability (DTC). The DTM is a
digital representation of the CPS, including heterogeneous models
corresponding to various components, e.g., software, hardware, and
communication. The DTC is the DT’s functionality, e.g., predicting non-
functional properties, detecting uncertainties, and preventing failures. PPT
adopts this conceptual model for DT construction and evolves the DTM and DTC.

Figure 1: Digital Twin for Cyber-Physical System

###  2.4 Uncertainty Quantification

UQ is intensively studied in both academia and industry, underpinning numerous
applications such as trustworthy decision-making [13] and software risk
analysis [14]. In our context, we use UQ to select the most uncertain samples
$U$ from a dataset $\mathcal{D}$. To that end, UQ assigns an uncertainty score
$\xi$ to each sample $x\in\mathcal{D}$. We have defined a comprehensive UQ
metric CS score in our prior work [7]. In this paper, we further investigate
UQ by adding two mainstream UQ approaches: Bayesian and ensemble UQ. Let
$\mathcal{M}_{I}$ be an indicator model which assesses the uncertainty of each
sample and shares the same structure as the DTM (Section 3.1.2). We introduce
each UQ approach below.

CS score combines the calibration and sharpness metrics. Calibration
represents the consistency between the prediction distribution and the
observation, while sharpness assesses the concentration of the prediction
distribution [15]. By combining these two metrics with a weighted sum
parameterized by $\lambda$ (decided empirically), we follow [7] and define the
comprehensive uncertainty metric CS score $\xi^{cs}$ as shown in Equation 8.

$\xi_{i}^{ut}=\lambda c(x_{i})+(1-\lambda)s(x_{i})$ (8)

Bayesian Method probabilistically interprets predictions, which can be
leveraged to derive UQ metrics. One popular Bayesian method for UQ in neural
network models is the Monte Carlo (MC) dropout. The MC dropout randomly sets
the activation of neurons to 0 with a fixed probability for a subset of
layers, resulting in a set of indicator models with dropout
$\\{\mathcal{M}^{B}_{d}\\}_{d=1}^{N_{B}}$, where $N_{B}$ is the number of
indicator models. Dropout represents randomly set some neurons inactive in one
indicator model. Each indicator model makes an individual prediction, and we
define the uncertainty of each sample as the standard deviation of these
predictions as in Equation 9.

$\displaystyle\bar{y}_{i}$
$\displaystyle=\frac{1}{N_{B}}\sum_{d=1}^{N_{B}}\mathcal{M}_{d}^{B}(x_{i})$
(9) $\displaystyle\xi_{i}^{bm}$
$\displaystyle=\sqrt{\frac{1}{N_{B}}\sum_{d=1}^{N_{B}}(\mathcal{M}_{d}^{B}(x_{i}-y_{i})^{2}}$
(10)

Ensemble Method is a commonly used technique in machine learning to combat
overfitting by training multiple models with different configurations
simultaneously. Building on the idea of ensemble learning, ensemble UQ divides
the dataset into $N^{E}$ subsets and trains a distinct indicator model $M^{E}$
on each subset. Similar to the MC dropout approach, each indicator model
generates predictions independently, and the uncertainty of each sample
$x_{i}$ is determined by calculating the standard deviation of the
predictions, as shown in Equation 11.

$\displaystyle\bar{y}_{i}$
$\displaystyle=\frac{1}{N_{E}}\sum_{d=1}^{N_{E}}\mathcal{M}_{d}^{E}(x_{i})$
(11) $\displaystyle\xi_{i}^{em}$
$\displaystyle=\sqrt{\frac{1}{N^{E}}\sum_{d=1}^{N_{E}}(\mathcal{M}_{d}^{E}(x_{i}-y_{i})^{2}}$
(12)

##  3 Approach

Figure 2: Overall architecture of PPT

PPT is a closed-loop deep learning approach, which requires training on the
relevant dataset. We introduce the architecture of PPT in Section 3.1 and the
training method in Section 3.1.3.

###  3.1 Overall Architecture

Figure 2 depicts the overall architecture of PPT, comprising the Data
Processing Component, the Digital Twin Component and the Transfer Learning
Component (denoted as TL component in the figure). Let the source and target
subject systems be $\Sigma_{S}$ and $\Sigma_{T}$ and their corresponding
datasets be $\mathcal{D}_{S}$ and $\mathcal{D}_{T}$. The data processing
component takes as input $\mathcal{D}_{S}$ and $\mathcal{D}_{T}$ and selects
the most uncertain contextualized samples $CU_{S}$ and $CU_{T}$. The source
and target digital twin components use these samples to construct a source DT
(SDT) and a target DT (TDT). Finally, the Transfer Learning Component
transforms hidden representations in SDT and TDT into shared intermediate
spaces, which signify the shared knowledge between the source and target
domain.

####  3.1.1 Data Processing Component

The data processing component takes source and target data ($\mathcal{D}_{S}$
and $\mathcal{D}_{T}$) as input and selects the most uncertain samples with
their context information using the data processing module. According to
Information Theory, higher uncertainty entails richer information [16]. Hence,
machine learning models can greatly benefit from training with more
informative samples [17].

We illustrate the details of the data processing module in Figure 3. Given
datasets $\mathcal{D}_{1},\mathcal{D}_{2},...,\mathcal{D}_{N}$, we utilizes an
uncertainty quantification module to select the most uncertain samples
$U_{1},U_{2},...,U_{N}$. However, the context information of these samples is
also critical for TTE analysis since each sample is dependent on previous
samples and influences future samples. Picking out the most uncertain samples
has the possibility of losing the context information. As a remedy, we use a
multi-head attention module to calculate representations
$CU_{1},CU_{2},...,CU_{N}$ that fuse information from both the sample itself
and its context. We will detail uncertainty quantification module and multi-
head attention module in the rest of this section.

Figure 3: Structure of the data processing module

Uncertainty Quantification Module ranks and selects the most uncertain
samples. In our study, we explored three UQ methods: CS score, Bayesian and
ensemble UQ methods and select the most suitable one to perform UQ. UQ assigns
an uncertainty score $\xi_{i}\in\Xi$ to each sample $x_{i}\in\mathcal{D}$. We
then rank all samples based on their scores and select the top $K$ ranked
samples for transfer learning (Equation 13).

$U=topK(\mathcal{D},key=\Xi)$ (13)

Multi-head Attention Module UQ selects the most uncertain samples out from
dataset $\mathcal{D}$. However, the CPS data used in this study is contextual,
meaning that considering only the uncertain samples in isolation could
potentially harm our model’s performance. To preserve the contextual
information, we employed a technique called multi-head self-attention (MHSA).
We project the input data into a hidden space, where each vector contains
information about both the input itself and its surrounding context. This
approach allows our model to incorporate relevant contextual information, and
hence make more accurate predictions.

To facilitate parallel computation, MHSA discards the positional information,
which is critical in our case. As a remedy, we follow the common practices and
encode such information with a positional vector as in Equation 14.

$U=concat(U,PosEnc(U))$ (14)

We then utilize linear transformations to map the input matrix $U$ to three
distinct spaces, creating three new matrices: $Q$, $K$, and $V$. $Q$ and $K$
are the query and key matrices, and $V$ is the new vector representation of
the input. By multiplying $Q$ and $K$, we generate an attention weight matrix,
which is subsequently multiplied with $V$ and scaled with a softmax function
and a scaling factor of $d_{k}$, as formulated in Equation 15.

$\displaystyle Q$ $\displaystyle=W_{Q}U_{in}+b_{Q}$ (15) $\displaystyle K$
$\displaystyle=W_{K}U_{in}+b_{K}$ (16) $\displaystyle V$
$\displaystyle=W_{V}U_{in}+b_{V}$ (17) $\displaystyle U_{att}$
$\displaystyle=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V$ (18)

To enhance the representation, we apply a feed-forward neural network to
$U_{att}$. This network has a ReLU activation layer and a linear layer
(Equation 19). The trainable weight matrices $W_{ffn}$ and $b_{ffn}$ are used
in this transformation.

$U_{ffn}=W_{ffn}\cdot ReLU(U_{att})+b_{ffn}$ (19)

To mitigate the problem of catastrophic forgetting in deep learning models, we
utilize a residual connection (Equation 20), i.e., summing $U_{ffn}$ and
$U_{att}$.

$CU=U_{att}+U_{ffn}$ (20)

####  3.1.2 Digital Twin Component

As the backbone model of PPT, a generic DT has a DTM simulating the physical
twin and a DTC with functionalities, e.g., waiting time prediction.

DTM aims to approximate the underlying distribution of input data
($\mathcal{D}$). It has three layers: the transformer, GRU and prediction
layers, which extract features from each sample, capture temporal features and
project the intermediate representation vectors into the sample space to
predict the next sample with CNN, respectively.

The transformer layer takes the contextualized uncertain samples $CU$ as input
and feeds them into a stack of $L$ MHSA modules. Each MHSA module takes the
output of the prior MHSA module as input (Equation 21).

$\displaystyle CU^{M}_{1}$ $\displaystyle=CU$ (21) $\displaystyle CU_{L}^{M}$
$\displaystyle=MHSA(CU^{M}_{L-1})$

The GRU layer takes the output of the transformer layer as input. For each
data sample $x_{i}\in CU^{M}_{L}$, the layer computes the hidden
representation ($H^{G}_{M}$) using Equation 22.

$\displaystyle z_{t}$
$\displaystyle=\sigma_{g}(W_{z}x_{t}+U_{z}h_{t-1}+b_{z})$ (22) $\displaystyle
r_{t}$ $\displaystyle=\sigma_{g}(W_{r}x_{t}+U_{r}h_{t-1}+b_{r})$ (23)
$\displaystyle\hat{h}_{t}$ $\displaystyle=\phi_{h}(W_{h}x_{t}+U_{h}(r_{t}\cdot
h_{t-1})+b_{h})$ (24) $\displaystyle H^{G}_{M}[t]$
$\displaystyle=(1-z_{t})\cdot h_{t-1}+z_{t}\cdot\hat{h}_{t}$ (25)

The CNN layer is responsible for predicting the next state vector for the
subject system, where $S$ is the size of the state vector. Each of its
dimensions is a continuous scalar value. However, direct training with
continuous labels can result in overfitting issues with insufficient data. To
overcome this, we discretize continuous scalar values into 10 categories,
thereby transforming these continuous prediction tasks into classification
tasks. The core operation of the CNN layer is the kernel convolution, which
calculates the probability $P_{i,j}$ for $i$th label on the $j$th dimension
(Equation 26). $\mathcal{K}$ denotes the convolution kernel.

$P_{i,j}=\sum_{m}\sum_{n}X_{i-m,j-n}\cdot\mathcal{K}_{m,n}$ (26)

We hence predict the next data samples by assigning labels to each dimension
with the highest probabilities (Equation 27).

$CU^{M}=argmax(P_{i,j})$ (27)

DTC performs TTE analysis. Figure 2 presents the design of the source DTC
(SDTC) and target DTC (TDTC), whose architectural designs are identical. In
the following part, we only illustrate SDTC’s architecture for brevity. SDTC
combines the real data $\mathcal{D}_{S}$ and predicted data
$\mathcal{D}_{S}^{\prime}$ as input and feeds it into three layers
sequentially: the transformer, GRU and prediction layers.

The transformer layer concatenates the real sample $CU$ and predicted sample
$CU^{M}$ and feeds it to a size-L stack of MHSA modules, which are computed
recursively (Equation 28).

$\displaystyle CU^{C}_{1}$ $\displaystyle=concat([CU,CU^{M}])$ (28)
$\displaystyle CU^{C}_{L}$ $\displaystyle=MHSA(CU^{C}_{L-1}))$

The GRU layer captures the dependency between the current input and previous
inputs (Equation 29). The detailed structure of GRU has been described in
Equation 22, where $H^{G}_{C}$ denotes the output of DTC’s GRU layer.

$H^{G}_{C}=GRU(CU^{C}_{L})$ (29)

The prediction layer transforms the intermediate representations into the
estimated time (e.g., passenger waiting time in the elevator case study and
time-to-collision in the ADS case study) as in Equation 30, where $W_{\tau}$
and $b$ are weight matrices.

$\hat{\tau}=W_{\tau}^{T}H^{G}_{C}+b_{\tau}$ (30)

####  3.1.3 Transfer Learning Component

As shown in Figure 2, PPT uses a projection layer to map the hidden
representations in SDT and TDT to shared spaces.

The projection layer first uses a linear transformation to map the hidden
representations ($H$) to representations $H^{P}$ in the shared spaces and
benefits from an activation function $tanh$ to add non-linearity (Equation
31).

$H^{P}=tanh(W_{P}H+b_{P})$ (31)

Then, we perform transfer learning by aligning SDT and TDT in the intermediate
spaces, aiming to reduce marginal and conditional losses.

Marginal loss is calculated as Kullback-Leibler (KL) divergence between hidden
layer representations of SDT and TDT. We first project the GRU layer outputs
of SDTM, SDTC, TDTM and TDTC ($H^{G}_{SM}$, $H^{G}_{SC}$, $H^{G}_{TM}$, and
$H^{G}_{TC}$) into an intermediate space using Equation 31, yielding
$H^{PG}_{SM}$, $H^{PG}_{TM}$, $H^{PG}_{SC}$ and $H^{PG}_{TC}$, respectively.
We then calculates the marginal loss in the intermediate space as in Equation
32 .

$\displaystyle\mathcal{L}_{mar}^{M}$
$\displaystyle=\sum_{t}{H^{G}_{SM}[t]\cdot logH^{G}_{TM}[t]}$ (32)
$\displaystyle\mathcal{L}_{mar}^{C}$
$\displaystyle=\sum_{t}{H^{G}_{SC}[t]\cdot logH^{G}_{TC}[t]}$

Conditional loss $\mathcal{L}_{cond}^{M}$ is calculated between the prediction
layer representations of SDTC and TDTC. We first transform the output of SDTC
and TDTC into an intermediate space as in Equation 33.

$\displaystyle P_{S}^{P}$ $\displaystyle=proj(P_{S})$ (33) $\displaystyle
P_{T}^{P}$ $\displaystyle=proj(P_{T})$

We then calculate the Maximum Mean Discrepancy (MMD) in the intermediate space
as in Equation 34.

$\mathcal{L}_{cond}^{M}=||\frac{1}{n^{s}}\sum_{i=1}^{{n^{s}}}{P_{S}^{P}[i]}+\sum_{i=1}^{{n^{t}}}{P_{T}^{P}[i]}\frac{1}{n^{t}}||$
(34)

###  3.2 Training Process of PPT

This process includes the pretraining phase (Section 3.2.1) and prompt tuning
(Section 3.2.2) phase. The former induces better initializations for the model
parameters, while the later quickly adapts the model to the target subject
system.

####  3.2.1 Pretraining Phase

Neural network methods, including PPT, tend to be trapped in local optimal
easily. Pretraining on large datasets can alleviate this issue by inducing the
optimizer more towards the global optimal. In this phase, we aim to find the
optimal parameters for PPT, as described in Algorithm 1. We collect source and
target dataset pairs
$\langle\mathcal{D}_{S}^{pre},\mathcal{D}_{T}^{pre}\rangle$ and output the
pretrained $SDT^{pre}$ and $TDT^{pre}$ (Lines 2-3). In each pair, we first
perform UQ to select the $K$ most uncertain samples for transfer learning
(Lines 4-9). SDT and TDT take these samples as input and make predictions
(Lines 10-13). We calculate the marginal loss and conditional loss (Lines
14-15) to accomplish transfer learning. Additionally, we calculate the Huber
loss between the predicted TTE ($\tau^{\prime}_{S}$ and $\tau^{\prime}_{T}$)
and the real TTE ($\tau_{S}$ and $\tau_{T}$). Minimizing Huber loss (Section
4.3) can induce the DTC to make more accurate TTE analysis. The last step of
Algorithm 1 minimizes all losses by adjusting the model parameters (Line 18).

Input: $\Sigma^{SPre}$ and $\Sigma^{TPre}$: source and target subject systems;
$N$: Number of source and target system pairs.

Output: $SDT^{pre}$ and $TDT^{pre}$: the pretrained source and target DTs.

1

2for  _i in 1:N_ do

3 $\mathcal{D}^{S}_{i}$=collect_from($\Psi^{SPre}_{i}$);

4 $\mathcal{D}^{T}_{i}$=collect_from($\Psi^{TPre}_{i}$);

/* Data processing */

5 $\omega^{S}$=UQ($\mathcal{D}^{SPre}_{i}$);

6 $\omega^{T}$=UQ($\mathcal{D}^{TPre}_{i}$);

7 $U^{S}=topK(\mathcal{D}^{SPre}_{i},key=\omega_{i}^{S}$);

8 $U_{T}=topK(\mathcal{D}^{TPre}_{i},key=omega_{i}^{T}$);

9 $CU^{S}$=MHSA($U^{S}$);

10 $CU^{T}$=MHSA($U^{T}$);

/* Train $SDT$ and $TDT$ with transfer learning */

11 $H_{SM}^{G},P_{S},CU_{SM}=SDTM(CU_{S})$;

12 $H_{TM}^{G},P_{T},CU_{TM}=SDTM(CU_{T})$;

13 $H_{SC}^{G},\hat{\tau}_{S}=SDTC(CU_{S},CU_{SM})$;

14 $H_{TC}^{G},\hat{\tau}_{T}=TDTC(CU_{T},CU_{TM})$;

15

16 $\mathcal{L}_{mar}=mar(H_{SM}^{G},H_{TM}^{G})+mar(H_{SC}^{G},H_{TC}^{G})$;

17 $\mathcal{L}_{marginal}=conditional(P_{S},P_{T})$;

18
$\mathcal{L}_{huber}=huber(\hat{\tau}_{S},\tau_{S})+huber(\hat{\tau}_{T},\tau_{T})$;

19 $\mathcal{L}=\mathcal{L}_{huber}+\mathcal{L}_{cond}+\mathcal{L}_{mar}$;

20 $minimize(\mathcal{L})$;

21

22 end for

Algorithm 1 Pretraining Phase of PPT

####  3.2.2 Prompt Tuning Phase

Figure 4: PPT’s prompt tuning. Note that we only display components related to
prompt tuning and omit the detailed DT structure and UQ strategies for brevity
since it is identical to that in the pretraining phase.

The pretrained DTs are trained on the pretraining datasets
$\mathcal{D}_{pre}$, which does not include the dataset collected from the
target subject system $\mathcal{D}_{T}$. To acquire expertise in the target
subject system $\Sigma$, supervised learning with a sufficient dataset
collected from $\Sigma$ is required. For this purpose, we employed the fine-
tuning technique in our previous work RISE-DT [7]. However, recent research
has shown that prompt tuning can be even more effective [18]. Hence, in PPT,
we employ prompt tuning to enhance its overall performance. Prompt tuning
involves designing prompts to test a pretrained model’s ability to distinguish
between the source and target domain data. The feedback from the test helps
the model learn more about the salient features in the target domain,
potentially leading to improved performance. A typical prompt tuning phase has
three steps: prompt template designing, answer generation, and answer mapping
[19], as depicted in Figure 4.

Step 1: Prompt template designing. Cloze-style prompts are a well-studied
technique, where certain parts of the data are masked, leaving a blank for the
model to fill in. At each time point $i$, we collect a L-lengthed sequence of
CPS data $x_{i},x_{i-1},x_{i-2},...,x_{i-L+1}$. We generate a prompt template
by masking the time interval $\tau_{i}$ for the current time point $i$ and the
previous one $i-1$ (denoted as "[ MASK ]" in Figure 4). Using this template,
we generate a positive prompt, where we fill in the true time interval
$\tau^{T}_{i-1}$ for time point $i-1$ from the target domain (denoted as
$\tau_{T}$ in Figure 4), and a negative prompt, where we fill in the same
blank with the time interval predicted with SDTC (denoted as $\tau^{S}_{i-1}$
in Figure 4).

Step 2: Answer generation. We ask the target DTC (TDTC) to fill in the blank
in both positive and negative prompts. We hypothesize that SDTC captures
knowledge about the source domain, while TDTC captures knowledge about the
target domain. Therefore, we expect to make accurate predictions on the
positive prompt while making noticeable errors when predicting the negative
prompt, as it has been "tampered" by SDTC. As shown in Figure 4, TDTC fills in
the two prompts, yielding a positive answer (denoted as Positive Answer in
Figure 4) and a negative answer (denoted as Negative Answer in Figure 4). We
calculate the Huber loss by comparing these two answers with the actual time
interval $W\tau_{i}$ using Equation 35. Note that we reverse the sign of the
Huber loss for the negative prompt by multiplying it by $-1$ because we assume
a well-adapted TDTC should be able to distinguish between source and target
domain data. This approach helps the model learn the salient features of the
target domain and improves its performance, for instance, in predicting
waiting times for elevator passengers in the target domain for our elevator
case study and predicting the time-to-collision in the target domain for the
ADS case study.

$\mathcal{L}_{prompt}=(\tau-\tau_{+})^{2}-(\tau-\tau_{-})^{2}$ (35)

Step 3: Answer mapping. Compared to fine-tuning, prompt tuning can reduce or
even obviate the need for extra model extensions. In prompt tuning, downstream
task predictions are acquired by mapping prompt answers to the prediction
space. In our context, we do not need to perform such mapping since the
positive prompt prediction can be considered as TTE analysis directly.

Algorithm 2 describes the prompt tuning phase in pseudo-code. For each time
point $i$, we consider not only the current data point but also history data
points within a time window of $K$. We generate positive and negative prompts
with the help of the prompt generator (Lines 3-5). SDT and TDT make
predictions with $\mathcal{D}_{S}$ and the latest $\omega$ $\mathcal{D}_{T}$
data, respectively (Lines 6-9). In Lines 10-12, TDTC fills in the positive
prompt and negative prompt. We calculate the prompt loss function as in
Equation 35, and optimize the parameters of DTs stochasticly (Lines 13-14).

Input: $\mathcal{D}_{S}$ and $\mathcal{D}_{T}$: source and target subject
system data; $SDT_{pre}$ and $TDT_{pre}$: pretrained source and target DTs;
$K$: history window size

1

Output: $TDT$: Trained target DT

2 $i=0$;

3 while  _True_ do

/* Generate prompt and prompt answer */

4 $\rho=PromptGenerator(\mathcal{D}_{T}[i-\omega:i+1])$;

5 $\rho_{+}=fill\\_in(\rho,\mathcal{D}_{T}[i])$;

6 $\rho_{-}=fill\\_in(\rho,SDTC(\mathcal{D}_{T}[i-\omega:i]))$;

/* Train SDT and TDT with transfer learning and prompt tuning */

7 $H_{SM}^{G},P_{S},CU_{SM}=SDTM(CU_{S})$;

8 $H_{TM}^{G},P_{T},CU_{TM}=SDTM(CU_{T})$;

9 $H_{SC}^{G},\hat{\tau}_{S}=SDTC(CU_{S},CU_{SM})$;

10 $H_{TC}^{G},\hat{\tau}_{T}=TDTC(CU_{T},CU_{TM})$;

11 $\tau_{+}=TDTC(\rho_{+})$;

12 $\tau_{-}=TDTC(\rho_{-})$;

13 $\tau=\mathcal{D}_{T}[i]$;

14 $\mathcal{L}_{prompt}=(\tau-\tau_{+})^{2}-(\tau-\tau_{-})^{2}$;

15 $minimize(\mathcal{L}_{prompt})$;

16 $i=i+1$;

17

18 end while

Algorithm 2 Prompt Tuning Phase of PPT

##  4 Experiment Design

In Section 4.1, we introduce four research questions, followed by detailing
the case studies in Section 4.2. Section 4.3 shows the evaluation metrics,
while Section 4.4 introduces the statistical tests employed. Finally, we
introduce the settings and execution environment in Section 4.5.

###  4.1 Research Questions (RQs)

In this paper, we plan to answer four RQs as follows.

  * •

RQ1: How effective is PPT in TTE analysis, as compared to RISE-DT?

  * •

RQ2: How efficient and effective is transfer learning?

  * •

RQ3: Does UQ help to improve the performance of transfer learning? If so,
which UQ method is the best?

  * •

RQ4: Is prompt tuning effective and efficient for improving the performance of
transfer learning?

RQ1 aims to compare PPT with the baseline, i.e., RISE-DT in TTE analysis.
RQ2-RQ4 dissect PPT and assess the cost-effectiveness of introducing transfer
learning, UQ, and prompt tuning to it. Specifically, RQ2 evaluates the
efficiency and effectiveness of transfer learning by comparing the performance
of PPT with/without transfer learning (denoted as w/o TL). With RQ3, we study
the impact of using or not using UQ on the performance of PPT (denoted as w/o
UQ) and select the most suitable UQ method from three UQ methods: CS score,
Bayesian method and ensemble methods (denoted as CS, BUQ and EUQ,
respectively). With RQ4, we plan to assess the improvement brought about by
prompt tuning by comparing PPT with and without prompt tuning (denoted as w/o
PT).

###  4.2 Case Studies

####  4.2.1 Orona Elevator System

Orona elevator system was studied in our prior work, where we evaluate RISE-DT
with 11 versions dispatchers $d_{*},d_{1},d_{2},...,d_{10}$, and two traffic
templates, i.e., Lunchpeak $\Gamma_{L}$ and Uppeak traffic templates
$\Gamma_{U}$. Notice the dispatcher $d*$ denotes the best dispatcher and
$d_{1:10}$ denotes ten previous versions. In this work, we acquire 20 more
dispatchers $d_{11},d_{12},...,d_{30}$ for a more comprehensive evaluation of
PPT. In total, we have access to 62 different subject systems
$\Sigma^{E}_{1}=\langle d_{*},\Gamma_{L}\rangle,\Sigma^{E}_{2}=\langle
d_{*},\Gamma_{L}\rangle,\Sigma^{E}_{3}=\langle
d_{1},\Gamma_{L}\rangle,...,\Sigma^{E}_{32}=\langle
d_{30},\Gamma_{L}\rangle,\Sigma^{E}_{33}=\langle
d_{1},\Gamma_{U}\rangle,...,\Sigma^{E}_{62}=\langle d_{30},\Gamma_{U}\rangle$.
We collect 62 datasets $\mathcal{D}_{1},..,\mathcal{D}_{62}$ from these
subject systems by performing simulation on Elevate, a commercial simulator
used by Orona to test their dispatchers in software in the loop simulation
environment. These 62 subject systems can be categorized into four types of
subject systems: LunchBest (or LunchWorse) denoting that the elevator
dispatcher with the highest performance (or sub-par) operates during lunch
rush (12:15 - 13:15 p.m.); UpBest (or UpWorse) representing that the best (or
an under-performing) elevator dispatcher operates during the morning rush hour
(8:30 a.m. - 9:30 a.m.).

Evolution dataset construction, in this case study, encompasses four types of
evolutions: (1) $LunchBest\rightarrow UpBest$;(2) $UpBest\rightarrow
LunchBest$; (3) $LunchWorse\rightarrow LunchBest$; (4) $UpWorse\rightarrow
UpBest$. (1) and (2) are dispatcher-variant evolutions where the dispatcher
has undergone changes in the evolution. (3) and (4) are traffic-variant
evolutions, where the source and target subject systems only differ in traffic
templates, while the elevator dispatcher remains unchanged.

####  4.2.2 Autonomous Driving Systems Dataset

ADS dataset is taken from DeepScenario [20]– an open-source dataset containing
33530 driving scenarios. These scenarios were generated with different
strategies (e.g., greedy search and reinforcement learning) to achieve various
objectives (e.g., reducing time-to-collision and distance to obstacles). Each
driving scenario is characterized by the properties and behaviors of the self-
driving car under study (known as the ego vehicle) and other objects in the
driving environment, such as pedestrians and other cars (known as NPC
vehicles). One example can be described as "A red BoxTruck is overtaking the
ego vehicle and maintaining the lane." DeepScenario provides us with 19
features of the ego and NPC vehicles with regard to their speed, location,
rotation, etc. The complexity of driving scenarios differs in terms of the
number of NPCs involved. For example, a driving scenario without any NPC
vehicle is much less challenging for the ADS of the ego vehicle to make
decisions as compared to a scenario with NPC vehicles around. We acquired two
datasets with different complexity levels from DeepScenario. We name the
dataset with fewer NPC vehicles on average as the Simple dataset and the one
with more NPC vehicles as the Complex dataset. Their descriptive statistics
are given in Table 1.

Table 1: Descriptive statistics of the number of NPC vehicles in the Simple and Difficult datasets. Q1, Q2 and Q3 denote 25%, 50% and 75% quantiles. Dataset |  Mean |  Std |  Min |  Q1 |  Q2 |  Q3 |  Max
---|---|---|---|---|---|---|---
Easy | 4.81 | 3.59 | 0 | 2 | 4 | 7 | 21
Difficult | 6.51 | 3.96 | 0 | 3 | 6 | 9 | 36

ADS evolution dataset construction encompasses the bidirectional evolution
between the Simple and Complex datasets, i.e., $Simple\rightarrow Complex$ and
$Complex\rightarrow Simple$.

###  4.3 Metrics

We introduce the metrics to evaluate the predictive performance in Section
4.3.1 and efficiency metrics in Section 4.3.2. Metrics for assessing the UQ
methods are introduced in Section 4.3.3. In Section 4.4, we present the
statistical tests used in the evaluation.

####  4.3.1 Predicative Performance Evaluation Metrics

TTE analysis is essentially a regression prediction task. In this study, we
prefer to use Huber loss because, unlike Mean Squared Error (MSE), Huber loss
does not heavily penalize data points that deviate significantly from the
rest, thus making the prediction model more robust in handling outliers in the
data. Huber loss is calculated using the following formula [21], which
involves two conditions:

$L_{\delta}(y,f(x))=\begin{cases}\frac{1}{2}(y-f(x))^{2}&\text{for
}|y-f(x)|\leq\delta,\\\
\delta\cdot|y-f(x)|-\frac{1}{2}\delta^{2}&\text{otherwise.}\end{cases}$ (36)

In this equation, $y$ is the true value, $f(x)$ is the predicted value, and
$\delta$ is a hyperparameter that controls the transition between the loss for
small and large residuals.

####  4.3.2 Efficiency Performance Evaluation Metrics

We evaluate the efficiency with training time spent by the pretraining and
prompt tuning phases of PPT process. We denote $\mathcal{D}^{S}$ as the source
dataset and $\mathcal{D}^{T}$ as the target dataset for transfer learning.
PPT’s pretraining is executed on the pretraining dataset
$\mathcal{D}^{pre}=\\{\langle\mathcal{D}^{SPre}_{1},\mathcal{D}^{TPre}_{1}\rangle,\langle\mathcal{D}^{SPre}_{2},\mathcal{D}^{TPre}_{2}\rangle,...,\langle\mathcal{D}^{SPre}_{N},\mathcal{D}^{TPre}_{N}\rangle\\}$
of $N$ individual transfers. We determine the convergence time for one
transfer with Equation 37, where $time_{early\\_stopping\\_end}$ denotes the
point at which early stopping occurs (i.e., no improvement for five
consecutive epochs), while $time_{start}$ signifies the commencement point of
training.

$time_{convergence}=time_{early\\_stopping\\_end}-time_{start}$ (37)

Pretraining time is computed by aggregating the convergence times for $N$
individual transfers with Equation 38.

$time_{pretrain}=\sum_{i=1}^{N}time_{convergence}(\mathcal{D}^{SPre}_{i},\mathcal{D}^{TPre}_{i})$
(38)

Prompt tuning time is determined by the convergence time on source dataset
$\mathcal{D}_{S}$ and target dataset $\mathcal{D}_{T}$, as defined by Equation
39.

$time_{prompttuning}=time_{convergence}(\mathcal{D}^{S},\mathcal{D}^{T})$ (39)

####  4.3.3 UQ Method Evaluation Metrics

UQ Effectiveness Metric. We compare samples selected by each UQ method with
Precision@K [22]; Let $l_{A}$ and $l_{B}$ denote the samples selected by
method A and method B, respectively and Precision@K measures to what extent
$l_{A}$ and $l_{B}$ overlap in the top $K$ samples (Equation 40).

$\text{Precision@K}=\frac{overlap(l_{A},l_{B})}{K}$ (40)

UQ Efficiency Metric measures the efficiency of a UQ method as the total time
$\tau_{UQ}$ required for sample selection. We denote $\tau_{UQ}$ required by
CS score, Bayesian and ensemble UQ as $\tau_{CS}$, $\tau_{BUQ}$ and
$\tau_{EUQ}$, respectively.

###  4.4 Statistical testing

To counteract the inherent variability associated with training neural
networks, we conducted each experiment 30 times. Subsequently, we employed the
Mann-Whitney U test [23] to investigate the statistical significance of
observed improvements of PPT over the baseline RISE-DT. This was done for all
pair-wise comparisons within each RQ. The baseline assumption or null
hypothesis presumes no significant distinction between PPT and RISE-DT under
comparison. If this null hypothesis is dispelled, we deduce that they are not
equivalent. We choose the significance level as 0.01; thus, $p-value<0.01$
denotes a significant improvement $\Delta$.

As recommend in [23], we chose Vargha and Delaney’s A12 as the measure of
effect size. This metric illustrates the probability of PPT outperforming
RISE-DT. If the A12 value exceeds 0.5, we can infer that PPT is more likely to
yield superior results compared to RISE-DT, and vice versa. We consider the
effect size in the range $[0.56,0.64)$ as _Small_ $\downarrow$, $[0.64,0.71)$
as _Medium_ $\rightarrow$, and $[0.71,1]$ as _Large_ $\uparrow$.

###  4.5 Settings and Execution

Assigning hyperparameter values manually can potentially introduce bias. To
mitigate this issue, we carried out a 10-fold cross-validation process to
select optimal hyperparameters. This involved partitioning the dataset into 10
sequential segments, using the first nine for training and the final one for
validation. Due to the difference in complexity, we set the hyperparameters
differently for the elevator and ADS subject systems. We present some key
values in Table 2

Table 2: Hyperparameter values for PPT. d_model, n_heads, dim_feedforward, n_layers denote the hidden dimension, number of heads, feedforward network dimension, and number of the MHSA modules in the transformer. proj_dim represents the dimension of the projection module in the transfer learning component. Parameter |  For Elevator System |  For ADS
---|---|---
d_model | 16 | 128
batch size | 1 | 1
n_heads | 1 | 32
dim_feedforward | 128 | 1024
n_layers | 1 | 24
proj_dim | 32 | 128

Our code is written in Python with Pytorch 2.0 library [24]. CS score is
calculated with Uncertainty Toolbox [25]. We execute our code on a national,
experimental, heterogeneous computational cluster called eX3. This node
contains 2x Intel Xeon Platinum 8186, 1x NVIDIA V100 GPUs.

##  5 Results and Analysis

In this section, we answer each RQ. A replication package of PPT is provided
here for reference 111https://github.com/qhml/ppt.

###  5.1 RQ1 - PPT’s Overall Effectiveness

RQ1 aims to evaluate the overall effectiveness of PPT in TTE analysis by
comparing it to RISE-DT. The Huber loss results are shown in Table 3, which
includes the results of both the elevator and ADS case studies. In the case
study, we find PPT outperforms RISE-DT in both traffic-variant evolutions
(i.e., $UpBest\rightarrow LunchBest$ and $LunchBest\rightarrow UpBest$) and
dispatcher-variant evolutions i.e., $LunchWorse\rightarrow LunchBest$ and
$UpWorse\rightarrow UpBest$). The minimum improvement is 5.70, for the case
$LunchBest\rightarrow UpBest$), while the maximum improvement is 9.62 for the
case of $UpWorse\rightarrow UpBest$. The average improvement reaches 7.31 in
all four evolutions. In the ADS case study, we observe larger improvements
compared to RISE-DT. The average improvement in this case study is 12.58.

Table 3: Huber loss of TTE analysis and statistical testing results. The "Difference" column shows the difference between RISE-DT and PPT. |  Evolution |  RISE-DT |  PPT |  Difference
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 87.89 | 80.14 | 7.75
LunchBest$\rightarrow$UpBest | 109.49 | 103.79 | 5.70
LunchWorse$\rightarrow$LunchBest | 89.11 | 82.95 | 6.16
UpWorse$\rightarrow$UpBest | 114.88 | 105.26 | 9.62
Average | 100.35 | 93.03 | 7.31
ADS | Simple$\rightarrow$Complex | 230.26 | 215.85 | 14.41
Complex$\rightarrow$Simple | 118.81 | 108.06 | 10.75
Average | 174.54 | 161.95 | 12.58

Table 4 presents the statistical testing results of comparing PPT with RISE-
DT. In the elevator case study, we find that the improvements in the traffic-
variant evolutions are significant ($p-value<0.01$) with strong effect sizes
($A12>0.71$). The majority of the improvements in the dispatcher-variant
evolutions are significant (21 out of 30) and the effect sizes are mostly
strong (17 out of 30 for $LunchWorse\rightarrow LunchBest$ and 20 out of 30
for the case $UpWorse\rightarrow UpBest$). In the ADS case study, we observe
significance and very strong effect sizes (close to 1) in both improvements.

Table 4: Mann-Whitney statistical test results and A12 effect size of comparing RISE-DT and PPT. $\Delta$ denotes a significant testing result. $\downarrow,\rightarrow,\uparrow$ represent small, medium, and large effect sizes, respectively. |  Evolution |  p-value |  A12
---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | $\Delta$ | 0.828 ($\uparrow$)
LunchBest$\rightarrow$UpBest | $\Delta$ | 0.774 ($\uparrow$)
LunchWorse$\rightarrow$LunchBest | $\Delta\times 21$ | $\downarrow\times 4;\rightarrow\times 3;\uparrow\times 17$
UpWorse$\rightarrow$UpBest | $\Delta\times 21$ | $\downarrow\times 2;\rightarrow\times 1;\uparrow\times 20$
ADS | Simple$\rightarrow$Complex | $\Delta$ | 0.993 ($\uparrow$)
Complex$\rightarrow$Simple | $\Delta$ | 0.927 ($\uparrow$)

We conclude that PPT is effective in TTE analysis in both elevator and ADS
case studies, averagely outperforming RISE-DT by 7.31 and 12.58, respectively.
Improvements are significant and have large effect sizes in both case studies.

###  5.2 RQ2 - Transfer Learning Effectiveness and Efficiency

With RQ2, we aim to investigate the contribution of the transfer learning in
PPT. Table 5 presents the experiment results of comparing PPT and PPT without
transfer learning (denoted as "w/o TL"). We find that the Huber loss increases
sharply after removing transfer learning. Specifically, in the elevator case
study, the average increase reaches 21.32, with a minimum increase of 12.96
for the case $LunchWorse\rightarrow LunchBest$. In the ADS case study, the
increases in Huber loss are even higher with an average value of 28.26.

Table 5: Huber loss of PPT and PPT without transfer learning (denoted as "w/o TL"). Column "Difference" represents the difference between PPT and "w/o TL" |  Evolution |  w/o TL |  PPT |  Difference
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 95.91 | 80.14 | 15.77
LunchBest$\rightarrow$UpBest | 132.80 | 103.79 | 29.01
LunchWorse$\rightarrow$LunchBest | 95.91 | 82.95 | 12.96
UpWorse$\rightarrow$UpBest | 132.80 | 105.26 | 27.54
Average | 114.35 | 93.03 | 21.32
ADS | Simple$\rightarrow$Complex | 241.19 | 215.85 | 25.34
Complex$\rightarrow$Simple | 139.23 | 108.06 | 31.17
Average | 190.21 | 161.95 | 28.26

We also investigated the efficiency of transfer learning as depicted in Table
6. In this table, we report the training time of transfer learning, comprising
the pretraining phase (denoted as "Pre") and prompt tuning phase (denoted as
"PT"). We find that pretraining consumes marginally more time compared to
prompt tuning. The average pretraining times in the elevator and ADS case
study are 70 hours and 97.5 hours, respectively. Whereas the prompt tuning
time in these two cases is merely 2.77 hours and 5.05 hours, respectively.
Such results are expected since the pretraining dataset is larger than the
prompt tuning dataset. Moreover, the pretraining phase only requires a single
execution before the transfer learning, making the large pretraining time
acceptable for the production environment.

Table 6: Time cost of each training phase in PPT. "Pre", "PT" and "Total" denote the time cost for the pretraining phase, prompt tuning phase, and sum of both. |  Evolution |  Pre |  PT |  Total
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 76h | 2.9h | 78.9h
LunchBest$\rightarrow$UpBest | 76h | 4.1h | 80.1h
LunchWorse$\rightarrow$LunchBest | 61h | 2.5h | 63.5h
UpWorse$\rightarrow$UpBest | 67h | 1.6h | 68.6h
| Average | 70h | 2.77h | 72.77h
ADS | Simple$\rightarrow$Complex | 98h | 5.1h | 103.1h
Complex$\rightarrow$Simple | 97h | 5.0h | 102h
| Average | 97.5h | 5.05h | 102.05h

We conclude that transfer learning is effective in both elevator and ADS case
studies. Removing transfer learning from PPT leads to surges in Huber loss,
i.e., 21.32 and 28.26 hours in the elevator and ADS case studies. The majority
of the time cost of transfer learning is spent in the pretraining phase, which
we believe acceptable as one only needs to pretrain PPT once.

###  5.3 RQ3 - UQ Effectiveness and efficiency

RQ3 aims to evaluate the effectiveness and efficiency of UQ by comprehensively
assessing its influence on TTE analysis, selected samples, and time cost.

UQ’s Influence on TTE Analysis. To highlight the effectiveness of UQ, we
compare PPT and PPT without UQ (denoted as "w/o UQ") in Table 7. In the
elevator case study, we see an average increase of 4.08 after removing UQ from
PPT. The maximum increase is 7.42 for case $UpBest\rightarrow LunchBest$. In
the ADS case study, we find the Huber loss boost from 215.85 to 224.80 for
case $Simple\rightarrow Complex$ and from 108.06 to 115.25 for case
$Complex\rightarrow Simple$.

Table 7: Huber loss of PPT and PPT without UQ (denoted as "w/o UQ"). Column "Difference" represents the difference between PPT and "w/o UQ" |  Evolution |  w/o UQ |  PPT |  Difference
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 87.56 | 80.14 | 7.42
LunchBest$\rightarrow$UpBest | 105.09 | 103.79 | 1.30
LunchWorse$\rightarrow$LunchBest | 87.04 | 82.95 | 4.09
UpWorse$\rightarrow$UpBest | 108.76 | 105.26 | 3.50
Average | 97.11 | 93.04 | 4.08
ADS | Simple$\rightarrow$Complex | 224.80 | 215.85 | 8.95
Complex$\rightarrow$Simple | 115.25 | 108.06 | 7.19
Average | 170.02 | 161.95 | 8.07

We also compare three UQ methods (i.e., CS score, Bayesian and Ensemble UQ,
denoted as CS, BUQ and EUQ) in terms of Huber loss in Table 8. In the elevator
case study, we find EUQ tends to be the most effective UQ method, achieving
the lowest Huber loss in all evolutions except for $LunchBest\rightarrow
UpBest$. However, the difference between CS and EUQ is nominal with a maximum
of 1.59 (105.26-103.67) for case $UpWorse\rightarrow UpBest$. In the ADS case
study, CS beats EUQ and shows the lowest Huber loss for both evolutions:
$Simple\rightarrow Complex$ and $Complex\rightarrow Simple$.

Table 8: Huber loss for TTE analysis with different UQ methods. CS, BUQ, and EUQ denote CS score, Bayesian UQ, and Ensemble UQ, respectively. |  Evolution |  CS |  BUQ |  EUQ
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 80.14 | 85.23 | 79.40
LunchBest$\rightarrow$UpBest | 103.79 | 101.8 | 105.20
LunchWorse$\rightarrow$LunchBest | 82.95 | 85.79 | 80.09
UpWorse$\rightarrow$UpBest | 105.26 | 113.28 | 103.67
ADS | Simple$\rightarrow$Complex | 215.85 | 219.44 | 216.10
Complex$\rightarrow$Simple | 108.06 | 112.90 | 110.60

UQ’s Influence on Samples Selected. To compare the three UQ methods, we look
into samples selected by the UQ methods. We calculate the Precision@K metrics
to demonstrate the overlaps among the methods and the results are shown in
Table 9. The precision@1 (denoted as P@1) and precision@3 (denoted as P@3) are
all 100% in each evolution in the elevator and ADS case studies, indicating
that the top 1 and top 3 samples selected by one UQ method are always selected
by the other two methods. The precision@10 metric (denoted as P@10) gives
lower results. In the elevator case study, the precision@10 metric remains
100% in the traffic-variant evolutions (i.e., $UpBest\rightarrow LunchBest$
and $LunchBest\rightarrow UpBest$), indicating the top 10 samples selected by
one UQ method are also selected by the other two. As for the dispatcher-
variant evolutions, the precision@10 results are still high ($\geq 0.93$),
though not 100%. In the ADS case study, the precision@10 results are higher
than 82%, implying approximately 8 out of the top 10 samples selected by one
UQ method are also selected by the other two.

Table 9: Precision@K results for samples selected by the UQ methods. "P@1", "P@3" and "P@10" denote Precision@1, Precision@3 and Precision@10. |  Evolution |  P@1 |  P@3 |  P@10
---|---|---|---|---
Elevator | UpBest$\rightarrow$ LunchBest | 100% | 100% | 100%
LunchBest$\rightarrow$UpBest | 100% | 100% | 100%
LunchWorse$\rightarrow$LunchBest | 100% | 100% | 95%
UpWorse$\rightarrow$UpBest | 100% | 100% | 93%
ADS | Simple$\rightarrow$Complex | 100% | 100% | 82%
Complex$\rightarrow$Simple | 100% | 100% | 86%

UQ’s Time Cost. To assess the efficiency of UQ methods, we calculate the time
cost for each UQ method in the elevator and ADS case studies. We find that the
time cost for ADS (127.04 seconds on average) is much higher than that for the
elevator case study (56.40 seconds on average). CS score spends the least time
(denoted as $\tau_{CS}$), while the ensemble UQ method (denoted as
$\tau_{EUQ}$) takes the most time to perform UQ in both case studies.

Table 10: Time cost of the three UQ methods. $\tau^{CS}$, $\tau^{BUQ}$ and $\tau^{EUQ}$ denote the time cost for CS score, Bayesian and ensemble UQ methods.  Case Study | $\tau_{CS}$ | $\tau_{BUQ}$ | $\tau_{EUQ}$ | Average
---|---|---|---|---
Elevator System | 10.27s | 77.00s | 81.92s | 56.40s
ADS | 25.81s | 156.02s | 199.3s | 127.04s

We conclude that CS score is the most efficient UQ approach compared to
ensemble and Bayesian UQ, while retaining comparable effectiveness in TTE
analysis. Hence we use CS score as the UQ method in PPT.

###  5.4 RQ4 - Prompt Tuning Effectiveness and Efficiency

RQ4 aims to demonstrate the effectiveness and efficiency of prompt tuning. We
first compare PPT with PPT without prompt tuning (denoted as "w/o" PT) as
shown in Table 11. In the elevator case study, PPT outperforms PPT without
prompt tuning by 4.89 on average. The maximum Huber loss reduction is 6.17 for
the case $UpBest\rightarrow LunchBest$. In the ADS case study, we find similar
reductions in both evolutions of $Simple\rightarrow Complex$ and
$Complex\rightarrow Simple$ with an average of 3.14.

Table 11: Huber loss of TTE analysis and TTE analysis without prompt tuning. Column "Difference" shows the difference between PPT and PPT w/o PL. |  Evolution |  w/o PT |  PPT |  Difference
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 86.31 | 80.14 | 6.17
LunchBest$\rightarrow$UpBest | 109.07 | 103.79 | 5.28
LunchWorse$\rightarrow$LunchBest | 88.65 | 82.95 | 5.70
UpWorse$\rightarrow$UpBest | 107.68 | 105.26 | 2.42
Average | 97.93 | 93.03 | 4.89
ADS | Simple$\rightarrow$Complex | 219.25 | 215.85 | 3.40
Complex$\rightarrow$Simple | 110.94 | 108.06 | 2.88
Average | 165.10 | 161.95 | 3.14

As for efficiency, we compare prompt tuning (denoted as PT) with fine-tuning
(denoted as FT) and report the time cost in Table 12. Notice that fine-tuning
is used in RISE-DT. We find both fine-tuning and prompt tuning times in the
ADS case study are higher than those in the elevator case study. However,
there is no dominating choice in terms of time cost since the time spent on
fine-tuning and prompt tuning is quite close. Fine-tuning takes less time for
cases $UpBest\rightarrow LunchBest$ and $LunchWorse\rightarrow LunchBest$ in
the elevator case study and $Complex\rightarrow Simple$ in the ADS case study.

Table 12: Time cost of fine tuning and prompt tuning. Column "Difference" shows the difference between the time cost of fine-tuning and prompt tuning. |  Evolution |  FT |  PT |  Difference
---|---|---|---|---
Elevator | UpBest$\rightarrow$LunchBest | 2.7h | 2.9h | -0.2h
LunchBest$\rightarrow$UpBest | 4.1h | 4.1h | 0h
LunchWorse$\rightarrow$LunchBest | 2.4h | 2.5h | -0.1h
UpWorse$\rightarrow$UpBest | 1.8h | 1.6h | 0.2h
| Average | 2.75h | 2.77h | -0.02h
ADS | Simple$\rightarrow$Complex | 5.3h | 5.1h | 0.2h
Complex$\rightarrow$Simple | 4.9h | 5.0h | -0.1h
| Average | 5.1h | 5.05h | 0.1h

We conclude that prompt tuning effectively reduces Huber loss in TTE analysis,
and the time cost is approximately on the same level as fine-tuning.

###  5.5 Threats to Validity

Construct Validity concerns whether the metrics we choose can reflect the
quality of TTE analysis. We know that other metrics can be used to evaluate a
regression task like TTE analysis, such as MSE and RMSE. We choose the Huber
loss instead of MSE or RMSE because of its robustness against outliers. Unlike
MSE and RMSE, which are sensitive to outliers, Huber loss is a smoothed metric
that can provide a more stable evaluation of PPT.

Internal Validity refers to the extent to which the cause-and-effect
relationship aiming to be established in our study is not due to other
factors. One possible threat lies in the choices of hyperparameters, which
might introduce biases into the experiment. To alleviate this issue, we
performed a 10-fold cross-validation to select the hyperparameters
automatically.

Conclusion Validity pertains to the validity of the conclusions. PPT is a
neural network-based method, which tends to introduce randomness into the
experiments. To reduce the influence of randomness, we repeat each experiment
30 times and perform statistical testing to draw more significant conclusions.

External Validity is about to what extent PPT can generalize to other domains.
We design PPT to be applicable for TTE analysis in CPSs. To demonstrate the
generalizability of PPT, we evaluate PPT with datasets collected from two
different real-world domains, namely elevator and ADS.

##  6 Related work

We discuss the related work from five aspects: CPS safety and security in
Section 6.1, DT in CPSs in Section 6.2, transfer learning in Section 6.3, UQ
in Section 6.4 and prompt tuning in Section 6.5.

###  6.1 Cyber-physical Systems Security and Safety

CPSs inherently bear susceptibilities that originate from both physical and
cyber dimensions. To ameliorate these risks, numerous security and safety
improvement methodologies have been suggested [26, 27, 28, 29]. The
multifaceted and heterogeneous nature of CPS permits adversaries to launch
attacks from various points of entry, such as physical devices [30, 31, 32],
cyber networks [33, 34, 35], or even a combination of both [36]. This adds to
the challenge of guaranteeing comprehensive system safety and security for
CPS, especially given environmental uncertainties, security breaches, and
physical device errors [37].

With the growing application of deep learning for enhancing CPS security and
safety [38, 39, 40], a critical bottleneck experienced by researchers and
practitioners is the high cost of collecting data and, in some cases, the
infeasibility of obtaining labeled data for real-world CPSs. This shortage of
labeled data forms a roadblock for the effective training of deep learning
models, which often require sufficient data generated from the subject system
- a condition that cannot always be ensured. To combat this, we propose PPT in
this paper, a deep learning methodology tailored to tackle the issue of scarce
labeled CPS data by incorporating transfer learning and prompt tuning.

###  6.2 Digital Twins in Cyber-physical Systems

DT technologies facilitate real-time synchronization with CPSs [41, 42, 4, 43,
5, 44, 45]. For instance, Bécue et al. [41] proposed to use DTs for analyzing
the appropriate engineering of CPSs under attack scenarios. Eckhart et al.
[46] incorporated rules into DTs to determine whether an attacker could
compromise programmable logic controllers. Bitton et al. [42] recommended
conducting tests on a DT as a safer alternative to testing real CPS.
Furthermore, Damjanovic-Behrendt [47] employed DTs for the privacy assessment
of actual smart car systems. These examples attest to the remarkable
advantages of DT technologies. However, to the best of our understanding, our
work is unique in its emphasis on the evolution and development of DTs.

###  6.3 Transfer Learning

Transfer learning involves four predominant strategies. The first, known as
the model control strategy, implements transfer learning at the model tier.
For instance, Duan et al. [48] introduced the Domain Adaptation Machine (DAM)
which utilizes data from several source domains, constructs a classifier for
each, and employs regularizers to maintain the final model’s complexity. The
second strategy, the parameter control strategy, operates assuming that a
model’s parameters embody the knowledge it has assimilated. For instance,
Zhuang et al. [49] proposed directly sharing parameters between the source and
target models in the context of text classification. The model ensemble
strategy is the third approach, where transfer learning is achieved by
amalgamating various source models. For instance, Gao et al. [50] trained
several weak classifiers with different model structures on multiple source
domains and determined the final model based on a weighted vote from these
weak classifiers. Lastly, deep learning transfer techniques facilitate
knowledge transfer between two deep learning models by aligning corresponding
layers from source and target models. Zhuang et al. [49] proposed a transfer
learning method with autoencoder that aligns reconstruction, distribution, and
regression representations. This method was later expanded by Tzeng et al.
[51], who introduced an adaptation layer. Long et al. [52] took it a step
further by aligning multiple layers in their Deep Adaptation Networks model.

In summary, early strategies such as model and parameter control performed
knowledge transfer using intuitive methods such as adding regularizers and
sharing parameters. Their performance is comparable to the more recent model
ensemble and deep learning transfer techniques. Model ensemble is particularly
efficient when dealing with multiple heterogeneous source domains [53],
although it demands substantial computing resources. Deep learning transfer
techniques are apt for transferring knowledge between two neural network
models. Given that PPT is a neural network-based DT, we follow this latter
research trend, aligning the representation of the GRU layer and the
prediction layer.

###  6.4 Uncertainty Quantification

Numerous UQ methods are derived from Bayesian methods. For example, Wang et
al. [54] suggested using probability theory to interpret the parameters of
neural networks. Later, Srivastava et al. [55] incorporated Monte Carlo
dropout as a regularization term for calculating prediction uncertainty,
eliminating the need for posterior probability computation. In a further
development, Salakhutdinov et al. [56] proposed a stochastic gradient Markov
chain Monte Carlo (SG-MCMC) method, which only necessitates estimating the
gradient on small mini-batch sets, significantly reducing computational load
compared to direct posterior distribution estimation. Neural networks have
also been employed for posterior distribution estimation, such as the
variational autoencoder (VAE) proposed by Ghosh et al. [57], featuring both an
encoder and decoder based on neural network structure. Other notable UQ
techniques include deep Gaussian processes [58] and ensemble-based UQ [59].

Several open-source UQ tools exist for practical implementation. For instance,
Uncertainty Wizard [60] is a TensorFlow Keras plugin that supports commonly
used quantification methods, including Bayesian and ensemble-based methods.
Similarly, Uncertainty Toolbox [25], built on Pytorch, provides common
Bayesian and ensemble UQ methods, alongside additional metrics such as
calibration, sharpness, and accuracy.

The availability of UQ methods has fostered their applications in various
application domains. For instance, Catak et al. [61] proposed NIRVANA
validating deep learning model predictions based on MC dropout. Regarding
uncertainty-aware analyses, Han et al. [62] presented approaches to
systematically classify uncertainties based on the Cynefin framework and
evaluated the robustness of industrial elevator systems based on the results
of uncertainty classification. Zhang et al. [63, 64, 65] proposed a series of
methods for specifying, modeling and quantifying uncertainties in CPSs and
testing CPSs under environmental uncertainties.

###  6.5 Prompt Tuning

Prompt tuning, a burgeoning field in the machine learning domain, has
attracted significant attention in recent years. Multiple techniques have been
proposed to enhance the effectiveness of prompt tuning. At the foundational
level, several works have explored using prompts for language models. For
example, Brown et al. [8] proposed GPT-3, which utilizes prompts to facilitate
natural language processing tasks without any explicit supervision. Inspired
by this, Shin et al. [66] proposed AutoPrompt, an automated process to
discover efficient prompts for language models. In terms of prompt selection
strategies, a line of work focuses on generating diverse and effective
prompts. Liu et al. [18] proposed P-tuning, a method incorporating trainable
continuous prompts into pre-trained models, which showed impressive
improvements on multiple benchmark datasets.

Though prompt tuning is an exciting and active area of research, with various
strategies, toolkits, and applications being proposed, there is no prior work
that has focused on using prompt tuning in DT construction and evolution. In
our work, we take advantage of advances in this field to develop our prompt-
based learning method, aligning it with the needs of our application domain.

##  7 Conclusion and Future Work

In this paper, we propose a novel method PPT to evolve the DTs for Time-to-
Event prediction in CPSs. To alleviate the data scarcity problem, we utilize
transfer learning to transfer knowledge across different subject systems, with
the help of uncertainty quantification and prompt tuning. We evaluate PPT on
two CPSs, namely an elevator system and an autonomous driving system (ADS).
The experiment results show that PPT is effective in TTE analysis in both
elevator and ADS case studies, averagely outperforming the baseline by 7.31
and 12.58, respectively, in terms of Huber loss. Further analysis into
transfer learning, uncertainty quantification, and prompt tuning demonstrate
their individual contribution to reducing the Huber loss. In the future, we
plan to investigate more prompt tuning techniques by exploring other prompt
designing methods. We are also interested in applying our method in other
CPSs, such as power grids and railway systems.

## Acknowledgment

Qinghua Xu is supported by the security project funded by the Norwegian
Ministry of Education and Research. The work is also partially supported by
the Horizon 2020 project ADEPTNESS (871319) funded by the European Commission
and the Co-tester project (No. 314544) funded by the Research Council of
Norway. The experiment has benefited from the Experimental Infrastructure for
Exploration of Exascale Computing (eX3), which is financially supported by the
Research Council of Norway under contract 270053.

## References

  * [1] S. Biffl, M. Eckhart, A. Lüder, and E. Weippl, Eds., _Security and Quality in Cyber-Physical Systems Engineering: With Forewords by Robert M. Lee and Tom Gilb_. Cham: Springer International Publishing, 2019. [Online]. Available: http://link.springer.com/10.1007/978-3-030-25312-7
  * [2] A. Boyd, R. Bamler, S. Mandt, and P. Smyth, “User-Dependent Neural Sequence Models for Continuous-Time Event Data,” no. NeurIPS, pp. 1–19, 2020. [Online]. Available: http://arxiv.org/abs/2011.03231
  * [3] A. A. Musa, A. Hussaini, W. Liao, F. Liang, and W. Yu, “Deep Neural Networks for Spatial-Temporal Cyber-Physical Systems: A Survey,” _Future Internet_ , vol. 15, no. 6, p. 199, May 2023. [Online]. Available: https://www.mdpi.com/1999-5903/15/6/199
  * [4] M. Eckhart and A. Ekelhart, “Securing Cyber-Physical Systems through Digital Twins,” _Ercim News_ , no. 115, pp. 22–23, 2018.
  * [5] ——, “A specification-based state replication approach for digital twins,” 2018, pp. 36–47.
  * [6] B. R. Barricelli, E. Casiraghi, and D. Fogli, “A survey on digital twin: Definitions, characteristics, applications, and design implications,” _IEEE Access_ , vol. 7, no. Ml, pp. 167 653–167 671, 2019, publisher: IEEE.
  * [7] Q. Xu, S. Ali, T. Yue, and M. Arratibel, “Uncertainty-aware transfer learning to evolve digital twins for industrial elevators,” in _Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering_. Singapore Singapore: ACM, Nov. 2022, pp. 1257–1268. [Online]. Available: https://dl.acm.org/doi/10.1145/3540250.3558957
  * [8] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language Models are Few-Shot Learners,” Jul. 2020, arXiv:2005.14165 [cs]. [Online]. Available: http://arxiv.org/abs/2005.14165
  * [9] X. Zhang, J. Tao, K. Tan, M. Törngren, J. M. G. Sánchez, M. R. Ramli, X. Tao, M. Gyllenhammar, F. Wotawa, N. Mohan, M. Nica, and H. Felbinger, “Finding Critical Scenarios for Automated Driving Systems: A Systematic Mapping Study,” _IEEE Transactions on Software Engineering_ , vol. 49, no. 3, pp. 991–1026, Mar. 2023. [Online]. Available: https://ieeexplore.ieee.org/document/9763411/
  * [10] R. Queiroz, D. Sharma, R. Caldas, K. Czarnecki, S. García, T. Berger, and P. Pelliccione, “A Driver-Vehicle Model for ADS Scenario-based Testing,” May 2022, arXiv:2205.02911 [cs]. [Online]. Available: http://arxiv.org/abs/2205.02911
  * [11] A. El Saddik, “Digital Twins: The Convergence of Multimedia Technologies,” _IEEE MultiMedia_ , vol. 25, no. 2, pp. 87–92, Jun. 2018.
  * [12] T. Yue, P. Arcaini, and S. Ali, “Understanding Digital Twins for Cyber-Physical Systems: A Conceptual Model,” in _Leveraging Applications of Formal Methods, Verification and Validation: Tools and Trends_ , T. Margaria and B. Steffen, Eds. Cham: Springer International Publishing, 2021, pp. 54–71.
  * [13] M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya, V. Makarenkov, and S. Nahavandi, “A review of uncertainty quantification in deep learning: Techniques, applications and challenges,” _Information Fusion_ , vol. 76, pp. 243–297, 2021.
  * [14] L. Lin, H. Bao, and N. Dinh, “Uncertainty quantification and software risk analysis for digital twins in the nearly autonomous management and control systems: A review,” _Annals of Nuclear Energy_ , vol. 160, pp. 108 362–108 362, 2021, publisher: Elsevier Ltd. [Online]. Available: https://doi.org/10.1016/j.anucene.2021.108362
  * [15] T. Gneiting, F. Balabdaoui, and A. E. Raftery, “Probabilistic forecasts, calibration and sharpness,” _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , vol. 69, no. 2, pp. 243–268, 2007, publisher: Oxford University Press.
  * [16] G. J. Klir, “Uncertainty and information: foundations of generalized information theory,” _Kybernetes_ , vol. 35, no. 7/8, pp. 1297–1299, 2006, publisher: Emerald Group Publishing Limited.
  * [17] Y. Yang and M. Loog, “Active learning using uncertainty information,” in _2016 23rd International Conference on Pattern Recognition (ICPR)_ , Dec. 2016, pp. 2646–2651, journal Abbreviation: 2016 23rd International Conference on Pattern Recognition (ICPR).
  * [18] X. Liu, K. Ji, Y. Fu, W. Tam, Z. Du, Z. Yang, and J. Tang, “P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks,” in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_. Dublin, Ireland: Association for Computational Linguistics, 2022, pp. 61–68. [Online]. Available: https://aclanthology.org/2022.acl-short.8
  * [19] P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing,” Jul. 2021, arXiv:2107.13586 [cs]. [Online]. Available: http://arxiv.org/abs/2107.13586
  * [20] C. Lu, T. Yue, and S. Ali, “DeepScenario: An Open Driving Scenario Dataset for Autonomous Driving System Testing.”
  * [21] G. P. Meyer, “An alternative probabilistic interpretation of the huber loss,” 2021, pp. 5261–5269.
  * [22] P. Sujatha and P. Dhavachelvan, “Precision at K in Multilingual Information Retrieval,” _International Journal of Computer Applications_ , vol. 24, no. 9, pp. 40–43, Jun. 2011. [Online]. Available: http://www.ijcaonline.org/volume24/number9/pxc3873929.pdf
  * [23] A. Arcuri and L. Briand, “A practical guide for using statistical tests to assess randomized algorithms in software engineering,” in _Proceedings of the 33rd International Conference on Software Engineering_. Waikiki, Honolulu HI USA: ACM, May 2011, pp. 1–10. [Online]. Available: https://dl.acm.org/doi/10.1145/1985793.1985795
  * [24] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” Dec. 2019, arXiv:1912.01703 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1912.01703
  * [25] Y. Chung, I. Char, H. Guo, J. Schneider, and W. Neiswanger, “Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantification,” Sep. 2021, arXiv:2109.10254 [cs, stat]. [Online]. Available: http://arxiv.org/abs/2109.10254
  * [26] A. Banerjee, K. K. Venkatasubramanian, T. Mukherjee, and S. K. S. Gupta, “Ensuring safety, security, and sustainability of mission-critical cyber–physical systems,” _Proceedings of the IEEE_ , vol. 100, no. 1, pp. 283–299, 2011, publisher: IEEE.
  * [27] A. Humayed, J. Lin, F. Li, and B. Luo, “Cyber-physical systems security—A survey,” _IEEE Internet of Things Journal_ , vol. 4, no. 6, pp. 1802–1831, 2017, publisher: IEEE.
  * [28] C. Lv, Y. Xing, J. Zhang, X. Na, Y. Li, T. Liu, D. Cao, and F.-Y. Wang, “Levenberg–Marquardt backpropagation training of multilayer neural networks for state estimation of a safety-critical cyber-physical system,” _IEEE Transactions on Industrial Informatics_ , vol. 14, no. 8, pp. 3436–3446, 2017, publisher: IEEE.
  * [29] G. Sabaliauskaite and A. P. Mathur, “Aligning cyber-physical system safety and security,” in _Complex Systems Design & Management Asia_. Springer, 2015, pp. 41–53.
  * [30] S. Amin, “Securing the electricity grid,” _The Bridge_ , vol. 40, no. 1, pp. 19–20, 2010.
  * [31] H. Zeynal, M. Eidiani, and D. Yazdanpanah, “Intelligent Substation Automation Systems for robust operation of smart grids,” _2014 IEEE Innovative Smart Grid Technologies - Asia, ISGT ASIA 2014_ , no. July, pp. 786–790, 2014.
  * [32] T. M. Chen, J. C. Sanchez-Aarnoutse, and J. Buford, “Petri net modeling of cyber-physical attacks on smart grid,” _IEEE Transactions on Smart Grid_ , vol. 2, no. 4, pp. 741–749, 2011.
  * [33] K. Coffey, R. Smith, L. Maglaras, and H. Janicke, “Vulnerability Analysis of Network Scanning on SCADA Systems,” _Security and Communication Networks_ , vol. 2018, 2018.
  * [34] H. H. Safa, D. M. Souran, M. Ghasempour, and A. Khazaee, “Cyber security of smart grid and SCADA systems, threats and risks,” _IET Conference Publications_ , vol. 2016, no. CP686, 2016.
  * [35] C. W. Ten, C. C. Liu, and M. Govindarasu, “Vulnerability assessment of cybersecurity for SCADA systems using attack trees,” _2007 IEEE Power Engineering Society General Meeting, PES_ , no. November 2014, 2007.
  * [36] J. Radcliffe, “Hacking medical devices for fun and insulin: Breaking the human SCADA system,” _Black Hat Conference presentation slides_ , pp. 13–13, 2011. [Online]. Available: http://cs.uno.edu/~dbilar/BH-US-2011/materials/Radcliffe/BH_US_11_Radcliffe_Hacking_Medical_Devices_WP.pdf%5Cnhttp://www.aicas.com/cms/sites/default/files/BH_US_11_Radcliffe_Hacking_Medical_Devices_WP.pdf
  * [37] R. Rajkumar, I. Lee, L. Sha, and J. Stankovic, “Cyber-physical systems: The next computing revolution,” 2010, pp. 731–736.
  * [38] J. Song, Q. Xu, W. Liu, Y. Zu, and M. Chen, “Semantic and Morphological Information Guided Chinese Text Classification.” Springer, 2020, pp. 14–26.
  * [39] C. S. Wickramasinghe, D. L. Marino, K. Amarasinghe, and M. Manic, “Generalization of deep learning for cyber-physical system security: A survey,” _Proceedings: IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society_ , vol. 1, pp. 745–751, 2018, publisher: IEEE.
  * [40] T. K. Lee, T. W. Wang, W. X. Wu, Y. C. Kuo, S. H. Huang, G. S. Wang, C. Y. Lin, J. J. Chen, and Y. C. Tseng, “Building a V2X Simulation Framework for Future Autonomous Driving,” _2019 20th Asia-Pacific Network Operations and Management Symposium: Management in a Cyber-Physical World, APNOMS 2019_ , pp. 1–6, 2019, publisher: IEICE.
  * [41] A. Bécue, Y. Fourastier, I. Praça, A. Savarit, C. Baron, B. Gradussofs, E. Pouille, and C. Thomas, “CyberFactory#1 — Securing the industry 4.0 with cyber-ranges and digital twins,” 2018, pp. 1–4.
  * [42] R. Bitton, T. Gluck, O. Stan, M. Inokuchi, Y. Ohta, Y. Yamada, T. Yagyu, Y. Elovici, and A. Shabtai, “Deriving a Cost-Effective Digital Twin of an ICS to Facilitate Security Evaluation: 23rd European Symposium on Research in Computer Security, ESORICS 2018, Barcelona, Spain, September 3-7, 2018, Proceedings, Part I,” Aug. 2018, pp. 533–554.
  * [43] M. Eckhart and A. Ekelhart, _Security and Quality in Cyber-Physical Systems Engineering_ , ser. Security and Quality in Cyber-Physical Systems Engineering, 2019, issue: March 2020.
  * [44] M. Tauber and C. Schmittner, “Enabling security and safety evaluation in industry 4.0 use cases with digital twins,” _ERCIM News_ , 2018, publisher: ERCIM EEIG.
  * [45] V. Damjanovic-Behrendt, “A digital twin architecture for security, privacy and safety,” _ERCIM NEWS_ , no. 115, pp. 25–26, 2018, publisher: EUROPEAN RESEARCH CONSORTIUM INFORMATICS & MATHEMATICS 2004, ROUTE LUCIOLES ….
  * [46] M. Eckhart and A. Ekelhart, “Towards security-aware virtual environments for digital twins,” 2018, pp. 61–72.
  * [47] V. Damjanovic-Behrendt, “A digital twin-based privacy enhancement mechanism for the automotive industry.” IEEE, 2018, pp. 272–279.
  * [48] L. Duan, D. Xu, and I. W. Tsang, “Learning with Augmented Features for Heterogeneous Domain Adaptation.”
  * [49] J. Zhuang, Z. Chen, P. Wei, G. Li, and L. Lin, “Open Set Domain Adaptation By Novel Class Discovery,” Mar. 2022. [Online]. Available: http://arxiv.org/abs/2203.03329
  * [50] X. Gao, C. Shan, C. Hu, Z. Niu, and Z. Liu, “An Adaptive Ensemble Machine Learning Model for Intrusion Detection,” _IEEE Access_ , vol. 7, pp. 82 512–82 521, 2019. [Online]. Available: https://ieeexplore.ieee.org/document/8740962/
  * [51] E. Tzeng, J. Hoffman, T. Darrell, and K. Saenko, “Simultaneous Deep Transfer Across Domains and Tasks.”
  * [52] M. Long, H. Zhu, J. Wang, and M. I. Jordan, “Deep Transfer Learning with Joint Adaptation Networks,” Aug. 2017, arXiv:1605.06636 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1605.06636
  * [53] A. Farahani, S. Voghoei, K. Rasheed, and H. R. Arabnia, “A Brief Review of Domain Adaptation,” 2020. [Online]. Available: http://arxiv.org/abs/2010.03978
  * [54] K.-C. Wang, P. Vicol, J. Lucas, L. Gu, R. Grosse, and R. Zemel, “Adversarial Distillation of Bayesian Neural Network Posteriors,” Jun. 2018, arXiv:1806.10317 [cs, stat]. [Online]. Available: http://arxiv.org/abs/1806.10317
  * [55] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting.”
  * [56] R. Salakhutdinov and A. Mnih, “Bayesian probabilistic matrix factorization using Markov chain Monte Carlo,” in _Proceedings of the 25th international conference on Machine learning - ICML ’08_. Helsinki, Finland: ACM Press, 2008, pp. 880–887. [Online]. Available: http://portal.acm.org/citation.cfm?doid=1390156.1390267
  * [57] S. Ghosh, A. Pal, A. Nag, S. Sadhu, and R. Pati, “Network anomaly detection using a fuzzy rule-based classifier,” in _Computer, Communication and Electrical Technology_ , 1st ed., D. Guha, B. Chakraborty, and H. Sekhar Dutta, Eds. CRC Press, Mar. 2017, pp. 61–65. [Online]. Available: https://www.taylorfrancis.com/books/9781315400617/chapters/10.1201/9781315400624-12
  * [58] A. C. Damianou and N. D. Lawrence, “Deep Gaussian Processes,” Mar. 2013, arXiv:1211.0358 [cs, math, stat]. [Online]. Available: http://arxiv.org/abs/1211.0358
  * [59] J. Liu, J. Paisley, M.-A. Kioumourtzoglou, and B. Coull, “Accurate Uncertainty Estimation and Decomposition in Ensemble Learning.”
  * [60] M. Weiss and P. Tonella, “Uncertainty Quantification for Deep Neural Networks: An Empirical Comparison and Usage Guidelines,” 2022. [Online]. Available: http://arxiv.org/abs/2212.07118
  * [61] F. O. Catak, T. Yue, and S. Ali, “Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data,” _ACM Trans. Softw. Eng. Methodol._ , vol. 31, no. 4, Jul. 2022, place: New York, NY, USA Publisher: Association for Computing Machinery. [Online]. Available: https://doi.org/10.1145/3527451
  * [62] Liping Han; Tao Yue; Shaukat Ali; Aitor Arrieta; Maite Arratibel., “Are Elevator Software Robust Against Uncertainties? Results and Experiences from an Industrial Case Study,” vol. 15, 2016, pp. 1–23, issue: 2.
  * [63] X. Zhang, G. Kumar, H. Khayrallah, K. Murray, J. Gwinnup, M. J. Martindale, P. McNamee, K. Duh, and M. Carpuat, “An Empirical Exploration of Curriculum Learning for Neural Machine Translation,” 2018. [Online]. Available: http://arxiv.org/abs/1811.00739
  * [64] J. M. Zhang, M. Harman, L. Ma, and Y. Liu, “Machine Learning Testing: Survey, Landscapes and Horizons,” _IEEE Transactions on Software Engineering_ , vol. X, no. X, pp. 1–1, 2020.
  * [65] M. Zhang, T. Yue, S. Ali, B. Selic, O. Okariz, R. Norgre, and K. Intxausti, “Specifying uncertainty in use case models,” _Journal of Systems and Software_ , vol. 144, pp. 573–603, 2018, publisher: Elsevier.
  * [66] T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, “AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts,” Nov. 2020, arXiv:2010.15980 [cs]. [Online]. Available: http://arxiv.org/abs/2010.15980