# A Federated Data Fusion-Based Prognostic Model for Applications with Multi-
Stream Incomplete Signals

Madi Arabi
Edward P. Fitts Department of Industrial and Systems Engineering,
North Carolina State University, Raleigh, U.S.
and
Xiaolei Fang
Edward P. Fitts Department of Industrial and Systems Engineering,
North Carolina State University, Raleigh, U.S

###### Abstract

Most prognostic methods require a decent amount of data for model training. In
reality, however, the amount of historical data owned by a single organization
might be small or not large enough to train a reliable prognostic model. To
address this challenge, this article proposes a federated prognostic model
that allows multiple users to jointly construct a failure time prediction
model using their multi-stream, high-dimensional, and incomplete data while
keeping each user’s data local and confidential. The prognostic model first
employs multivariate functional principal component analysis to fuse the
multi-stream degradation signals. Then, the fused features coupled with the
times-to-failure are utilized to build a (log)-location-scale regression model
for failure prediction. To estimate parameters using distributed datasets and
keep the data privacy of all participants, we propose a new federated
algorithm for feature extraction. Numerical studies indicate that the
performance of the proposed model is the same as that of classic non-federated
prognostic models and is better than that of the models constructed by each
user itself.

Keywords: Federated Learning, Federated Data Fusion, Remaining Useful Life

##  1 Introduction

Industrial prognostic aims to predict the failure time of machines by
utilizing their degradation signals. This is typically achieved by
establishing a statistical learning model that maps the degradation signals of
machines to their time-to-failure (TTFs) [1, 2]. Similar to that of many other
statistical learning models, the implementation of prognostic models usually
consists of two steps: model training and real-time monitoring (also known as
model testing or deployment). Model training focuses on using a historical
dataset that comprises the degradation signals and TTFs of some failed
machines to estimate the parameters of the prognostic model; real-time
monitoring feeds the real-time degradation signals from a partially degraded
onsite machine into the prognostic model trained earlier to predict its TTF or
TTF distribution. Most existing prognostic models assume that a historical
dataset from a decent number of failed machines is available for model
training [3, 4, 5, 6, 7]. In reality, however, the amount of historical data
owned by a single organization (e.g., a company, a university lab, a factory,
etc.) might be small or not large enough to train a reliable prognostic model.
Thus, it is beneficial to investigate the possibility of allowing multiple
geographically distributed organizations/users/participants to collaboratively
train a prognostic model using their data together.

Cloud-based computing is a straightforward solution that enables multiple
users to jointly train a prognostic model. Specifically, all the users first
upload their data to a server on the cloud. Next, the server aggregates the
data from all users and uses the aggregated data to train a prognostic model
on the cloud. Finally, the trained model is shared with all users for real-
time monitoring. Unfortunately, cloud computing-based prognostic is usually
not feasible in real-world applications since it cannot protect the data
privacy of participants. This is because many companies have concerns over
sharing their data, and some are not even allowed to upload their data to the
cloud due to various privacy protection policies and regulations [8, 9, 10].
One solution to address this challenge is Federated Learning, which allows
multiple organizations to use their isolated data to jointly train machine
learning models while keeping each participant’s data local and confidential
[11, 12, 13, 14, 15, 16, 17, 18, 19]. However, integrating federated learning
into prognostic is nontrivial since it requires the development of new
privacy-preserving statistical learning models or parameter estimation
algorithms under the distributed computing framework.

Recently, some research works have investigated federated learning-based
prognostics [20, 21, 22, 23, 24, 25, 26, 27]. However, many of these models
focus on modeling single stream degradation signals [21, 22, 23, 24], which
means they cannot be applied to applications with degradation signal from
multiple sensors. As discussed and verified in many articles [28, 7, 6],
prognostic models based on multi-stream degradation signals usually achieve a
better performance than those work on single-stream signals since multi-stream
data may capture different aspects of the underlying degradation process.
Also, advances in sensing technology have enabled the industry to monitor
their equipment using multiple sensors, which generate multi-stream
degradation signals. Some other works focus on developing prognostic models
based on multi-stream degradation signals [20, 25, 26, 27]. However, many of
them fail to fuse the information from multi-stream data, which might
compromise the performance of their prognostic models. As discussed in
articles [7, 28], data fusion or dimension reduction is usually necessary for
the prognostic of machines with multi-stream degradation signals. This is
because degradation signals are typically high-dimensional, which implies the
number of parameters to be estimated in prognostic models is large and often
far more than the size of samples in the historical training data (i.e., the
small $n$ large $p$ issue in statistical learning). Also, high-dimensional
degradation signals usually contain redundant/overlapped information that
provides the possibility for data fusion to reduce the number of features used
to construct prognostic models. The redundant information is manifested in
terms of correlations within and among degradation signals: the observations
of a single degradation signal are usually auto-correlated and observations
among different degradation signals are often cross-correlated. A few papers
have developed federated data fusion algorithms before constructing federated
prognostic models. For example, the authors in [27] proposed a federated
feature extraction algorithm based on randomized singular value decomposition.
However, this article assumes that degradation signals are complete, which
implies it cannot be applied to applications whose degradation signals have
missing observations. In real-world applications, incompleteness is a common
characteristic of degradation signals. One of the reasons is that many complex
assets operate in harsh environments that often have a significant impact on
the quality of the raw data, due to errors in data acquisition, communication,
read/write operations, etc, which results in degradation signals with
significant levels of missing and corrupt observations. Another reason for
data incompleteness is the failure time truncation. Specifically, the failure
times of machines vary from one to another, and no degradation signal can be
observed beyond the failure times. As a result, the lengths of degradation
signals of different machines are usually different. Unfortunately, many
prognostic models require the lengths of degradation signals to be the same.
To the best of our knowledge, few works have focused on developing prognostic
models that work with incomplete degradation signals. To address this
challenge, this paper proposes a federated prognostic model that can fuse
multi-stream incomplete degradation signals and use the fused features for TTF
distribution prediction.

The proposed federated prognostic model comprises two sequential steps: data
fusion and prognostic model construction. Data fusion focuses on fusing multi-
stream high-dimensional degradation signals and provides low-dimensional
features, while prognostic model construction is accomplished by mapping TTFs
to the features extracted earlier using a statistical learning model. In this
article, we use multivariate functional principal component analysis (MFPCA)
for data fusion. MFPCA is a nonparametric dimension reduction method that
captures the joint variation of multi-stream functional data (multi-stream
degradation signals in this paper) and provides low-dimensional features known
as MFPC-scores [6]. It can capture the auto-correlation within each
degradation signal as well as cross-correlation among different degradation
signal streams. MFPCA provides low-dimensional fused features known as MFPC-
scores, which are then used for prognostic model construction. In this
article, we use (log)-location-scale (LLS) regression to build the prognostic
model, in which the response variable is the TTF and the predictors are MFPC-
scores. LLS regression has been widely used in reliability engineering and
survival analysis since their response variables are general and include a
variety of TTF distributions, such as (log)-normal, (log)-logistics, Smallest
Extreme Value (SEV), and Weibull [29]. Prognostic models based on MFPCA and
LLS regression have been discussed and their effectiveness has been
extensively validated in existing articles [6, 7, 30, 31]. However, all of
these models use a centralized dataset for model training, and none of them
can be used under federated learning settings. Our contribution in this paper
is to propose a federated algorithm that enables multiple users to use their
(multi-stream) incomplete data to collaboratively conduct MFPCA and extract
fused features (i.e., MFPC-scores) while keeping every user’s data local and
confidential.

The proposed federated feature extraction algorithm is inspired by the classic
singular value decomposition (SVD)-based MFPC-scores computation method. The
classic SVD-based method works with a degradation signal matrix that
aggregates signals from all failed machines in the training dataset. First, it
removes the mean of signals, which is known as data centralization. Next, it
conducts SVD on the centered signal matrix to get singular values and singular
vectors. Third, MFPC-scores are computed by multiplying the centered signal
matrix by singular vectors, and the eigenvalues are often used to determine
the number of dominant principal components to be kept. Apparently, the
classic SVD-based method cannot be applied in federated learning settings
since it requires data from all users to be aggregated first. Also, it assumes
the aggregated signal matrix is complete since the centralization and
multiplication cannot be done if signals have missing entries. Thus, a new
federated algorithm that can work with incomplete degradation signals needs to
be developed. From the perspective of linear algebra and geometry, the
singular vectors of the classic SVD-based method span the low-dimensional
dominant subspace in the high-dimensional space spanned by the centered
degradation signals. Also, the singular vectors are a special set of
orthonormal bases of the dominant subspace (the dominant subspace has numerous
sets of basis vectors, and the singular vectors are one of these sets).
Inspired by this, we first propose a federated dominant subspace detection
algorithm that can use uncentered and incomplete degradation signals to
extract a general set of basis vectors of the dominant subspace. This is
achieved by developing a new federated incremental SVD algorithm, which is an
iterative method that allows each user to update basis vectors locally using
its own data. Since only the basis vectors are shared among different users,
data privacy can be protected. With the general set of basis vectors provided
by the federated dominant subspace detection algorithm, we then propose a
federated algorithm for MFPC-scores computation.

The rest of the paper is organized as follows. In section 2, we introduce the
MFPCA- and LLS regression-based prognostic model. Section 3 discusses the
feature extraction method, in which Section 3.1 summarizes the computation of
MFPC-scores with an aggregated dataset while Section 3.2 presents our proposed
federated feature extraction method. In Sections 4 and 5, the performance of
the proposed federated prognostic model is evaluated using a simulated dataset
and a degradation dataset from the NASA data repository. Finally, Section 6
concludes.

##  2 The Prognostic Model and Parameter Estimation

This article proposes a federated prognostic model that can be jointly
constructed by multiple geographically distributed users using their
incomplete degradation signals while keeping each user’s data local and
confidential. Similar to some existing articles [7, 31], the prognostic model
used in this paper is based on functional (log)-location-scale regression
[32], which regresses a system’s TTF against its multi-stream degradation
signals. To improve the performance of the prognostic model and simplify its
parameter estimation, we transform functional LLS regression into classic LLS
regression. This is achieved by first employing Multivariate Functional
Principal Component Analysis (MFPCA) to fuse the (multi-stream) high-
dimensional degradation signals and provide low-dimensional features; then,
the TTF of each system is regressed against its features from MFPCA using
regular LLS regression. Articles [7, 31] have proved that such a
transformation neither results in the loss of information nor compromises the
performance of the prognostic model.

We assume there are $I$ users jointly constructing the prognostic model, and
user $i$ has the historical degradation signals and TTFs of $J_{i}$ failed
systems/assets/components. Also, we assume that the failed systems of all
users are identical, and each of them is monitored by $P$ sensors, which
generate $P$-channel degradation signals (without loss of generality, we
assume that each sensor generates one channel degradation signal). Let
${x}_{ijp}(t)$ be the degradation signal from sensor $p$ of system $j$ of user
$i$, where $i=1,\ldots,I$, $j=1,\ldots,J_{i}$, and $p=1,\ldots,P$,
$t\in[0,T]$. Also, let $\tilde{y}_{ij}$ be the TTF of the system $j$ from user
$i$. To map TTF to the degradation signals, we employ the following functional
LLS regression model:

${y}_{ij}=\gamma_{0}+\int_{0}^{T}\boldsymbol{\gamma}(t)^{\top}\boldsymbol{x}_{ij}(t)dt+\sigma\epsilon_{ij},$
(1)

where ${y}_{ij}=\tilde{y}_{ij}$ if the TTF follows a location-scale
distribution and ${y}_{ij}=\log(\tilde{y}_{ij})$ if it follows a
(log)-location-scale distribution. $\gamma_{0}$ is the intercept,
$\boldsymbol{\gamma}(t)=(\gamma_{1}(t),\gamma_{2}(2),...,\gamma_{P}(t))^{\top}$
is the regression coefficient function, and
$\boldsymbol{x}_{ij}(t)=(x_{ij1}(t),x_{ij2}(t),...,x_{ijP}(t))^{\top}$ is the
concatenated degradation signals from all $P$ sensors of system $j$ of user
$i$. $\sigma$ is the scale parameter, and $\epsilon_{ij}$ is the random noise
term with a standard location-scale density $f(\epsilon)$. For example
$f(\epsilon)=1/\sqrt{2\pi}\exp(-\epsilon^{2})$ for normal distribution and
$f(\epsilon)=\exp(\epsilon-\exp(\epsilon))$ for SEV distribution.

Based on the Kosambi-Karhunen-Loève theorem [33], the concatenated degradation
signal $\boldsymbol{x}_{ij}(t)$ can be decomposed as follows:

$\boldsymbol{x}_{ij}(t)=\boldsymbol{\mu}(t)+\sum_{k=1}^{\infty}\boldsymbol{\zeta}_{ijk}\boldsymbol{\Psi}_{k}(t),\quad
t\in[0,T]$ (2)

where $\boldsymbol{\mu}(t)=(\mu_{1}(t),\mu_{2}(t),...,\mu_{P}(t))^{\top}$ is
the mean function;
${\zeta}_{ijk}=\int_{0}^{T}(\boldsymbol{x}_{ij}(t)-\boldsymbol{\mu}(t))^{\top}\boldsymbol{\Psi}_{k}(t)dt$
is the feature (known as MFPC-score).
$\boldsymbol{\Psi}_{k}(t)=(\Psi_{k1}(t),...,\Psi_{kP}(t))$ is the $k$th eigen
function of a $P\times P$ block covariance matrix
$\boldsymbol{C}(t,t^{\prime})=\begin{bmatrix}C_{1,1}(t,t^{\prime})&...&C_{1,P}(t,t^{\prime})\\\
\vdots&\ddots&\vdots\\\ C_{P,1}(t,t^{\prime})&...&C_{P,P}(t,t^{\prime})\\\
\end{bmatrix}$, where the $(g,h)$th entry ${C}_{g,h}(t,t^{\prime})$ is the
covariance function between sensor $g$ and $h$, $g=1,\ldots,P,h=1,\ldots,P$.
Usually, it is sufficient to keep the first $K$ eigenfunctions corresponding
to the $K$ largest eigenvalues. As a result, Equation (2) can be expressed as
follows:
$\boldsymbol{x}_{ij}(t)\approx\boldsymbol{\mu}(t)+\sum_{k=1}^{K}\boldsymbol{\zeta}_{ijk}\boldsymbol{\Psi}_{k}(t)$,
where $K$ can be determined using Fraction-of-Variance Explained (FVE) [30] or
cross-validation [7]. Since the set of eigenfunctions
$\\{\boldsymbol{\Psi}_{k}(t)\\}_{k=1}^{\infty}$ forms a complete set of
orthonormal basis functions, the unknown regression coefficient function
$\boldsymbol{\gamma}(t)$ can be decomposed as
$\boldsymbol{\gamma}(t)=\sum_{k=1}^{\infty}\beta_{k}\boldsymbol{\Psi}_{k}(t)\approx\sum_{k=1}^{K}\beta_{k}\boldsymbol{\Psi}_{k}(t)$
[34]. As a result, the function LLS regression model in Equation (1) can be
simplified to the following classic LLS regression (the proof can be found in
[7]):

$y_{ij}=\beta_{0}+\boldsymbol{\beta}^{\top}\boldsymbol{\zeta}_{ij}+\sigma\epsilon_{ij},$
(3)

where $\beta_{0}$ is the intercept,
$\boldsymbol{\beta}=(\beta_{1},\beta_{2},...,\beta_{K})^{\top}\in\mathbb{R}^{K}$
is the coefficient vector, and
$\boldsymbol{\zeta}_{ij}=(\zeta_{ij1},...,\zeta_{ijK})^{\top}\in\mathbb{R}^{K}$
are the MFPC-scores of the $j$th system of user $i$.

The parameter estimation of the prognostic model consists of two steps: (i)
feature extraction and (ii) regression parameters estimation. The first step,
feature extraction, focuses on using the decentralized historical degradation
signals from multiple users (i.e., $\boldsymbol{x}_{ij}(t)$) to calculate the
MFPC-scores (i.e., $\boldsymbol{\zeta}_{ij}$ in Equation (3)). The second
step, regression parameters estimation, uses the features from the first step
(i.e., $\boldsymbol{\zeta}_{ij}$) and the historical TTFs (i.e., $y_{ij}$ in
Equation (3)) to estimate the regression parameters (i.e.,
$\beta_{0},\boldsymbol{\beta},\sigma$). Since the historical data is
decentralized, both the feature extraction and coefficient estimation
algorithms should be federated, which means that they can use data from all
the users while keeping each user’s data local and confidential. In this
article, our contribution is to propose a federated feature extraction
algorithm for applications with incomplete degradation signals, which will be
discussed in Section 3. Regarding the second step, regression parameters
estimation, we will employ the federated algorithm developed in [27].

##  3 The Federated Feature Extraction Algorithm

In this section, we propose a federated feature extraction algorithm that
computes the MFPC-scores (i.e., $\boldsymbol{\zeta}_{ij}$ in Equation (3))
using the decentralized incomplete degradation signals from multiple users
while keeping each user’s data local and confidential. To help understand the
proposed federated algorithm, we will first review how to extract the MFPC-
scores using centralized complete data (i.e., the data from all users are
merged together for feature extraction) in Section 3.1. After that, the
proposed federated algorithm using decentralized data will be discussed in
Section 3.2.

###  3.1 Feature Extraction Using Centralized Data

Recall that the degradation signal from senor $p$ of the system $j$ of user
$i$ is denoted as ${x}_{ijp}(t),t\in[0,T]$. Let vector
$\boldsymbol{x}_{ijp}=({x}_{ijp}(\tau_{1}^{(p)}),{x}_{ijp}(\tau_{2}^{(p)}),\ldots,{x}_{ijp}(\tau_{n_{p}}^{(p)}))^{\top}\in\mathbb{R}^{n_{p}}$
be its complete discrete observations, where
$0\leq\tau_{1}^{(p)}\leq\tau_{2}^{(p)}\leq\ldots\leq\tau_{n_{p}}^{(p)}\leq T$
are the discrete observation time stamps, $n_{p}$ is the number of
observations for sensor $p$, $p=1,\ldots,P$, $i=1,\ldots,I$, and
$j=1,\ldots,J_{i}$. Then, the concatenated degradation signal observations
from all $P$ sensors of system $j$ belonging to user $i$ can be denoted as
$\boldsymbol{x}_{ij}=(\boldsymbol{x}_{ij1}^{\top},\boldsymbol{x}_{ij2}^{\top},\ldots,\boldsymbol{x}_{ijP}^{\top})^{\top}\in\mathbb{R}^{\sum_{p=1}^{P}n_{p}}$.
Furthermore, the degradation signal matrix after combing the data from all
users can be expressed as
$\boldsymbol{X}=(\boldsymbol{x}_{11},\boldsymbol{x}_{12},\ldots,\boldsymbol{x}_{1J_{1}},\boldsymbol{x}_{21},\boldsymbol{x}_{22},\ldots,\boldsymbol{x}_{2J_{2}},\ldots,\boldsymbol{x}_{I1},\boldsymbol{x}_{I2},\ldots,\boldsymbol{x}_{IJ_{I}})\in\mathbb{R}^{N\times
J}$, where $N=\sum_{p=1}^{P}n_{p}$ is the total number of observations from
all $P$ sensors, and $J=\sum_{i=1}^{I}J_{i}$ is the total number of systems
from all $I$ users.

Given the centralized signal matrix $\boldsymbol{X}$, the MFPC-scores can be
computed following the steps below[6, 7, 30]:

  1. (a)

Compute the mean vector of the signal matrix, i.e.,
$\boldsymbol{\bar{x}}=\frac{1}{J}\sum_{i=1}^{I}\sum_{j=1}^{J_{i}}\boldsymbol{x}_{ij}\in\mathbb{R}^{N}$.

  2. (b)

Center the signal matrix by subtracting the mean vector from each column of
the signal matrix, i.e.,
$\boldsymbol{\tilde{X}}=\boldsymbol{X}-\boldsymbol{\bar{X}}\in\mathbb{R}^{N\times
J}$, where $\boldsymbol{\bar{X}}$ has $J$ columns, each of which is
$\boldsymbol{\bar{x}}$.

  3. (c)

Compute the covariance matrix of the centered signal matrix
$\boldsymbol{C}=\boldsymbol{\tilde{X}}\boldsymbol{\tilde{X}}^{\top}\in\mathbb{R}^{N\times
N}$ and conduct eigen decomposition
$\boldsymbol{C}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{U}^{\top}$, where
$\boldsymbol{U}\in\mathbb{R}^{N\times N}$ is a matrix whose columns are the
eigenvectors, and $\boldsymbol{\Sigma}\in\mathbb{R}^{N\times N}$ is a diagonal
matrix whose diagonal entries are the eigen values
$\lambda_{1},\lambda_{2},\ldots,\lambda_{N}$ and
$\lambda_{1}\geq\lambda_{2}\geq\ldots\geq\lambda_{N}$. Another way to compute
the eigenvectors and eigenvalues is to conduct SVD on
$\boldsymbol{\tilde{X}}$. That is,
$\boldsymbol{\tilde{X}}=\boldsymbol{U}\boldsymbol{S}\boldsymbol{V}$, where
$\boldsymbol{U}\in\mathbb{R}^{N\times N}$ is a matrix whose columns are the
left singular vectors (same as the eigenvectors from the eigendecomposition of
$\boldsymbol{C}$), $\boldsymbol{S}\in\mathbb{R}^{N\times J}$ is a diagonal
matrix whose entries are the singular values (i.e., the square root of
eigenvalues
$\sqrt{\lambda_{1}}\geq\sqrt{\lambda_{2}}\geq\ldots\geq\sqrt{\lambda_{N}}$),
and $\boldsymbol{V}\in\mathbb{R}^{N\times J}$.

  4. (d)

Use the fraction-of-variance-explained [30] or cross-validation [7] to select
the number of dominant principal components (denoted as $K$,
$K\leq\min\\{N,J\\}$) and keep the first $K$ eigenvectors – that is –
$\boldsymbol{U}_{dominant}=\boldsymbol{U}_{[1:N,1:K]}\in\mathbb{R}^{N\times
K}$. Here, the subscript [1:N,1:K] represents the first $K$ columns.

  5. (e)

Calculate the MFPC-scores using
$\boldsymbol{Z}=\boldsymbol{U}_{dominant}^{\top}\boldsymbol{\tilde{X}}\in\mathbb{R}^{K\times
J}$.

Steps (a)-(e) summarize the process of computing MFPC-scores. Here, we give
some remarks on the dominant eigenvectors $\boldsymbol{U}_{dominant}$. First,
the dominant eigenvectors $\boldsymbol{U}_{dominant}\in\mathbb{R}^{N\times K}$
span a special $K$-dimensional subspace in $\mathbb{R}^{N}$. Here, “special”
refers to that, among all the $K$-dimensional subspaces in $\mathbb{R}^{N}$,
the one spanned by $\boldsymbol{U}_{dominant}$ is the best subspace with
respect to preserving information of $\boldsymbol{X}$. Specifically, among the
numerous $K$-dimensional subspaces in $\mathbb{R}^{N}$, the one spanned by
$\boldsymbol{U}_{dominant}$ preserves most information of $\boldsymbol{X}$
after projecting $\boldsymbol{X}$ onto it, i.e.,
$\boldsymbol{U}_{dominant}=\arg\min_{\boldsymbol{U}\in\mathbb{R}^{N\times
K}}\|\boldsymbol{X}-\boldsymbol{U}\boldsymbol{U}^{\top}\boldsymbol{X}\|_{F}^{2}\;\;s.t.\;\;\boldsymbol{U}^{\top}\boldsymbol{U}=\boldsymbol{I}$,
where $\|\cdot\|_{F}$ is the Frobenius norm. For the sake of brevity, we refer
to the $K$-dimensional subspace spanned by $\boldsymbol{U}_{dominant}$ as
dominant subspace hereafter. Second, the dominant subspace has numerous sets
of basis vectors since any $K$ linearly independent vectors in the dominant
subspace can span it, and $\boldsymbol{U}_{dominant}$ is one of these sets of
basis vectors.

###  3.2 Federated Feature Extraction Using Decentralized Data

In this section, we propose a federated feature extraction method that enables
multiple users to utilize their incomplete degradation signals to compute
their MFPC-scores (i.e., $\boldsymbol{\zeta}_{ij}$ in Equation (3)) while
keeping each user’s data local and confidential. Inspired by the feature
extraction method and the remarks discussed in Section 3.1, we first develop a
federated algorithm to detect the dominant subspace of the degradation signal
matrix $\boldsymbol{X}$ (Section 3.2.1). This provides a general set of basis
vectors of the dominant subspace. Next, we propose a federated algorithm that
uses the general set of basis vectors of the dominant subspace and (incomplete
and uncentered) degradation signals to calculate MFPC-scores (Section 3.2.2).

####  3.2.1 Federated Dominant Subspace Identification

The federated dominant subspace identification algorithm was inspired by the
incremental singular value decomposition (ISVD) method [35, 36, 37]. ISVD is
an iterative algorithm that performs SVD on a matrix by successively using
each row (or column) of the matrix to update the singular values and singular
vectors. It first starts with a randomly initialized subspace. Next, the rows
(or columns) of the matrix are projected onto the subspace one by one. After
projecting a row (or column) onto the current subspace, the residual vector is
analyzed. If no residual exists (i.e., the residual vector is a zero vector),
it implies that the row (or column) vector is already within the current
subspace. As a result, the current subspace does not need to be updated.
However, if a residual does exist, it means that the current subspace needs to
be revised. The revision is achieved by updating the current subspace so that
after projecting the row (or column) vector onto the revised subspace, the
norm of the new residual is minimized. The updating process is repeated (i.e.,
each row or column possibly gets involved more than once) until the norms of
the residuals of all the rows (or columns) are zero or smaller than a
predefined threshold.

Following the notation definitions in Section 3.1, the concatenated
degradation signal of user $i$’s $j$th system is denoted as
$\boldsymbol{x}_{ij}\in\mathbb{R}^{N}$, where $i=1,\ldots,I$ and
$j=1,\ldots,J_{i}$. Here, $I$ is the total number of users, and $J_{i}$ is the
number of systems that user $i$ has. Due to data missing and failure
truncation, the signal observations are incomplete. We use the set
$\boldsymbol{\Omega}_{ij}$ and its complementary set
$\boldsymbol{\Omega}_{ij}^{C}\subset\\{1,2,\ldots,N\\}\setminus\boldsymbol{\Omega}_{ij}$
to denote the indices of observations that are available and missing in
$\boldsymbol{x}_{ij}$, respectively. The proposed federated dominant subspace
identification algorithm works as follows. First, we randomly initialize the
subspace to be detected, which can be done by one of the users. Then, each
user uses their own data to update the subspace sequentially until
convergence.

We take the signal from system $j$ of user $i$, $\boldsymbol{x}_{ij}$, as an
example to explain how the subspace is updated. Let
$\boldsymbol{U}_{old}\in\mathbb{R}^{N\times K}$ be the orthonormal matrix
whose columns span the old subspace (before using $\boldsymbol{x}_{ij}$) and
$\boldsymbol{U}_{new}\in\mathbb{R}^{N\times K}$ be the matrix spans the
updated subspace (after using $\boldsymbol{x}_{ij}$). Here, $K$ is the
dimension of the subspace, $0<K\leq N$, the value of which can be determined
using cross-validation. We first project the incomplete signal
$\boldsymbol{x}_{ij}$ onto the old subspace. This is achieved by identifying a
linear combination of the basis vectors in $\boldsymbol{U}_{old}$ that can
best approximate $\boldsymbol{x}_{ij}$:

$\hat{\boldsymbol{w}}=\arg\min_{\boldsymbol{w}}\|\boldsymbol{U}_{old}^{\boldsymbol{\Omega}_{ij}}\boldsymbol{w}-\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}\|_{2}^{2},$
(4)

where $\hat{\boldsymbol{w}}\in\mathbb{R}^{K}$ is the weight vector,
$\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}\in\mathbb{R}^{|\boldsymbol{\Omega}_{ij}|}$
is the vector containing the observed entries of $\boldsymbol{x}_{ij}$, and
$\boldsymbol{U}_{old}^{\boldsymbol{\Omega}_{ij}}\in\mathbb{R}^{|\boldsymbol{\Omega}_{ij}|\times
K}$ is the matrix keeping the rows of $\boldsymbol{U}_{old}$ indexed by
$\boldsymbol{\Omega}_{ij}$. As a result, the projected vector in the old
subspace is $\boldsymbol{U}_{old}\hat{\boldsymbol{w}}$, and the missing
entries of $\boldsymbol{x}_{ij}$ can be imputed as follows:

$[\tilde{\boldsymbol{x}}_{ij}]_{n}=\begin{cases}[{\boldsymbol{x}}_{ij}]_{n},&n\in\boldsymbol{\Omega}_{ij}\\\
[\boldsymbol{U}_{old}\hat{\boldsymbol{w}}]_{n},&n\in\boldsymbol{\Omega}_{ij}^{C}\end{cases}\\\
$ (5)

where $[\boldsymbol{a}]_{n}$ represents the $n$th entry of vector
$\boldsymbol{a}$, and $\tilde{\boldsymbol{x}}_{ij}$ is the imputed signal of
user $i$’s system $j$. Thus, the residual vector can be computed as follows:

${\boldsymbol{r}}=\tilde{\boldsymbol{x}}_{ij}-\boldsymbol{U}_{old}\hat{\boldsymbol{w}}.$
(6)

If the norm of the residual vector $\boldsymbol{r}\in\mathbb{R}^{N}$ (i.e.,
$\|\boldsymbol{r}\|\in\mathbb{R}$) is not zero, it means that vector
$\boldsymbol{x}_{ij}$ is not fully within the subspace spanned by the columns
of $\boldsymbol{U}_{old}$. Consequently, the subspace needs to be revised such
that $\boldsymbol{x}_{ij}$ will be in the revised subspace or the residual
after projecting $\boldsymbol{x}_{ij}$ onto the revised subspace will be
minimized. The subspace revision can be achieved as follows. First, we
construct a new matrix by appending $\tilde{\boldsymbol{x}}_{ij}$ to the end
of $\boldsymbol{U}_{old}$ – that is –
${\boldsymbol{C}}=[\boldsymbol{U}_{old}\;\;\tilde{\boldsymbol{x}}_{ij}]\in\mathbb{R}^{N\times(K+1)}$.
By doing so, we are sure that $\tilde{\boldsymbol{x}}_{ij}$ is within the
$(K+1)$-dimensional subspace spanned by the columns of ${\boldsymbol{C}}$.
However, since the dimension of the subspace we want is $K$, we will conduct
SVD on ${\boldsymbol{C}}$ (i.e.,
${\boldsymbol{C}}=\tilde{\boldsymbol{U}}\boldsymbol{\Sigma}\boldsymbol{V}$)
and keep the first $K$ left singular vectors – that is– the updated subspace
is spanned by the columns of
$\boldsymbol{U}_{new}=\tilde{\boldsymbol{U}}{(1:N,1:K)}$, where $(1:N,1:K)$
represents all the $N$ rows and the first $K$ columns.

One of the limitations of the above subspace updating method is that it is
computationally intensive since SVD has to be conducted on an
$N\times(K+1)$-dimensional matrix ${\boldsymbol{C}}$ for each update.
Typically, $N$ is an extremely large number given it is the number of
observations in the longest degradation signal, while $K$ is usually a small
number since high-dimensional signals are often embedded in a relatively low-
dimensional subspace. To address this challenge, we may decompose matrix
${\boldsymbol{C}}$ as follows based on Equation (6):

${\boldsymbol{C}}=\left[\boldsymbol{U}\;\;\;\;\tilde{\boldsymbol{x}}\right]=\left[\boldsymbol{U}_{old}\;\;\;\;\frac{\boldsymbol{r}}{\|\boldsymbol{r}\|}\right]\begin{bmatrix}\boldsymbol{I}&\hat{\boldsymbol{w}}\\\
\boldsymbol{0}&\|\boldsymbol{r}\|\end{bmatrix},$ (7)

where $\boldsymbol{I}\in\mathbb{R}^{K\times K}$ is an identity matrix,
$\boldsymbol{0}\in\mathbb{R}^{1\times K}$ is a zero vector,
$\|\boldsymbol{r}\|\in\mathbb{R}$ is the norm of the residual vector
$\boldsymbol{r}$. Equation (7) implies that, instead of conducting SVD on
${\boldsymbol{C}}$, we may conduct SVD on
$\begin{bmatrix}\boldsymbol{I}&\hat{\boldsymbol{w}}\\\
\boldsymbol{0}&\|\boldsymbol{r}\|\end{bmatrix}$, which is a $(K+1)\times(K+1)$
matrix. Let $\begin{bmatrix}\boldsymbol{I}&\hat{\boldsymbol{w}}\\\
\boldsymbol{0}&\|\boldsymbol{r}\|\end{bmatrix}=\hat{\boldsymbol{U}}\hat{\boldsymbol{\Sigma}}\hat{\boldsymbol{V}}$,
where $\hat{\boldsymbol{U}}\in\mathbb{R}^{(K+1)\times(K+1)}$, then the left
singular vector matrix of ${\boldsymbol{C}}$ is
$\tilde{\boldsymbol{U}}=\left[\boldsymbol{U}\;\;\frac{\boldsymbol{r}}{\|\boldsymbol{r}\|}\right]\hat{\boldsymbol{U}}$.
As a result, $\boldsymbol{U}_{new}$ can be yielded by keeping the first $K$
columns of $\tilde{\boldsymbol{U}}$.

1User 1 Randomly initializes $\boldsymbol{U}_{new}$, the columns of which span
the subspace

2if  _The convergence criterion not met_ then

3 for  _User $i=1$ to $I$_ do

4 for  _System $j=1$ to $J_{i}$_ do

5

6 $\boldsymbol{U}_{old}\coloneqq\boldsymbol{U}_{new}$

7 Project signal $\boldsymbol{x}_{ij}$ onto the current subspace
$\hat{\boldsymbol{w}}\coloneqq\arg\min_{\boldsymbol{w}}\|\boldsymbol{U}_{old}^{\boldsymbol{\Omega}_{ij}}\boldsymbol{w}-\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}\|_{2}^{2}$

8 Impute missing entries of the signal
$[\tilde{\boldsymbol{x}}_{ij}]_{n}\coloneqq\begin{cases}[{\boldsymbol{x}}_{ij}]_{n},&n\in\boldsymbol{\Omega}_{ij}\\\
[\boldsymbol{U}_{old}\hat{\boldsymbol{w}}]_{n},&n\in\boldsymbol{\Omega}_{ij}^{C}\end{cases}$

9 Compute the residual vector
${\boldsymbol{r}}\coloneqq\tilde{\boldsymbol{x}}_{ij}-\boldsymbol{U}_{old}\hat{\boldsymbol{w}}$

10 Conduct SVD on $\begin{bmatrix}\boldsymbol{I}&\hat{\boldsymbol{w}}\\\
\boldsymbol{0}&\|\boldsymbol{r}\|\end{bmatrix}=\hat{\boldsymbol{U}}\hat{\boldsymbol{\Sigma}}\hat{\boldsymbol{V}}$

11
$\tilde{\boldsymbol{U}}\coloneqq\left[\boldsymbol{U}\;\;\frac{\boldsymbol{r}}{\|\boldsymbol{r}\|}\right]\hat{\boldsymbol{U}}$

12 $\boldsymbol{U}_{new}\coloneqq\tilde{\boldsymbol{U}}{(1:N,1:K)}$

13 end for

14

15 Share the new matrix $\boldsymbol{U}_{new}$ with the next user

16 end for

17

18 end if

Algorithm 1 Federated Dominant Subspace Identification

We summarize the proposed Federated Dominant Subspace Identification in
Algorithm 1. Similar to some of the matrix completion and incremental singular
value decomposition algorithms [37, 38, 39, 40], one of the possible metrics
to determine if Algorithm 1 has converged or not is the residual norm, which
is defined as follows:

$e=\sum_{i=1}^{I}\sum_{j=1}^{J_{i}}\frac{\|\boldsymbol{r}_{ij}\|}{\|\tilde{\boldsymbol{x}}_{ij}\|}=\sum_{i=1}^{I}\sum_{j=1}^{J_{i}}\frac{\|\boldsymbol{r}_{ij}^{\boldsymbol{\Omega}_{ij}}\|}{\|\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}\|}$
(8)

where $\|\cdot\|$ is the $\ell_{2}$ norm, and $\boldsymbol{r}_{ij}$ is the
residual of projecting vector $\boldsymbol{x}_{ij}$ onto the current subspace
(see Row 9 in Algorithm 1). To determine if Algorithm 1 has converged or not,
the first user will compute the summation of its residual norm (i.e.,
$\sum_{j=1}^{J_{1}}\frac{\|\boldsymbol{r}_{1j}\|}{\|\tilde{\boldsymbol{x}}_{1j}\|}$)
and pass it to the second user; the second user will add its residual norm to
the norm from the previous user and pass it to next user, so on so forth; the
last user will calculate the total residual norm $e$ from all the users. If
$e$ is smaller than a predefined small number, say $10^{-6}$, then Algorithm 1
is considered to be converged. Another possible way to terminate Algorithm 1
is the total number of iterations. Specifically, if the total number of
iterations exceeds a predefined threshold, say $100$, then Algorithm 1 stops
[36].

####  3.2.2 Federated Algorithm for MFPC-scores Computation

Algorithm 1 provides a (general) set of basis vectors (i.e.,
$\boldsymbol{U}_{new}$) for the $K$-dimensional dominant subspace of signal
matrix $\boldsymbol{{X}}$. As discussed in Section 3.1, MFPC-scores are
computed using
$\boldsymbol{Z}=\boldsymbol{U}_{dominant}^{\top}\boldsymbol{\tilde{X}}$, which
requires the centered signal matrix (i.e., $\boldsymbol{\tilde{X}}$) and its
first $K$ left singular vectors (i.e., the columns of
$\boldsymbol{U}_{dominant}$). However, both $\boldsymbol{\tilde{X}}$ and
$\boldsymbol{U}_{dominant}$ are unknown and cannot be easily computed under a
federated learning framework. To address this challenge, we will first show
that MFPC-scores can be computed using basis vectors $\boldsymbol{U}_{new}$
and the uncentered signal matrix $\boldsymbol{{X}}$. Next, we will propose a
federated algorithm that uses $\boldsymbol{U}_{new}$ and $\boldsymbol{{X}}$ to
compute MFPC-scores. This allows multiple users to jointly compute their MFPC-
scores while keeping each user’s data local and confidential.

To show that MFPC-scores can be computed using basis vectors
$\boldsymbol{U}_{new}$ and the uncentered signal matrix $\boldsymbol{{X}}$, we
first introduce Proposition 1.

###### Proposition 1

Given an uncentered matrix $\boldsymbol{X}\in\mathbb{R}^{N\times J}$ and
$\boldsymbol{U}_{new}\in\mathbb{R}^{N\times K}$ whose orthonormal columns span
the $K$-dimensional dominant subspace of $\boldsymbol{X}$, the MFPC-scores
discussed in Section 3.1 can be computed from $\boldsymbol{X}$ and
$\boldsymbol{U}_{new}$ as follows: (i) compute
$\boldsymbol{W}=\boldsymbol{U}_{new}^{\top}\boldsymbol{X}\in\mathbb{R}^{K\times
J}$, (ii) center $\boldsymbol{W}$ by computing
$\boldsymbol{\tilde{W}}=\boldsymbol{W}-\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times
J}$, where $\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times J}$ has $J$ identical
columns, each of which is the column mean of $\boldsymbol{{W}}$, (iii) conduct
SVD on $\boldsymbol{\tilde{W}}$–that
is–$\boldsymbol{\tilde{W}}=\boldsymbol{P}\boldsymbol{D}\boldsymbol{Q}^{\top}$,
where $\boldsymbol{P}\in\mathbb{R}^{K\times K}$ is the left singular vector
matrix, and (iv) compute MFPC-scores
$\boldsymbol{Z}=\boldsymbol{P}^{\top}\boldsymbol{\tilde{W}}\in\mathbb{R}^{K\times
J}$.

The proof of Proposition 1 can be found in the Appendix. Geometrically,
Proposition 1 can be explained as follows. The columns of
$\boldsymbol{U}_{new}\in\mathbb{R}^{N\times K}$ constructs a new coordinate
system in the $K$-dimensional dominant subspace of $\boldsymbol{X}$, which is
different from the default coordinate system in which $\boldsymbol{X}$ is
defined. Steps (i)-(iv) compute MFPC-scores in the new coordinate system.
Specifically, step (i),
$\boldsymbol{W}=\boldsymbol{U}_{new}^{\top}\boldsymbol{X}\in\mathbb{R}^{K\times
J}$, tries to express the the $J$ data points (a.k.a. samples) in
$\boldsymbol{X}\in\mathbb{R}^{N\times J}$ in the new coordinate system, and
$\boldsymbol{W}\in\mathbb{R}^{K\times J}$ contains the coordinates of the $J$
data points. In step (ii), we center $\boldsymbol{W}$, which yields
$\boldsymbol{\tilde{W}}=\boldsymbol{W}-\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times
J}$. This is the same as centering the original signal matrix $\boldsymbol{X}$
in the default coordinate system. This is because
$\boldsymbol{\tilde{W}}=\boldsymbol{U}_{new}^{\top}\boldsymbol{\tilde{X}}$,
where $\boldsymbol{\tilde{X}}=\boldsymbol{X}-\boldsymbol{\bar{X}}$ is the
centered signal matrix (see Section 3.1 for definitions). In step (iii), we
conduct SVD on $\boldsymbol{\tilde{W}}$, which gives
$\boldsymbol{\tilde{W}}=\boldsymbol{P}\boldsymbol{D}\boldsymbol{Q}^{\top}$,
where $\boldsymbol{P}\in\mathbb{R}^{K\times K}$ is the left singular vector
matrix. Since $\boldsymbol{\tilde{W}}$ contains the coordinates of the $J$
data points in the new coordinate system, the singular vectors in
$\boldsymbol{P}$ are defined in the new coordinate system as well. Step (iii)
is equivalent to conducting SVD on $\boldsymbol{\tilde{X}}$ in the default
coordinate system, the left $K$ dominant singular vectors of which are the
columns of $\boldsymbol{U}_{dominant}\in\mathbb{R}^{N\times K}$. Notice that
$\boldsymbol{P}\in\mathbb{R}^{K\times K}$ and
$\boldsymbol{U}_{dominant}\in\mathbb{R}^{N\times K}$ are the same set of
singular vectors defined in two different coordinate systems, where
$\boldsymbol{P}\in\mathbb{R}^{K\times K}$ is defined in the new coordinate
system constructed by the columns of
$\boldsymbol{U}_{new}\in\mathbb{R}^{N\times K}$, while
$\boldsymbol{U}_{dominant}\in\mathbb{R}^{N\times K}$ is defined in the default
coordinate system. The transformation of the two coordinate systems can be
easily achieved by using
$\boldsymbol{U}_{new}\boldsymbol{P}=\boldsymbol{U}_{dominant}$. Given the
singular vectors, step (iv) computes MFPC-scores
$\boldsymbol{Z}=\boldsymbol{P}^{\top}\boldsymbol{\tilde{W}}\in\mathbb{R}^{K\times
J}$. The geometric meaning of computing MFPC-scores is that the singular
vectors (the columns of $\boldsymbol{P}$) define another coordinate system,
and we express the centered $J$ data points $\boldsymbol{\tilde{W}}$ in this
coordinate system ($\boldsymbol{P}^{\top}\boldsymbol{\tilde{W}}$ are the new
coordinates) such that the first dimension will have the maximum variance, the
second dimension has the secondary maximum variance, so on so forth.

Proposition 1 indicates that MFPC-scores can be computed using basis vectors
$\boldsymbol{U}_{new}$ and the uncentered signal matrix $\boldsymbol{{X}}$.
Based on Proposition 1, we propose a federated algorithm that allows multiple
users to use $\boldsymbol{U}_{new}$ and their uncentered signals to compute
MFPC-scores. Following the notations defined earlier, there are $I$ users, and
the degradation signal of user $i$’s $j$th system is denoted as
$\boldsymbol{x}_{ij}\in\mathbb{R}^{N}$, where $i=1,\ldots,I$ and
$j=1,\ldots,J_{i}$. $I$ is the total number of users, and $J_{i}$ is the
number of systems that user $i$ has. Our proposed federated algorithm requires
a server to coordinate the MFPC-scores computation. It can be a standalone
server on the cloud or any of the users participating in federated learning.
The proposed federated learning algorithm works as follows.

First, each user computes the coordinates of its signals in the new coordinate
systems constructed by the columns of
$\boldsymbol{U}_{new}\in\mathbb{R}^{N\times K}$. The new coordinates of signal
$\boldsymbol{x}_{ij}$ is computed using
$\boldsymbol{w}_{ij}=\boldsymbol{U}_{new}^{\top}\boldsymbol{x}_{ij}\in\mathbb{R}^{K}$
if $\boldsymbol{x}_{ij}$ is complete. However, degradation signal observations
are often incomplete, so not all the observations of $\boldsymbol{x}_{ij}$ are
available. To address this challenge, we follow the suggestion of article [41]
and compute $\boldsymbol{w}_{ij}$ by solving the following optimization
problem
$\boldsymbol{w}_{ij}\coloneqq\min_{\boldsymbol{w}_{ij}}{\|\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}-\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}\boldsymbol{w}_{ij}\|_{2}^{2}}$,
which yields the solution
$\boldsymbol{w}_{ij}=({\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}}^{\top}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}})^{-1}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}$.
Here $\boldsymbol{\Omega}_{ij}$ is the set of indices of available
observations in $\boldsymbol{x}_{ij}$. This is a reasonable estimator because
the elements of vector $\boldsymbol{w}_{ij}$ are the weights when expressing
$\boldsymbol{x}_{ij}$ as a linear weighted combination of the columns of
$\boldsymbol{U}_{new}$. After computing the weights from all $J_{i}$ signals
(i.e.,
$\boldsymbol{w}_{i1},\boldsymbol{w}_{i2},\ldots,\boldsymbol{w}_{iJ_{i}}$),
user $i$ will send
$\boldsymbol{w}_{(i)}=[\boldsymbol{w}_{i1},\boldsymbol{w}_{i2},\ldots,\boldsymbol{w}_{iJ_{i}}]\in\mathbb{R}^{K\times
J_{i}}$ to the server.

Second, the server receives $\boldsymbol{w}_{i}$ from all $I$ users and
integrates them into matrix
$\boldsymbol{W}=[\boldsymbol{w}_{(1)},\boldsymbol{w}_{(2)},\ldots,\boldsymbol{w}_{(I)}]\in\mathbb{R}^{K\times
J}$, where $J=\sum_{i=1}^{I}J_{i}$. Then, MFPC-scores can be computed by
following steps (ii)-(iv) in Proposition 1. Specifically, the server first
centers $\boldsymbol{W}$ by computing
$\boldsymbol{\tilde{W}}=\boldsymbol{W}-\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times
J}$, where $\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times J}$ has $J$ identical
columns, each of which is
$\bar{\boldsymbol{w}}=\sum_{i=1}^{I}\sum_{j=1}^{J_{i}}\boldsymbol{w}_{ij}\in\mathbb{R}^{K}$.
Next, it conducts SVD on $\boldsymbol{\tilde{W}}$–that
is–$\boldsymbol{\tilde{W}}=\boldsymbol{P}\boldsymbol{D}\boldsymbol{Q}^{\top}$,
where $\boldsymbol{P}\in\mathbb{R}^{K\times K}$ is the left singular vector
matrix. Finally, MFPC-scores are computed by using
$\boldsymbol{Z}=\boldsymbol{P}^{\top}\boldsymbol{\tilde{W}}\in\mathbb{R}^{K\times
J}$. Matrix $\boldsymbol{Z}$ consists of the MFPC-scores of all $I$ users,
where the first $J_{1}$ columns are the MFPC-scores of user $1$ (denoted as
$\boldsymbol{z}_{1}\in\mathbb{R}^{K\times J_{1}}$), the following $J_{2}$
columns are the scores of user $2$ (denoted as
$\boldsymbol{z}_{2}\in\mathbb{R}^{K\times J_{2}}$), so on so forth. In other
words, $\boldsymbol{Z}=[\boldsymbol{z}_{1}\in\mathbb{R}^{K\times
J_{1}},\boldsymbol{z}_{2}\in\mathbb{R}^{K\times
J_{2}},\ldots,\boldsymbol{z}_{I}\in\mathbb{R}^{K\times J_{I}}]$. We summarize
the proposed federated algorithm for MFPC-scores computation in Algorithm 2.

1User side

2for  _User $i=1$ to $I$_ do

3 for  _$j=1$ to $J_{i}$_ do

4 Compute
$\boldsymbol{w}_{ij}=({\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}}^{\top}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}})^{-1}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}\in\mathbb{R}^{K}$

5 end for

6 Merge
$\boldsymbol{w}_{(i)}=[\boldsymbol{w}_{i1},\boldsymbol{w}_{i2},\ldots,\boldsymbol{w}_{iJ_{i}}]\in\mathbb{R}^{K\times
J_{i}}$ and send it to server

7 end for

8

9Server side

10Integrate $\boldsymbol{w}_{(i)}$ from all users, which yields
$\boldsymbol{W}=[\boldsymbol{w}_{(1)},\boldsymbol{w}_{(2)},\ldots,\boldsymbol{w}_{(I)}]\in\mathbb{R}^{K\times
J}$, where $J=\sum_{i=1}^{I}J_{i}$.

11Center $\boldsymbol{W}$ by computing
$\boldsymbol{\tilde{W}}=\boldsymbol{W}-\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times
J}$, where $\boldsymbol{\bar{W}}\in\mathbb{R}^{K\times J}$ has $J$ identical
columns, each of which is
$\bar{\boldsymbol{w}}=\sum_{i=1}^{I}\sum_{j=1}^{J_{i}}\boldsymbol{w}_{ij}\in\mathbb{R}^{K}$.

12Conduct SVD on $\boldsymbol{\tilde{W}}$, i.e.,
$\boldsymbol{\tilde{W}}=\boldsymbol{P}\boldsymbol{D}\boldsymbol{Q}^{\top}$,
where $\boldsymbol{P}\in\mathbb{R}^{K\times K}$ is the left singular vector
matrix.

13Compute MFPC-scores
$\boldsymbol{Z}=\boldsymbol{P}^{\top}\boldsymbol{\tilde{W}}\in\mathbb{R}^{K\times
J}$.

14Send $\boldsymbol{P}$, $\bar{\boldsymbol{w}}$, and each user’s MFPC-scores
back.

Algorithm 2 Federated Algorithm for MFPC-scores Computation

Notice that user $i$ sends $\boldsymbol{w}_{(i)}$ instead of its signals
$\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}$ to the server. Since
$\boldsymbol{w}_{ij}=({\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}}^{\top}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}})^{-1}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{ij}}\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}$
and $\boldsymbol{\Omega}_{ij}$ is confidential (user $i$’s
$\boldsymbol{\Omega}_{ij}$ is not shared with other users), signal
$\boldsymbol{x}_{ij}^{\boldsymbol{\Omega}_{ij}}$ cannot be derived from
$\boldsymbol{w}_{ij}$. This protects the data privacy of each user’s data.

###  3.3 Prognostic Model Training and Time-To-Failure Prediction

The federated algorithm for MFPC-scores computation in Section 3.2.2 sends
$\boldsymbol{P}$, $\bar{\boldsymbol{w}}$, and MFPC-scores
$\boldsymbol{z}_{i}\in\mathbb{R}^{K\times J_{i}}$ back to user $i$,
$i=1,\ldots,I$. All the $I$ users will jointly train a prognostic model based
on (log)-location-scale (LLS) regression using their MFPC-scores
$\\{\boldsymbol{z}_{i}\\}_{i=1}^{I}$ and TTFs
$\\{\boldsymbol{y}_{i}\in\mathbb{R}^{J_{i}}\\}_{i=1}^{I}$. In this article, we
will employ the federated LLS regression model developed in [27], which allows
the $I$ users to collaboratively train the prognostic model while keeping each
user’s MFPC-scores and TTFs local and confidential. As a result, each user
will have the prognostic model trained using the data from all users and can
use it for real-time TTF prediction.

Assume user $i$ has a partially degraded system whose real-time (incomplete)
degradation signal is denoted as
$\boldsymbol{x}_{i,r}^{\boldsymbol{\Omega}_{i,r}}$. To predict the system’s
TTF, we first compute
$\boldsymbol{w}_{i,r}=({\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{i,r}}}^{\top}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{i,r}})^{-1}\boldsymbol{U}_{new}^{\boldsymbol{\Omega}_{i,r}}\boldsymbol{x}_{i,r}^{\boldsymbol{\Omega}_{i,r}}\in\mathbb{R}^{K}$.
Next, MFPC-scores can be computed
$\boldsymbol{z}_{i,r}=\boldsymbol{P}^{\top}(\boldsymbol{w}_{i,r}-\bar{\boldsymbol{w}})$.
Finally, $\boldsymbol{z}_{i,r}$ is fed into the trained prognostic model, and
TTF can be predicted.

##  4 Simulation Study

In this section, we conduct some simulation studies to validate the
effectiveness of the proposed federated prognostic model.

###  4.1 Data Generation and Benchmarking Models

We generate degradation signals and the TTFs of $120$ systems and assign $54$
systems to user 1, $27$ to user 2, and $9$ to user 3. The remaining $30$
systems are considered test data. Similar to [31], we will first generate the
underlying degradation trajectories, based on which TTFs are computed. Next,
we will generate degradation signals, which are noisy discrete observations of
underlying degradation trajectories. Specifically, the underlying degradation
trajectory of system $i$ is simulated using the equation below:

$s_{i}(t)=-c_{i}/\ln{(t)},$ (9)

where $c_{i}\sim N(1,0.25)$, $0\leq t<1$, $i=1,\ldots,100$. The TTFs are
computed as the first time that $s_{i}(t)$ reaches or crosses a predefined
threshold $D$: $s_{i}(t)=-c_{i}/\ln{(\tilde{y_{i}})}=D$. This yields
$\ln{(\tilde{y_{i}})}=-c_{i}/D$, where $\tilde{y_{i}}$ is the TTF of system
$i$. To mimic data acquisition errors, we add noise to the true TTFs. As a
result, the observed TTFs are computed from
$\ln{(\tilde{y_{i}})}=-c_{i}/D+\epsilon_{i}$, where $\epsilon_{i}\sim
N(0,0.025^{2})$. It can be easily shown that the relationship between TTFs and
degradation signals is lognormal, and thus the lognormal regression will be
employed in this simulation study to verify the performance of the proposed
federated prognostic model. With the underlying degradation trajectory, the
noisy discrete observations (i.e., the observed degradation signal from a
condition monitoring sensor) of system $i$, $i=1,\ldots,100$, are generated as
follows:

${x}(\tau_{i})=-c_{i}/\ln{(\tau_{i})}+{\varepsilon}(\tau_{i}),$ (10)

where ${\varepsilon}(\tau_{i})\sim N(0,0.2)$ is the random observation noise,
$\tau_{i}=0,1\times 15e^{-4},2\times 15e^{-4},3\times
15e^{-4},\ldots,\lfloor\tilde{y_{i}}/15e^{-4}\rfloor\times 15e^{-4}$, and
$\lfloor\cdot\rfloor$ is the floor function.

We compare the performance of the proposed federated prognostic model
(designated as “Federated Model”) with two benchmarking methods. The first
baseline is referred to as the “Non-Federated Model”, which works by first
aggregating the degradation data from all the users. Then, the prognostic
model proposed in [31] is applied to the aggregated data for failure time
prediction. To the best of our knowledge, the model in [31] is the prognostic
model that is closest to the model proposed in this article. It has three
steps: matrix completion for missing data imputation, functional principal
component analysis for data fusion, and LLS regression for failure time
prediction. However, the model in [31] is not federated and thus cannot
protect users’ data privacy. The second baseline, which we refer to as the
“Individual Model”, means each user trains its own prognostic model using the
data it has. In other words, the data from different users are not used to
jointly train the prognostic model.

We consider three levels of data incompleteness, $30\%$, $50\%$, and $70\%$,
where, for example, $30\%$ means that $30\%$ observations of the generated
degradation signals are randomly removed. Under each incompleteness level, we
employ 15 random permutations of the data to evaluate the performance of the
proposed method as well as the benchmarks. The rank of the dominant subspace
in the proposed method and the number of MFPC-scores in all the models are
chosen using 5-fold cross-validation. The predictor errors are computed using
the following formula:

$\text{Prediction Error}=\frac{|\text{Predicted TTF - Real TTF}|}{\text{Real
TTF}}\times 100\%.$ (11)

###  4.2 Results and Analysis

Figure 1 summarizes the prediction errors of the proposed federated model and
the first baseline, which is a non-federated model that works on the
aggregated data. It can be seen from Figure 1 that the proposed federated
model achieves the same prediction accuracy as the non-federated benchmark at
all data missing levels. For example, when $30\%$ observations of degradation
signals are missing, the median (and the Interquartile Range, i.e., IQR) of
the federated model and non-federated model are both $0.013(0.016)$; when the
data missing level is $50\%$, the median (and IQR) of both models are
$0.017(0.023)$; furthermore, the median (and IQR) are $0.026(0.034)$ for both
models when the data missing level is $70\%$. This implies that the proposed
federated method does not compromise the prediction performance of prognostic
models. Another observation from Figure 5 is that the performance of the
proposed model and the first benchmark degrades as the data missing level
increases. This is reasonable since a higher level of data incompleteness
means more data observations are missing, which results in a more inaccurate
feature extraction, parameter estimation, and thus TTF prediction.

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 1: The prediction errors of federated and non-federated models.

Figures 2, 3, and 4 report the prediction errors of the proposed federated
model and the second benchmark, i.e., the individual model. We can conclude
from Figures 2, 3, and 4 that the federated model outperforms individual
models at all levels of data incompleteness. For example, it can be seen in
Figure 2 that the median (and IQR) of prediction errors of the proposed
federated model and the individual model constructed by User 1 are
$0.013(0.016)$ and $0.015(0.019)$ respectively when data missing level is
$30\%$; when $50\%$ data are missing, they are respectively $0.017(0.023)$ and
$0.017(0.0.24)$; when data missing level is $70\%$, they are $0.026(0.034)$
and $0.027(0.034)$, respectively. As another example, in Figure 4, the median
(and IQR) of the proposed federated model and the individual model built by
User 3 are $0.013(0.016)$ and $0.017(0.024)$, $0.017(0.023)$ and
$0.025(0.029)$, and $0.026(0.034)$ and $0.067(0.089)$ when data missing levels
are $30\%$, $50\%$, and $70\%$, respectively. A similar observation can also
be found in Figure 3 as well. This is reasonable since the proposed federated
model uses samples from all three users for model training, while the
individual models use a smaller number of training samples owned by each
individual user.

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 2: The prediction errors of federated model and individual model (User
1)

Another conclusion can be drawn from Figures 2, 3, and 4 is that the
performance of individual models constructed by each user is significantly
affected by the number of training samples owned by the user. For example,
when the data missing level is $30\%$, the median (and IQR) of prediction
errors of the individual model constructed by User 1 (Figure 2), User 2
(Figure 3), and User 3 (Figure 4) are respectively $0.015(0.019)$,
$0.016(0.020)$, and $0.017(0.024)$. This is reasonable since the sample sizes
of Users 1, 2, and 3 are $54$, $27$, and $9$, respectively, and when the
training sample size is relatively limited, a larger number of training data
usually help improve the model performance. This confirms the importance of
combining training data from multiple users for a joint prognostic model
construction, which also affirms the value of federated failure time
prediction.

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 3: The prediction errors of federated model and individual model (User
2)

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 4: The prediction errors of federated model and individual model (User
3)

##  5 Case Study

###  5.1 Data Introduction and Benchmarking Models

In this section, we use a dataset from the NASA data repository to verify the
performance of the proposed federated prognostic model. The dataset contains
multi-sensor degradation signals and TTFs of commercial aircraft turbofan
engines [42]. It includes 100 training engines that were run to failure and
100 test engines that were shut down at a random time before failure. In total
21 sensors were used to monitor the health condition of each of the engines,
which generated 21-channel degradation signals. Readers may refer to [42],
[6], and [7] for detailed information regarding the dataset. According to [7],
not all 21 sensors are informative for prognostics, and the four crucial
sensors suggested by the authors are Total temperature at LPT outlet, Bypass
Ratio, Bleed Enthalpy, and HPT coolant bleed (senor indices 4,15,17, and 20).
In this article, we will use degradation signals from these 4 sensors to
validate the performance of the proposed federated prognostic model.

Similar to the simulation study in Section 4, we randomly assign the training
data to three users, where Users 1, 2, and 3 have $60$, $30$, and $10$
training engines, respectively. We also compare the performance of the
proposed “Federated Model” with two benchmarks, the “Non-Federated Model” and
the “Individual Model”. The proposed model and benchmarks use the same test
dataset, i.e., the $100$ test engines. We consider three levels of data
incompleteness, $30\%$, $50\%$, and $70\%$. Under each incompleteness level,
15 random permutations of data are employed to evaluate the performance of the
proposed method as well as the benchmarks. The rank of the dominant subspace
in the proposed method and the number of MFPC-scores in all the models are
chosen using 5-fold cross-validation.

###  5.2 Results and Analysis

The prediction performance of the proposed “Federated Model” and the “Non-
Federated Model” is reported in Figure 5, which indicates that the two models
achieve the same prediction accuracy and precision. For example, when the data
missing level is $30\%$, the median (and IQR) of the two models are both
0.081(0.125); the median (and IQR) of them are 0.096(0.135) when the data
missing level changes to $50\%$ and 0.117(0.157) when $70\%$ observations are
missing. This suggests that the proposed “Federated Model” can accomplish the
same prediction accuracy and precision as the “Non-Federated Model” which
works on an aggregated dataset.

The performance of the proposed “Federated Model” and the second baseline
model, “Individual Model”, are reported in Figures 6, 7, and 8, which suggest
that the conclusions drawn from the simulation study still hold. The first
conclusion is that the proposed “Federated Model” works better than the
“Individual Model” at all levels of data incompleteness. For example, the
medians (and IQRs) of the federated model and the individual model constructed
by User 1 are 0.081(0.125) and 0.086(0.127), 0.096(0.135) and 0.102(0.144), as
well as 0.117(0.157) and 0.126(0.169), when the data incompleteness levels are
$30\%$, $50\%$, and $70\%$, respectively; the medians (and IQRs) of the
federated model and the individual model constructed by User 2 are
0.081(0.125) and 0.121(0.162), 0.096(0.135) and 0.118(0.166), as well as
0.117(0.157) and 0.136(0.183) when $30\%$, $50\%$, and $70\%$ data are
missing, respectively. We believe this is reasonable since the federated model
uses training data from all three users, and thus the size of its samples for
model training is larger than that of each user. Specifically, each of the
three users has a small number of samples for model training–that is–$60$ for
User 1, $30$ for User 2, and $10$ for User 3, so the federated model has $100$
samples in total for model establishment. As a result, the trained federated
model is more accurate than the individual models trained by each user itself.

Another conclusion is that the performance of the individual models is
affected by the size of the training samples. For example, when the data
missing level is $30\%$, the medians (and IQRs) of the models constructed by
Users 1, 2, and 3 are 0.086(0.127), 0.121(0.162), and 0.129(0.169),
respectively. Similarly, when $50\%$ data are missing, the medians (and IQRs)
are respectively 0.102(0.144), 0.118(0.166), and 0.140(0.176); when the data
incompleteness level is $70\%$, they are respectively 0.126(0.169),
0.136(0.183), and 0.159(.188). This again proves the importance of sample size
in establishing a high-performance prognostic model and thus verifies the
value and necessity of federated learning-based prognostic when the size of
samples owned by each individual user is small.

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 5: The prediction errors of federated model and non-federated model.

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 6: The prediction errors of federated model and individual model (User
1)

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 7: The prediction errors of federated model and individual model (User
2)

(a) 30% missing values

(b) 50% missing values

(c) 70% missing values

Figure 8: The prediction errors of federated model and individual model (User
3)

##  6 Conclusions

In this article, we proposed a federated prognostic model that allows multiple
users to collaboratively train the model while keeping the data of each user
local and confidential. The proposed model works for applications with
multiple stream degradation signals, which may be incomplete (i.e., some
observations are missing). As some of the existing works in remaining useful
lifetime prediction, the prognostic model used in this paper first uses
multivariate functional principal component analysis (MFPCA) to fuse the
multi-stream degradation signals; the fused features (known as MFPC-scores)
coupled with the times-to-failure (TTFs) are utilized to build a
(log)-location-scale regression model for failure time prediction. To address
the challenges that 1) some observations of degradation signals are missing
and thus classic algorithms for MFPC-score computation cannot be used, and 2)
the feature extraction (i.e., MFPC-score computation) should be federated, we
proposed a federated feature extraction method that enables multiple users to
utilize their incomplete degradation signals to compute their MFPC-scores
while keeping each user’s data local and confidential.

The proposed federated feature extraction method consists of two steps. First,
we developed a federated algorithm to detect the dominant subspace of the
degradation signal matrix, which provides a general set of basis vectors of
the dominant subspace. Next, we propose a federated algorithm that uses the
general set of basis vectors of the dominant subspace to calculate the
dominant eigenvectors as well as the MFPC-scores. Simulated datasets as well
as a degradation dataset from aircraft engines were used to evaluate the
performance of the proposed federated prognostic model. The results indicated
that the proposed federated prognostic model accomplished the same accuracy
and precision for failure time prediction as that of the non-federated model
using an aggregated dataset from all users. This suggests that the proposed
model can protect the data privacy of users while not compromise the
performance of failure time prediction. In addition, the results showed that
the performance of the proposed federated model is better than that of the
prognostic models constructed by each user itself, especially when a user has
a small number of data samples for model training. This further verifies the
value and necessity of the developed federated prognostic model.

##  7 Appendix

####  7.0.1 Proof of Proposition 1

It is known that $\boldsymbol{U}_{dominant}$ and $\boldsymbol{P}$ represent
the same set of eigenvectors defined in two different coordinate systems.
$\boldsymbol{P}\in\mathbb{R}^{K\times K}$ is defined in the new coordinate
system constructed by the columns of
$\boldsymbol{U}_{new}\in\mathbb{R}^{N\times K}$. In the meanwhile,
$\boldsymbol{U}_{dominant}\in\mathbb{R}^{N\times K}$ and
$\boldsymbol{U}_{new}$ itself are defined in the default coordinate system
(the same coordinate system in which the signal matrix $\boldsymbol{X}$ is
defined). Since $\boldsymbol{U}_{dominant}$ is a set of vectors defined in the
default coordinate system,
$\boldsymbol{U}_{new}^{\top}\boldsymbol{U}_{dominant}$ gives its
representation in the new coordinate system, which is known as
$\boldsymbol{P}$. In other words,
$\boldsymbol{P}=\boldsymbol{U}_{new}^{\top}\boldsymbol{U}_{dominant}$ or
$\boldsymbol{U}_{new}\boldsymbol{P}=\boldsymbol{U}_{dominant}$. MFPC-scores
$\boldsymbol{Z}=\boldsymbol{U}_{dominant}^{\top}\boldsymbol{\tilde{X}}=(\boldsymbol{U}_{new}\boldsymbol{P})^{\top}\boldsymbol{\tilde{X}}=\boldsymbol{P}^{\top}\boldsymbol{U}_{new}^{\top}\boldsymbol{\tilde{X}}=\boldsymbol{P}^{\top}\boldsymbol{U}_{new}^{\top}(\boldsymbol{X}-\boldsymbol{\bar{X}})=\boldsymbol{P}^{\top}\boldsymbol{U}_{new}^{\top}\boldsymbol{X}-\boldsymbol{P}^{\top}\boldsymbol{U}_{new}^{\top}\boldsymbol{\bar{X}}=\boldsymbol{P}^{\top}\boldsymbol{W}-\boldsymbol{P}^{\top}\boldsymbol{\bar{W}}=\boldsymbol{P}^{\top}(\boldsymbol{W}-\boldsymbol{\bar{W}})=\boldsymbol{P}^{\top}\boldsymbol{\tilde{W}}$.

##  8 Data Availability Statement

The data that support the findings of this study are openly available in NASA
Prognostics Center of Excellence Data Set Repository at
https://www.nasa.gov/content/prognostics-center-of-excellence-data-set-
repository.

## References

  * [1] Nagi Gebraeel, Mark Lawley, Richard Liu, and Vijay Parmeshwaran.  Residual life predictions from vibration-based degradation signals: a neural network approach.  IEEE Transactions on industrial electronics, 51(3):694–700, 2004\.
  * [2] Joanna Z Sikorska, Melinda Hodkiewicz, and Lin Ma.  Prognostic modelling options for remaining useful life estimation by industry.  Mechanical systems and signal processing, 25(5):1803–1836, 2011\.
  * [3] Yuanyuan Gao, Yuxin Wen, and Jianguo Wu.  A neural network-based joint prognostic model for data fusion and remaining useful life prediction.  IEEE Transactions on Neural Networks and Learning Systems, 32(1):117–127, 2021.
  * [4] Nagi Z Gebraeel and Mark A Lawley.  A neural network degradation model for computing and updating residual life distributions.  IEEE Transactions on Automation Science and Engineering, 5(1):154–163, 2008.
  * [5] Nagi Gebraeel, Alaa Elwany, and Jing Pan.  Residual life predictions in the absence of prior degradation knowledge.  IEEE Transactions on Reliability, 58(1):106–117, 2009.
  * [6] Xiaolei Fang, Nagi Z Gebraeel, and Kamran Paynabar.  Scalable prognostic models for large-scale condition monitoring applications.  IISE Transactions, 49(7):698–710, 2017.
  * [7] Xiaolei Fang, Kamran Paynabar, and Nagi Gebraeel.  Multistream sensor fusion-based prognostics model for systems with single failure modes.  Reliability Engineering & System Safety, 159:322–331, 2017.
  * [8] Stuart L Pardau.  The california consumer privacy act: towards a european-style privacy regime in the united states.  J. Tech. L. & Pol’y, 23:68, 2018.
  * [9] Paul Voigt and Axel Von dem Bussche.  The eu general data protection regulation (gdpr).  A Practical Guide, 1st Ed., Cham: Springer International Publishing, 10:3152676, 2017.
  * [10] Warren B Chik.  The singapore personal data protection act and an assessment of future trends in data privacy reform.  Computer Law & Security Review, 29(5):554–575, 2013.
  * [11] Raed Kontar, Naichen Shi, Xubo Yue, Seokhyun Chung, Eunshin Byon, Mosharaf Chowdhury, Jionghua Jin, Wissam Kontar, Neda Masoud, Maher Nouiehed, et al.  The internet of federated things (ioft).  IEEE Access, 9:156071–156113, 2021.
  * [12] Jakub Konečnỳ, H Brendan McMahan, Daniel Ramage, and Peter Richtárik.  Federated optimization: Distributed machine learning for on-device intelligence.  arXiv preprint arXiv:1610.02527, 2016.
  * [13] Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon.  Federated learning: Strategies for improving communication efficiency.  arXiv preprint arXiv:1610.05492, 2016.
  * [14] H Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Agüera y Arcas.  Federated learning of deep networks using model averaging.(2016).  arXiv preprint arXiv:1602.05629, 2016.
  * [15] Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He.  A survey on federated learning systems: vision, hype and reality for data privacy and protection.  arXiv preprint arXiv:1907.09693, 2019.
  * [16] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong.  Federated machine learning: Concept and applications.  ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1–19, 2019.
  * [17] Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin.  A review of applications in federated learning.  Computers & Industrial Engineering, page 106854, 2020.
  * [18] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith.  Federated learning: Challenges, methods, and future directions.  IEEE Signal Processing Magazine, 37(3):50–60, 2020.
  * [19] Christopher Briggs, Zhong Fan, and Peter Andras.  A review of privacy-preserving federated learning for the internet-of-things.  Federated Learning Systems, pages 21–50, 2021.
  * [20] Behrad Bagheri, Maryam Rezapoor, and Jay Lee.  A unified data security framework for federated prognostics and health management in smart manufacturing.  Manufacturing Letters, 24:136–139, 2020.
  * [21] Xi Chen, Hui Wang, Siliang Lu, and Ruqiang Yan.  Bearing remaining useful life prediction using federated learning with taylor-expansion network pruning.  IEEE Transactions on Instrumentation and Measurement, 72:1–10, 2023\.
  * [22] Ji-Yan Wu, Min Wu, Zhenghua Chen, Xiao-Li Li, and Ruqiang Yan.  Degradation-aware remaining useful life prediction with lstm autoencoder.  IEEE Transactions on Instrumentation and Measurement, 70:1–10, 2021\.
  * [23] Seokhyun Chung and Raed Al Kontar.  Federated condition monitoring signal prediction with improved generalization.  IEEE Transactions on Reliability, pages 1–13, 2023.
  * [24] Liang Guo, Yaoxiang Yu, Mengui Qian, Ruiqi Zhang, Hongli Gao, and Zhe Cheng.  Fedrul: A new federated learning method for edge-cloud collaboration based remaining useful life prediction of machines.  IEEE/ASME Transactions on Mechatronics, 28(1):350–359, 2023.
  * [25] Xi Chen, Hui Wang, Siliang Lu, and Ruqiang Yan.  Bearing remaining useful life prediction using federated learning with taylor-expansion network pruning.  IEEE Transactions on Instrumentation and Measurement, 72:1–10, 2023\.
  * [26] Liang Guo, Yaoxiang Yu, Mengui Qian, Ruiqi Zhang, Hongli Gao, and Zhe Cheng.  Fedrul: A new federated learning method for edge-cloud collaboration based remaining useful life prediction of machines.  IEEE/ASME Transactions on Mechatronics, 2022.
  * [27] Yuqi Su and Xiaolei Fang.  Federated (log)-location-scale regression.  2022\.
  * [28] Kaibo Liu, Nagi Z Gebraeel, and Jianjun Shi.  A data-level fusion model for developing composite health indices for degradation modeling and prognostic analysis.  IEEE Transactions on Automation Science and Engineering, 10(3):652–664, 2013.
  * [29] Louis Doray.  Ibnr reserve under a loglinear location-scale regression model.  In Casualty Actuarial Society Forum, volume 2, pages 607–652. Citeseer, 1994.
  * [30] Xiaolei Fang, Rensheng Zhou, and Nagi Gebraeel.  An adaptive functional regression-based prognostic model for applications with missing data.  Reliability Engineering & System Safety, 133:266–274, 2015.
  * [31] Xiaolei Fang, Hao Yan, Nagi Gebraeel, and Kamran Paynabar.  Multi-sensor prognostics modeling for applications with highly incomplete signals.  IISE Transactions, 53(5):597–613, 2021.
  * [32] WQ Meeker and LA Escobar.  Statistical methods for reliability data john wiley & sons new york.  New York, 1998.
  * [33] K Karhunen.  Ueber lineare methoden in der wahrscheinlichkeitsrechnung issue 37 of annales academiae scientiarum fennicae.  Series A, 1, 1947.
  * [34] Fang Yao, Hans-Georg Müller, and Jane-Ling Wang.  Functional data analysis for sparse longitudinal data.  Journal of the American statistical association, 100(470):577–590, 2005.
  * [35] James R Bunch and Christopher P Nielsen.  Updating the singular value decomposition.  Numerische Mathematik, 31(2):111–129, 1978.
  * [36] Laura Balzano and Stephen J Wright.  On grouse and incremental svd.  In 2013 5th IEEE international workshop on computational advances in multi-sensor adaptive processing (CAMSAP), pages 1–4. IEEE, 2013\.
  * [37] Matthew Brand.  Incremental singular value decomposition of uncertain data with missing values.  In Computer Vision—ECCV 2002: 7th European Conference on Computer Vision Copenhagen, Denmark, May 28–31, 2002 Proceedings, Part I 7, pages 707–720. Springer, 2002.
  * [38] Laura Balzano, Robert Nowak, and Benjamin Recht.  Online identification and tracking of subspaces from highly incomplete information.  In 2010 48th Annual allerton conference on communication, control, and computing (Allerton), pages 704–711. IEEE, 2010.
  * [39] Wei Dai and Olgica Milenkovic.  Set: An algorithm for consistent matrix completion.  In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 3646–3649. IEEE, 2010.
  * [40] Raghunandan H Keshavan, Andrea Montanari, and Sewoong Oh.  Matrix completion from a few entries.  IEEE transactions on information theory, 56(6):2980–2998, 2010\.
  * [41] Laura Balzano, Benjamin Recht, and Robert Nowak.  High-dimensional matched subspace detection when data are missing.  In 2010 IEEE International Symposium on Information Theory, pages 1638–1642. IEEE, 2010.
  * [42] Abhinav Saxena, Kai Goebel, Don Simon, and Neil Eklund.  Damage propagation modeling for aircraft engine run-to-failure simulation.  In 2008 international conference on prognostics and health management, pages 1–9. IEEE, 2008.