# P3LS: Partial Least Squares under Privacy Preservation

Nguyen Duy Du  Ramin Nikzad-Langerodi<EMAIL_ADDRESS>
###### Abstract

Modern manufacturing value chains require intelligent orchestration of
processes across company borders in order to maximize profits while fostering
social and environmental sustainability. However, the implementation of
integrated, systems-level approaches for data-informed decision-making along
value chains is currently hampered by privacy concerns associated with cross-
organizational data exchange and integration. We here propose Privacy-
Preserving Partial Least Squares (P3LS) regression, a novel federated learning
technique that enables cross-organizational data integration and process
modeling with privacy guarantees. P3LS involves a singular value decomposition
(SVD) based PLS algorithm and employs removable, random masks generated by a
trusted authority in order to protect the privacy of the data contributed by
each data holder. We demonstrate the capability of P3LS to vertically
integrate process data along a hypothetical value chain consisting of three
parties and to improve the prediction performance on several process-related
key performance indicators. Furthermore, we show the numerical equivalence of
P3LS and PLS model components on simulated data and provide a thorough privacy
analysis of the former. Moreover, we propose a mechanism for determining the
relevance of the contributed data to the problem being addressed, thus
creating a basis for quantifying the contribution of participants.

###### keywords:

Cross-organizational Process Modeling , Partial Least Squares Regression ,
Privacy Preserving Machine Learning , Vertical Federated Learning

††journal: Journal of Process Control

[inst1]organization=Software Competence Center Hagenberg, country=Austria

Highlight 1: We developed an approach that allows multiple companies along a
value chain to collaboratively construct and exploit a cross-organizational
process model while protecting their data privacy and security.

Highlight 2: We proposed an incentive mechanism that quantifies participants’
contributions, thus creating a basis for a profit-sharing scheme and
motivating the companies to join the federation and contribute high-quality
data.

Highlight 3: We experimented on synthetic datasets to evaluate the
effectiveness of the proposed method. The results show that our method
outperformed the local model and yielded the same performance as the
conventional approach of centralizing data.

##  1 Introduction

Value chains involving sequential manufacturing stages operated by different
companies have become increasingly prevalent, e.g., in the automotive,
semiconductor, pharmaceutical, and waste management industry [1]. The
transition away from (petroleum-based) virgin - and towards increasing use of
recycled - feedstock further increases the mutual interaction and dependence
of the involved processes. Thus, achieving consistent product characteristics
requires closer collaboration of the stakeholders along value chains to be
able to dynamically adapt the corresponding processes to increasingly variable
input streams.

Distributed ledger technology (DLT) has become the technology of choice for
trustworthy, cross-organizational information exchange and has been adopted in
several industries such as manufacturing, automotive, real estate, energy, or
healthcare [2]. DLT also forms the basis of various (non-) corporate digital
passport initiatives, e.g., the European Blockchain Partnership (EBP). With
the upcoming EU Battery regulation that calls for a mandatory digital battery
passport by 2026, the pace for implementing digital passports is also picking
up from the regulatory side [3]. Although digital passports have been promoted
as the key enabler of the circular economy and successfully implemented to
realize circular business models, they come with several limitations [4].
First and foremost, digital passports, whether DLT-based and equipped with
encryption technology to protect sensitive data or not, always contain only
those pieces of information that are _a priori_ deemed relevant for process
and value chain optimization. However, it is not always clear which
material/process/product specifications have an impact on downstream
processes, and thus, it can be challenging to determine which information
should be included in digital passports to ensure their maximum effectiveness
in supporting a circular economy and sustainable development goals [4].

With the ever-increasing complexity of industrial processes and amounts of
data produced along value chains, AI and big data analytics have become
obvious tools to help reveal such interactions across company borders. Joint
analysis of (process) data can provide stakeholders with a more comprehensive
view of the entire value chain, including the relationships between different
process stages and the factors that impact the sustainability and circularity
of the system as a whole. However, despite the overwhelming consensus that
cross-organizational information exchange is imperative for increasing
efficiency, resilience as well as social and environmental sustainability of
the processes involved along value chains, privacy concerns related to data
sharing still prevail among the relevant stakeholders.

Federated learning (FL) is a relatively new machine learning paradigm, first
proposed by researchers at Google in 2016, that addresses privacy and security
concerns associated with centralized data storage and processing. FL allows
multiple parties to jointly train data-driven models while preserving data
privacy – either by entirely avoiding data exchange or by means of data
encryption [5]. Vertical Federated Learning (VFL) refers to scenarios where
the private datasets share the same sample space but differ in the feature
space. Cross-organizational processes naturally fit this scenario because even
though local datasets can be mapped using the product or batch IDs, the local
features are often distinct because each company usually operates a different
type of sub-process. Although some research efforts have been devoted to
adopting VFL in the field of process modeling, limited progress has been made.
In [6] [7], the authors have proposed federated variants of Principal
Component Analysis (PCA), which is a widely used method for fault detection
and diagnosis. Nevertheless, in these studies, the application of PCA to
process monitoring was not investigated. In [8], Du _et al._ recently proposed
a multiblock principal component analysis (MPCA) based federated multivariate
statistical process control (FedMSPC) approach to jointly model a
semiconductor fabrication process over multiple stages that are operated by
different companies. Not only was the joint model more efficient (compared to
the model trained on the downstream process only) for predicting process
faults, but it also revealed important interactions between the two process
stages that led to faulty batches in the downstream process. This study is
among the first to demonstrate the prospects of (vertical) federated data
analysis and process modeling for collaborative risk identification, decision-
making, and value chain optimization.

However, as an unsupervised technique, FedMSPC’s application is limited to
scenarios where the relationship between process variables and some specific
response variables (e.g., product quality, KPIs, etc.) is not of primary
interest. Furthermore, previous methodology lack mechanisms to incentivize
data federation, which is pivotal to fostering collaboration among
stakeholders along the value chain. In particular, participants should be
rewarded based on their contribution to the overall outcomes (e.g., cost
savings) resulting from data federation. Accessing contribution based merely
on data quantity is certainly not enough, as one party may contribute a large
volume of data that doesn’t help much in solving the problem. Recent studies
have proposed the use of the Shapley value in evaluating data contribution
[9][10]. However, since the Shapley value has an exponential time complexity,
this approach poses significant inefficiency [11].

To overcome the crucial shortcomings of MPCA-based MPSC, we here propose
Privacy-preserving Partial Least Squares (P3LS). P3LS is a federated variant
of Partial Least Squares (PLS), a popular supervised technique for process
modeling [12][13]. P3LS leverages Federated Singular Vector Decomposition
(FedSVD) [14] to enable different companies to jointly train a PLS model while
protecting private datasets. To create a basis for incentive mechanisms, we
also propose a method for securely quantifying the contribution of
participants based on the variance explained by the contributed data.

In the next section, we first cover the background of the proposed method,
including the concepts of PLS, FL, and FedSVD. In Section 3, the methodology
and implementation of P3LS are described in detail. We then provide a security
analysis in Section 4 and discuss potential real-world applications of P3LS in
manufacturing in Section 5. After that, in Section 6, we provide an empirical
evaluation of P3LS on simulated data when its performance is compared with
local and centralized PLS models. Finally, conclusions drawn from the results
and potential topics for future research and development are presented in
Section 7.

##  2 Background

###  2.1 Preliminaries

We use boldfaced lower-case letters, e.g., $\bm{x}$, to denote vectors and
boldfaced upper-case letters, e.g., $\bm{X}$, to denote matrices. The
subscript is associated with indices of different parties, e.g., the matrix
$\bm{X}_{i}$ denotes the $i^{th}$ party’s data.

###  2.2 Partial Least Squares

Suppose the process data of $m$ samples (e.g., batches) and $n$ variables is
denoted as $\bm{X}\in\mathbb{R}^{m\times n}$, and the corresponding response
variables (e.g., critical quality attributes of the final product) is stored
in $\bm{Y}\in\mathbb{R}^{m\times l}$. The general underlying model of PLS
applied on $\bm{X}$ and $\bm{Y}$ is:

$\displaystyle\bm{X}$ $\displaystyle=\bm{T}\bm{P}^{\top}+\bm{\Theta}$
$\displaystyle\bm{Y}$ $\displaystyle=\bm{U}\bm{Q}^{\top}+\bm{\Phi},$

where

  * 1.

$\bm{T}\in\mathbb{R}^{m\times k}$ and $\bm{U}\in\mathbb{R}^{m\times k}$ are,
respectively, the $\bm{X}$ scores and the $\bm{Y}$ scores.

  * 2.

$k$ is the number of latent variables (LVs), often determined by means of
cross-validation [15].

  * 3.

$\bm{P}\in\mathbb{R}^{n\times k}$ and $\bm{Q}\in\mathbb{R}^{l\times k}$ are,
respectively, $\bm{X}$ loadings and $\bm{Y}$ loadings.

  * 4.

$\bm{\Theta}\in\mathbb{R}^{m\times n}$ and $\bm{\Phi}\in\mathbb{R}^{m\times
l}$ are the residuals matrices.

The decompositions of $\bm{X}$ and $\bm{Y}$ are made in a way that maximizes
the covariance between $\bm{T}$ and $\bm{U}$. Different implementations of PLS
exist, among which the most popular are NIPALS [16], SIMPLS [17], and Singular
Vector Decomposition (SVD) [18][19]. In this study, we propose a privacy-
preserving variant of PLS that leverages a federated version of SVD. A
description of SVD-based PLS is provided in Algorithm 1.

1

Input: $\bm{X}$, $\bm{Y}$

Output: $\bm{T}$, $\bm{Q}$, $\bm{W}$, $\bm{P}$, $\bm{B}$, $\bm{R}$

2 Function  _PLS.train(_$\bm{X}$ , $\bm{Y},k$_)_:

3

4 Initialize two matrices: $\bm{E}:=\bm{X}$, and $\bm{F}:=\bm{Y}$

5 for  _$j=1\rightarrow k$_ do

6 Calculate the cross-product matrix $\bm{S}$: $\bm{S}=\bm{E}^{\top}\bm{F}$.

7 Perform SVD on $\bm{S}$ and extract the first left and right singular
vectors, $\bm{w}$ and $\bm{v}$.

8 Calculate scores $\bm{t}$ and $\bm{u}$:
$\bm{t}=\bm{E}\bm{w},\quad\bm{u}=\bm{F}\bm{v}$

9 Obtain loadings by regressing against the same vector $\bm{t}$:
$\bm{p}=\frac{\bm{E}^{\top}\bm{t}}{\bm{t}^{\top}\bm{t}},\quad\bm{q}=\frac{\bm{F}^{\top}\bm{t}}{\bm{t}^{\top}\bm{t}}$

10 Deflate the data matrices $\bm{E}$ and $\bm{F}$:
$\bm{E}=\bm{E}-\bm{t}\bm{p}^{\top},\quad\bm{F}=\bm{F}-\bm{t}\bm{q}^{\top}$

11 Save $\bm{w}$, $\bm{t}$, $\bm{p}$, and $\bm{q}$.

12 end for

13 Use vectors $\bm{w}$, $\bm{t}$, $\bm{p}$ and $\bm{q}$ as columns to form
matrices $\bm{W}$, $\bm{T}$, $\bm{P}$ and $\bm{Q}$, respectively.

14 Compute the rotation matrix (denoted as $\bm{R}$) and regression
coefficients (denoted as $\bm{B}$):
$\bm{R}=\bm{W}(\bm{P}^{\top}\bm{W})^{-1},\quad\bm{B}=\bm{R}\bm{Q}^{\top}$

15 End

Algorithm 1 PLS Training

After the model is trained, it can be used to calculate scores and predict the
output for unseen data $\bm{X}_{new}$:

$\displaystyle\bm{T}_{new}$ $\displaystyle=\bm{X}_{new}\bm{R}$
$\displaystyle\hat{\bm{Y}}_{new}$ $\displaystyle=\bm{X}_{new}\bm{B}$

The PLS algorithms commonly require centralized data, which poses challenges
in cross-organizational processes where each company possesses data only from
its own production line. Privacy and security concerns further restrict the
sharing of data among companies. While some variations of PLS have aimed to
handle multi-source data [20][21], they still assume full data accessibility,
failing to address the critical issue of data privacy.

###  2.3 Federated Learning

FL is a collaborative learning paradigm that allows multiple clients to
jointly train a global machine learning model without revealing private data
[22]. Depending on how data is distributed among the clients, FL can be
generally categorized into two settings, namely Horizontal Federated Learning
(HFL) and Vertical Federated Learning (VFL). HFL refers to scenarios where
private datasets share the same feature space but differ in the sample space.
Conversely, VFL consists of situations where private datasets share the same
sample space but differ in the feature space. The difference between the two
scenarios is demonstrated in Figure 1. As noted by [23][11][24], most of the
current research in the field of privacy-preserving machine learning
concentrates on the horizontal schema, while the vertical setting has received
inadequate devotion.

Figure 1: Federated Learning categorization.

###  2.4 Federated Singular Vector Decomposition

FedSVD, proposed in [14], is a lossless approach for securely conducting SVD
on distributed data without data centralization.

Suppose $\bm{H}\in\mathbb{R}^{m\times m}$ and $\bm{G}\in\mathbb{R}^{m\times
m}$ are two random, orthogonal matrices. For any matrix
$\bm{X}\in\mathbb{R}^{m\times n}$, it holds that
$\bm{X}^{\prime}=\bm{H}^{\top}\bm{X}\bm{G}$ has the same singular values as
$\bm{X}$, and their singular vectors can be transformed into each other by a
linear transformation, i.e.,

$\displaystyle\begin{aligned} \bm{w}&=\bm{H}\bm{w}^{\prime}\\\
\bm{v}&=\bm{G}\bm{v}^{\prime},\end{aligned}$ (1)

with $\bm{w}$ and $\bm{v}$, and $\bm{w}^{\prime}$ and $\bm{v}^{\prime}$
denoting the first left and right singular vectors of $\bm{X}$ and
$\bm{X}^{\prime}$, respectively.

In order to operate, FedSVD requires a trusted authority (TA) to handle the
key generation (i.e., matrices $\bm{H}$ and $\bm{G}$) and a computation
service provider (CSP) to perform data aggregation and matrix decomposition.
The detailed descriptions of the method can be found in [14]. Briefly, FedSVD
has four main steps:

  * 1.

Step 1: The TA generates random orthogonal matrices and creates shared and
private keys.

  * 2.

Step 2: Each data holder downloads its proprietary key and masks the private
data.

  * 3.

Step 3: The CSP aggregates the masked data using a secure aggregation
protocol, performs SVD, and obtains masked singular vectors.

  * 4.

Step 4: The data holders collaborate with the CSP to recover the real results
using its given keys.

The approach was proved to be confidential under the semi-honest setting where
the TA is fully trusted while the CSP and data holders are semi-honest [14].
More specifically, it assumes that the CSP and data holders will strictly
follow the pre-specified protocol, even though they may try to deduce the
private knowledge of others. In addition, The security protocol also requires
the absence of collusion among the participants. Collusion denotes any secret
or illicit agreement between entities aiming to compromise the system’s
security. For instance, collusion might occur if the TA or a data holder
shares encryption keys with the CSP, allowing the decryption of masked data or
masked model components, thereby undermining security.

##  3 Privacy-Preserving Partial Least Squares

###  3.1 Participants and roles

The hypothetical scenario involves $g$ data holders, each holding a partition
of features of the same samples. Let $\bm{X}_{i}\in\mathbb{R}^{m\times n_{i}}$
denotes the features owned by the $i^{th}$ data holder, where $m$ is the
number of samples (batches) and $n_{i}$ is the number of private features. And
let $\bm{Y}\in\mathbb{R}^{m\times l}$ represent the labels of the $m$ samples.
Without loss of generality, we assume that $\bm{Y}$ is privately owned by the
$g^{th}$ data holder.

Based on their contribution, we categorize data holders into two types:
Feature Contributor (FC) and Label Contributor (LC). A data holder that
contributes only a feature block (i.e., $\bm{X}_{i}$) is designated as a
Feature Contributor. On the other hand, a data holder that contributes to the
target block (i.e., $\bm{Y}$) is referred to as a Label Contributor. A feature
contributor may also play the role of a label contributor. While there can be
multiple label contributors, the present work focuses on scenarios where there
is only one single $\bm{Y}$ block from one label contributor. An extension to
handle multi-label-contributor applications will be investigated in future
work.

Besides data holders, P3LS also requires the presence of a trusted authority
and a central service provider. Their functions within the system closely
mirror the roles they play in FedSVD.

###  3.2 Overall workflow

Suppose the data holders in the data federation aim at training a PLS model on
the concatenated matrix $\bm{X}=[\bm{X}_{1},\bm{X}_{2},...,\bm{X}_{g}]$ and
the matrix $\bm{Y}$, where $\bm{X}\in\mathbb{R}^{m\times n}$ and
$n=\sum_{i=1}^{g}n_{i}$. Assume that the matrices have been standardized to
have zero means and unit variance. In this case, the P3LS decomposition
results are the following:

$\displaystyle[\bm{X}_{1},...,\bm{X}_{g}]$
$\displaystyle=\bm{T}[\bm{P}_{1}^{\top},...,\bm{P}_{g}^{\top}]+[\bm{\Theta}_{1},...,\bm{\Theta}_{g}]$
(2) $\displaystyle\bm{Y}$ $\displaystyle=\bm{U}\bm{Q}^{\top}+\bm{\Phi}$ (3)

Accordingly, FC-$i$ gets $\bm{X}_{i}=\bm{T}\bm{P}^{\top}_{i}+\bm{\Theta}_{i}$,
where $\bm{T}$ is a shared result among all participants, and
$\bm{P}^{\top}_{i}\in\mathbb{R}^{n_{i}\times k}$ and $\bm{\Theta}_{i}$ are the
private results. Meanwhile, since the LC is the sole contributor of the
$\bm{Y}$-block, it must be the only party who has access to $\bm{U}$,
$\bm{Q}$, and $\bm{\Phi}$. Additionally, in order to make predictions for new
samples, all FCs must also know their local coefficients (i.e., $\bm{B}_{i}$,
see Algorithm 1). Table 1 summarizes the shared and private components of the
P3LS model. It is strictly required that FC-$i$’s contributed data and private
model components cannot be leaked to any other parties during the computation.

Table 1: Shared and private components of the P3LS model. Role |  | Shared
---
components
Private components
FC-$i$ | $\bm{T}$ (X-scores) |  |  $\bm{X}_{i}$ (Local features),
---
$\bm{P}_{i}$ (Local X-loadings),
$\bm{B}_{i}$ (Local coefficients),
$\bm{\Theta}_{i}$ (Local X-residuals)
LC |  |  $\bm{U}$ (Y-scores),
---
$\bm{Q}$ (Y-loadings),
$\bm{Y}$ (Target block),
$\bm{\Phi}$ (Y-residuals)

Overall, P3LS has the following three steps:

  * 1.

Secure data aggregation: The TA generates common and private keys, which are
random orthogonal matrices. Each data holder downloads its proprietary key,
masks their local data using the key, and then forwards the encrypted data to
the CSP.

  * 2.

Sequential matrix decomposition: The CSP aggregates the encrypted data,
performs the standard SVD-based PLS algorithm, and obtains all encrypted model
components.

  * 3.

Model decryption: The data holders collaborate with the TA and the CSP to
recover the real model components using both shared and private keys.

The pseudo-code of P3LS is shown in Algorithm 2. The following subsections
will elaborate on the specifics of each step.

1

Input: $\bm{X}=[\bm{X}_{1},\ldots,\bm{X}_{g}]$, $\bm{Y}$

Output: $\bm{T}$, $\bm{Q}$, $\bm{W}=[\bm{W}_{1},\ldots,\bm{W}_{g}]$,
$\bm{P}=[\bm{P}_{1},\ldots,\bm{P}_{g}]$,
$\bm{B}=[\bm{B}_{1},\ldots,\bm{B}_{g}]$

2 Function  _P3LS.train(_$[\bm{X}_{1},\ldots,\bm{X}_{g}]$ , $\bm{Y},k$_)_:

3 TA do:

4 Generate orthogonal matrices $\bm{A}\in\mathbb{R}^{m\times m}$,
$\bm{H}\in\mathbb{R}^{n\times n}$, $\bm{G}\in\mathbb{R}^{l\times l}$.

5 Split $\bm{H}^{\top}$ into $[\bm{H}_{1}^{\top},\ldots,\bm{H}_{g}^{\top}]$
where $\bm{H}_{i}^{\top}\in\mathbb{R}^{n\times n_{i}}$.

6 end

7

8 Feature Contributor do:

9 for  _$i=1\rightarrow g$ , Feature Contributor $i$_ do

10 Download $\bm{A}$, $\bm{H}^{\top}_{i}$ from TA and compute
$\bm{X}_{i}^{\prime}=\bm{A}\bm{X}_{i}\bm{H}_{i}$

11 Send $\bm{X}_{i}^{\prime}$ to the CSP

12 end for

13

14 end

15

16 Label Contributor do:

17 Download $\bm{G}$ from the TA and computes
$\bm{Y}^{\prime}=\bm{A}\bm{Y}\bm{G}$

18 Send $\bm{Y}^{\prime}$ to the CSP

19 end

20

21 CSP do:

22 Receive $\bm{X}_{i}^{\prime}$ and $\bm{Y}^{\prime}$, then aggregate
$\bm{X}^{\prime}$ :  $\bm{X}^{\prime}=\sum_{i=1}^{g}\bm{X}_{i}^{\prime}$

23 Assign $\bm{X}^{\prime}$ and $\bm{Y}^{\prime}$ to $\bm{E}^{\prime}$ and
$\bm{F}^{\prime}$ respectively: $\bm{E}^{\prime}:=\bm{X}^{\prime}$, and
$\bm{F}^{\prime}:=\bm{Y}^{\prime}$

24 for  _$j=1\rightarrow k$_ do

25 Calculate the cross-product matrix $\bm{S}^{\prime}$:
$\bm{S}^{\prime}=\bm{E}^{\prime\top}\bm{F}^{\prime}$.

26 Perform SVD on $\bm{S}^{\prime}$ and extract the first left and right
singular vectors, $\bm{w}^{\prime}$ and $\bm{v}^{\prime}$.

27 Calculate scores $\bm{t}^{\prime}$ and $\bm{u}^{\prime}$:
$\bm{t}^{\prime}=\bm{E}^{\prime}\bm{w}^{\prime},\quad\bm{u}^{\prime}=\bm{F}^{\prime}\bm{v}^{\prime}$

28 Obtain loadings by regressing against the same vector $\bm{t}^{\prime}$:
$\bm{p}^{\prime}=\frac{\bm{E}^{\prime\top}\bm{t}^{\prime}}{\bm{t}^{\prime\top}\bm{t}^{\prime}},\quad\bm{q}^{\prime}=\frac{\bm{F}^{\prime\top}\bm{t}^{\prime}}{\bm{t}^{\prime\top}\bm{t}^{\prime}}$

29 Deflate the data matrices $\bm{E}^{\prime}$ and $\bm{F}^{\prime}$:
$\bm{E}^{\prime}=\bm{E}^{\prime}-\bm{t}^{\prime}\bm{p}^{\prime\top},\quad\bm{F}^{\prime}=\bm{F}^{\prime}-\bm{t}^{\prime}\bm{q}^{\prime\top}$

30 Save $\bm{w}^{\prime}$, $\bm{t}^{\prime}$, $\bm{p}^{\prime}$, and
$\bm{q}^{\prime}$.

31 end for

32 Use vectors $\bm{w}^{\prime}$, $\bm{t}^{\prime}$, $\bm{p}^{\prime}$ and
$\bm{q}^{\prime}$ as columns to form matrices $\bm{W^{\prime}}$,
$\bm{T}^{\prime}$, $\bm{P}^{\prime}$ and $\bm{Q}^{\prime}$, respectively.

33 Compute the masked rotation matrix and regression coefficients:
$\displaystyle\bm{R}^{\prime}$
$\displaystyle=\bm{W}^{\prime}(\bm{P}^{\prime\top}\bm{W}^{\prime})^{-1}$
$\displaystyle\bm{B}^{\prime}$
$\displaystyle=\bm{R}^{\prime}\bm{Q}^{\prime\top}$

34 end

35

36 FCs and the LC collaborate with the CSP to recover real model components
using Algorithm 3.

37 End

Algorithm 2 P3LS Training

###  3.3 Secure data aggregation

The TA generates two random orthogonal matrices $\bm{A}\in\mathbb{R}^{m\times
m}$, $\bm{H}\in\mathbb{R}^{n\times n}$, $\bm{G}\in\mathbb{R}^{l\times l}$
using the block-based efficient mask generation method [14], and then it
splits $\bm{H}^{\top}$ into $[\bm{H}_{1}^{\top},...,\bm{H}_{g}^{\top}]$ where
$\bm{H}_{i}^{\top}\in\mathbb{R}^{n\times n_{i}}$.

FC-$i$ downloads $\bm{A}$ and $\bm{H}_{i}$ from the TA, masks their private
$\bm{X}$-block such that $\bm{X}_{i}^{\prime}=\bm{A}\bm{X}_{i}\bm{H}_{i}$, and
sends $\bm{X}_{i}^{\prime}$ to the CSP. At the same time, the LC downloads
$\bm{G}$ from the TA and masks the $\bm{Y}$-block through
$\bm{Y}^{\prime}=\bm{A}\bm{Y}\bm{G}$, and sends $\bm{Y}^{\prime}$ to the CSP.

The CSP aggregates $\bm{X}^{\prime}$ which is the sum of all
$\bm{X}_{i}^{\prime}$. According to the rule of matrix block multiplication,
we have:

$\displaystyle\bm{X}^{\prime}=\sum_{i=1}^{g}\bm{X}_{i}^{\prime}=\sum_{i=1}^{g}\bm{A}\bm{X}_{i}\bm{H}_{i}$
$\displaystyle=\bm{A}[\bm{X}_{1},...,\bm{X}_{g}][\bm{H}_{1}^{\top},...,\bm{H}_{g}^{\top}]^{\top}$
$\displaystyle=\bm{A}\bm{X}\bm{H}$

After this step, even though the CSP receives $\bm{X}^{\prime}$ and
$\bm{Y}^{\prime}$, it has been proved that the CSP cannot recover the original
data [14].

###  3.4 Sequential matrix decomposition

During this step, the latent variables are sequentially estimated. In what
follows, we outline the methodology involved in this process and define the
relations between the outcomes of P3LS and PLS. As described in 2, in the
first iteration, we have

$\displaystyle\bm{E}^{\prime}$
$\displaystyle:=\bm{X}^{\prime}=\bm{A}\bm{E}\bm{H}$
$\displaystyle\bm{F}^{\prime}$
$\displaystyle:=\bm{Y}^{\prime}=\bm{A}\bm{F}\bm{G}$

with the cross-product matrix

$\bm{S}^{\prime}=\bm{E}^{\prime\top}\bm{F}^{\prime}=\bm{H}^{\top}\bm{E}^{\top}\bm{A}^{\top}\bm{A}\bm{F}\bm{G}.$

Since $\bm{A}$ is an orthogonal matrix, $\bm{A}^{\top}\bm{A}=I$, therefore

$\bm{S}^{\prime}=\bm{H}^{\top}\bm{E}^{\top}\bm{F}\bm{G}=\bm{H}^{\top}\bm{S}\bm{G}.$

Note that both $\bm{H}^{\top}$ and $\bm{G}$ are orthogonal matrices. Thus,
according to Equation 1, the results of performing SVD on $\bm{S}$ and
$\bm{S}^{\prime}$ are convertible.

Next, we calculate X-scores and Y-scores. Since $\bm{H}^{\top}$ and $\bm{G}$
are orthogonal matrices, $\bm{H}\bm{H}^{\top}=I$ and $\bm{G}\bm{G}^{\top}=I$.

$\displaystyle\bm{t}^{\prime}$
$\displaystyle=\bm{E}^{\prime}\bm{w}^{\prime}=\bm{A}\bm{E}\bm{H}\bm{H}^{\top}\bm{w}=\bm{A}\bm{E}\bm{w}=\bm{A}\bm{t}$
$\displaystyle\bm{u}^{\prime}$
$\displaystyle=\bm{F}^{\prime}\bm{v}^{\prime}=\bm{A}\bm{F}\bm{G}\bm{G}^{\top}\bm{v}=\bm{A}\bm{F}\bm{v}=\bm{A}\bm{u}$

The X-scores $\bm{t}^{\prime}$ are normalised:

$\bm{t}^{\prime}=\frac{\bm{t}^{\prime}}{\sqrt{\bm{t}^{\prime\top}\bm{t}^{\prime}}}=\frac{\bm{A}\bm{t}}{\sqrt{\bm{t}^{\top}\bm{A}^{\top}\bm{A}\bm{t}}}=\frac{\bm{A}\bm{t}}{\sqrt{\bm{t}^{\top}\bm{t}}}.$

Next, X- and Y-loadings are obtained by regressing $\bm{E}^{\prime}$ and
$\bm{F}^{\prime}$ against the same vector $\bm{t}^{\prime}$:

$\displaystyle\bm{p}^{\prime}$
$\displaystyle=\bm{E}^{\prime\top}\bm{t}^{\prime}=\bm{H}^{\top}\bm{E}^{\top}\bm{A}^{\top}\bm{A}\bm{t}=\bm{H}^{\top}\bm{E}^{\top}\bm{t}=\bm{H}^{\top}\bm{p}$
$\displaystyle\bm{q}^{\prime}$
$\displaystyle=\bm{F}^{\prime\top}\bm{t}^{\prime}=\bm{G}^{\top}\bm{F}^{\top}\bm{A}^{\top}\bm{A}\bm{t}=\bm{H}^{\top}\bm{F}^{\top}\bm{t}=\bm{G}^{\top}\bm{q}$

Finally, $\bm{E}^{\prime}$ and $\bm{F}^{\prime}$ are deflated:

$\displaystyle\bm{E}^{\prime}_{i+1}=\bm{E}^{\prime}_{i}-\bm{t}^{\prime}\bm{p}^{\prime\top}=\bm{A}\bm{E}_{i}\bm{H}-\bm{A}\bm{t}\bm{p}^{\top}\bm{H}$
$\displaystyle=\bm{A}(\bm{E}_{i}-\bm{t}\bm{p}^{\top})\bm{H}$
$\displaystyle=\bm{A}\bm{E}_{i+1}\bm{H}$
$\displaystyle\bm{F}^{\prime}_{i+1}=\bm{F}^{\prime}_{i}-\bm{t}^{\prime}\bm{q}^{\prime\top}=\bm{A}\bm{F}_{i}\bm{G}-\bm{A}\bm{t}\bm{q}^{\top}\bm{G}$
$\displaystyle=\bm{A}(\bm{F}_{i}-\bm{t}\bm{p}^{\top})\bm{G}$
$\displaystyle=\bm{A}\bm{F}_{i+1}\bm{G}$

The estimation of the next latent variables then can start from the SVD of the
cross-product matrix
$\bm{S}^{\prime}_{i+1}=\bm{E}^{\prime\top}_{i+1}\bm{F}^{\prime}_{i+1}=\bm{H}^{\top}\bm{S}_{i+1}\bm{G}$.
After every iteration, the vectors $\bm{w}^{\prime}$, $\bm{t}^{\prime}$,
$\bm{p}^{\prime}$ and $\bm{q}^{\prime}$ are saved as columns to form matrices
$\bm{W^{\prime}}$, $\bm{T}^{\prime}$, $\bm{P}^{\prime}$ and $\bm{Q}^{\prime}$,
respectively.

The X-rotation matrix $\bm{R}^{\prime}$ is defined as:

$\displaystyle\bm{R}^{\prime}$
$\displaystyle=\bm{W}^{\prime}(\bm{P}^{\prime\top}\bm{W}^{\prime})^{-1}$
$\displaystyle=\bm{H}^{\top}\bm{W}(\bm{P}^{\top}\bm{H}\bm{H}^{\top}\bm{W})^{-1}$
$\displaystyle=\bm{H}^{\top}\bm{W}(\bm{P}^{\top}\bm{W})^{-1}$
$\displaystyle=\bm{H}^{\top}\bm{R}$

Meanwhile, the coefficient matrix $\bm{B}^{\prime}$ is defined as:

$\bm{B}^{\prime}=\bm{R}^{\prime}\bm{Q}^{\prime\top}=\bm{H}^{\top}\bm{R}\bm{Q}^{\top}\bm{G}=\bm{H}^{\top}\bm{B}\bm{G}$

In summary, the relationships between the PLS and P3LS model components are:

$\displaystyle\bm{T}$ $\displaystyle=\bm{A}^{\top}\bm{T}^{\prime}$ (4)
$\displaystyle\bm{W}$ $\displaystyle=\bm{H}\bm{W}^{\prime}$ (5)
$\displaystyle\bm{P}$ $\displaystyle=\bm{H}\bm{P}^{\prime}$ (6)
$\displaystyle\bm{Q}$ $\displaystyle=\bm{G}\bm{Q}^{\prime}$ (7)
$\displaystyle\bm{R}$ $\displaystyle=\bm{H}\bm{R}^{\prime}$ (8)
$\displaystyle\bm{B}$ $\displaystyle=\bm{H}\bm{B}^{\prime}\bm{G}^{\top}$ (9)

###  3.5 Decryption

A challenge that P3LS faces is how to remove the masks securely from the
encrypted results computed by the CSP. Motivated by FedSVD, we propose
Algorithm 3 to recover the real results of P3LS.

Since all FCs have the keys $\bm{A}$, they can download $\bm{T}^{\prime}$ and
get the real $\bm{T}$ using Equation 4. Similarly, the LC can get
$\bm{Q}^{\prime}$ from the CSP and recover $\bm{Q}$ using Equation 7. On the
other hand, since FC-$i$ only possesses a portion of $\bm{H}$, it can recover
$\bm{W}_{i}$, $\bm{P}_{i}$, and $\bm{B}_{i}$ through:

$\displaystyle\bm{W}_{i}$ $\displaystyle=\bm{H}_{i}\bm{W}^{\prime}$ (10)
$\displaystyle\bm{P}_{i}$ $\displaystyle=\bm{H}_{i}\bm{P}^{\prime}$ (11)
$\displaystyle\bm{B}_{i}$
$\displaystyle=\bm{H}_{i}\bm{B}^{\prime}\bm{G}^{\top}$ (12)

However, during the computation, we want to guarantee the confidentiality of
$\bm{H}_{i}$, $\bm{G}$, $\bm{W}^{\prime}$, $\bm{P}^{\prime}$,
$\bm{Q}^{\prime}$, and $\bm{B}^{\prime}$, i.e., the FCs should not directly
get $\bm{W}^{\prime}$, $\bm{P}^{\prime}$, $\bm{Q}^{\prime}$, and
$\bm{B}^{\prime}$ and the CSP should not be able to learn $\bm{H}_{i}$ and
$\bm{G}$. If the CSP knew $\bm{H}_{i}$ and $\bm{G}$, they could easily
estimate $\bm{W}_{i}$, $\bm{P}_{i}$, $\bm{Q}$, and $\bm{B}_{i}$.

We propose to first mask $\bm{H}_{i}\in\mathbb{R}^{n_{i}\times n}$ using a
locally-generated random matrix $\bm{C}_{i}\in\mathbb{R}^{n_{i}\times n_{i}}$
according to Equation 13. Next, FC-$i$ sends $[\bm{H}_{i}]^{\bm{C}_{i}}$
(i.e., the masked matrix) to the CSP, which will subsequently compute
$[\bm{W}_{i}]^{\bm{C}_{i}}$ according to Equation 14 and send
$[\bm{W}_{i}]^{\bm{C}_{i}}$ back to FC-$i$. Then FC-$i$ can remove the random
mask according to Equation 15 and get the final result (i.e., $\bm{W}_{i}$).

$\displaystyle[\bm{H}_{i}]^{C}$ $\displaystyle=\bm{C}_{i}\bm{H}_{i}$ (13)
$\displaystyle[\bm{W}_{i}]^{C}$
$\displaystyle=[\bm{H}_{i}]^{\bm{C}_{i}}\bm{W}^{\prime}(=\bm{C}_{i}\bm{W}_{i})$
(14) $\displaystyle\bm{W}_{i}$
$\displaystyle=\bm{C}_{i}^{-1}[\bm{W}_{i}]^{\bm{C}_{i}}$ (15)

This same approach can be applied to recover $\bm{P}_{i}$.

It is more complicated when it comes to recovering $\bm{B}_{i}$ since
$\bm{B}_{i}$ is protected by two keys, one of which is known only to the LC
(.i.e, $\bm{G}$). Our idea is to replace $\bm{G}$ with a common key known to
all FCs, then perform a similar procedure as recovering $\bm{W}_{i}$. First,
the TA generates a random matrix $\bm{N}\in\mathbb{R}^{n\times n}$. Next, the
LC downloads $\bm{N}$ and masks $\bm{G}^{\top}$ through
$[\bm{G}^{\top}]^{\bm{N}}=\bm{G}^{\top}\bm{N}$, and then sends
$[\bm{G}^{\top}]^{\bm{N}}$ to the CSP. Then, the CSP calculates
$[\bm{B}_{i}]^{\bm{C}_{i}}$ by the operation
$[\bm{B}_{i}]^{\bm{C}_{i}}=[\bm{H}_{i}]^{\bm{C}_{i}}\bm{B}^{\prime}[\bm{Q}^{\top}]^{\bm{N}}$.
Mathematically,
$[\bm{H}_{i}]^{\bm{C}_{i}}\bm{B}^{\prime}[\bm{Q}^{\top}]^{\bm{N}}=\bm{C}_{i}\bm{H}_{i}\bm{B}^{\prime}\bm{G}^{\top}\bm{N}=\bm{C}_{i}\bm{B}_{i}\bm{N}$.
After that, the CSP sends $[\bm{B}_{i}]^{\bm{C}_{i}}$ to FC-$i$. Finally,
FC-$i$ downloads $\bm{N}$ from the TA and recovers $\bm{B}_{i}$ through
$\bm{B}_{i}=\bm{C}_{i}^{-1}[\bm{B}_{i}]^{\bm{C}_{i}}\bm{N}^{-1}$.

1

2Function  _P3LS.recover(__)_ :

3 TA do:

4 Generate a random matrix $\bm{N}\in\mathbb{R}^{l\times l}$

5 end

6

7 Feature Contributor do:

8 for  _$i=1\rightarrow g$ , FC-$i$_ do

9 Generate a random matrix $\bm{C}_{i}\in\mathbb{R}^{n_{i}\times n_{i}}$

10 Mask $\bm{H}_{i}$ through: $[\bm{H}_{i}]^{\bm{C}_{i}}=\bm{C}_{i}\bm{H}_{i}$

11 Send $[\bm{H}_{i}]^{\bm{C}_{i}}$ to the CSP

12 end for

13

14 end

15

16 Label Contributor do:

17 Download $\bm{N}$ from the TA

18 Mask $\bm{G}^{\top}$ through:
$[\bm{G}^{\top}]^{\bm{N}}=\bm{G}^{\top}\bm{N}$

19 Send $[\bm{G}^{\top}]^{\bm{N}}$ to the CSP

20 end

21

22 CSP wait to receive data then do:

23 if  _Receive $[\bm{H}_{i}]^{C}$_ then

24 Compute the following:
$[\bm{W}_{i}]^{\bm{C}_{i}}=[\bm{H}_{i}]^{\bm{C}_{i}}\bm{W}^{\prime},\quad[\bm{P}_{i}]^{\bm{C}_{i}}=[\bm{H}_{i}]^{\bm{C}_{i}}\bm{P}^{\prime},\quad[\bm{B}_{i}]^{\bm{C}_{i}}=[\bm{H}_{i}]^{\bm{C}_{i}}\bm{B}^{\prime}[\bm{Q}^{\top}]^{\bm{N}}$

25 Send $[\bm{W}_{i}]^{\bm{C}_{i}}$, $[\bm{P}_{i}]^{\bm{C}_{i}}$, and
$[\bm{B}_{i}]^{\bm{C}_{i}}$ to FC-$i$

26 end if

27 Send $\bm{Q}^{\prime}$ only to the LC

28 end

29

30 Label Contributor do:

31 Recover $\bm{Q}$ by: $\bm{Q}=\bm{G}\bm{Q}^{\prime}$

32 end

33

34 Feature Contributor do:

35 for  _$i=1\rightarrow g$ , FC-$i$_ do

36 Download $\bm{N}$ from the TA

37 Get $[\bm{W}_{i}]^{C}$, $[\bm{P}_{i}]^{C}$, and $[\bm{B}_{i}]^{C}$ from the
CSP

38 Recover $\bm{W}_{i}$, $\bm{P}_{i}$, and $\bm{B}_{i}$ through:
$\bm{W}_{i}=\bm{C}_{i}^{-1}[\bm{W}_{i}]^{\bm{C}_{i}},\quad\bm{P}_{i}=\bm{C}_{i}^{-1}[\bm{P}_{i}]^{\bm{C}_{i}},\quad\bm{B}_{i}=\bm{C}_{i}^{-1}[\bm{B}_{i}]^{\bm{C}_{i}}\bm{N}^{-1}$

39 end for

40

41 end

42

43 End

Algorithm 3 P3LS Recover

###  3.6 Incentive mechamisms

Incentive mechanisms are pivotal in real-world applications, to promote
participant engagement, enhance collaboration, and ensure the truthful sharing
of data. Intuitively, data providers with high contributions deserve a better
payoff. Considering only the volume of contributed data is certainly not
sufficient because data quality is equally essential as data quantity in
building robust and effective models.

The explained variance, represented as $R^{2}_{i}$, quantifies the amount of
variance present in the data block $\bm{X}_{i}$ that can be described by the
latent variables $\bm{T}$ within the P3LS model. This calculation is derived
from the $\bm{X}$-loadings matrix $\bm{P}_{i}$ and can be expressed through
the following equation:

$R^{2}_{X_{i}}=\frac{SS(\bm{P}_{i})}{SS(X)}=\frac{SS(\bm{P}_{i})}{(m-1)n}$
(16)

Similarly, the explained variance in $\bm{Y}$ can be quantified by the
following equation:

$R^{2}_{Y}=\frac{SS(\bm{Q})}{SS(\bm{Y})}=\frac{SS(\bm{Q})}{(m-1)l}$ (17)

where $SS(\cdot)$ denotes the sum of squares of the term in parentheses [15].
Since $m$ and $l$ are known to all data holders, the above equations can be
calculated locally.

To evaluate the relevance of data, it is intuitive to examine how well each
data block (i.e., $\bm{X}_{i}$) is able to predict the target block (i.e.,
$\bm{Y}$). The standard way to assess predictive power involves determining
how much variance in $\bm{Y}$ can be accounted for by $\bm{X}_{i}$. Algorithm
4 outlines the steps to compute this metric.

Mathematically, $R^{2}_{X_{i}Y}$ can be computed through the following
equation:

$R^{2}_{X_{i}Y}=1-\frac{SS(\bm{Y}-\hat{\bm{Y}}_{i})}{SS(\bm{Y})}=1-\frac{SS(\bm{Y}-\bm{X}_{i}\bm{B}_{i})}{ml}$
(18)

To keep $\hat{\bm{Y}}_{i}$ and $\bm{Y}$ private to FC-$i$ and the LC,
respectively, we propose a procedure that involves the TA and the CSP. First,
the TA generates two orthogonal matrices $\bm{M}\in\mathbb{R}^{m\times m}$ and
$\bm{N}\in\mathbb{R}^{l\times l}$. Next, all FCs download $\bm{M}$ and
$\bm{N}$ from TA and use them to mask their local predictions
$\hat{\bm{Y}}_{i}$ via
$\hat{\bm{Y}}^{\prime}_{i}=\bm{M}\hat{\bm{Y}}_{i}\bm{N}$ and send
$\hat{\bm{Y}}^{\prime}_{i}$ to the CSP. Meanwhile, the LC mask $\bm{Y}$ via
$\bm{Y}^{\prime}=\bm{M}\bm{Y}\bm{N}$ and sends $\bm{Y}^{\prime}$ to the CSP.
Then, the CSP calculates the residual matrix
$\bm{E}^{\prime}_{i}=\bm{Y}^{\prime}-\hat{\bm{Y}}^{\prime}_{i}$ which is
equivalent of $\bm{M}(\bm{Y}-\hat{\bm{Y}}_{i})\bm{N}$ or
$\bm{M}\bm{E}_{i}\bm{N}$. After that, the CSP performs SVD on
$\bm{E}^{\prime\top}_{i}\bm{E}^{\prime}_{i}$
($=\bm{N}^{\top}\bm{E}^{\top}_{i}\bm{E}_{i}\bm{N}$). Since $\bm{N}^{\top}$ and
$\bm{N}$ are two random orthogonal matrices, it has been proved that the SVD
results of $\bm{N}^{\top}\bm{E}_{i}^{\top}\bm{E}_{i}\bm{N}$ and
$\bm{E}_{i}^{\top}\bm{E}_{i}$ share the same eigenvalues [14]. Thus, the CSP
can calculate the sum of eigenvalues of $\bm{E}_{i}^{\top}\bm{E}_{i}$, which
is equivalent to the trace of $\bm{E}_{i}^{\top}\bm{E}_{i}$ and
$SS(\bm{E}_{i})$. Finally, the CSP forwards $SS(\bm{E}_{i})$ to FC-$i$, who
proceeds to calculate $R^{2}_{X_{i}Y}$ using Equation 18.

1

2Function  _P3LS.recover(__)_ :

3 TA do:

4 Generate two orthogonal matrices $\bm{M}\in\mathbb{R}^{m\times m}$ and
$\bm{N}\in\mathbb{R}^{l\times l}$

5 end

6

7 Feature Contributor do:

8 for  _$i=1\rightarrow g$ , FC-$i$_ do

9 Download $\bm{M}$ and $\bm{N}$ from the TA

10 Calculate $\hat{\bm{Y}}_{i}$ through:
$\hat{\bm{Y}}_{i}=\bm{X}_{i}\bm{B}_{i}$

11 Mask $\hat{\bm{Y}}_{i}$ through:
$\hat{\bm{Y}}^{\prime}_{i}=\bm{M}\hat{\bm{Y}}_{i}\bm{N}$

12 Send $\hat{\bm{Y}}^{\prime}_{i}$ to the CSP

13 end for

14

15 end

16

17 Label Contributor do:

18 Download $\bm{M}$ and $\bm{N}$ from the TA

19 Mask $\bm{Y}$ through: $\bm{Y}^{\prime}=\bm{M}\bm{Y}\bm{N}$

20 Send $\bm{Y}^{\prime}$ to the CSP

21 end

22

23 CSP wait to receive data then do:

24 Compute the residual matrix:
$\bm{E}^{\prime}_{i}=\bm{Y}^{\prime}-\hat{\bm{Y}}^{\prime}_{i}$

25 Perform SVD on $\bm{E}^{\prime\top}_{i}\bm{E}^{\prime}_{i}$ and extract all
eigenvalues ($\lambda_{i}$)

26 Calculate $SS(\bm{E}^{\prime}_{i})$ by:
$SS(\bm{E}^{\prime}_{i})=\sum\lambda_{i}$

27 Send $SS(\bm{E}^{\prime}_{i})$ to FC-$i$

28 end

29

30 Feature Contributor do:

31 for  _$i=1\rightarrow g$ , FC-$i$_ do

32 Download $SS(\bm{E}^{\prime}_{i})$ from the CSP

33 Calculate $R^{2}_{X_{i}Y}$ by:
$R^{2}_{X_{i}Y}=1-SS(\bm{E}^{\prime}_{i})/ml$

34 end for

35

36 end

37

38 End

Algorithm 4 Variance explained in $\bm{Y}$ by $\bm{X}_{i}$

##  4 Security analysis

As P3LS incorporates FedSVD at its core, it inherits the security assurances
provided by FedSVD’s proof of security [14], which includes the following:

  * 1.

The TA only knows the keys and never sees the real/encrypted data.

  * 2.

The CSP solely receives encrypted data and learns encrypted model components
within the framework. Importantly, it has been established that it is not
feasible to directly deduce or infer the raw data from the encrypted data
within this context.

  * 3.

The data holders only see their private data and the relevant components of
the model that correspond to the data that they contribute.

Despite their potential usefulness, specific model components could be
withheld from the data holders to ensure the preservation of security and
confidentiality within the framework.

The decision not to disclose the local rotation matrix $\bm{R}_{i}$ to FC-$i$
is intentional. By withholding $\bm{R}_{i}$, it prevents FC-$i$ from accessing
both $\bm{B}_{i}$ and $\bm{R}_{i}$ simultaneously, as knowledge of both
enables estimation of $\bm{Q}^{\top}$ through the formula
$\bm{Q}^{\top}=(\bm{R}_{i}^{\top}\bm{R}_{i})^{-1}\bm{R}_{i}^{\top}\bm{B}_{i}$.
As $\bm{Q}$ contains the loadings of the response variables, it is crucial for
this information to be known exclusively by the LC. While hiding $\bm{R}_{i}$,
we allow FC-$i$ to learn $\bm{B}_{i}$ due to specific functional purposes
within the framework:

  * 1.

Joint estimation of target variables: FC-$i$ can still jointly estimate the
target variables.

  * 2.

Insights into cross-organizational process interactions: Access to
$\bm{B}_{i}$ allows FC-$i$ to learn how their process variables interact with
the product quality of the downstream company. Such information might be
useful when they try to optimize their processes and want to take the
downstream company’s KPIs into account.

In addition, we strategically decide to hide the local scores $\bm{T}_{i}$
from data holders to prevent potential inference of the local rotation matrix
($\bm{R}_{i}$).

$\bm{T}_{i}=\bm{X}_{i}\bm{R}_{i}\rightarrow\bm{R}_{i}=(\bm{X}_{i}^{\top}\bm{X}_{i})^{-1}\bm{X}_{i}^{\top}\bm{T}_{i}$

Even if the left inverse isn’t unique, it doesn’t necessarily imply that data
holders are unable to find a suitable solution. As highlighted earlier, the
primary reason for restricting access to $\bm{R}_{i}$ from FCs is due to their
potential ability to estimate $\bm{Q}$, which encompasses crucial information
related to the target variables.

##  5 Applications of P3LS

As a variant of PLS, P3LS naturally lends itself to applications where PLS is
commonly employed (e.g., in MSPC). One notable advantage of utilizing P3LS
lies in its ability to account for variations from preceding stages.
Consequently, it has the potential to enhance the quality control performance
[25]. Some of the applications that we consider relevant for P3LS include:

Process monitoring: PLS is widely acknowledged as one of the most popular
methods in process monitoring [26]. P3LS can be directly employed for this
purpose. Algorithm 5 outlines the steps involved in estimating the global
X-scores matrix ($\bm{T}$). By utilizing $\bm{T}$ along with the residual
matrix ($\bm{\Theta}_{i}$), it becomes feasible to compute commonly used
monitoring metrics such as Hotelling’s $T^{2}$ and $Q$-statistics, and the
contributions from original variables. These indices serve as valuable tools
in process monitoring and fault detection and diagnosis [12].

Soft sensors: The purpose of an inferential sensor is to deduce a property
that is challenging or expensive to measure directly — usually a laboratory
measurement or a costly parameter — by leveraging a blend of process data and
software-driven algorithms. Once validated, a soft sensor can significantly
minimize process costs by enabling swift feedback control over the estimated
property. This control mechanism aids in reducing the production of off-
specification products. Moreover, an additional benefit of soft sensors is
their capacity to curtail the need for extensive laboratory sampling, thus
reducing manpower costs. Soft sensors leveraging latent variables
predominantly rely on PLS models. Algorithm 5 outlines the steps involved in
generating predictions for unseen data. However, unlike the global score
matrix $\bm{T}$, the prediction of the response variables is constrained to
the Label Contributors (LC) exclusively.

Improved process understanding: The regression coefficients allow all data
holders to grasp the correlation of their process variables with some
particular KPIs of the final product. This acquired knowledge proves immensely
valuable, particularly during process optimization, as it provides insights
into how their variables influence the desired outcomes. Moreover, this
valuable understanding is only accessible to the participants through their
involvement in the data federation.

1

Input: $\bm{X}=[\bm{X}_{1},...,\bm{X}_{g}]\in\mathbb{R}^{m\times n}$, where
$\bm{X}_{i}\in\mathbb{R}^{m\times n_{i}}$

Output: $\bm{T}$, $\hat{\bm{Y}}$

2 Function  _P3LS.predict(_$[\bm{X}_{1},...,\bm{X}_{g}]$_)_:

3 TA do:

4 Generate a random matrix $\bm{M}\in\mathbb{R}^{m\times m}$

5 end

6

7 Feature Contributor do:

8 for  _$i=1\rightarrow g$ , FC-$i$_ do

9 Download $\bm{M}$ from the TA

10 Make local predictions: $\hat{\bm{Y}}_{i}=\bm{X}_{i}\bm{B}_{i}$

11 Mask local predictions: $\hat{\bm{Y}}^{\prime}_{i}=\bm{M}\hat{\bm{Y}}_{i}$

12 Encrypt local data using $\bm{M}$ and $\bm{H}_{i}$:
$\bm{X}^{\prime}_{i}=\bm{M}\bm{X}_{i}\bm{H}_{i}$

13 Send $\hat{\bm{Y}}^{\prime}_{i}$ and $\bm{X}^{\prime}_{i}$ to the CSP

14 end for

15

16 end

17

18 CSP do:

19 Aggregate $\bm{X}^{\prime}$:
$\bm{X}^{\prime}=\sum_{i=1}^{g}\bm{X}^{\prime}_{i}\quad(=\bm{M}\bm{X}\bm{H})$

20 Aggregate
$\hat{\bm{Y}}^{\prime}=\sum_{i=1}^{g}\hat{\bm{Y}}^{\prime}_{i}\quad(=\bm{M}\hat{\bm{Y}})$

21 Calculate the masked global scores:
$\bm{T}^{\prime}=\bm{X}^{\prime}\bm{R}^{\prime}\quad(=\bm{M}\bm{X}\bm{H}\bm{H}^{\top}\bm{R}=\bm{M}\bm{X}\bm{R}=\bm{M}\bm{T})$

22 end

23

24 Feature Contributor do:

25 for  _$i=1\rightarrow g$ , FC-$i$_ do

26 Download $\bm{T}^{\prime}$ from the CSP

27 Recover the real global scores: $\bm{T}=\bm{M}^{-1}\bm{T}^{\prime}$

28 end for

29

30 end

31

32 Label Contributor do:

33 Download $\hat{\bm{Y}}^{\prime}$ from the CSP

34 Recover the real predictions:
$\hat{\bm{Y}}=\bm{M}^{-1}\hat{\bm{Y}}^{\prime}$

35 end

36

37 End

Algorithm 5 P3LS Inference

##  6 Experimental evaluation

To assess the efficacy of P3LS, an experiment utilizing synthetic datasets has
been designed. The primary objective of this experiment is to address the
following questions:

  * 1.

Q1: What is the performance gap between P3LS and a centralized PLS model that
lacks privacy protection?

  * 2.

Q2: Does introducing more data from other participants improve the model
performance?

In essence, Q1 delves into the inherent trade-offs associated with
safeguarding privacy within a vertical federated setting, while Q2 examines
the potential enhancements resulting from collaborative training involving
diverse data sources. We continue by describing the datasets and then the
experiment settings. Finally, the experiment results will be employed to
address the previously posed questions.

###  6.1 Datasets

Due to the absence of realistic datasets in VFL as indicated by [27], we
implemented a multistage process simulator outlined in [28] to generate five
synthetic datasets of varying sizes. For comprehensive details regarding the
simulator itself and the specific configurations adopted for generating these
synthetic datasets, an extensive explanation is available in A.

Each of the generated datasets simulates a three-stage process. In the context
of a vertical federated scenario, the assumption made is that there are three
distinct manufacturing companies, with each company controlling and possessing
data from a specific stage of the process. Although quality characteristics
are recorded across all stages, the primary objective of the data federation
is to construct a predictive model solely for the output quality of the final
stage. The process variables specific to company $i$’s are denoted as
$\bm{X}_{i}$, while the quality variables of the final product are represented
as $\bm{Y}$. An illustration of the structure of the simulated dataset is
provided in Figure 2. Furthermore, Table 2 presents a summary detailing the
dataset owned by each individual company.

Figure 2: The structure of the simulated datasets. Table 2: Process variables and company assignment. Dataset |  $\bm{X}$-blocks |  $\bm{Y}$-block
---|---|---
#1 |  | Comp 1: $\bm{X}_{1}\in\mathbb{R}^{1000\times 10}$
---
Comp 2: $\bm{X}_{2}\in\mathbb{R}^{1000\times 20}$
Comp 3: $\bm{X}_{3}\in\mathbb{R}^{1000\times 20}$
Comp 3: $\bm{Y}\in\mathbb{R}^{1000\times 7}$
#2 |  | Comp 1: $\bm{X}_{1}\in\mathbb{R}^{1000\times 20}$
---
Comp 2: $\bm{X}_{2}\in\mathbb{R}^{1000\times 40}$
Comp 3: $\bm{X}_{3}\in\mathbb{R}^{1000\times 40}$
#3 |  | Comp 1: $\bm{X}_{1}\in\mathbb{R}^{1000\times 50}$
---
Comp 2: $\bm{X}_{2}\in\mathbb{R}^{1000\times 100}$
Comp 3: $\bm{X}_{3}\in\mathbb{R}^{1000\times 100}$
#4 |  | Comp 1: $\bm{X}_{1}\in\mathbb{R}^{1000\times 100}$
---
Comp 2: $\bm{X}_{2}\in\mathbb{R}^{1000\times 200}$
Comp 3: $\bm{X}_{3}\in\mathbb{R}^{1000\times 200}$
#5 |  | Comp 1: $\bm{X}_{1}\in\mathbb{R}^{1000\times 200}$
---
Comp 2: $\bm{X}_{2}\in\mathbb{R}^{1000\times 400}$
Comp 3: $\bm{X}_{3}\in\mathbb{R}^{1000\times 400}$

###  6.2 Experiment settings

All runs were conducted on a workstation using an 11th Gen Intel(R) Core(TM)
i7-1185G7 with four cores at 3.00GHz, 32.0 GB RAM, and PyCharm 2022.1.

Each private dataset was randomly partitioned into three separate subsets:
training set (60%), validation set (20%), and test set (20%). The training set
was dedicated to constructing the models. The validation set was utilized for
optimizing the number of latent variables in the constructed models. In
particular, the optimal number is the one that returns the highest accuracy on
the validation set. The test set was designated to estimate the model’s
performance on unseen data.

For each of the generated datasets, we constructed three distinct models, each
reflecting a scenario commonly observed in cross-organizational manufacturing
processes:

  * 1.

CenPLS: A PLS model trained by the last company using the centralized dataset.
It simulates a scenario where a single company has access to and utilizes all
available data along the value chain.

  * 2.

LocalPLS: A PLS model trained exclusively on the private dataset of the last
company (i.e., $\bm{X}_{3}^{train}$ and $\bm{Y}^{train}$).

  * 3.

P3LS: A P3LS model trained on all available data along the value chain without
direct data sharing between companies.

A detailed description of the data used for developing and testing each model
is described in Table 3. To answer Q1 and Q2, we considered the following
factors:

  * 1.

Losslessness: Even though it has been proved that the outcomes of CenPLS and
P3LS are equivalent, we wanted to empirically compare the components of CenPLS
and P3LS models using the mean squared distance.

  * 2.

Prediction performance: We measured the $R^{2}$ of P3LS, CenPLS, and LocalPLS
when applying these models to the test set.

  * 3.

Computation time: We measured the run time of all three models when making
inferences.

Since P3LS involves using random matrices to gain a comprehensive evaluation,
we repeated the experiment 100 times for each dataset.

Table 3: Training set, validation set, and test set for each model. Model |  Training Data |  Validation Data |  Test Data
---|---|---|---
CenPLS |  $\bm{X}^{train}$, $\bm{Y}^{train}$ |  $\bm{X}^{val}$, $\bm{Y}^{val}$ |  $\bm{X}^{test}$, $\bm{Y}^{test}$
LocalPLS |  $\bm{X}_{3}^{train}$, $\bm{Y}^{train}$ |  $\bm{X}_{3}^{val}$, $\bm{Y}^{val}$ |  $\bm{X}_{3}^{test}$, $\bm{Y}^{test}$
P3LS |  $\bm{X}_{1}^{train}$, $\bm{X}_{2}^{train}$, $\bm{X}_{3}^{train}$, $\bm{Y}^{train}$ |  $\bm{X}_{1}^{val}$, $\bm{X}_{2}^{val}$, $\bm{X}_{3}^{val}$, $\bm{Y}^{val}$ |  $\bm{X}_{1}^{test}$, $\bm{X}_{2}^{test}$, $\bm{X}_{3}^{test}$, $\bm{Y}^{test}$

###  6.3 Results

The similarity between P3LS and CenPLS is empirically confirmed when comparing
corresponding model components. Figure 3 displays the mean squared distance
between different components of the two models across all datasets. It
illustrates that the difference between the two models is negligible.
Consequently, as shown in Figure 4, the difference in their $R^{2}$ scores is
insignificant.

It can also be seen in Figure 4 that for the same dataset, even though the
results of different runs are not the same, the variations are ignorable. It
is important to note that P3LS is theoretically lossless. However, minor
deviations in the experiment might occur due to floating-point number
representation in computers.

Figure 5 compares the overall $R^{2}$ of LocalPLS and P3LS when evaluating
these models on the test set. Across all datasets, it shows that P3LS models
outperform the local PLS models. It can be explained by the fact that the
underlying process models involve variables from all stages. Since LocalPLS
models only learn from the last company’s data, they miss crucial information
needed to make good predictions. However, in practice, there might be cases
where the target variables are totally independent of preceding production
steps, or the information provided by other data providers is not relevant. In
such cases, the advantages of using P3LS might be less significant.

Figure 6 illustrates the computation time of the three models. As expected,
compared to CenPLS or LocalPLS, it took P3LS more time to produce inferences.
This results from processing more data and the additional complexity
introduced by the data encryption and decryption steps. It should be
highlighted that the experiment was performed in simulation mode, and the
communication time was not considered. Therefore, the results might differ
when deploying in real systems with more advanced hardware.

Figure 3: Compare model components of CenPLS and P3LS. Figure 4: Difference in
test $R^{2}$ between CenPLS and P3LS. Figure 5: Comparison of performance on
the test set of CenPLS, LocalPLS, and P3LS. Figure 6: Computation time in the
inference phase.

##  7 Discussion and conclusions

This paper proposes P3LS, which allows multiple companies to collaboratively
build and exploit an integrated cross-organizational process model without
revealing private datasets. The empirical experiment on simulated data shows
that the proposed approach can yield a model with comparable performance to
the conventional approach of centralizing data while protecting the data
privacy of the participants. Besides, we also proposed an approach for
quantifying the contribution of participants. This approach creates a basis
for a profit-sharing scheme and, at the same time, motivates companies to join
the federation and contribute high-quality data.

Even though the paper focuses on the vertical setting, the proposed method can
also be extended to apply to the horizontal setting. In the horizontal
setting, each data holder acts as both a feature contributor and a label
contributor as they own both process variables (i.e., $\bm{X}_{i}$) and
response variables (i.e., $\bm{Y}_{i}$). Under this setting, the shared model
components are the weights, loadings, and coefficients. All steps in Algorithm
2 are kept the same, except the following:

  * 1.

Step 2-4: Instead of splitting $\bm{H}$, the TA splits $\bm{A}^{\top}$ into
$[\bm{A}_{1}^{\top},...,\bm{A}_{g}^{\top}]$ where
$\bm{A}_{i}^{\top}\in\mathbb{R}^{m_{i}\times n}$.

  * 2.

Step 6-15: FC $i$ downloads $\bm{A}_{i}^{\top}$ and $\bm{H}$ from the TA and
computes $\bm{X}^{\prime}_{i}=\bm{A}_{i}\bm{X}_{i}\bm{H}$,
$\bm{Y}^{\prime}_{i}=\bm{A}_{i}\bm{Y}_{i}\bm{G}$.

  * 3.

Step 17: In addition to aggregating $\bm{X}^{\prime}$, the CSP needs to
aggregates $\bm{Y}^{\prime}=\sum_{i=1}^{g}\bm{Y}_{i}^{\prime}$.

At the end of the training phase, all data holders can download the encrypted
loadings, encrypted weights, and encrypted coefficients from the CSP. Then,
they can use the given keys (i.e., $\bm{A}_{i}$, $\bm{H}$ and $\bm{G}$) to
recover the real model components. After that, they can use these model
components to estimate scores and predict the response variables of unseen
data.

We will also investigate the scenarios where there are multiple label
contributors. In particular, multiple companies along the value chain can
contribute their response variable blocks (e.g., $\bm{Y}_{i}$). Ultimately,
companies can learn the interaction between their process variables and all
other companies’ KPIs, not just the last company. This extension will require
a change in key generation and encrypted data aggregation and will be in the
scope of our future works.

Another extension of P3LS that will be investigated is Multiway P3LS, which
can handle batch process data where $\bm{X}_{i}$ is in a 3D format with the
additional dimension of time. In an offline mode, batch data can be unfolded
into a 2D format using the batch-wise unfolding [13], and P3LS can be
naturally applied. However, it is more challenging to estimate the response
variables in online mode. It is because the $\bm{X}_{new}$ matrix is not
complete until the end of the batch operation. Suppose it takes $K$ time units
to complete a batch. At time interval $k$, the matrix $\bm{X}_{new}$ has only
its first $k$ rows complete, and it is missing all the future observations
($K-k$ rows). A solution to this problem is to leverage the ability of PLS to
handle missing data. Multiway PLS does this by projecting the already known
observations up to time interval $k$ into the reduced space defined by the
$\bm{W}$ and $\bm{P}$ matrices [13]. A similar approach will be adopted to
enable P3LS to operate in online mode.

Furthermore, in our future research and development, we aim to validate the
effectiveness of P3LS in real-life datasets, which tend to be more intricate.
Factors that are crucial for practical application in real production
scenarios, such as infrastructure requirements and computational time, will
also go through a comprehensive assessment.

## Acknowledgment

The research reported in this paper has been partly funded by the Federal
Ministry for Climate Action, Environment, Energy, Mobility, Innovation and
Technology (BMK), the Federal Ministry for Digital and Economic Affairs
(BMDW), and the State of Upper Austria in the frame of SCCH, a center in the
COMET - Competence Centers for Excellent Technologies Program managed by the
Austrian Research Promotion Agency FFG and the FFG project circPlast-mr (Grant
No. 889843).

##  Appendix A Multistage process simulator

We implemented the multistage process simulator proposed in [25] to generate
synthetic data. Figure 7 illustrates a typical diagram of a multistage system.
At stage-$i$, the process variables and the measured responses are denoted as
$\bm{X}_{i}$ and $\bm{Y}_{i}$, respectively. Meanwhile, $\bm{V}_{i}$
represents the measurement noises associated with responses. To introduce the
interaction between stages, some responses of upstream stages, denoted as
$\bm{\acute{Y}}_{i-1}$, will be used as inputs of stage-$i$’s process model.
In this simulator, all stages’ process models are set to quadratic functions
that can be expressed as:

$\bm{Y}_{i}=\bm{A}_{i}\bm{U}_{i.lin}+\bm{B}_{i}\bm{U}_{i.qd}+\bm{V}_{i}$ (19)

where $\bm{U}_{i.lin}$ and $\bm{U}_{i.qd}$ are matrices containing the linear
terms and the quadratic terms of the model, respectively. Meanwhile,
$\bm{A}_{i}$ and $\bm{B}_{i}$ represent the linear and quadratic model
coefficients, respectively. The elements of $\bm{A}_{i}$ and $\bm{B}_{i}$ are
drawn randomly from a specific distribution. While $\bm{A}_{i}$ is a dense
matrix, $\bm{B}_{i}$ is intentionally made sparse to imitate actual physical
process models, often containing only a sparse subset of quadratic terms.

Figure 7: Multistage process simulator diagram.

Following are the configurations used for generating Dataset-1:

  * 1.

Stage 1: $\bm{X}_{1}\in\mathbb{R}^{1000\times 10}$,
$\bm{Y}_{1}\in\mathbb{R}^{1000\times 5}$, $\bm{A}_{1}\sim\mathcal{U}(-1,\,2)$,
$\bm{B}_{1}\sim\mathcal{U}(-0.01,\,0.02)$. $\bm{A}_{1}$ has sparsity
probability of 0.15.

  * 2.

Stage 2: $\bm{X}_{2}\in\mathbb{R}^{1000\times 20}$,
$\bm{Y}_{2}\in\mathbb{R}^{1000\times 6}$,
$\bm{\acute{Y}}_{1}\in\mathbb{R}^{1000\times 3}$,
$\bm{A}_{2}\sim\mathcal{U}(-3,\,3)$,
$\bm{B}_{2}\sim\mathcal{U}(-0.03,\,0.03)$. $\bm{A}_{2}$ has sparsity
probability of 0.2.

  * 3.

Stage 3: $\bm{X}_{3}\in\mathbb{R}^{1000\times 20}$,
$\bm{Y}_{3}\in\mathbb{R}^{1000\times 7}$,
$\bm{\acute{Y}}_{2}\in\mathbb{R}^{1000\times 3}$,
$\bm{A}_{3}\sim\mathcal{U}(-3,\,2)$,
$\bm{B}_{3}\sim\mathcal{U}(-0.03,\,0.02)$. $\bm{A}_{3}$ has sparsity
probability of 0.25.

For all stages, $\bm{B}_{i}$ has a sparsity probability of 0.999. The noise
matrix $\bm{V}_{i}\sim\mathcal{N}(0,\,0.001)$. To reproduce the correlations
between process variables often observed in practice, we made $\bm{X}_{i}$ a
low-rank matrix with bell-shaped singular values where approximately four
singular vectors are required to explain 90% of the data.

We used the same configurations to generate other datasets except the number
of process variables. In particular, for Dataset-2,-3, -4, and -5, the number
of process variables is multiplied by 2, 5, 10, and 100 respectively.

## References

  * [1] M. Colledani, T. Tolio, A. Fischer, B. Iung, G. Lanza, R. Schmitt, J. Váncza, Design and management of manufacturing systems for production quality, Cirp Annals 63 (2) (2014) 773–796.
  * [2] J. Lohmer, E. Ribeiro da Silva, R. Lasch, Blockchain technology in operations & supply chain management: a content analysis, Sustainability 14 (10) (2022) 6192\.
  * [3] Proposal for a regulation of the european parliament and of the council concerning batteries and waste batteries, repealing directive 2006/66/ec and amending regulation (eu) no 2019/1020, accessed on December 11, 2023.
URL https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52020PC0798

  * [4] J. Walden, A. Steinbrecher, M. Marinkovic, Digital product passports as enabler of the circular economy, Chemie Ingenieur Technik 93 (11) (2021) 1717–1727.
  * [5] D. Nguyen Duy, M. Affenzeller, R. Nikzad-Langerodi, Towards vertical privacy-preserving symbolic regression via secure multiparty computation, in: Proceedings of the Companion Conference on Genetic and Evolutionary Computation, 2023, pp. 2420–2428.
  * [6] A. Hartebrodt, R. Nasirigerdeh, D. B. Blumenthal, R. Röttger, Federated principal component analysis for genome-wide association studies, in: 2021 IEEE International Conference on Data Mining (ICDM), IEEE, 2021, pp. 1090–1095.
  * [7] A. Grammenos, R. Mendoza Smith, J. Crowcroft, C. Mascolo, Federated principal component analysis, Advances in Neural Information Processing Systems 33 (2020) 6453–6464.
  * [8] D. N. Duy, D. Gabauer, R. Nikzad-Langerodi, Towards federated multivariate statistical process control (fedmspc), arXiv preprint arXiv:2211.01645 (2022).
  * [9] G. Wang, C. X. Dang, Z. Zhou, Measure contribution of participants in federated learning, in: 2019 IEEE international conference on big data (Big Data), IEEE, 2019, pp. 2597–2604.
  * [10] T. Wang, J. Rausch, C. Zhang, R. Jia, D. Song, A principled approach to data valuation for federated learning, Federated Learning: Privacy and Incentive (2020) 153–167.
  * [11] L. Yang, D. Chai, J. Zhang, Y. Jin, L. Wang, H. Liu, H. Tian, Q. Xu, K. Chen, A survey on vertical federated learning: From a layered perspective, arXiv preprint arXiv:2304.01829 (2023).
  * [12] J. F. MacGregor, T. Kourti, Statistical process control of multivariate processes, Control engineering practice 3 (3) (1995) 403–414.
  * [13] P. Nomikos, J. F. MacGregor, Multivariate spc charts for monitoring batch processes, Technometrics 37 (1) (1995) 41–59.
  * [14] D. Chai, L. Wang, J. Zhang, L. Yang, S. Cai, K. Chen, Q. Yang, Practical lossless federated singular vector decomposition over billion-scale data, in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 46–55.
  * [15] H. Abdi, Partial least square regression (pls regression), Encyclopedia for research methods for the social sciences 6 (4) (2003) 792–795.
  * [16] P. Geladi, B. R. Kowalski, Partial least-squares regression: a tutorial, Analytica chimica acta 185 (1986) 1–17.
  * [17] S. De Jong, Simpls: an alternative approach to partial least squares regression, Chemometrics and intelligent laboratory systems 18 (3) (1993) 251–263.
  * [18] R. Wehrens, B.-H. Mevik, The pls package: principal component and partial least squares regression in r (2007).
  * [19] K.-A. Lê Cao, D. Rossouw, C. Robert-Granié, P. Besse, A sparse pls for variable selection when integrating omics data, Statistical applications in genetics and molecular biology 7 (1) (2008).
  * [20] J. F. MacGregor, C. Jaeckle, C. Kiparissides, M. Koutoudi, Process monitoring and diagnosis by multiblock pls methods, AIChE Journal 40 (5) (1994) 826–838.
  * [21] L. Wangen, B. Kowalski, A multiblock partial least squares algorithm for investigating complex chemical systems, Journal of chemometrics 3 (1) (1989) 3–20.
  * [22] B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial intelligence and statistics, PMLR, 2017, pp. 1273–1282.
  * [23] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu, B. He, A survey on federated learning systems: Vision, hype and reality for data privacy and protection, IEEE Transactions on Knowledge and Data Engineering (2021).
  * [24] A. Khan, M. t. Thij, A. Wilbik, Vertical federated learning: A structured literature review, arXiv preprint arXiv:2212.00622 (2022).
  * [25] J. Shi, S. Zhou, Quality control and improvement for multistage systems: A survey, IIE transactions 41 (9) (2009) 744–753.
  * [26] Z. Ge, Z. Song, Multivariate statistical process control: Process monitoring methods and applications, Springer Science & Business Media, 2012.
  * [27] Y. Liu, Y. Kang, T. Zou, Y. Pu, Y. He, X. Ye, Y. Ouyang, Y.-Q. Zhang, Q. Yang, Vertical federated learning, arXiv preprint arXiv:2211.12814 (2022).
  * [28] J. Jin, J. Shi, State space modeling of sheet metal assembly for dimensional control (1999).