11institutetext: Ning Ge 22institutetext: School of Software, Beihang
University, Beijing, China
22email<EMAIL_ADDRESS>
_She was also with the Key Laboratory of Safety-Critical Software (Nanjing
University of Aeronautics and Astronautics), Ministry of Industry and
Information Technology._ 33institutetext: Guanghao Li 44institutetext: School
of Software, Beihang University, Beijing, China
44email<EMAIL_ADDRESS>55institutetext: Li Zhang 66institutetext:
School of Computer Science and Engineering, Beihang University, Beijing, China
66email<EMAIL_ADDRESS>
_Li Zhang is the corresponding author._ 77institutetext: Yi
Liu88institutetext: School of Computer Science and Engineering, Beihang
University, Beijing, China
88email<EMAIL_ADDRESS>
# Failure Prediction in Production Line Based on Federated Learning: An
Empirical Study

Ning Ge  Guanghao Li  Li Zhang  Yi Liu

(Received: date / Accepted: date)

###### Abstract

Data protection across organizations is limiting the application of
centralized learning (CL) techniques. Federated learning (FL) enables multiple
participants to build a learning model without sharing data. Nevertheless,
there is very few research works on FL in intelligent manufacturing. This
paper presents the results of an empirical study on failure prediction in the
production line based on FL. This paper (1) designs Federated Support Vector
Machine (FedSVM) and Federated Random Forest (FedRF) algorithms for the
horizontal FL and vertical FL scenarios, respectively; (2) proposes an
experiment process for evaluating the effectiveness between the FL and CL
algorithms; (3) finds that the performance of FL and CL are not significantly
different on the global testing data, on the random partial testing data, and
on the estimated unknown Bosch data, respectively. The fact that the testing
data is heterogeneous enhances our findings. Our study reveals that FL can
replace CL for failure prediction.

###### Keywords:

Empirical study Federated learning Failure prediction Production line
Manufacturing Bosch dataset

##  1 Introduction

Artificial intelligence (AI) is one of the core techniques in the fourth
industrial revolution Zhong et al. (2017); Li et al. (2017). For instance, AI
techniques have been employed to improve the early detection or fault
prediction within production lines Tao et al. (2018); Kusiak (2017) in
intelligent manufacturing (IM). A prominent achievement is that in 2012, Intel
saved $3 million in manufacturing costs through the use of predictive
analytics to prioritize its silicon chip inspections Ronen and Burns . To this
end, the Bosch company has published a dataset of product quality prediction
on the data analysis competition platform
Kaggle111https://www.kaggle.com/c/bosch-production-line-performance. The
dataset reflects the relevant parameters and equipment operation of each
product during the production process, hoping to reduce defective products. By
using this dataset, several studies have proposed product quality prediction
methods based on centralized learning (CL) algorithms Carbery et al. (2019,
2018); Zhang et al. (2016); Khoza and Grobler (2019); Kotenko et al. (2019);
Mangal and Kumar (2016); Hebert (2016); Maurya (2016); Huang et al. (2019b);
Moldovan et al. (2019); Liu et al. (2020d). The results of these work revealed
two important facts. First, data preprocessing on the Bosch dataset has a
significant impact on the prediction result. Second, among the CL algorithms,
the Random Forest algorithm performed better than others when time-series
features are excluded Zhang et al. (2016); Khoza and Grobler (2019) and the
Long Short Term Memory (LSTM) network can improve the prediction result when
time-series features are included Huang et al. (2019b). CL methods usually
require that clients upload data to the server. The server trains an
integrated model based on the shared data. However, in real-world scenarios,
private data belongs to multiple independent organizations and cannot be
shared with others. Therefore, data sharing has arisen to be an obstacle to
the application of CL methods. As UC Berkeley pointed out in a report
published in 2017, learning from private data is one of the challenges
encountered by the current AI field Stoica et al. (2017).

In response to the above problem, Google proposed the concept of federated
learning (FL) in 2016 to decentralize the collaborative learning McMahan et
al. (2016). FL trains models across multiple decentralized devices holding
local private data samples, without exchanging or sharing these samples.
Clients then encrypt and transmit the trained model or parameters back to the
server to build an integrated model. On the one hand, FL addresses critical
issues such as data privacy and data security Yang et al. (2019a); Li et al.
(2020a). On the other hand, as FL trains model locally without removing the
private data features, the data samples can be more complete than CL. FL has
received widespread attention recently and has been applied in many fields.
For example, Google launched a project on the prediction of mobile keyboard
input in 2018 and obtained better prediction results using FL Hard et al.
(2018). Intel started to support the hardware architecture for
FL222https://www.intel.com/content/www/us/en/artificial-
intelligence/posts/federated-learning-for-medical-imaging.html. FL has also
been applied to various fields like medical systems Sheller et al. (2018b);
Brisimi et al. (2018); Boughorbel et al. (2019b); Huang et al. (2019a),
Internet finance Suzumura et al. (2019); Yang et al. (2019b), smart city
Samarakoon et al. (2019); Saputra et al. (2019); Liu et al. (2020b), edge
computing Bakopoulou et al. (2019), IOT Nguyen et al. (2019); Chen et al.
(2020), Cyber Physical Systems (CPS) Mowla et al. (2019); Aussel et al.
(2020), etc.

In the field of IM, the manufacturing process often crosses enterprises,
workshops, or even production lines. Due to data privacy and security, the
application of CL is becoming the bottleneck. The studies on the failure
prediction in the production line have been conducted under the assumption
that all data are shared. Under the premise of data protection between
production lines, these methods become unfeasible. Nevertheless, according to
our investigation Liu et al. (2020c), applying FL to the field of IM is still
in its infancy, and there are very few research works on it. Although FL has
achieved success in many fields, whether it can achieve similar performance
and replace CL in real-world manufacturing is still an open question.
Accordingly, this paper bridges this gap by reporting an empirical study of
defective product prediction in the production line by comparing FL with CL.
This study is the first attempt to design FL algorithms for failure prediction
in the production line.

In addressing our research goal, we construct a horizontal FL (HFL) scenario
and a vertical FL (VFL) scenario, respectively. In the former, we compared the
Federated Support Vector Machine (FedSVM) and the Support Vector Machine (SVM)
algorithms. In the latter, we compare the Federated Random Forest (FedRF) and
the Random Forest (RF) algorithms. Each group of contrast experiment focuses
on four research questions (RQs).

  * $\circ$

_RQ1_ , we are interested in whether FL can replace CL on the whole Bosch
testing data. Hence, we ask whether the average performance of FedSVM (FedRF
resp.) is similar to that of SVM (RF resp.) on the whole Bosch testing data?

  * $\circ$

_RQ2_ , we are interested in whether FL can replace CL on part of the given
Bosch testing data. Therefore, we ask whether the average performance of
FedSVM (FedRF resp.) is similar to that of SVM (RF resp.) on random partial
Bosch testing data?

  * $\circ$

_RQ3_ , if FL is applied to predicting defective products on other unknown
Bosch data, whether FL can replace CL. Hence, we ask whether the average
performance of FedSVM (FedRF resp.) is similar to that of SVM (RF resp.) on
the estimated unknown Bosch testing data?

  * $\circ$

_RQ4_ , we want to know whether the Bosch testing data is heterogeneous or
not. If the data is heterogeneous and the answers of RQ1-RQ3 are positive, it
means that the selected FL algorithms are robust. Therefore, we ask, is there
local heterogeneity within the given testing data?

Answers to these questions are helpful for manufacturing as they provide
insights into how to evaluate an FL algorithm can achieve similar effects as a
CL one in a real-world application, and then replace it. To answer these
questions, this paper makes methodological, substantive, and theoretical
contributions to the literature on failure prediction in the production line.

First, in terms of algorithm design, to conduct empirical research on the
failure prediction based on HFL and VFL, we first select SVM and RF from
previous works as CL baseline because both algorithms performed better than
others in the previous works. Then, we design FedSVM and FedRF algorithms for
our problem based on existing works Bakopoulou et al. (2019); Liu et al.
(2020a). In terms of novelty in the algorithm design, our work improved FedRF
by introducing optimal feature selection and pruning steps.

Second, we propose a set of experimental methodology to compare the average
performance of FL and CL in manufacturing from three aspects: on the whole
testing data, on the random partial testing data, and on the estimated unknown
testing data. To compare the performance of FL and CL on the estimated unknown
Bosch data, we follow the process of Measurement Systems Analysis (MSA) to fit
a Markov process model of prediction error $\mathtt{M_{F}}$ ($\mathtt{M_{C}}$
resp.) based on FL (CL resp.) and then compare the two fitting models. The
experimental methods also assess the heterogeneity of the testing data to
enhance the results of RQ1-RQ3 on whether FL can replace CL.

Last, our empirical research shows that our FL solution is not significantly
different from the CL solution for failure prediction, on the whole, on the
random partial, and on estimated unknown Bosch testing data. FL can replace CL
in the application of failure prediction within the manufacturing process.

In the remainder of this paper, we first introduce the background knowledge of
our empirical research and related work in Section 2, present the designed HFL
and VFL algorithms in Section 3, and outline the experimental methodology in
Section 4. Section 5 presents the results of our empirical study. Section 6
discusses the threat of validity. Section 7 discusses whether FL can replace
CL. Section 8 concludes this paper and looks forward to future work.

##  2 Background and Related Work

This section introduces previous works on Bosch product quality prediction
based on the CL algorithms and discusses the necessity of conducting empirical
research on the FL algorithms for this problem; and then, briefly explains the
principles of federated learning.

###  2.1 Failure Prediction in Bosch Production Line

Bosch is one of the worldwide leading manufacturing companies. It ensures high
quality of the production by monitoring its parts in the manufacturing
processes. Because Bosch records detailed data for each step on the assembly
lines, they can apply advanced techniques to improve the manufacturing
processes. To this end, Bosch has published a dataset on the Kaggle
competition platform to predict internal failures by thousands of measurements
and tests made for each component along the assembly line. Some studies have
analyzed the dataset and carried out approaches to predicting product quality
based on CL algorithms excluding time-series features Carbery et al. (2019,
2018); Zhang et al. (2016); Khoza and Grobler (2019); Kotenko et al. (2019);
Mangal and Kumar (2016); Hebert (2016); Maurya (2016) or including time-series
features Huang et al. (2019b); Moldovan et al. (2019); Liu et al. (2020d), as
summarized in Table 1. These works were conducted based on a set of learning
methods, including Logistic Regression (LR), Gradient Boosting Machine (GBM),
Random Forest (RF), Gradient Boosted Trees (GBT), Naive Bayes (NB), Bayesian
Network (BN), K-Nearest Neighbors (KNN), Support Vector Machines (SVM),
Multilayer Perceptron Classifier (MPC), Majority Voting (MV), Decision Tree
(DT), Statistical Process Control (SPC), etc.

Table 1: Research Works on Failure Prediction in Bosch Production Line Objectives | Ref |  Learning Method |  Contribution
---|---|---|---
Improving predictive model without time-series features | Carbery et al. (2018) |  XGBoost, BN |  BN model performs well in failure prediction.
Zhang et al. (2016) |  RF, Gradient Boosting, LR, NB, DT |  RF performs better on different clusters.
Khoza and Grobler (2019) |  RF, SVM, NB, SPC |  RF outperforms other models.
Kotenko et al. (2019) |  SVM, KNN, Perceptron, LR, DT, MV |  SVM and MV outperform other models.
Mangal and Kumar (2016) |  LR, Extra Trees Classifier, RF, XGBoost |  XGBoost and RF outperform other methods.
Hebert (2016) |  LR, RF, XGBoost |  RF and XGBoost can properly identify conditions leading to failure events.
Maurya (2016) |  XGBoost |  Optimize MCC by using GBM as a base classifier.
Improving predictive model with time-series features | Huang et al. (2019b) |  Ontology-based LSTM neural network |  Ontology-based LSTM neural network yields a better performance
Moldovan et al. (2019) |  RF, GBT, NB, KNN, SVM, and MPC |  The LSTM RNN model outperform others.
Liu et al. (2020d) |  SP-LSTM models |  The A-Bi-SP-LSTM model outperforms other models.

Carbery et al. Carbery et al. (2019) conducted a systematic feature analysis
on the Bosch dataset and used BN to predict product quality Carbery et al.
(2018). Zhang et al. Zhang et al. (2016) weakened data heterogeneity through
clustering, and then applied RF, Boosting, LR, NB, DT to each cluster. They
showed that RF performed better than other algorithms. Based on Zhang et al.
(2016), Khoza et al. Khoza and Grobler (2019) compared RF, NB, SVM, and SPC.
They also showed that RF performed better. Kotenko et al. Kotenko et al.
(2019) used SVM, KNN, LR, Perceptron, DT, and MV and showed that SVM achieved
relatively higher prediction accuracy. Mangal et al. Mangal and Kumar (2016)
conducted a visual analysis of the three types of data: categorical, numeric,
and time-series features on the Bosch dataset, and used LR, Extra Trees
Classifier, RF, XGBoost for quality prediction. Hebert Hebert (2016) found
that RF and XGBoost could properly identify conditions that lead to failure
events. Maurya Maurya (2016) optimized MCC measure by using GBM as a base
classifier.

The studies Moldovan et al. (2019); Huang et al. (2019b); Liu et al. (2020d)
have considered time-series features. Huang et al. Huang et al. (2019b)
constructed an LSTM network model based on the time-series features of the
data, which has great enlightening significance for other researchers.
Moldovan et al. Moldovan et al. (2019) found that the LSTM RNN model
outperformed other machine learning models. Liu et al. Liu et al. (2020d)
proposed an end-to-end unified quality prediction framework to capture
temporal interactions among the features of different processes. They showed
that the A-Bi-SP-LSTM model outperforms the existing data-driven methods.

Despite these studies on learning features for failure prediction, they have
been all conducted under the assumption that all data are shared. Under the
premise of data protection between production lines, these methods become
unpractical. An open question is whether there exist some FL algorithms that
can replace CL methods. Thus, our study is a first attempt to design FL
algorithms for failure prediction in the assembly line.

###  2.2 Overview of Federated Learning

According to the relationship between the datasets provided by the clients, we
usually classify FL into HFL, VFL, and federated transfer learning (TFL)
Kairouz et al. (2019). This article focuses on the HFL and VFL scenarios. In
HFL, data are partitioned by user IDs or device IDs. For example, in the
context of manufacturing, clients A and B are two independent factories having
the same production line structure and machine configuration. The data
features in each production line are roughly the same, while the product IDs
on their production lines do not overlap. VFL is applicable to the cases that
two datasets share the same sample ID space but differ in feature space. For
example, production lines A and B are independent, but one belongs to the
upstream, and the other belongs to the downstream of the entire production
line. Their product IDs are likely to be the same, which guarantees the
intersection of their product space. However, since both A and B record part
of manufacturing behavior, their feature spaces are very different. In this
paper, we design FedSVM as the HFL model, and design FedRF as the VFL model,
respectively, to compare with centralized SVM and RF models.

##  3 Federated SVM and Federated RF

This section introduces the design of the federation learning algorithms in
this work. Through the investigation in Sect. 2, we selected SVM and RF
models, which performed well on the Bosch dataset, as the baseline. In the HFL
scenario, we reuse the existing FedSVM algorithm to compare it with SVM. In
the VFL scenario, we improve the existing Federated Forest algorithm Liu et
al. (2020a) to compare with RF.

###  3.1 Federated SVM

As a supervised learning algorithm, SVM is suitable for classification and
regression analysis. The effectiveness of SVM mainly depends on how to select
1) the kernel, 2) the kernel’s parameters, and 3) the soft margin parameter.
In this work, we use FedSVM with a linear kernel. SVM specifies parameters and
intercepts, which can be directly weighted. In FL, the SVM models generated on
different clients can be integrated by averaging the parameters and intercepts
to meet the need of the server. The FedSVM algorithm is first proposed in
Bakopoulou et al. (2019) for mobile packet classification. We have adapted
their FedSVM to our failure prediction problem. Its training process is shown
in Fig. 1. The algorithm of the client and server is given by Algo. 1 and
Algo. 2, respectively. The main steps of FedSVM are explained hereafter.

Figure 1: The Training Process of FedSVM

  * $\circ$

S1. Parameter initialization: The server ($S$) notifies each client ($C_{i}$)
to train models using the local data with random value of parameters and
intercepts ($w_{0}$).

  * $\circ$

S2. Local training: $C_{i}$ trains the local SVM model independently.

  * $\circ$

S3. Return local SVM parameters: $C_{i}$ returns locally trained parameters
and intercepts ($w^{i}_{t}$) in this round to $S$.

  * $\circ$

S4. Compute global SVM parameters:  $S$ averages the trained values in two
successive round $(\sum w^{i}_{t}+w_{t-1})/2$ as the value $w_{t}$ in the
global model $\mathtt{M_{G}}$.

  * $\circ$

S5. Send global SVM parameter: $S$ sends the global value $w_{t}$ in the
$\mathtt{M_{G}}$ to $C_{i}$ to train the model of next round.

  * $\circ$

S6. Repeat until convergence: Repeat steps S2 - S5 after a specified number of
iterations until convergence. The global model in the last round is sent to
all clients as the training result.

Algorithm 1 FedSVM - Client

1:Data set $D_{i}$ on client $i\ (i=1\ to\ k)$

2:Local SVM model on client $i$

3:Initialize $w_{0}^{i}$

4:$w_{1}^{i}\leftarrow$ SVM($w_{0}^{i}$, $D_{i}$) $\triangleright$ S2

5:Send $w_{1}^{i}$ to server $\triangleright$ S3

6:for $epoch\ t=2\ to\ N$ do $\triangleright$ S6

7: Receive global model $w_{t-1}$ $\triangleright$ S5

8: $w_{t}^{i}\leftarrow$ SVM($w_{t-1}$, $D_{i}$) $\triangleright$ S2

9: Send $w_{t}^{i}$ to server $\triangleright$ S3

10:end for

Algorithm 2 FedSVM - Server

1:Local SVM model

2:Global SVM model

3:Inform all clients start training $\triangleright$ S1

4:Receive $w_{1}^{i}$ from all clients

5:$w_{1}\leftarrow\sum_{i=1}^{k}\frac{w_{1}^{i}}{k}$ $\triangleright$ S4

6:Send $w_{1}$ to all clients $\triangleright$ S5

7:for $epoch\ t=2\ to\ N$ do $\triangleright$ S6

8: Receive $w_{t}^{i}$ from all clients

9: $w_{t}\leftarrow\frac{1}{2}(w_{t-1}+\sum_{i=1}^{k}\frac{w_{t}^{i}}{k})$
$\triangleright$ S4

10: Send $w_{t}$ to all clients $\triangleright$ S5

11:end for

###  3.2 Federated Random Forest

The Federated Forest algorithm was first proposed by Liu et al. in 2020 Liu et
al. (2020a). The random forest can be regarded as an integrated implementation
of the decision tree. The core steps of the decision tree (calculating the
purity of each feature, represented by Gini coefficients) only involve all
data in a single feature and the classification label. Each client in VFL can
do this on its local data. In our work, we have optimized the federated random
forest algorithm in Liu et al. (2020a) to get better accuracy and efficiency
by improving the steps of optimal feature selection and pruning. The improved
method is illustrated hereafter.

Optimal feature selection: The optimal feature is the one with the smallest
Gini coefficient according to the CART method in the decision tree. For
continuous variables, it is necessary to try to divide all possible values
when calculating the Gini coefficient. Here we reduce the calculation steps of
this function and improve the efficiency of the algorithm.

Pruning: There is no explicit pruning process in Liu et al. (2020a). Here we
add a pre-pruning function, which enhances robustness and improves the
efficiency of the overall algorithm. When the decision tree finds that the new
node will not improve the accuracy, it will change this node to a leaf node.

The training process of our improved FedRF algorithm is given in Fig. 2. The
algorithm of client and server is given by Algo. 3 and Algo. 4, respectively.
The main steps in its training process are explained hereafter.

Figure 2: The Training Process of FedRF

  * $\circ$

S1. Select initial information: The server ($S$) randomly selects the feature
subset $\mathtt{F^{\prime}}$, the sample ID subset $\mathtt{D^{\prime}}$ and
test set $\mathtt{T}$, and notifies them to each client ($C_{i}$).

  * $\circ$

S2. Local training and pruning: $C_{i}$ knows which features have been
selected, and does not know the number of features selected globally. $C_{i}$
calculates the Gini coefficient of each feature in $\mathtt{F^{\prime}}$.
Because the feature is only on the client, the client also knows the
classification labels of all samples, so the Gini coefficient calculated by
the client represents the global Gini coefficient. $C_{i}$ obtains the optimal
division feature and corresponding Gini coefficient, and retain the division
threshold of this feature. $C_{i}$ also checks whether the accuracy of the
decision tree is improved on this optimal feature partition, and if there is
no improvement, it judges that pre-pruning is needed.

  * $\circ$

S3. Upload local Gini coefficient: $C_{i}$ uploads the Gini coefficient and
pre-pruning information to $S$.

  * $\circ$

S4. Select global optimal features: $S$ selects the globally optimal feature.

  * $\circ$

S5. Send optimal global features: If pruning is not required, $S$ notifies the
corresponding client to return the ID division result; otherwise, $S$ notifies
all clients to perform pruning processing to form a leaf node.

  * $\circ$

S6. Send local decision: When pruning is not required, the notified client
returns the result of dividing the dataset with this feature (the ID set that
falls into the left and right subtrees).

  * $\circ$

S7. Broadcast global decision: $S$ notifies other clients of the ID division
and forms the first node of the decision tree. $C_{i}$ knows the results of
each ID division, but only knows the existence of local features in the global
tree. $S$ knows the structure of the entire tree and the corresponding feature
names of each node. $S$ and $C_{i}$ recursively establish new nodes and their
left and right subtrees according to the current $\mathtt{F^{\prime}}$ and
$\mathtt{D^{\prime}}$ until they form leaf nodes.

  * $\circ$

S8. Repeat until N tree built: Iteratively build N decision trees to form a
random forest.

Algorithm 3 FedRF - Client

1:Data set $D$, feature set $F_{i}$ on client $i\ (i=1\ to\ k)$

2:Local FedRF model on $Ci$

3:for $epoch\ t=1\ to\ N$ do

4: Receive $F_{i}^{{}^{\prime}}\subseteq F_{i}$, $D^{{}^{\prime}}\subseteq D$,
$T=D-D^{{}^{\prime}}$ from $S$

5: function TreeBuild($D^{{}^{\prime}},F_{i}^{{}^{\prime}},T$)

6: if all samples in $D^{{}^{\prime}}$ have the same label then

7: return $leaf\ node$

8: end if

9: $BestF_{i},GiniPara_{i},threshold_{i}\leftarrow$
Gini($D^{{}^{\prime}},F_{i}^{{}^{\prime}}$)

10: if $BestF_{i}$ cannot improve the accuracy on $T$ then

11: Send $needPruning_{i}$ to $S$ $\triangleright$ S2

12: end if

13: Send $BestF_{i},GiniPara_{i}$ to $S$ $\triangleright$ S3

14: if Receive ${}^{\prime}Prune^{\prime}$ message from $S$ then

15: return $leaf\ node$

16: else if Receive ${}^{\prime}Success^{\prime}$ message from $S$ then

17: Split $D^{{}^{\prime}}$ with $BestF_{i}$, get
$D_{l}^{{}^{\prime}},D_{r}^{{}^{\prime}},T_{l},T_{r}$

18: Send $D_{l}^{{}^{\prime}},D_{r}^{{}^{\prime}},T_{l},T_{r}$ to $S$
$\triangleright$ S6

19: leftTree $\leftarrow$
TreeBuild($D_{l}^{{}^{\prime}},F_{i}^{{}^{\prime}},T_{l}$)

20: rightTree $\leftarrow$
TreeBuild($D_{r}^{{}^{\prime}},F_{i}^{{}^{\prime}},T_{r}$) $\triangleright$ S7

21: else

22: Receive $D_{l}^{{}^{\prime}},D_{r}^{{}^{\prime}},T_{l},T_{r}$ from $S$

23: leftTree $\leftarrow$
TreeBuild($D_{l}^{{}^{\prime}},F_{i}^{{}^{\prime}},T_{l}$)

24: rightTree $\leftarrow$
TreeBuild($D_{r}^{{}^{\prime}},F_{i}^{{}^{\prime}},T_{r}$) $\triangleright$ S7

25: end if

26: return tree node

27: end function

28: Add this decision tree to forest

29:end for$\triangleright$ S8

Algorithm 4 FedRF - Server

1:Sample ID, feature names on all clients

2:Global FedForest model

3:for $epoch\ t=1\ to\ N$ do

4: for $i=1\ to\ k$ do

5: Send $F_{i}^{{}^{\prime}}\subseteq F_{i}$, $D^{{}^{\prime}}\subseteq D$,
$T=D-D^{{}^{\prime}}$ to $C_{i}$ $\triangleright$ S1

6: end for

7: function TreeBuild($D^{{}^{\prime}},T$)

8: if all samples in $D^{{}^{\prime}}$ have the same label then

9: return $leaf\ node$

10: end if

11: Receive $BestF_{i}^{k},GiniPara_{i}^{k},needPruning_{i}^{k}$
$\triangleright$ S3

12: $BestF_{s}=Max(BestF_{i=1}^{k},key=GiniPara_{i})$ $\triangleright$ S4

13: if $needPruning_{s}==True$ then

14: Send ${}^{\prime}Prune^{\prime}$ message to $C_{i}$

15: return $leaf\ node$

16: end if

17: Send ${}^{\prime}Success^{\prime}$ message to $C$ $\triangleright$ S5

18: Receive $D_{l}^{{}^{\prime}},D_{r}^{{}^{\prime}},T_{l},T_{r}$ from $C_{i}$

19: Send $D_{l}^{{}^{\prime}},D_{r}^{{}^{\prime}},T_{l},T_{r}$ to other
$C_{i}$

20: leftTree $\leftarrow$ TreeBuild($D_{l}^{{}^{\prime}},T_{l}$)

21: rightTree $\leftarrow$ TreeBuild($D_{r}^{{}^{\prime}},T_{r}$)

22: return tree node $\triangleright$ S7

23: end function

24: Add this decision tree to forest

25:end for$\triangleright$ S8

The prediction process is described as follows. $C_{i}$ analyzes the new data
ID according to the stored tree. For each sample, when an unknown node is
encountered, the sample enters the left and right subtrees. When the
encountered node is known (that means, the dividing feature of this node
belongs to this client’s dataset), the sample enters the corresponding subtree
according to the threshold of the feature. The final data that falls on the
leaf node _l_ of the global tree _t_ is the intersection of all client data
that falls on this leaf node.

We have experimented to evaluate the improvement of FedRF. In this experiment,
we constructed a VFL scenario with two clients sharing data features using the
Bosch dataset. There are 50 independent features in each client, which are
extracted after performing the principal component analysis (PCA) from
different production lines and the same 11154 samples. The experiment results
are shown in Table 2. It can be seen that we get better prediction results
than the federated random forest algorithm proposed in Liu et al. (2020a).

Table 2: Improvement of FedRF Algo. | ACC | PRE | F1 | MCC | AUC
---|---|---|---|---|---
FedForest Liu et al. (2020a) | 0.552 | 0.493 | 0.550 | 0.118 | 0.559
FedRF (Ours) | 0.825 | 0.802 | 0.883 | 0.593 | 0.866

##  4 Methodology

###  4.1 Experiment Overview and Research questions

This paper studies whether there is a big difference between the effectiveness
of FL and CL algorithms for failure prediction on the production line, and
then whether FL algorithms can replace CL algorithms on this problem. To this
end, we construct two production scenarios. One is HFL, where we compare
FedSVM and SVM. The other is VFL, where we compare the effectiveness of FedRF
and RF. We design four research questions (RQ), each of which concerns both
HFL and VFL. The logical association between the four RQs is depicted, as
shown in Fig. 3.

Figure 3: Experiment Overview and Research Questions

RQ1: On the whole Bosch testing dataset, is there a significant difference
between FedSVM and SVM? We ask the same question for FedRF and RF.

RQ1 is aimed at comparing the average performance of FL and CL algorithms on
the whole Bosch testing dataset. If the difference between the value of each
measurement is within the threshold $\delta$ ( $\delta=0.1$), it can be
considered that there is no significant difference between this pair of
algorithms on the Bosch testing dataset.

RQ2: On the random partial Bosch testing dataset, is there a significant
difference between FedSVM and SVM? We ask the same question for FedRF and RF.

RQ2 is aimed at comparing the average performance of the FL and CL algorithms
on the random partial Bosch testing dataset that contains consecutive time-
series samples. If the prediction results have no significant difference, it
means that the pair of algorithms perform similarly on the random partial
testing data.

RQ3: On the estimated unknown Bosch data, is there a significant difference
between FedSVM and SVM? We ask the same question for FedRF and RF.

RQ3 is aimed at comparing the average performance of FL and CL algorithms on
the estimated unknown Bosch data. To answer this RQ, we have designed a method
to compare the error distribution of FL and CL algorithm on the estimated
unknown data based on the given data, as shown in Fig. 4. This method consists
of four main steps (S1-S4), explained hereafter.

Figure 4: Comparing FL and CL algorithms on the Estimated Unknown Bosch Data

S1. Construct prediction table. We first construct a prediction table
containing the ground truth (GT) label, the prediction results (Pre) of CL and
FL algorithms of products ordered by timestamp (ts).

S2. Construct timed state sequence. The states represent the prediction error
compared to the GT labels: 1) _hit_ state: GT positive + Pre positive or GT
negative + Pre negative; 2) _miss_ state: GT positive + Pre negative; 3)
_mistake_ state: GT negative + Pre positive.

S3. Fit Markov models. We fit Markov model using the timed state sequence, and
obtain $\mathtt{M_{FL}}$ and $\mathtt{M_{CL}}$, respectively.

S4. Compare FL and CL Markov models. If the difference between the parameters
of $\mathtt{M_{FL}}$ and $\mathtt{M_{CL}}$ is within the threshold $\delta$
($\delta=0.1$), the difference between FL and CL predictive models will
maintain similar on estimated unknown Bosch data if the production line
structure, manufacturing process, and quality control methods are not changed.

RQ4: Is Bosch testing data heterogeneous? And what is the impact of data
heterogeneity on the results of RQ1-RQ3?

RQ4 analyzes the impact of data heterogeneity on the results of RQ1-RQ3. To
evaluate the heterogeneity of data, we first obtain N groups of randomly
selected continuous samples $\mathtt{D^{i}}$, and then uses the GT labels as
states (GT positive state and GT negative state) to construct the state
transition sequence $\mathtt{S^{i}_{GT}}$. Then we fit the Markov model
$\mathtt{M^{i}_{GT}}$ using $\mathtt{S^{i}_{GT}}$, and perform DBSCAN
clustering Ester et al. (1996) on the parameter matrix of
$\mathtt{M^{i}_{GT}}$. If the clustering result shows multiple different
clusters, it means that the data heterogeneity is strong. DBSCAN calculates
the distance between two parameter matrices by converting the two matrices
into a one-dimensional vector and calculating the angle between the two
vectors. If the included angle is less than the specified neighborhood
distance threshold, the two-parameter matrices can be regarded as on the same
cluster.

Based on the results of heterogeneity, the conclusions of RQ1-RQ3 (_Y_ means
there is no significant difference between FL and CL, _N_ means there is
significant difference between FL and CL) will be analyzed from the following
four aspects.

  * $\circ$

If the data heterogeneity is strong, and the answers to RQ1-RQ3 are _Y_ , then
the conclusion will be further strengthened. In our manufacturing scenario,
the FL and CL algorithms can obtain similar prediction results and can replace
each other.

  * $\circ$

If the data heterogeneity is strong and some answer in RQ1-RQ3 is _N_ , it
means that data heterogeneity may be one of the reasons for disturbing the
conclusions.

  * $\circ$

If the data heterogeneity is weak and the answers to RQ1-RQ3 are _Y_ , it can
be concluded that the FL algorithm can replace the CL one under the premise of
data homogeneity.

  * $\circ$

If the data heterogeneity is weak and some answer in RQ1-RQ3 is _N_ , it means
that the FL algorithm cannot replace CL even with homogeneous data.

###  4.2 Measurements

MSA is one of the commonly used analysis methods in manufacturing Montgomery
(2007). The learning-based failure prediction method can be seen as a product
quality measurement system. Taking the Bosch company’s product quality testing
results as a benchmark, we can analyze the FL and CL algorithms through the
MSA method. The evaluation measurements in MSA mainly include Accuracy (ACC),
Precision (PRE), and Stability. On this basis, F1, AUC, and Matthew’s
Correlation Coefficient (MCC) are also involved. The six measurements we use
in the empirical study are provided in Table 3, where P represents the number
of GT positive cases, N is the number of GT negative cases, TP (hit) is true
positive, TN (correct rejection) is true negative, FP (false alarm) is false
positive, and FN (miss) is false negative.

Table 3: Measurements Measurement |  Formula
---|---
ACC |  $\mathtt{\dfrac{TP+TN}{TP+FN+TN+FP}}$
PRE |  $\mathtt{\dfrac{TP}{TP+FP}}$
F1 |  $\mathtt{\dfrac{2TP}{2TP+FP+FN}}$
MCC |  $\mathtt{\dfrac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}$
AUC |  The area enclosed by the ROC curve and the reference line
Stability |  The tendency of accuracy for N groups of random selected continuous data group.

MCC is suitable for evaluating the prediction results of unbalanced data. AUC
is a commonly used two-category evaluation method, and its value is the area
enclosed by the ROC curve and the reference line. The x-axis of a ROC curve is
the false positive rate, and the y-axis of a ROC curve is the true positive
rate. It shows the relationship between clinical sensitivity and specificity
for every possible cut-off. It is a graph with: The x-axis showing
1-specificity ($\mathtt{=FP/(FP+TN)}$) The y-axis showing sensitivity
($\mathtt{=TP/(TP+FN)}$). The range of AUC value is between 0 and 1.

Stability characterizes the ability of the measurement system to maintain a
constant performance within a certain time range. We divide the test set into
N groups (N = 10) on average according to the time-series and calculate the
accuracy of these N groups to analyze the stability of the target algorithm.

###  4.3 Bosch Dataset

The Bosch dataset is one of the largest public manufacturing datasets (14.3Gb)
on Kaggle. The Bosch dataset consists of a training set (1184687 samples) and
a test set (1183748 samples). Each sample has three types of features:
categorical feature, numeric feature, and date feature, and a two-category
classification label indicating the faulty status of a product.

The date feature gives the time stamp of each site through which the product
passes. Based on the date feature, it is possible to analyze the timed
behavior. Existing studies have constructed LSTM neural network models based
on time-series information Huang et al. (2019b); Moldovan et al. (2019); Liu
et al. (2020d). Since we focus on empirical research on FL without time
information, the date features are excluded from the scope of our study.
Nevertheless, the time-series information can still provide time-continuity
data for empirical research on the estimated unknown data.

The data features are shown in Table 4. There are 1184687 products in the
dataset, including 1177808 positive samples (Pos) and 6879 negative ones
(Neg). In the feature space, the dataset has 968 features. These features are
collected from 51 workstations (S1-S51) on 4 production lines L0-L3 that
contain 168, 513, 42, and 245 features, respectively. L1_S24_F1695 indicates
that the feature No.1695 was observed at the No.24 workstation of the L1
production line. The label Response represents whether the final product is
qualified (label = 0) or not (label = 1). From the perspective of data items,
only a few items (0.58%) in the dataset result are qualified, and the data is
extremely unbalanced. The data is also extremely sparse, with only a few
workstations collecting data from most products Carbery et al. (2019).

Table 4: Bosch dataset Sample Num (1184687) | Features Num (968)
---|---
Positive | Negative | L0 | L1 | L2 | L3
1177808 | 6879 | 168 | 513 | 42 | 245

###  4.4 Experiment Scenarios of HFL and VFL

To compare FedSVM and SVM on the testing data, we form four workshops A, B, C,
and D that do not share manufacturing data but have the same structure and
configuration of the production line. Under the premise that the product
features are aligned, they contain different samples, and each sample has a
unique ID. Each workshop can be regarded as an independent client. Table 5
shows the data used in this experiment. The number of samples on the 4 clients
is basically the same. In the experiment, the data set is randomly distributed
to all clients on average, each of which possesses 3000 positive and 1500
negative samples. The four workshops have 713 common features of data.

Table 5: Data Description of HFL Scenario Sample Num. | Positive | Negative | Feature Num.
---|---|---|---
Total | 12000 | 6000 | 713 selected
Per client | 3000 | 1500

To compare FedRF and RF on the given testing data, it is assumed that the
first three production lines (L0, L1, L2) of the Bosch dataset belong to the
same organization O1, and the L3 production line belongs to another
organization O2. O1 and O2 are independent. Since the samples in O1 and O2
have the same ID and different characteristics, VFL is applied. The data set
used in the FedRF experiment is shown in Table 6. We extract the features
belonging to L0, L1, and L2 as a group, and the features belonging to L3 as
another group. Then we perform principal component analysis (PCA) to reduce
the feature dimension in O1/O2 from 723/245 to 22/22, respectively. According
to the overall analysis of PCA results, the first 22 dimensions can represent
more than 95% of the variance, so reducing to 22 dimensions, respectively,
which will not cause too much information loss Zhang et al. (2016). The
positive and negative sample numbers are 12000 and 6000, respectively.

Table 6: Data Description for VFL Scenario Sample Num | Positive | Negative | Feature Num
---|---|---|---
Total | 12000 | 6000 | 44 (F1 - F44)
Client O1 | 22 (F1 - F22)
Client O2 | 22 (F23 - F44)

##  5 Experiment Results

Our experiments investigate and answer four research questions.

###  5.1 RQ1: Comparison on Bosch Testing Data

####  5.1.1 FedSVM vs. SVM

Table 7 shows the experimental results. To ensure that the data set and the
algorithm kernel are consistent, the baseline is the result of FedSVM when the
number of clients is set to 1, ie., FedSVM degenerates into SVM. Under this
premise, the results of FedSVM and SVM are compared. When analyzing the
stability, the experiment data is ordered by time-series and then divided into
10 groups on average. The measurements of each group are calculated. The
overall results of stability are shown in Fig. 5. The average value of
stability is above 0.7 for FedSVM and SVM. The variance of the stability is
given as Stability (Var) in Table 7.

Table 7: Experiment Results of RQ1 on FedSVM/SVM Algo. | ACC | PRE | F1 | MCC | AUC |  Stab. (Var)
---|---|---|---|---|---|---
FedSVM | 0.825 | 0.859 | 0.869 | 0.607 | 0.807 |  0.005
SVM | 0.859 | 0.828 | 0.903 | 0.690 | 0.902 |  0.003
Diff | -0.034 | 0.031 | -0.046 | -0.083 | -0.095 |  0.002
Figure 5: FedSVM vs. SVM on Stability

The value difference of ACC, Precision, F1, MCC, AUC, and Stability (Var)
between FedSVM and SVM are all within a threshold value of 0.1, respectively.
Therefore, the conclusion of RQ1 on FedSVM and SVM is that _the prediction
results of FedSVM and SVM on the Bosch testing data are not significantly
different_.

In Figure 5, FedSVM performs slightly better than SVM on two test groups,
namely 1 and 7. Similar results are also shown for the Precision measure in
Table 7. This effect is only shown in the HFL scenario, where the clients
contribute local data features to the central server. If the amount of data on
each client is small, FL’s effect can be better than CL because FL expands the
number of IID data samples in the HFL scenario.

####  5.1.2 FedRF vs. RF

The experimental results are shown in Table 8. FedRF degenerates into RF by
combining two clients so that the effect of FedRF and RF can be compared under
the premise of ensuring the consistency of the testing data and the algorithm
kernel. When analyzing the stability, the test set is divided into 10 groups
on average according to the time-series, and the measurements of each group
are calculated. The variance of the stability is given in Table 8, and the
overall results of stability are shown in Fig. 6.

Table 8: Experiment Results of RQ1 on FedRF/RF Algo. | ACC | PRE | F1 | MCC | AUC |  Stab. (Var)
---|---|---|---|---|---|---
FedRF | 0.843 | 0.808 | 0.894 | 0.659 | 0.902 |  0.004
RF | 0.868 | 0.836 | 0.909 | 0.713 | 0.912 |  0.002
Diff | -0.025 | -0.028 | -0.015 | -0.054 | -0.010 |  0.002
Figure 6: FedRF vs. RF on Stability

The value difference of ACC, Precision, F1, MCC, AUC, and Stability (Var) on
each group are all within 0.1. Therefore, _the prediction results of FedRF and
RF on the Bosch testing data are not significantly different._

###  5.2 RQ2: Comparison on Random Partial Bosch Testing Data

In order to compare the average difference between the FL and CL algorithms in
randomly sampling partial data, we randomly select N groups (N=100) of testing
samples. Each group contains L time-continuous samples. The starting point S
of the group is randomly set. L is randomly generated from the interval [300,
1000].

####  5.2.1 FedSVM vs. SVM

The line charts of the prediction results of FedSVM and SVM on 100 groups are
shown in Fig. 7. In all charts, the abscissa is the serial number of the 100
groups. The ordinate is the value of the measurement on each group. The solid
blue line and red dot-dash line represent the prediction result of FedSVM and
SVM, respectively. Through the charts, we can see the range and difference of
the FedSVM and SVM on the prediction results of each group.

(a) ACC Values of FedSVM and SVM

(b) PRE Values of FedSVM and SVM

(c) F1 Values of FedSVM and SVM

(d) MCC Values of FedSVM and SVM

(e) AUC Values of FedSVM and SVM

Figure 7: FedSVM vs. SVM on Random Partial Data

We draw a histogram to visualize the average difference between FedSVM and SVM
on each measurement for 100 groups of data, as shown in Fig. 8. Based on all
results, the difference values in ACC, PRE, and F1 are all within 0.1. The
difference values in MCC and AUC are mostly within 0.2. It is thus validated
that _the performance of FedSVM and SVM on random partial Bosch testing data
is not significantly different_.

(a) ACC Difference

(b) PRE Difference

(c) F1 Difference

(d) MCC Difference

(e) AUC Difference

Figure 8: Difference of FedSVM and SVM on Random Partial Testing Data

####  5.2.2 FedRF vs. RF

The line charts of FedRF and RF on 100 groups of samples are shown in Fig. 9.
In all line charts, the abscissa is the number of groups. The ordinate is the
value of the measurements on each group. The solid blue line and red dot-dash
line represent the prediction result of the FedRF and RF, respectively.

(a) ACC Values of FedRF and RF

(b) PRE Values of FedRF and RF

(c) F1 Values of FedRF and RF

(d) MCC Values of FedRF and RF

(e) AUC Values of FedRF and RF

Figure 9: FedRF vs. RF on Random Partial Data

We draw a histogram of the differences between FedRF and RF on each
measurement for 100 groups of data, as shown in Fig. 10. Based on all results,
the difference values in ACC, PRE, F1, and AUC are all within 0.1. The
difference values in MCC are within 0.2, and the MCC values of over 80% groups
are within 0.1. It is thus validated that _the performance of FedRF and RF on
the random partial Bosch testing data is not significantly different._

(a) ACC Difference

(b) PRE Difference

(c) F1 Difference

(d) MCC Difference

(e) AUC Difference

Figure 10: Difference of FedRF and RF on Random Partial Testing Data

###  5.3 RQ3: Comparison on Estimated Unknown Bosch Data

This experiment is used to analyze whether the FL and CL algorithms can
maintain no significant difference on the estimated unknown Bosch data. This
experiment is conducted following the method presented in Sect. 4.

####  5.3.1 FedSVM vs. SVM

We first constructed a timed state sequence indicating the prediction error
using FedSVM and SVM, respectively, based on which their Markov models are
fitted. The parameters of two Markov models are given in Table
LABEL:tab:RQ3-SVM_resa and LABEL:tab:RQ3-SVM_resa. The value difference
between each pair of parameters is calculated in Table LABEL:tab:RQ3-SVM_resc.
The average difference value between each pair of parameters is 0.054. The
maximum difference value is 0.096. It means that there exists no significant
difference between the two Markov models, and the prediction error between
FedSVM and SVM are almost equally distributed on the estimated unknown Bosch
data. Therefore, _the performance of FedSVM and SVM in the subsequent Bosch
production was not significantly different._

Table 9: Experiment Results of RQ3 on FedSVM/SVM

State |  Hit |  Miss |  Mistake
---|---|---|---
Hit |  0.882 |  0.113 |  0.005
Miss |  0.715 |  0.280 |  0.005
Mistake |  0.813 |  0.062 |  0.125
(a) Markov Model Parameters of Prediction Error using FedSVM

|  Hit |  Miss |  Mistake
---|---|---|---
Hit |  0.851 |  0.077 |  0.072
Miss |  0.690 |  0.225 |  0.085
Mistake |  0.717 |  0.125 |  0.158
(b) Markov Model Parameters of Prediction Error using SVM

|  Hit |  Miss |  Mistake
---|---|---|---
Hit |  0.031 |  0.036 |  0.067
Miss |  0.025 |  0.055 |  0.080
Mistake |  0.096 |  0.063 |  0.033
(c) Comparison of Two Markov Model Parameters

####  5.3.2 FedRF vs. RF

We first constructed a timed state sequence indicating the prediction error
using FedRF and RF, respectively, based on which their Markov models are
fitted. The parameters of two Markov models are given in Table
LABEL:tab:RQ3-RF_resa and LABEL:tab:RQ3-RF_resa. The value difference between
each pair of parameters is calculated in Table LABEL:tab:RQ3-RF_resc. The
average difference value between each pair of parameters is 0.035. The maximum
difference value is 0.100. It means that there exists no significant
difference between the two Markov models, and the prediction error between
FedRF and RF are equally distributed on the estimated unknown Bosch data.
Therefore, _the performance of FedRF and RF in the subsequent Bosch production
was not significantly different._

Table 10: Experiment Results of RQ3 on FedRF/RF

|  Hit |  Miss |  Mistake
---|---|---|---
Hit |  0.869 |  0.130 |  0.001
Miss |  0.701 |  0.299 |  0.000
Mistake |  1.000 |  0.000 |  0.000
(a) Markov Model Parameters of Prediction Error using FedRF

|  Hit |  Miss |  Mistake
---|---|---|---
Hit |  0.888 |  0.109 |  0.003
Miss |  0.735 |  0.263 |  0.002
Mistake |  0.900 |  0.100 |  0.000
(b) Markov Model Parameters of Prediction Error using RF

|  Hit |  Miss |  Mistake
---|---|---|---
Hit |  0.019 |  0.021 |  0.002
Miss |  0.034 |  0.036 |  0.002
Mistake |  0.100 |  0.100 |  0.000
(c) Comparison of Two Markov Model Parameters

###  5.4 RQ4: Heterogeneity of Bosch Testing Data

This experiment is used to evaluate the heterogeneity of the testing data used
in the experiments of RQ1-RQ3. We follow the method proposed in Sect. 4 to
experiment. One hundred groups of time-ordered consecutive samples were
randomly selected. According to the GT label of the sample, the qualified
sample is set to state S0, and the unqualified sample is set to state S1,
forming a state sequence. For the state sequence of each group of samples, the
Markov model of k=1 step and k=2 steps are established, respectively. The
DBSCAN algorithm is used to cluster and analyze all the 1-step and 2-step
parameter matrices. If the clustering results show multiple different
clusters, it can explain that there is heterogeneity within the testing data.

####  5.4.1 Heterogeneity of the Testing Data used in FedSVM

The experimental results are shown in Table 11. There are outliers in the test
results, so the sum of the samples in each cluster is less than 100. It can be
seen that the testing data is divided into 2 clusters. The difference between
clusters is no less than the distance threshold, which means that the data
used in the experiment of FedSVM is heterogeneous. It enforces the conclusion
that _FedSVM and SVM have no significant difference on heterogeneous
manufacturing data for the problem of failure prediction_.

Table 11: Experiment Results of RQ4 on FedSVM Step | Clusters |  Distance threshold (∘) |  Contour factor |  Samples / cluster
---|---|---|---|---
k = 1 | 2 |  5 |  0.456 |  92 / 6
k = 2 | 2 |  8 |  0.421 |  89 / 6

According to the experiment results and the statement of RQ4 in Section 4.1,
the data heterogeneity in the VFL scenario is relatively strong and the answer
to RQ1-RQ3 are positive. This means in our VFL manufacturing scenarios, the FL
and CL algorithms can obtain similar prediction results and can thus replace
each other under the premise of data heterogeneity.

####  5.4.2 Heterogeneity of the Testing Data used in FedRF

The experimental results are shown in Table 12. It can be seen that the
testing data is divided into 2 clusters. The difference between clusters is no
less than the distance threshold, which means that the data used in the
experiment of FedRF is heterogeneous. It enforces the conclusion that _FedRF
and RF have no significant difference on heterogeneous manufacturing data for
the problem of failure prediction_.

Table 12: Experiment Results of RQ4 on FedRF Step | Clusters |  Distance threshold (∘) |  Contour factor |  Samples / cluster
---|---|---|---|---
k = 1 | 2 |  4 |  0.524 |  94 / 6
k = 2 | 2 |  8 |  0.473 |  78 / 22

According to the experiment results and the statement of RQ4 in Section 4.1,
the data heterogeneity in the HFL scenario is relatively strong and the answer
to RQ1-RQ3 are positive. This means in our HFL manufacturing scenarios, the FL
and CL algorithms can obtain similar prediction results and can thus replace
each other under the premise of data heterogeneity.

##  6 Threats to Validity

Following common guidelines for empirical studies Runeson and Höst (2009); Yin
(2017), we discuss threats to the validity of our study.

Threats to internal validity are mainly concerned with uncontrolled factors.
These factors may impact the results and reduce their creditability. In this
work, the main threat to internal validity is potential defects in the
implementation of our own algorithm and the reimplementation of other
algorithms (mainly the federated SVM and federated random forest algorithms).
To reduce this threat, we reimplement the baseline techniques by following
their papers. We carefully review all our code and experiment scripts by
several developers to ensure their correctness. However, there is always a
small possibility of defects, which introduces risk to the result’s
correctness.

Threats to external validity are mainly concerned with whether the performance
of our techniques can still hold in other experimental settings. In this work,
the main threat to external validity is the impact of data preprocessing
method. As we mentioned in Sect. 1, the existing works revealed that the data
preprocessing on the Bosch dataset has a significant impact on the prediction
result. In our work, we have applied PCA to find the principle features. For
example, in the VFL scenario, according to the PCA analysis, only 22 features
are kept in each client. But, these features represent more than 95% of the
variance of data. Another threat to external validity is the experimental
methodology we designed. In order to answer whether the FL algorithm can
replace the CL algorithm or not, we conduct experiment on the whole testing
data, on the partial random testing data, and on the estimated unknown data,
respectively. This method is aimed at simulating possible data set used in
manufacturing. Therefore, the answer tends to be complete.

The first threat to construct validity is the suitability of our evaluation
metrics. To reduce this risk, we follow the same measurements (ACC, precision,
F1, MCC, AUC, stability) as used in other works on failure prediction in
production line Carbery et al. (2019, 2018); Zhang et al. (2016); Khoza and
Grobler (2019); Kotenko et al. (2019); Mangal and Kumar (2016); Hebert (2016);
Maurya (2016); Huang et al. (2019b); Moldovan et al. (2019); Liu et al.
(2020d) to measure the quality of the learning model, which reduces the risk
of construction validity.

Another threat to construct validity is the threshold $\delta$ ( $\delta=0.1$
or $\delta=0.2$) used in the experiment. In the industry, the threshold value
is set according to the production needs. We have performed a hypothesis test
for the difference between the two approaches’ performances to explain that
the difference between FL and CL is within the threshold. The original
hypothesis H0 is difference $>\delta$, and the alternative hypothesis H1 is
difference $<\delta$, where the difference = CL – FL on the same random
partial testing data group. Suppose $\alpha$ = 0.05. The p-value of the
evaluation metrics for FedSVM vs. SVM and FedForest vs. RForest is shown in
Table 13 and Table 14, respectively. As $alpha>$ p-value for all metrics, the
alternative hypothesis H1 is accepted, which means that the difference between
CL and FL is less than the threshold $\delta$ ( $\delta=0.1$ or $\delta=0.2$).

Table 13: P-value of FedSVM vs. SVM ($\alpha$ = 0.05) Metrics | Threshold | Statistics | p-value
---|---|---|---
ACC | 0.1 | -22.48877 | 5.44616e-41
Precision | 0.1 | -83.57313 | 6.33511e-94
F1 | 0.1 | -17.48523 | 2.45813e-32
MCC | 0.2 | -20.44888 | 1.27870e-37
AUC | 0.2 | -26.63278 | 3.08126e-47
stability | 0.1 | -5.40692 | 0.00021
Table 14: P-value of FedForest vs. RForest ($\alpha$ = 0.05) Metrics | Threshold | Statistics | p-value
---|---|---|---
ACC | 0.1 | -34.44843 | 3.16706e-57
Precision | 0.1 | -48.40227 | 5.04976e-71
F1 | 0.1 | -75.16218 | 1.95987e-89
MCC | 0.2 | -12.90324 | 3.11933e-23
AUC | 0.2 | -16.77851 | 5.26331e-31
stability | 0.1 | -9.83780 | 2.04964e-06

##  7 Discussion

Due to data privacy and security, we are facing a bottleneck when applying CL
in the industry. In this work, we have adopted two FL algorithms to predict
failure in the production line. According to our investigation of existing
results of this problem, SVM and RF performed better on Bosch data among CL
algorithms that did not consider time series. Therefore, we choose Federated-
SVM and Federated-RF in the study. We expect that both will perform as well as
the CL algorithms. The results showed that our intuition is not wrong on the
given dataset.

An interesting question is whether FL can achieve the same or similar
prediction results as CL. If yes, what are the conditions (e.g., the the
dataset’s size)? Otherwise, what are the guidelines on when we can use FL in
place of CL? As our study is limited to the Bosch dataset from the
manufacturing industry, we have conducted a literature investigation to answer
this question. According to existing studies Kairouz et al. (2019); Duan et
al. (2019), FL’s effect is mainly related to the data distribution and data
amount. Two important facts are summarized as follows.

  1. 1.

When the data is independent and identically distributed, the learning results
of FL and CL are similar. For instance, the work Lu et al. (2019); Chen et al.
(2020); Sozinov et al. (2018); Olivia et al. (2019) showed that when the
training data is independent and identically distributed (IID), the difference
between FL and CL is within 3%. If the amount of data on each client is small,
FL’s effect is better than CL because FL expands the number of IID data
samples Ickin et al. (2019); Bakopoulou et al. (2019); Guha et al. (2019);
Xinle et al. (2019); Shaoqi et al. (2020); Suzumura et al. (2019).

  2. 2.

When the training data is unevenly distributed, FL may not achieve the same
effect as CL Sheller et al. (2018a); Hu et al. (2018); Feng et al. (2020);
Dianbo et al. (2018); Luca and M (2019); Chen et al. (2020); Boughorbel et al.
(2019a); Sozinov et al. (2018). The review article Kairouz et al. (2019)
claimed that if the amount of data on a client is small, the CL model may be
better than the FL model trained using data on multi-clients in the case of
uneven distribution of training data.

Besides, some works have shown that the effect of FL is also related to
encryption algorithms R et al. (2019); Tao et al. (2020); Feng et al. (2020);
Olivia et al. (2019); Lu et al. (2020); Li et al. (2019, 2020b). If the
encryption strength increases, the information loss on the data will be worse.
So that the effect of FL decreases. The trade-off between data privacy
protection and the accuracy of the trained model is still inevitable. It is
still considered an open question of how the effect of FL is related to other
factors.

##  8 Conclusion

There is increasing attention for federated learning in the manufacturing
industry, but few works have studied how federated learning methods perform in
practice. In this work, we have conducted an empirical study of comparing FL
and CL methods for problem of the failure prediction in the production line.
We constructed a horizontal and vertical federated learning scenario based on
the Bosch dataset. We implemented FedSVM to compare with SVM in the HFL
scenario and designed FedRF to compare it with RF in the VFL scenario. We also
designed an experiment process for evaluating the effectiveness of FL and CL
algorithms, which can be reused in future studies. Our results reveal that
FedSVM and FedRF can replace SVM and RF, respectively, for failure prediction
in the Bosch production line. Because the CL algorithm can be replaced by the
FL one (1) on the global testing dataset; (2) on the random partial testing
dataset; (3) on the estimated unknown Bosch data. The value of each evaluation
metric for FL and CL algorithms differ within 0.1. Moreover, the fact that the
testing data is heterogeneous enhances the above three conclusions.

The results of our empirical study reveal that FL can replace CL in some
applications and FL is a desirable way of protecting data in the manufacturing
industry. More FL techniques can also be investigated in other manufacturing
applications. The studies Moldovan et al. (2019); Huang et al. (2019b); Liu et
al. (2020d) have considered time-series features and showed that the Long
Short Term Memory (LSTM) network can improve the prediction result when time-
series features are included Huang et al. (2019b). A future work is to design
federated LSTM models to compare with centralized LSTM model, and study
whether FL can replace CL within this context.

###### Acknowledgements.

This work was supported by National Key Research and Development Program of
China Grant No. 2019YFB1703903, National Natural Science Foundation of China
Grant No. 61732019 and No. 61902011, and the Grant No. NJ2018014 of the Key
Laboratory of Safety-Critical Software (Nanjing University of Aeronautics and
Astronautics), Ministry of Industry and Information Technology.

## References

  * Aussel et al. [2020] Nicolas Aussel, Sophie Chabridon, and Yohan Petetin.  Combining federated and active learning for communication-efficient distributed failure prediction in aeronautics.  _arXiv preprint arXiv:2001.07504_ , 2020.
  * Bakopoulou et al. [2019] Evita Bakopoulou, Balint Tillman, and Athina Markopoulou.  A federated learning approach for mobile packet classification.  _arXiv preprint arXiv:1907.13113_ , 2019.
  * Boughorbel et al. [2019a] Sabri Boughorbel, Fethi Jarray, Neethu Venugopal, Shabir Moosa, Haithum Elhadi, and Michel Makhlouf.  Federated uncertainty-aware learning for distributed hospital EHR data.  _arXiv preprint arXiv:1910.12191_ , 2019a.
  * Boughorbel et al. [2019b] Sabri Boughorbel, Fethi Jarray, Neethu Venugopal, Shabir Moosa, Haithum Elhadi, and Michel Makhlouf.  Federated uncertainty-aware learning for distributed hospital ehr data.  _arXiv preprint arXiv:1910.12191_ , 2019b.
  * Brisimi et al. [2018] Theodora S Brisimi, Ruidi Chen, Theofanie Mela, Alex Olshevsky, Ioannis Ch Paschalidis, and Wei Shi.  Federated learning of predictive models from federated electronic health records.  _International journal of medical informatics_ , 112:59–67, 2018.
  * Carbery et al. [2018] Caoimhe M Carbery, Roger Woods, and Adele H Marshall.  A bayesian network based learning system for modelling faults in large-scale manufacturing.  In _2018 IEEE international conference on industrial technology (ICIT)_ , pages 1357–1362. IEEE, 2018.
  * Carbery et al. [2019] Caoimhe M Carbery, Roger Woods, and Adele H Marshall.  A new data analytics framework emphasising preprocessing of data to generate insights into complex manufacturing systems.  _Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science_ , 233(19-20):6713–6726, 2019.
  * Chen et al. [2020] Dawei Chen, Linda Jiang Xie, BaekGyu Kim, Li Wang, Choong Seon Hong, Li-Chun Wang, and Zhu Han.  Federated learning based mobile edge computing for augmented reality applications.  In _2020 International Conference on Computing, Networking and Communications (ICNC)_ , pages 767–773. IEEE, 2020.
  * Dianbo et al. [2018] Liu Dianbo, Miller Timothy, Sayeed Raheel, and Mandl Kenneth D.  Fadl: Federated-autonomous deep learning for distributed electronic health record.  _arXiv preprint arXiv:1811.11400_ , 2018.
  * Duan et al. [2019] Moming Duan, Duo Liu, Xianzhang Chen, Yujuan Tan, Jinting Ren, Lei Qiao, and Liang Liang.  Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications.  In _37th IEEE International Conference on Computer Design, ICCD 2019, Abu Dhabi, United Arab Emirates, November 17-20, 2019_ , pages 246–254. IEEE, 2019.
  * Ester et al. [1996] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al.  A density-based algorithm for discovering clusters in large spatial databases with noise.  In _Kdd_ , volume 96, pages 226–231, 1996.
  * Feng et al. [2020] Jie Feng, Can Rong, Funing Sun, Diansheng Guo, and Yong Li.  PMF: A privacy-preserving human mobility prediction framework via federated learning.  _IMWUT_ , 4(1):10:1–10:21, 2020.
  * Guha et al. [2019] Roy Abhijit Guha, Siddiqui Shayan, Pölsterl Sebastian, Navab Nassir, and Wachinger Christian.  Braintorrent: A peer-to-peer environment for decentralized federated learning.  _arXiv preprint arXiv:1905.06731_ , 2019.
  * Hard et al. [2018] Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Françoise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage.  Federated learning for mobile keyboard prediction.  _arXiv preprint arXiv:1811.03604_ , 2018.
  * Hebert [2016] Jeff Hebert.  Predicting rare failure events using classification trees on large scale manufacturing data with complex interactions.  In _2016 ieee international conference on big data (big data)_ , pages 2024–2028. IEEE, 2016.
  * Hu et al. [2018] Binxuan Hu, Yujia Gao, Liang Liu, and Huadong Ma.  Federated region-learning: An edge computing based framework for urban environment sensing.  In _IEEE Global Communications Conference, GLOBECOM 2018, Abu Dhabi, United Arab Emirates, December 9-13, 2018_ , pages 1–7. IEEE, 2018\.
  * Huang et al. [2019a] Li Huang, Andrew L Shea, Huining Qian, Aditya Masurkar, Hao Deng, and Dianbo Liu.  Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records.  _Journal of Biomedical Informatics_ , 99:103291, 2019a.
  * Huang et al. [2019b] Xin Huang, Cecilia Zanni-Merk, and Bruno Crémilleux.  Enhancing deep learning with semantics: an application to manufacturing time series analysis.  _Procedia Computer Science_ , 159:437–446, 2019b.
  * Ickin et al. [2019] Selim Ickin, Konstantinos Vandikas, and Markus Fiedler.  Privacy preserving qoe modeling using collaborative learning.  In Pedro Casas, Florian Wamser, Fabian E. Bustamante, and David R. Choffnes, editors, _Proceedings of the 4th Internet-QoE Workshop on QoE-based Analysis and Management of Data Communication Networks, Internet-QoE@MobiCom 2019, Los Cabos, Mexico, October 21, 2019_ , pages 13–18. ACM, 2019.
  * Kairouz et al. [2019] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.  Advances and open problems in federated learning.  _arXiv preprint arXiv:1912.04977_ , 2019.
  * Khoza and Grobler [2019] Sibusiso C Khoza and Jacomine Grobler.  Comparing machine learning and statistical process control for predicting manufacturing performance.  In _EPIA Conference on Artificial Intelligence_ , pages 108–119. Springer, 2019.
  * Kotenko et al. [2019] Igor Kotenko, Igor Saenko, and Alexander Branitskiy.  Improving the performance of manufacturing technologies for advanced material processing using a big data and machine learning framework.  _Materials Today: Proceedings_ , 11:380–385, 2019.
  * Kusiak [2017] Andrew Kusiak.  Smart manufacturing must embrace big data.  _Nature_ , 544(7648):23–25, 2017.
  * Li et al. [2017] Bo-hu Li, Bao-cun Hou, Wen-tao Yu, Xiao-bing Lu, and Chun-wei Yang.  Applications of artificial intelligence in intelligent manufacturing: a review.  _Frontiers of Information Technology & Electronic Engineering_, 18(1):86–96, 2017.
  * Li et al. [2020a] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith.  Federated learning: Challenges, methods, and future directions.  _IEEE Signal Processing Magazine_ , 37(3):50–60, 2020a.
  * Li et al. [2019] Wenqi Li, Fausto Milletarì, Daguang Xu, Nicola Rieke, Jonny Hancox, Wentao Zhu, Maximilian Baust, Yan Cheng, Sébastien Ourselin, M. Jorge Cardoso, and Andrew Feng.  Privacy-preserving federated brain tumour segmentation.  In Heung-Il Suk, Mingxia Liu, Pingkun Yan, and Chunfeng Lian, editors, _Machine Learning in Medical Imaging - 10th International Workshop, MLMI 2019, Held in Conjunction with MICCAI 2019, Shenzhen, China, October 13, 2019, Proceedings_ , volume 11861 of _Lecture Notes in Computer Science_ , pages 133–141. Springer, 2019.
  * Li et al. [2020b] Zengpeng Li, Vishal Sharma, and Saraju P. Mohanty.  Preserving data privacy via federated learning: Challenges and solutions.  _IEEE Consumer Electronics Magazine_ , 9(3):8–16, 2020b.
  * Liu et al. [2020a] Yang Liu, Yingting Liu, Zhijie Liu, Yuxuan Liang, Chuishi Meng, Junbo Zhang, and Yu Zheng.  Federated forest.  _IEEE Transactions on Big Data_ , 2020a.
  * Liu et al. [2020b] Yi Liu, JQ James, Jiawen Kang, Dusit Niyato, and Shuyu Zhang.  Privacy-preserving traffic flow prediction: A federated learning approach.  _IEEE Internet of Things Journal_ , 2020b.
  * Liu et al. [2020c] Yi Liu, Li Zhang, Ning Ge, and Guanghao Li.  A systematic literature review on federated learning: From a model quality perspective.  _arXiv preprint arXiv:2012.01973_ , 2020c.
  * Liu et al. [2020d] Zhenyu Liu, Donghao Zhang, Weiqiang Jia, Xianke Lin, and Hui Liu.  An adversarial bidirectional serial–parallel lstm-based qtd framework for product quality prediction.  _Journal of Intelligent Manufacturing_ , pages 1–19, 2020d.
  * Lu et al. [2019] Sidi Lu, Yongtao Yao, and Weisong Shi.  Collaborative learning on the edges: A case study on connected vehicles.  In Irfan Ahmad and Swaminathan Sundararaman, editors, _2nd USENIX Workshop on Hot Topics in Edge Computing, HotEdge 2019, Renton, WA, USA, July 9, 2019_. USENIX Association, 2019.
  * Lu et al. [2020] Yunlong Lu, Xiaohong Huang, Yueyue Dai, Sabita Maharjan, and Yan Zhang.  Differentially private asynchronous federated learning for mobile edge computing in urban informatics.  _IEEE Trans. Ind. Informatics_ , 16(3):2134–2143, 2020.
  * Luca and M [2019] Corinzia Luca and Buhmann Joachim M.  Variational federated multi-task learning.  _arXiv preprint arXiv:1906.06268_ , 2019.
  * Mangal and Kumar [2016] Ankita Mangal and Nishant Kumar.  Using big data to enhance the bosch production line performance: A kaggle challenge.  In _2016 IEEE International Conference on Big Data (Big Data)_ , pages 2029–2035. IEEE, 2016.
  * Maurya [2016] Abhinav Maurya.  Bayesian optimization for predicting rare internal failures in manufacturing processes.  In _2016 IEEE International Conference on Big Data (Big Data)_ , pages 2036–2045. IEEE, 2016.
  * McMahan et al. [2016] H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al.  Communication-efficient learning of deep networks from decentralized data.  _arXiv preprint arXiv:1602.05629_ , 2016.
  * Moldovan et al. [2019] Dorin Moldovan, Ionut Anghel, Tudor Cioara, and Ioan Salomie.  Time series features extraction versus lstm for manufacturing processes performance prediction.  In _2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD)_ , pages 1–10. IEEE, 2019.
  * Montgomery [2007] Douglas C Montgomery.  _Introduction to statistical quality control_.  John Wiley & Sons, 2007.
  * Mowla et al. [2019] Nishat I Mowla, Nguyen H Tran, Inshil Doh, and Kijoon Chae.  Federated learning-based cognitive detection of jamming attack in flying ad-hoc network.  _IEEE Access_ , 2019.
  * Nguyen et al. [2019] Thien Duc Nguyen, Samuel Marchal, Markus Miettinen, Hossein Fereidooni, N Asokan, and Ahmad-Reza Sadeghi.  Dïot: A federated self-learning anomaly detection system for iot.  In _2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)_ , pages 756–767. IEEE, 2019.
  * Olivia et al. [2019] Choudhury Olivia, Gkoulalas-Divanis Aris, Salonidis Theodoros, Sylla Issa, Park Yoonyoung, Hsu Grace, and Das Amar.  Differential privacy-enabled federated learning for sensitive health data.  _arXiv preprint arXiv:1910.02578_ , 2019.
  * R et al. [2019] Pfohl Stephen R, Dai Andrew M, and Heller Katherine.  Federated and differentially private learning for electronic health records.  _arXiv preprint arXiv:1911.05861_ , 2019.
  * [44] AC Einat Ronen and Kenneth Burns.  Utilizing predictive sales analytics with big data.  Technical report, Intel, Tech. Rep., 2013.[Online]. Available: http://www. intel. com/content ….
  * Runeson and Höst [2009] Per Runeson and Martin Höst.  Guidelines for conducting and reporting case study research in software engineering.  _Empirical software engineering_ , 14(2):131, 2009\.
  * Samarakoon et al. [2019] Sumudu Samarakoon, Mehdi Bennis, Walid Saad, and Mérouane Debbah.  Distributed federated learning for ultra-reliable low-latency vehicular communications.  _IEEE Transactions on Communications_ , 2019.
  * Saputra et al. [2019] Yuris Mulya Saputra, Dinh Thai Hoang, Diep N Nguyen, Eryk Dutkiewicz, Markus Dominik Mueck, and Srikathyayani Srikanteswara.  Energy demand prediction with federated learning for electric vehicle networks.  _arXiv preprint arXiv:1909.00907_ , 2019.
  * Shaoqi et al. [2020] Chen Shaoqi, Xue Dongyu, Chuai Guohui, Yang Qiang, and Liu Qi.  Fl-qsar: a federated learning based qsar prototype for collaborative drug discovery.  _bioRxiv_ , 2020.
  * Sheller et al. [2018a] Micah J. Sheller, G. Anthony Reina, Brandon Edwards, Jason Martin, and Spyridon Bakas.  Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation.  In Alessandro Crimi, Spyridon Bakas, Hugo J. Kuijf, Farahani Keyvan, Mauricio Reyes, and Theo van Walsum, editors, _Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I_ , volume 11383 of _Lecture Notes in Computer Science_ , pages 92–104. Springer, 2018a.
  * Sheller et al. [2018b] Micah J Sheller, G Anthony Reina, Brandon Edwards, Jason Martin, and Spyridon Bakas.  Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation.  In _International MICCAI Brainlesion Workshop_ , pages 92–104. Springer, 2018b.
  * Sozinov et al. [2018] Konstantin Sozinov, Vladimir Vlassov, and Sarunas Girdzijauskas.  Human activity recognition using federated learning.  In Jinjun Chen and Laurence T. Yang, editors, _IEEE International Conference on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018, Melbourne, Australia, December 11-13, 2018_, pages 1103–1111. IEEE, 2018.
  * Stoica et al. [2017] Ion Stoica, Dawn Song, Raluca Ada Popa, David Patterson, Michael W Mahoney, Randy Katz, Anthony D Joseph, Michael Jordan, Joseph M Hellerstein, Joseph E Gonzalez, et al.  A berkeley view of systems challenges for ai.  _arXiv preprint arXiv:1712.05855_ , 2017.
  * Suzumura et al. [2019] Toyotaro Suzumura, Yi Zhou, Natahalie Barcardo, Guangnan Ye, Keith Houck, Ryo Kawahara, Ali Anwar, Lucia Larise Stavarache, Daniel Klyashtorny, Heiko Ludwig, et al.  Towards federated graph learning for collaborative financial crimes detection.  _arXiv preprint arXiv:1909.12946_ , 2019.
  * Tao et al. [2018] Fei Tao, Qinglin Qi, Ang Liu, and Andrew Kusiak.  Data-driven smart manufacturing.  _Journal of Manufacturing Systems_ , 48:157–169, 2018.
  * Tao et al. [2020] Qi Tao, Wu Fangzhao, Wu Chuhan, Huang Yongfeng, and Xie Xing.  Fedrec: Privacy-preserving news recommendation with federated learning.  _arXiv preprint arXiv:2003.09592_ , 2020.
  * Xinle et al. [2019] Liang Xinle, Liu Yang, Chen Tianjian, Liu Ming, and Yang Qiang.  Federated transfer reinforcement learning for autonomous driving.  _arXiv preprint arXiv:1910.06001_ , 2019.
  * Yang et al. [2019a] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong.  Federated machine learning: Concept and applications.  _ACM Transactions on Intelligent Systems and Technology (TIST)_ , 10(2):1–19, 2019a.
  * Yang et al. [2019b] Wensi Yang, Yuhang Zhang, Kejiang Ye, Li Li, and Cheng-Zhong Xu.  Ffd: A federated learning based method for credit card fraud detection.  In _International Conference on Big Data_ , pages 18–32. Springer, 2019b.
  * Yin [2017] Robert K Yin.  _Case study research and applications: Design and methods_.  Sage publications, 2017.
  * Zhang et al. [2016] Darui Zhang, Bin Xu, and Jasmine Wood.  Predict failures in production lines: A two-stage approach with clustering and supervised learning.  In _2016 IEEE international conference on big data (big data)_ , pages 2070–2074. IEEE, 2016.
  * Zhong et al. [2017] Ray Y Zhong, Xun Xu, Eberhard Klotz, and Stephen T Newman.  Intelligent manufacturing in the context of industry 4.0: a review.  _Engineering_ , 3(5):616–630, 2017.


1 | 1 | 11 | -86 | -87 | 133.524 | 0.030 | 133.523 | 0.017 | 133.494 | 0.613 | 71.9
1 | -1 | 11 | -85 | -86 | 133.923 | 0.001 | 133.924 | 0.010 | 133.934 | 0.811 | 101.8
1 | 1 | 11 | -85 | -86 | 134.598 | 0.003 | 134.593 | 0.011 | 134.608 | 0.923 | 66.9
1 | -1 | 11 | -84 | -85 | 135.179 | 0.004 | 135.176 | 0.014 | 135.191 | 0.955 | 9.3
1 | 1 | 11 | … | -85 | … | … | 135.924 | 0.010 | 135.928 | 0.972 | 11.9
1 | 1 | … | -83 | … | 137.527 | 0.003 | … | … | … | … | …
1 | -1 | … | -82 | … | 137.917 | 0.409 | … | … | … | … | …
1 | 1 | 12 | -82 | -83 | 138.722 | 0.003 | 138.718 | 0.014 | 138.733 | 0.983 | 8.5
1 | -1 | 12 | -81 | -82 | 139.410 | 0.004 | 139.411 | 0.015 | 139.432 | 0.982 | 12.3
1 | -1 | 12 | -80 | -81 | 140.891 | 0.002 | 140.890 | 0.012 | 140.905 | 0.970 | 18.0
1 | 0 | 12 | … | -81 | … | … | 141.292 | 0.016 | 141.274 | 0.963 | 6.1
1 | -1 | 12 | -79 | -80 | 142.366 | 0.007 | 142.362 | 0.014 | 142.356 | 0.906 | 5.7
1 | 1 | 12 | -79 | -80 | 143.050 | 0.024 | 142.984 | 0.018 | 143.008 | 0.770 | 8.7
1 | -1 | 12 | -78 | -79 | 143.555 | 0.099 | 143.527 | 0.014 | 143.515 | 0.512 | 18.1
1 | 0 | 12 | … | -79 | … | … | 143.527 | 0.087 | 143.698 | 0.447 | 18.1
1 | 1 | 12 | -78 | -79 | 143.940 | 0.003 | 143.882 | 0.013 | 143.874 | 0.469 | 21.5
1 | -1 | 12 | -77 | -78 | 144.373 | 0.001 | 144.362 | 0.011 | 144.351 | 0.719 | 26.7
1 | 0 | 12 | … | -78 | … | … | 144.661 | 0.014 | 144.643 | 0.820 | 14.1
1 | 1 | 12 | -77 | -78 | 144.972 | 0.002 | 144.968 | 0.012 | 144.963 | 0.882 | 8.9
1 | -1 | 12 | … | -76 | … | … | 147.329 | 0.013 | 147.339 | 0.976 | 7.8
1 | 1 | 12 | … | -76 | … | … | 148.101 | 0.013 | 148.086 | 0.980 | 11.5
1 | 0 | 13 | … | -75 | … | … | 149.337 | 0.019 | 149.362 | 0.981 | 5.0
1 | 1 | 13 | … | -75 | … | … | 149.754 | 0.014 | 149.736 | 0.980 | 8.8
1 | 0 | 13 | … | -74 | … | … | 151.029 | 0.015 | 151.044 | 0.972 | 6.1
1 | 1 | 13 | … | -74 | … | … | 151.407 | 0.013 | 151.415 | 0.968 | 7.5
1 | -1 | 13 | … | -72 | … | … | 153.902 | 0.017 | 153.879 | 0.647 | 13.8
1 | 0 | 13 | -71 | -72 | 153.925 | 0.010 | 154.185 | 0.046 | 154.102 | 0.518 | 5.8
1 | 1 | 13 | … | -72 | … | … | 154.398 | 0.060 | 154.283 | 0.431 | 7.9
1 | -1 | 13 | … | -71 | … | … | 154.823 | 0.040 | 154.750 | 0.539 | 7.8
1 | 0 | 13 | … | -71 | … | … | 155.020 | 0.026 | 154.979 | 0.666 | 5.6
1 | 1 | 13 | … | -71 | … | … | 155.272 | 0.018 | 155.254 | 0.779 | 5.1
2 | 0 | 6 | 7 | … | 85.387 | 0.120 | … | … | … | … | …
2 | 0 | 7 | 8 | 0 | 95.395 | 0.036 | 95.449 | 0.135 | 94.778 | 0.000 | 8.4
2 | 0 | 8 | 9 | 0 | 105.537 | 0.026 | 105.579 | 0.095 | 105.106 | 0.000 | 14.0
2 | 0 | 9 | 10 | 0 | 116.058 | 0.004 | 116.072 | 0.112 | 115.512 | 0.000 | 95.1
2 | 0 | 10 | 11 | 0 | 126.474 | 0.018 | 126.462 | 0.095 | 125.988 | 0.000 | 68.9
2 | 0 | 11 | 12 | 0 | 136.935 | 0.028 | 136.986 | 0.089 | 136.544 | 0.000 | 34.0
2 | 0 | 12 | 13 | 0 | 147.637 | 0.035 | 147.676 | 0.102 | 147.170 | 0.000 | 22.7
2 | 0 | 13 | 14 | … | 158.462 | 0.099 | … | … | … | … | …
3 | 0 | 9 | 9 | … | 119.568 | 0.098 | … | … | … | … | …
3 | 0 | 10 | 10 | … | 130.013 | 0.026 | … | … | … | … | …
Table 6: Continued.

##  Appendix C Eclipse properties

Table LABEL:tab:eclipse_ppties presents the eclipses properties obtained with
the Lightcurvefactory.

1]

Table 7:  Eclipse properties.  n | $t_{1}$ | $O_{1}-nP-t_{0}$ | $D_{1}$ | $W_{1}$ | $t_{2}$ | $O_{2}-nP-t_{0}$ | $D_{2}$ | $W_{2}$
---|---|---|---|---|---|---|---|---
| (KJD) | day | % | hour | (KJD) | day | % | hour
8 | 127.4560 | 4.4034 | -0.55 | 2.10 | 134.9905 | 11.9379 | -0.50 | 2.24
9 | 142.8022 | 4.3911 | -0.52 | 2.15 | 150.3368 | 11.9121 | -0.51 | 2.20
10 | 158.1280 | 4.3778 | -0.53 | 2.32 | … | … | … | …
11 | 174.1688 | 4.3612 | -0.62 | 2.48 | 181.7034 | 11.8727 | -0.66 | 2.18
12 | 188.7792 | 4.3464 | -0.65 | 2.38 | 196.3137 | 11.8560 | -0.72 | 2.39
13 | 204.1046 | 4.3251 | -0.77 | 2.31 | 211.6391 | 11.8481 | -0.88 | 2.53
14 | 219.4298 | 4.3058 | -0.84 | 2.40 | 226.9644 | 11.8495 | -0.96 | 2.57
15 | 235.7970 | 4.3025 | -0.87 | 2.40 | 243.3315 | 11.8809 | -0.91 | 2.48
16 | 250.0798 | 4.4026 | -0.89 | 2.43 | 257.6144 | 11.9343 | -1.01 | 2.57
17 | 265.4046 | 4.4707 | -1.15 | 2.67 | 272.9391 | 11.9343 | -1.39 | 2.78
18 | … | … | … | … | 288.2638 | 11.9523 | -1.56 | 2.69
19 | 296.0539 | 4.4538 | -1.38 | 2.67 | 303.5884 | 11.9615 | -1.62 | 2.71
20 | 311.3988 | 4.4533 | -1.38 | 2.79 | 318.9334 | 11.9532 | -1.58 | 2.80
21 | 327.9493 | 4.4437 | -1.40 | 2.79 | 335.4839 | 11.9355 | -1.53 | 2.81
22 | 342.0480 | 4.4350 | -1.35 | 2.71 | … | … | … | …
23 | 357.3727 | 4.4217 | -1.42 | 2.63 | 364.9072 | 11.8882 | -1.68 | 2.79
24 | 372.6975 | 4.4051 | -1.39 | 2.55 | 380.2320 | 11.8657 | -1.71 | 2.70
25 | 388.7172 | 4.3921 | -1.52 | 2.81 | 396.2517 | 11.8462 | -1.81 | 2.79
26 | 405.2275 | 4.3752 | -1.60 | 2.75 | 412.7621 | 11.8327 | -1.93 | 2.87
27 | 418.6730 | 4.3593 | -1.64 | 2.70 | 426.2075 | 11.8260 | -2.04 | 2.79
28 | 433.9985 | 4.3395 | -1.75 | 2.65 | 441.5330 | 11.8378 | -1.98 | 2.78
29 | 449.3241 | 4.3562 | -1.68 | 2.73 | 456.8586 | 11.8931 | -1.94 | 2.95
30 | 464.6703 | 4.4809 | -1.73 | 2.74 | 472.2048 | 11.9169 | -2.15 | 2.94
31 | 480.6909 | 4.5004 | -1.97 | 2.85 | 488.2254 | 11.9218 | -2.52 | 2.98
32 | 495.3219 | 4.4902 | -2.16 | 2.88 | 502.8565 | 11.9427 | -2.58 | 2.93
33 | 510.6478 | 4.4881 | -2.15 | 2.79 | 518.1823 | 11.9468 | -2.53 | 2.92
34 | 525.9735 | 4.4836 | -2.15 | 2.80 | 533.5081 | 11.9353 | -2.61 | 2.85
35 | 541.2992 | 4.4736 | -2.10 | 2.81 | 548.8337 | 11.9158 | -2.50 | 2.93
36 | 556.6248 | 4.4625 | -2.13 | 2.79 | 564.1593 | 11.8930 | -2.54 | 2.85
37 | 571.9502 | 4.4480 | -2.11 | 2.81 | 579.4847 | 11.8694 | -2.63 | 2.90
38 | 587.2755 | 4.4327 | -2.21 | 2.79 | 594.8100 | 11.8469 | -2.72 | 2.88
39 | 602.6006 | 4.4184 | -2.26 | 2.82 | 610.1351 | 11.8267 | -2.79 | 2.91
40 | 617.9255 | 4.4025 | -2.34 | 2.82 | 625.4601 | 11.8149 | -2.88 | 2.98
41 | 633.2504 | 4.3860 | -2.35 | 2.82 | 640.7849 | 11.8126 | -2.94 | 2.85
42 | 648.5955 | 4.3689 | -2.40 | 2.78 | 656.1300 | 11.8403 | -2.81 | 2.87
43 | 663.9201 | 4.4192 | -2.28 | 2.77 | 671.4546 | 11.9076 | -2.86 | 2.85
44 | 679.2446 | 4.5282 | -2.46 | 2.82 | 686.7792 | 11.9015 | -3.18 | 2.86
45 | 694.5692 | 4.5233 | -2.60 | 2.87 | 702.1037 | 11.9192 | -3.28 | 2.84
46 | 709.8938 | 4.5202 | -2.61 | 2.83 | 717.4283 | 11.9373 | -3.21 | 2.83
47 | … | … | … | … | … | … | … | …
48 | 740.5432 | 4.5077 | -2.61 | 2.83 | 748.0777 | 11.9246 | -3.15 | 2.85
49 | 755.8682 | 4.4959 | -2.57 | 2.83 | … | … | … | …
50 | 771.1933 | 4.4826 | -2.66 | 2.77 | 778.7278 | 11.8789 | -3.26 | 2.86
51 | 786.5186 | 4.4682 | -2.60 | 2.81 | 794.0531 | 11.8559 | -3.27 | 2.83
52 | 801.8440 | 4.4541 | -2.64 | 2.82 | 809.3786 | 11.8329 | -3.30 | 2.86
53 | 817.1901 | 4.4394 | -2.68 | 2.80 | 824.7246 | 11.8117 | -3.26 | 2.82
54 | 832.5158 | 4.4247 | -2.68 | 2.88 | 840.0503 | 11.8028 | -3.35 | 2.82
55 | 847.8416 | 4.4062 | -2.67 | 2.85 | 855.3761 | 11.8107 | -3.34 | 2.85
56 | 863.1674 | 4.3991 | -2.65 | 2.81 | 870.7020 | 11.8615 | -3.31 | 2.84
57 | 878.4933 | 4.4867 | -2.63 | 2.79 | 886.0278 | 11.9122 | -3.37 | 2.82
58 | 893.8191 | 4.5508 | -2.71 | 2.83 | 901.3536 | 11.9018 | -3.52 | 2.82
59 | 909.1448 | 4.5441 | -2.71 | 2.83 | 916.6793 | 11.9259 | -3.40 | 2.80
60 | 924.4703 | 4.5415 | -2.71 | 2.83 | 932.0049 | 11.9388 | -3.46 | 2.78
61 | 939.7958 | 4.5342 | -2.72 | 2.85 | 947.3303 | 11.9356 | -3.36 | 2.83
62 | 955.1211 | 4.5223 | -2.72 | 2.78 | 962.6557 | 11.9193 | -3.40 | 2.78
63 | 970.4463 | 4.5100 | -2.63 | 2.66 | 977.9808 | 11.8984 | -3.42 | 2.79
64 | 985.7713 | 4.4957 | -2.73 | 2.77 | 993.3058 | 11.8751 | -3.40 | 2.80
65 | … | … | … | … | 1008.6307 | 11.8528 | -3.60 | 2.87
66 | 1016.4413 | 4.4668 | -2.79 | 2.83 | 1023.9758 | 11.8293 | -3.54 | 2.78
67 | 1031.7659 | 4.4523 | -2.74 | 2.79 | 1039.3004 | 11.8137 | -3.44 | 2.78
68 | 1047.0905 | 4.4373 | -2.75 | 2.85 | 1054.6250 | 11.8063 | -3.44 | 2.79
69 | 1062.4150 | 4.4218 | -2.71 | 2.86 | 1069.9495 | 11.8256 | -3.51 | 2.78
70 | 1077.7396 | 4.4325 | -2.60 | 2.78 | 1085.2741 | 11.9016 | -3.38 | 2.81
71 | 1093.0642 | 4.5402 | -2.74 | 2.80 | 1100.5987 | 11.9150 | -3.39 | 2.83
72 | 1108.3889 | 4.5620 | -2.70 | 2.88 | 1115.9235 | 11.9160 | -3.42 | 2.85
73 | 1123.7138 | 4.5594 | -2.70 | 2.78 | 1131.2484 | 11.9406 | -3.36 | 2.83
74 | 1139.0389 | 4.5551 | -2.74 | 2.85 | 1146.5734 | 11.9484 | -3.48 | 2.78
75 | … | … | … | … | … | … | … | …
76 | 1169.7100 | 4.5329 | -2.74 | 2.82 | 1177.2446 | 11.9232 | -3.41 | 2.79
77 | 1185.0356 | 4.5174 | -2.71 | 2.83 | 1192.5701 | 11.9026 | -3.45 | 2.77
78 | 1200.3613 | 4.5009 | -2.71 | 2.81 | 1207.8958 | 11.8782 | -3.48 | 2.82
79 | 1215.6870 | 4.4898 | -2.61 | 2.64 | 1223.2216 | 11.8551 | -3.42 | 2.83
80 | 1231.0129 | 4.4730 | -2.66 | 2.83 | 1238.5474 | 11.8341 | -3.45 | 2.78
81 | 1246.3387 | 4.4609 | -2.68 | 2.85 | 1253.8732 | 11.8182 | -3.38 | 2.76
82 | 1261.6645 | 4.4451 | -2.77 | 2.91 | … | … | … | …
83 | 1276.9903 | 4.4303 | -2.79 | 2.83 | 1284.5248 | 11.8558 | -3.42 | 2.72
84 | … | … | … | … | 1299.8504 | 11.9417 | -3.43 | 2.72
85 | 1307.6414 | 4.5621 | -2.64 | 2.90 | 1315.1759 | 11.9248 | -3.34 | 2.83
86 | 1322.9667 | 4.5669 | -2.66 | 2.96 | 1330.5013 | 11.9406 | -3.40 | 2.75
87 | 1338.2920 | 4.5667 | -2.74 | 2.87 | 1345.8265 | 11.9605 | -3.44 | 2.75
88 | 1353.6170 | 4.5586 | -2.74 | 2.83 | 1361.1515 | 11.9625 | -3.41 | 2.75
89 | 1368.9623 | 4.5468 | -2.74 | 2.85 | 1376.4969 | 11.9525 | -3.62 | 2.77
90 | 1384.2871 | 4.5313 | -2.83 | 2.85 | 1391.8216 | 11.9353 | -3.55 | 2.70
91 | 1399.6117 | 4.5150 | -2.82 | 2.79 | 1407.1463 | 11.9131 | -3.55 | 2.75
92 | … | … | … | … | 1422.4708 | 11.8905 | -3.43 | 2.74
93 | 1430.2608 | 4.4844 | -2.72 | 2.84 | 1437.7954 | 11.8663 | -3.43 | 2.76
94 | 1445.5854 | 4.4706 | -2.73 | 2.86 | 1453.1199 | 11.8456 | -3.43 | 2.78
95 | 1460.9100 | 4.4564 | -2.71 | 2.91 | 1468.4445 | 11.8321 | -3.41 | 2.73
96 | 1476.5208 | 4.4392 | -2.71 | 2.89 | 1484.0553 | 11.9078 | -3.43 | 2.70
97 | … | … | … | … | … | … | … | …
98 | 1506.9050 | 4.5093 | -2.77 | 2.85 | 1514.4395 | 11.9682 | -3.47 | 2.78
99 | 1522.2302 | 4.5590 | -2.73 | 2.92 | 1529.7647 | 11.9494 | -3.47 | 2.75
100 | 1537.5556 | 4.5621 | -2.64 | 2.94 | 1545.0901 | 11.9717 | -3.49 | 2.77
101 | 1552.8811 | 4.5624 | -2.69 | 2.90 | 1560.4156 | 11.9858 | -3.36 | 2.72
102 | 1568.2067 | 4.5518 | -2.66 | 2.89 | 1575.7413 | 11.9851 | -3.37 | 2.74
Table 7: Continued.