# Integrating Unsupervised and Supervised learning approaches to unveil
Critical Process Inputs

Paris Papavasileiou
Faculty of Science, Technology and Medicine
University of Luxembourg
Esch-sur-Alzette, L-4364, Luxembourg
<EMAIL_ADDRESS>
& Dimitrios G. Giovanis
Department of Civil & Systems Engineering, Whiting School of Engineering
Johns Hopkins University
Baltimore, MD 21218, USA
<EMAIL_ADDRESS>
&Martin Kathrein
CERATIZIT Luxembourg S.à r.l.
Mamer, L-8201, Luxembourg
<EMAIL_ADDRESS>
&Gabriele Pozzetti
CERATIZIT Luxembourg S.à r.l.
Mamer, L-8201, Luxembourg
<EMAIL_ADDRESS>
&Christoph Czettl
CERATIZIT Austria GmbH
Reutte, A-6600, Austria
<EMAIL_ADDRESS>
& Ioannis G. Kevrekidis
Department of Chemical and Biomolecular Engineering
& Department of Applied Mathematics and Statistics, Whiting School of
Engineering
Johns Hopkins University
Baltimore, MD 21218, USA
<EMAIL_ADDRESS>
& Andreas G. Boudouvis
School of Chemical Engineering
National Technical University of Athens
Zographos Campus, 15780, Attiki, Greece
<EMAIL_ADDRESS>
& Stéphane P.A. Bordas
Faculty of Science, Technology and Medicine
University of Luxembourg
Esch-sur-Alzette, L-4364
<EMAIL_ADDRESS>
& Eleni D. Koronaki
Faculty of Science, Technology and Medicine
University of Luxembourg
Esch-sur-Alzette, L-4364, Luxembourg
<EMAIL_ADDRESS>Author also affiliated with the School of Chemical
Engineering, National Technical University of Athens, Zographos Campus, 15780,
Attiki, Greece

###### Abstract

This study introduces a machine learning framework tailored to large-scale
industrial processes characterized by a plethora of numerical and categorical
inputs. The framework aims to (i) discern critical parameters influencing the
output and (ii) generate accurate out-of-sample qualitative and quantitative
predictions of production outcomes. Specifically, we address the pivotal
question of the significance of each input in shaping the process outcome,
using an industrial Chemical Vapor Deposition (CVD) process as an example. The
initial objective involves merging subject matter expertise and clustering
techniques exclusively on the process output, here, coating thickness
measurements at various positions in the reactor. This approach identifies
groups of production runs that share similar qualitative characteristics, such
as film mean thickness and standard deviation. In particular, the differences
of the outcomes represented by the different clusters can be attributed to
differences in specific inputs, indicating that these inputs are potentially
critical for the production outcome. Shapley Value analysis corroborates the
formed hypotheses. Leveraging this insight, we subsequently implement
supervised classification and regression methods using the identified critical
process inputs. The proposed methodology proves to be valuable in scenarios
with a multitude of inputs and insufficient data for the direct application of
deep learning techniques, providing meaningful insights into the underlying
processes.

_Keywords_ critical parameters $\cdot$ machine learning $\cdot$ industrial
process $\cdot$ data-driven approaches $\cdot$ chemical vapor deposition
$\cdot$ Shapley values

##  1 Introduction

Chemical vapor deposition (CVD) is a widely used chemical process for
producing thin films with various properties, applied in semiconductor
manufacturing [1, 2], membranes [3, 4], protective [5, 6] and wear-resistant
[7, 8] coatings. Although Computational Fluid Dynamics (CFD) models
traditionally explore CVD complexity [9, 10, 11, 12, 13, 14, 15, 16, 17] their
efficiency and adequacy are challenged in cases involving unknown chemical
reactions or intricate reactor geometries. The computational cost of large-
scale industrial process models and the nonlinear nature of competing physical
and chemical mechanisms further limit the utility of CFD as a viable “digital
twin”. It is also possible that there are different process outputs arise for
the same inputs, which is also linked to non-linearity [18, 19].

Recently, Machine Learning (ML) has emerged as a promising alternative in the
era of Industry 4.0 with abundant process data. ML applications range from
maintenance management [20, 21, 22] and production planning [23, 24] to
outcome prediction [25, 26], process control [27], and optimization [28]. ML
models can also be developed based on preexisting physics-based models, in
order to further investigate the modeled process [29, 30, 31].

Despite recent advances in explainable AI (XAI), challenges persist in
addressing the “black box” nature of ML models. However, tools such as SHAP
(SHapley Additive exPlanations) offer improved explainability using a game
theory approach [32, 33, 34, 35].

This study utilizes production data from an industrial CVD reactor for wear-
resistant coating production of cutting tools. The data encompass details
about reactor setup and process inputs; thickness measurements of the
Ti(C,N)/$\alpha$-Al2O3 coating in 15 positions within the reactor are
considered as process outputs.

Implementing state-of-the-art (SotA) methods faces challenges that include:

  * •

Process complexity, namely, multiscale interacting phenomena in intricate
geometries.

  * •

A multitude of numerical and categorical inputs, with little insight of their
impact on the process outcome.

  * •

Noisy and heterogeneous data, collected over months or years with varying
instrumentation and calibration, which cannot be categorized as “big”.

Several different options are available in the literature related to the
discovery of important process parameters and the facilitation of subsequent
modeling attempts. Variable Importance in Projection (VIP) parameters [36,
37], a byproduct of Partial Least Squares (PLS) models, have traditionally
been used to determine the impact of process inputs on the output [38, 37,
39].

Variable selection tools have been shown to enable improved performance and
subsequently lead to a greater understanding of the importance of input
variables on model output [40]. To this end, several powerful dimensionality
reduction techniques based on Principal Component Analysis (PCA) or Diffusion
Maps (DMaps) [41, 42] can lead to the discovery of effective process
parameters [43, 44].

Despite the effectiveness of existing methods for strictly numerical data,
challenges arise when dealing with datasets rich in categorical features, as
seen in this application. This work aims to propose an ML workflow for the
identification of critical process inputs without labeled data, an essential
contribution to control, optimization, and experimental design.

Our approach involves an unsupervised analysis of process outputs to identify
clusters of similar production runs. Subsequently, we analyze relevant process
input data to discern distinguishing characteristics within these clusters.
Our findings are supported by subject matter expertise. Shifting to supervised
learning, we use cluster labels to train a classifier for predicting these
labels given specific process inputs. Furthermore, we attempt to create a
regression model for predicting thickness measurements. Finally, we employ
SHAP and Shapley values to interpret the model output.

The manuscript is structured as follows. A brief overview of the process and
the available production data is given in Section 2. The various machine
learning methods implemented (supervised, unsupervised) are presented in
Section 3. The results are discussed in Section 4, followed by concluding
remarks in Section 5.

##  2 Process overview

The studied process involves two coating steps carried out inside a
commercial, industrial-scale Sucotec SCT600TH CVD reactor. To start with, a
Ti(C,N) base layer of approximately 9 µm is deposited on cemented carbide
cutting tool inserts, shown in Fig. 1(a). The second step involves the
deposition of an alumina layer under specific conditions: $T$=1005°C and
$p$=80 mbar, from a mixture of gas reactants that includes
AlCl3–CO2–HCl–H2–H2S. This step takes around 3 hours to complete [45].

The CVD reactor consists of 40-50 perforated disks, stacked one on top of the
other. The inserts to be coated are placed on each disk. For illustrative
purposes, a schematic of three such disks is shown in Fig. 1(b). Specially
designed perforations on a rotating cylindrical tube, which is placed in the
center of the reactor, ensure the uniform distribution of the gas reactants
over and around the inserts: the perforations are placed antipodally and there
is a 60° angle difference between the axis connecting the inlets at each disk
level. The feeding tube rotates at a fixed rotational speed of 2 RPM.

(a)

(b)

Figure 1: (a) Examples of the coated cutting tool inserts. (b) A 3D
representation of a 3-disk part of the reactor. In red: inlet perforations on
the rotating inlet tube. In blue: outlet perforations for each disk.

It is worth noting that each insert has a dedicated disk design which
ultimately suggests that the interior geometry of the CVD reactor changes
every time that it is set up.

The desired process outcome is uniform coating thickness distribution for the
same insert and also uniform mean thickness across all production runs, all
reactors and all production sites, as this ensures consistent product life
(quality) [46]. In practice, the desired uniformity is not always achieved,
and therefore a systematic way of identifying the influential aspects of
coating uniformity becomes necessary.

###  2.1 Available data

For each production run, thickness measurements are taken at three positions
on five disks of interest, schematically shown in a representative geometry in
Fig. 2. The thickness of the Ti(C,N) and $\alpha$-Al2O3 coating layers is
measured using the Calotest method [47]. These measurements have been utilized
in previous work, both for the development of a CFD model of the process [48],
and for the implementation of ML approaches for the prediction of coating
thickness [26].

Coating thickness is a vital measure of product quality for CVD applications.
The long-term experience of the practitioners led to the selection of these 15
measurements for testing the quality of each production run. It should be
noted that in case additional quality-related data (i.e. roughness of the
coating) become available, they can be easily incorporated in the framework
presented in this paper, in conjunction with thickness.

Figure 2: Most common measurement positions among the production data. These
measurements can be used for several tasks, such as the development of CFD or
ML approaches for the prediction of the process outcome.

Additionally, the available dataset contains information about a) the process
input parameters and b) the reactor geometry and setup. Some examples of these
features include, but are not limited to:

  * •

The components of the reactor setup that determine the overall interior
geometry, i.e. the sequence according to which the disk/inserts are stacked to
form the overall reactor.

  * •

The surface area of the inserts on each disk.

  * •

The production “recipe”, a feature that encodes several process parameters and
steps. We should note that there can be several versions of one recipe. There
are a total of four base recipes present in the dataset with five versions for
each (marked $V21$, $V20$, and older variants). This makes up a total of 20
recipes.

  * •

The serial number of the reactor used for the production run.

An important contribution of this work emerged in the context of data
exploration and pre-processing. It became necessary to engineer additional
features, based on our intuition (subject matter expertise) regarding the
existing inputs. These engineered features include the total surface area per
reactor, the standard deviation of the surface area within the reactor, and
the difference between the nominal and actual surface area within the reactor.
The nominal surface area is the surface area considered by the production
recipe and does not always coincide with the actual surface area. For more
information on the available data, its type and its characteristics, the
interested reader is referred to previous work by Papavasileiou et al. [26]. A
comparison of our approach with systematic methods for feature combinations,
such as polynomial combination or even symbolic regression, is underway and
out of the scope of this work.

##  3 Machine learning methods

###  3.1 Unsupervised learning

Unsupervised learning algorithms take unlabeled data as inputs to discover
interesting patterns in the data (e.g., association rule analysis) or try to
create subgroups - or clusters - of similar observations within the dataset
[49]. Dimensionality reduction techniques such as the widely used Principal
Component Analysis (PCA), autoencoders [50], and diffusion maps [41, 42] also
fall under unsupervised learning as they provide a reduced data representation
without requiring the corresponding response. The clustering and
dimensionality reduction techniques implemented are briefly discussed in the
following sections.

####  3.1.1 Clustering

Clustering algorithms are based on the concept of dissimilarity (or
similarity) between observations, which determines their grouping. Typically,
these algorithms utilize a similarity matrix, where pairwise similarities
between observations are represented. For quantitative variables, the commonly
employed metric is the Euclidean distance, while alternative distance metrics
can also be used [49, 51].

Clustering algorithms are categorized into various categories. Partitional
approaches, such as the k-means algorithm, involve assigning observations to
clusters based on distances to centroids iteratively, requiring an a priori
choice of the number of clusters and sensitive to initial centroid positions
[52]. Density-based algorithms, such as OPTICS and DBSCAN, identify clusters
by considering areas of high density separated by low-density regions. Certain
algorithm parameters, such as the minimum points that form a cluster and the
minimum distance between the core points require specification [53, 54, 55].
Hierarchical clustering methods link data points according to criteria,
progressively creating clusters until a single cluster is achieved in the case
of agglomerative clustering, or progressively splitting clusters starting
until each observation is its own cluster in the case of divisive clustering.
The results depend on the distance metric and the linkage criteria selected
[56, 57]. Additional methods include model-based and spectral methods [58,
59].

Here, we focus on agglomerative hierarchical clustering, implementing a Ward
linkage criterion for merging the clusters. This is an established variance
minimization approach [60] that works by minimizing the sum of squared
differences within all clusters. Agglomerative hierarchical clustering is
selected because it provides insight on how the data merges depending on the
number of clusters chosen. This information is readily available in the form
of a dendrogram, such as the one presented in Fig. 3(a).

For this specific problem, the 15 available thickness measurements of 603
production runs are used as inputs (cf. Section 2.1). The clustering results
are then interpreted based on the characteristics of the resulting clusters.
Our goal is to identify production runs that are similar to each other and to
try to uncover the discerning features of these clusters.

###  3.2 Supervised learning

Supervised learning algorithms, unlike unsupervised ones, require labeled
data, associating features ($x_{\\_}i$) with responses ($y_{\\_}i$).
Supervised learning tasks include regression for continuous variables and
classification for binary or ordinal responses [61].

The methods evaluated for this work include: (a) linear methods: for
regression, lasso [62], and ridge [63] regression and logistic regression for
classification tasks. (b) Support vector machines (SVMs) [64] that can be
categorized as linear or nonlinear methods based on the kernel used for
classification tasks. (c) Tree-based methods: involving classification and
regression trees [65] and their ensemble counterparts such as random forests
[66], gradient-boosted trees [67], extra trees [68], and XGBoost [69], which
combine numerous trees to enhance performance [70]. (d) Artificial neural
networks (ANN), whose diverse architectures [71] can provide valuable options
for both classification and regression tasks.

In this work, logistic regression, random forests, SVM, extra trees, gradient-
boosted trees, XGBoost, and ANNs are implemented for supervised learning
tasks. However, only the methods that demonstrate the best performance for our
dataset are presented in Section 4.

###  3.3 Shapley values

Shapley values, originally introduced by Shapley [32] and proposed as a tool
to analyze machine learning models in [33, 72] intricately assess the average
contribution of each feature’s value to predictions, providing an
understanding of how alterations to a variable might influence the ultimate
model output.

In the context of this work, a SHAP (Shapley value based) analysis is
conducted on the proposed regression models (cf. Sections 3.2 and 4.5) and the
resulting Shapley values will shed light on the importance of each feature to
the model output.

##  4 Results

###  4.1 Clustering

As mentioned in Section 3.1.1, the agglomerative hierarchical clustering
algorithm with a Ward linkage criterion is implemented for clustering the 603
production runs.

The clustering algorithm utilizes the 15 thickness measurements for each of
the 603 production runs, forming a $603\times 15$ matrix. Clusters are then
created solely based on the process outputs. Subsequently, the distinctive
features are identified by analyzing the process inputs for each production
run.

The hierarchical clustering algorithm generates a dendrogram that illustrates
cluster levels, member counts, and dissimilarities. The clusters are depicted
as branches of a tree, culminating in the "trunk," representing the final
cluster (by agglomerating smaller ones). In our case, the resulting dendrogram
is shown in Fig. 3(a). By selecting a dissimilarity threshold, we can discern
one, two, three, or more clusters. In Fig. 3(a), the three clusters are
colored purple, red, and green. A higher dissimilarity threshold merges the
red and green clusters into a single blue cluster (as shown in Fig. 3(a)). The
resulting clusters are visualized in a reduced three-dimensional space
(through projection on three principal components) in Fig. 3(b).

(a)

(b)

Figure 3: (a) Resulting dendrogram of the clusters output by the implemented
agglomerative hierarchical clustering algorithm using a Ward linkage
criterion. The three main clusters of interest are colored purple, red, and
green. We note that by selecting a slightly higher dissimilarity threshold,
the red and green clusters can be merged and viewed as a larger cluster (shown
in blue). (b) The three resulting clusters, visualized in a reduced 3D space.
The three clusters appear to be well-formed. PCA was used for finding the 3D
reduced space.

As mentioned above, the thickness and its uniformity throughout production
runs is a very effective process performance indicator and product quality
metric. Thus, production runs with a higher average thickness and a lower
standard deviation can be considered superior to those with a lower average
thickness and higher standard deviation. We observe that the thickness within
the clusters follows a normal distribution, and therefore we can calculate the
first and second statistical moments (that is, the mean
($\mu_{\\_}\mathrm{thick}$), and standard deviation
($\sigma_{\\_}\mathrm{thick}$) and visualize the thickness distributions as
shown in Fig. 4.

(a)

(b)

Figure 4: Thickness distribution in the case of: a) 2 clusters and b) 3
clusters. High average thickness and low standard deviation is a measure of
process efficiency and product quality. The production runs in the “purple”
cluster demonstrate superior quality characteristics.

###  4.2 Critical input identification

In this section, the focus shifts to the process inputs whose variation is
critical for each cluster. We propose three different ways for assessing the
relative importance of process inputs.

  1. 1.

Intuition-based approach: By finding characteristics that are predominantly
different in each cluster, we can assess their importance on the process
outcome (cf. Section 4.2.1).

  2. 2.

Supervised learning approach: Classification algorithms are trained using the
cluster labels of the clustering step as outputs and various inputs: some
process inputs lead to higher accuracy, which is an indication of their
importance. Conversely, less important inputs have an adverse effect on the
accuracy of the classifier (cf. Section 4.3).

  3. 3.

Shapley value approach: The importance of input features for classification or
regression can be assessed using Shapley values (cf. Section 4.5).

####  4.2.1 Combining clustering and subject matter expertise

When two clusters are considered (Fig. 4(a)), cluster 0 demonstrates superior
characteristics, with the highest average thickness and the lowest standard
deviation (Table 1). Further examination reveals that cluster 0 is
characterized by production runs predominantly using recipe version $V21$,
while cluster 1 comprises runs using version $V20$ and older versions,
indicating recipe version as the main distinguishing feature.

Table 1: Characteristics of each cluster in the case of two clusters. The
recipe version used for production is the discerning feature of the two
clusters.

Cluster $\boldsymbol{\mu_{\\_}\mathrm{thick}}$ (µm)
$\boldsymbol{\sigma_{\\_}\mathrm{thick}}$ (µm) Predominant recipe versions 0
16.35 1.355 $V21$ 1 15.08 1.578 $V20$ & older

When three clusters (Fig. 4(b)), are identified by the clustering algorithm,
cluster 0 exhibits superior characteristics, with the highest average
thickness and the lowest standard deviation (Table 2). In particular, cluster
0 comprises production runs using recipe version $V21$, and it is practically
the same as cluster 0 in the two-cluster case mentioned in the previous
paragraph. Clusters 1 and 2 predominantly use $V20$ and older versions and are
the result of the splitting of cluster 1 identified in the two-cluster case.
This cluster splitting, in essence, means that even among production runs
using recipe version $V20$ and older, there are certain cases where favorable
quality characteristics are achieved. This raises the question: which is the
critical input that led to this difference in quality?

Further assessment drew our attention to an engineered feature, the absolute
value of the difference between the nominal and actual total surface area to
be coated. The nominal surface area is the predetermined production setting,
specified for increments of $1m^{2}$. In practice, this rarely matches the
actual total surface area value and this discrepancy is evident when comparing
the distributions between clusters 1 and 2, as shown in Fig. 5; On average,
for the members of cluster 2, the difference between the nominal and actual
total surface area is greater than $0.5m^{2}$, while in cluster 1 it is less
than $0.5m^{2}$. This analysis suggests that when the value of this difference
is less than $0.5m^{2}$, the qualitative characteristics of the products are
superior, thus leading to a clear and cost-free suggestion for improvement:
define preset production parameters for increments of $0.5m^{2}$ (instead of
$1m^{2}$) of the total surface area.

Table 2: Characteristics of three clusters: Discerning features include the
recipe version used for production and the absolute difference between nominal
and actual surface area.

Cluster $\boldsymbol{\mu_{\\_}\mathrm{thick}}$ (µm)
$\boldsymbol{\sigma_{\\_}\mathrm{thick}}$ (µm) Predominant recipe versions
|Nominal recipe surface area \- actual surface area| (cm2) 0 16.35 1.354 $V21$
4892 1 15.53 1.386 $V20$ & older 4628 2 14.32 1.588 $V20$ & older 5526

(a)

(b)

Figure 5: Distributions of |Nominal recipe surface area - actual surface area|
for clusters 1 (in green) and 2 (in red). Cluster 2 includes relatively more
observations with values larger than 5000 cm2 when compared with cluster 1.

###  4.3 Classification

We train a classifier to predict cluster labels that resulted from the
clustering analysis, using as inputs the dominant features identified in the
previous section (clustering). This is useful in practice to predict the
overall quality characteristics of the production run, as these cluster labels
correspond to distinct thickness distributions.

The results for a binary (two-cluster case) and a multi-label classification
(three-cluster case) task are presented. For these tasks, we divide the 603
observations into a training set and a test set using an 80/20 ratio.

Initially, classification models take as input the two important features
identified through clustering. However, these are not the only discernible
differences between clusters; other features, such as the year of production,
the reactor used, and the standard deviation of the surface area within the
reactor, also have marked differences among clusters. Therefore, these inputs
are also considered when training the classifier.

The initial step involves training a random forest classification model
(n_estimators=1000, max_depth=6) to predict whether a production run belongs
to cluster 0 or 1 in Fig. 4(a), treating it as a binary classification
problem. The classifier, as shown in the confusion matrices in Figs. 6(a) and
6(b), accurately distinguishes between clusters 0 and 1 production runs both
for the training ($\mathrm{accuracy}=0.954$) and test set
($\mathrm{accuracy}=0.958$). The calculated accuracy, $f1$, precision, and
recall metrics are presented in detail in Table 3.

Subsequently, a random forest classification model (n_estimators=1000,
max_depth=6) is developed to determine if a production run belongs to cluster
0, 1, or 2 in Fig. 4(b), making it a multi-label classification problem. As
demonstrated in the confusion matrices in Figs. 6(c) and 6(d), the classifier
identifies cluster 0 members very accurately, for both training and test
datasets. However, it sometimes struggles to distinguish between members of
cluster 1 and cluster 2, often misclassifying them as members of the other
cluster. The accuracy of the classifier on the test set is 0.793. As in the
two-cluster case, all metrics are presented in Table 3. Since this is not a
binary classification problem, the $f1$, precision and recall metrics are
macro-averaged [73].

Table 3: Classification metrics for the two-cluster and three-cluster cases. The metrics for the three-cluster case have been macro-averaged. | Accuracy | $f1$ | Precision | Recall
---|---|---|---|---
2-cluster case
Training Set | 0.968 | 0.958 | 0.954 | 0.962
Test Set | 0.967 | 0.958 | 0.958 | 0.958
3-cluster case (macro-averaged metrics)
Training Set | 0.840 | 0.848 | 0.844 | 0.852
Test Set | 0.793 | 0.792 | 0.795 | 0.790

(a)

(b)

(c)

(d)

Figure 6: Confusion matrices for (a),(c) the training set and (b),(d) the test
set of the two-cluster and three-cluster classification cases, respectively.

###  4.4 Regression

In the present work, regression is used as a tool that allows for the
prediction of the average coating thickness for each production run, using
fewer measurements than the 15 currently used. Specifically, we use the
features identified through clustering and five thickness measurements (the
closest to the reactor’s inlet (R0)) as inputs. This leads to accurate
prediction of the mean coating thickness (average of R1/2 and R) for both
training ($\mathrm{R^{2}=0.914}$) and the test set ($\mathrm{R^{2}=0.722}$)
(cf. Fig. 7). This method proves valuable for streamlined post-production
quality control as it allows for precise quality assessment with only one
third of the previously required measurements.

Figure 7: Training set performance metrics: MSE: 0.067, MAE: 0.198, R2: 0.914,
MAPE: 1.26%. Test set performance metrics: MSE: 0.264, MAE: 0.409, R2: 0.722,
MAPE: 2.62%.

###  4.5 Shapley value analysis

The most influential features that affect the predicted average coating
thickness are identified by computing the SHAP values for the developed
regression model. The mean absolute SHAP values are shown in Fig. 8(b). The
five thickness measurements provided along with the year of production emerge
as the most crucial features. Of the five thickness measurements provided, the
lowest contribution comes from the measurement on the first disk from the top
of the reactor. They are followed by the four remaining features, i.e.,
recipe, difference between the nominal and actual substrate surface area
within the reactor (surf_area_diff), standard deviation of the surface area
(surface_area_std) and the reactor used for production. These four features
demonstrate a similar contribution to the model’s predictions.

(a)

(b)

Figure 8: (a) 2D representation of the reactor, indicating the positions of
the thickness measurements used as inputs for the regression problem. (b)
Calculated mean absolute SHAP values for each of the inputs to the regression
model. The five provided thickness measurements along with the year of
production appear to be the dominant features, followed by surfareadiff, the
reactor and the recipe used for production and the standard deviation of
surface area within the reactor.

##  5 Conclusions

This study introduces a data-driven approach for uncovering patterns and
influencing process inputs in an industrial Chemical Vapor Deposition (CVD)
process, addressing challenges associated with process complexity and dataset
characteristics.

Our analysis relies on subject matter expertise, combined with supervised and
unsupervised learning methods. The main premise is that the performance of
data-driven algorithms, given a specific dataset, is influenced by, and is
indicative of, the importance of the inputs used during training. This is
supported here by intuition about critical process inputs and some knowledge
about the important quality characteristics.

We use unsupervised learning to obtain meaningful data labels that correspond
to groups of production runs of similar quality. We then use these labels, in
the context of supervised learning, to predict the outcome for a new set of
inputs, thus providing a cost-efficient shortcut for quality control.

The importance of features in investigated using Shapley values, which
corroborates both subject matter expertise and also the conclusions drawn from
the accuracy of classification methods. The results of this study offer
opportunities to streamline post-production quality control and contribute to
the ongoing refinement of the manufacturing process.

It is worth noting that this framework is adaptable to other processes,
contingent on data availability. Even in cases with limited data, this
approach unveils potential process-determining inputs, corroborating the
insights of process experts in a purely data-driven manner.

Furthermore, consistent and improved data collection in the coming years will
not only aid in validating and enhancing the developed predictive models, but
also contribute to the continuous optimization of the overall process.

## Acknowledgments

This research was funded in part by the Luxembourg National Research Fund
(FNR), grant reference [16758846]. For the purpose of open access, the authors
have applied a Creative Commons Attribution 4.0 International (CC BY 4.0)
license to any Author Accepted Manuscript version arising from this
submission. PP gratefully acknowledges funding from the FSTM in the University
of Luxembourg. The work of IGK has been partly supported by the US Department
of Energy.

## References

  * Cote et al. [1999] D. R. Cote, S. V. Nguyen, A. K. Stamper, D. S. Armbrust, D. Tobben, R. A. Conti, G. Y. Lee,  Plasma-assisted chemical vapor deposition of dielectric thin films for ULSI semiconductor circuits,  IBM Journal of Research and Development 43 (1999) 5–38. doi:10.1147/rd.431.0005.
  * Biefeld [2002] R. M. Biefeld,  The metal-organic chemical vapor deposition and properties of III–V antimony-based semiconductor materials,  Materials Science and Engineering: R: Reports 36 (2002) 105–142. doi:10.1016/S0927-796X(02)00002-5.
  * Ha et al. [1996] H. Y. Ha, S. W. Nam, T. H. Lim, I.-H. Oh, S.-A. Hong,  Properties of the TiO2 membranes prepared by CVD of titanium tetraisopropoxide,  Journal of Membrane Science 111 (1996) 81–92. doi:10.1016/0376-7388(95)00278-2.
  * Khatib and Oyama [2013] S. J. Khatib, S. T. Oyama,  Silica membranes for hydrogen separation prepared by chemical vapor deposition (CVD),  Separation and Purification Technology 111 (2013) 20–42. doi:10.1016/j.seppur.2013.03.032.
  * Schmauder et al. [2006] T. Schmauder, K. D. Nauenburg, K. Kruse, G. Ickes,  Hard coatings by plasma CVD on polycarbonate for automotive and optical applications,  Thin Solid Films 502 (2006) 270–274. doi:10.1016/j.tsf.2005.07.296.
  * Jia et al. [2021] S. Jia, W. Chen, J. Zhang, C. Y. Lin, H. Guo, G. Lu, K. Li, T. Zhai, Q. Ai, J. Lou,  CVD growth of high-quality and large-area continuous h-BN thin films directly on stainless-steel as protective coatings,  Materials Today Nano 16 (2021) 100135. doi:10.1016/j.mtnano.2021.100135.
  * Karner et al. [1996] J. Karner, M. Pedrazzini, I. Reineck, M. E. Sjöstrand, E. Bergmann,  CVD diamond coated cemented carbide cutting tools,  Materials Science and Engineering: A 209 (1996) 405–413. doi:10.1016/0921-5093(95)10140-3.
  * Kathrein et al. [2003] M. Kathrein, W. Schintlmeister, W. Wallgram, U. Schleinkofer,  Doped CVD Al2O3 coatings for high performance cutting tools,  Surface and Coatings Technology 163–164 (2003) 181–188. doi:10.1016/s0257-8972(02)00483-8.
  * Mitrovic et al. [2007] B. Mitrovic, A. Gurary, W. Quinn,  Process conditions optimization for the maximum deposition rate and uniformity in vertical rotating disc MOCVD reactors based on CFD modeling,  Journal of Crystal Growth 303 (2007) 323–329. doi:10.1016/j.jcrysgro.2006.11.247.
  * Cheimarios et al. [2012] N. Cheimarios, E. D. Koronaki, A. G. Boudouvis,  Illuminating nonlinear dependence of film deposition rate in a CVD reactor on operating conditions,  Chemical Engineering Journal 181–182 (2012) 516–523. doi:10.1016/j.cej.2011.11.008.
  * Koronaki et al. [2014] E. D. Koronaki, N. Cheimarios, H. Laux, A. G. Boudouvis,  Non-Axisymmetric Flow Fields in Axisymmetric CVD Reactor Setups Revisited: Influence on the Film’s Non-Uniformity,  ECS Solid State Lett. 3 (2014) P37. doi:10.1149/2.002404ssl.
  * Gakis et al. [2015] G. Gakis, E. Koronaki, A. Boudouvis,  Numerical investigation of multiple stationary and time-periodic flow regimes in vertical rotating disc CVD reactors,  Journal of Crystal Growth 432 (2015) 152–159. doi:10.1016/j.jcrysgro.2015.09.026.
  * Koronaki et al. [2016] E. D. Koronaki, G. P. Gakis, N. Cheimarios, A. G. Boudouvis,  Efficient tracing and stability analysis of multiple stationary and periodic states with exploitation of commercial CFD software,  Chemical Engineering Science 150 (2016) 26–34. doi:10.1016/j.ces.2016.04.043.
  * Aviziotis et al. [2016] I. G. Aviziotis, N. Cheimarios, T. Duguet, C. Vahlas, A. G. Boudouvis,  Multiscale modeling and experimental analysis of chemical vapor deposited aluminum films: Linking reactor operating conditions with roughness evolution,  Chemical Engineering Science 155 (2016) 449–458. doi:10.1016/j.ces.2016.08.039.
  * Aviziotis et al. [2017] I. G. Aviziotis, T. Duguet, C. Vahlas, A. G. Boudouvis,  Combined Macro/Nanoscale Investigation of the Chemical Vapor Deposition of Fe from Fe(CO)5,  Advanced Materials Interfaces 4 (2017) 1601185. doi:10.1002/admi.201601185.
  * Psarellis et al. [2018] G. M. Psarellis, I. G. Aviziotis, T. Duguet, C. Vahlas, E. D. Koronaki, A. G. Boudouvis,  Investigation of reaction mechanisms in the chemical vapor deposition of al from DMEAA,  Chemical Engineering Science 177 (2018) 464–470. doi:10.1016/j.ces.2017.12.006.
  * Topka et al. [2022] K. C. Topka, H. Vergnes, T. Tsiros, P. Papavasileiou, L. Decosterd, B. Diallo, F. Senocq, D. Samelor, N. Pellerin, M.-J. Menu, C. Vahlas, B. Caussat,  An innovative kinetic model allowing insight in the moderate temperature chemical vapor deposition of silicon oxynitride films from tris(dimethylsilyl)amine,  Chemical Engineering Journal 431 (2022) 133350. doi:10.1016/j.cej.2021.133350.
  * Gkinis et al. [2017] P. Gkinis, I. Aviziotis, E. Koronaki, G. Gakis, A. Boudouvis,  The effects of flow multiplicity on GaN deposition in a rotating disk CVD reactor,  Journal of Crystal Growth 458 (2017) 140–148. doi:10.1016/j.jcrysgro.2016.10.065.
  * Koronaki et al. [2019] E. Koronaki, P. Gkinis, L. Beex, S. Bordas, C. Theodoropoulos, A. Boudouvis,  Classification of states and model order reduction of large scale Chemical Vapor Deposition processes with solution multiplicity,  Computers & Chemical Engineering 121 (2019) 148–157. doi:10.1016/j.compchemeng.2018.08.023.
  * Saxena and Saad [2007] A. Saxena, A. Saad,  Evolving an artificial neural network classifier for condition monitoring of rotating mechanical systems,  Applied Soft Computing 7 (2007) 441–454. doi:10.1016/j.asoc.2005.10.001.
  * Susto et al. [2015] G. A. Susto, A. Schirru, S. Pampuri, S. McLoone, A. Beghi,  Machine Learning for Predictive Maintenance: A Multiple Classifier Approach,  IEEE Trans. Ind. Inf. 11 (2015) 812–820. doi:10.1109/tii.2014.2349359.
  * Wu et al. [2019] H. Wu, Z. Yu, Y. Wang,  Experimental study of the process failure diagnosis in additive manufacturing based on acoustic emission,  Measurement 136 (2019) 445–453. doi:10.1016/j.measurement.2018.12.067.
  * Priore et al. [2018] P. Priore, B. Ponte, J. Puente, A. Gómez,  Learning-based scheduling of flexible manufacturing systems using ensemble methods,  Computers & Industrial Engineering 126 (2018) 282–291. doi:10.1016/j.cie.2018.09.034.
  * Agarwal et al. [2020] P. Agarwal, M. Tamer, M. H. Sahraei, H. Budman,  Deep Learning for Classification of Profit-Based Operating Regions in Industrial Processes,  Ind. Eng. Chem. Res. 59 (2020) 2378–2395. doi:10.1021/acs.iecr.9b04737.
  * Papananias et al. [2019] M. Papananias, T. E. McLeay, M. Mahfouf, V. Kadirkamanathan,  A Bayesian framework to estimate part quality and associated uncertainties in multistage manufacturing,  Computers in Industry 105 (2019) 35–47. doi:10.1016/j.compind.2018.10.008.
  * Papavasileiou et al. [2023] P. Papavasileiou, E. D. Koronaki, G. Pozzetti, M. Kathrein, C. Czettl, A. G. Boudouvis, S. P. Bordas,  Equation-based and data-driven modeling strategies for industrial coating processes,  Computers in Industry 149 (2023) 103938. doi:10.1016/j.compind.2023.103938.
  * Ma et al. [2019] Y. Ma, W. Zhu, M. G. Benton, J. Romagnoli,  Continuous control of a polymerization system with deep reinforcement learning,  Journal of Process Control 75 (2019) 40–47. doi:10.1016/j.jprocont.2018.11.004.
  * Humfeld et al. [2021] K. D. Humfeld, D. Gu, G. A. Butler, K. Nelson, N. Zobeiry,  A machine learning framework for real-time inverse modeling and multi-objective process optimization of composites for active manufacturing control,  Composites Part B: Engineering 223 (2021) 109150. doi:10.1016/j.compositesb.2021.109150.
  * Gkinis et al. [2019] P. Gkinis, E. Koronaki, A. Skouteris, I. Aviziotis, A. Boudouvis,  Building a data-driven reduced order model of a chemical vapor deposition process from low-fidelity CFD simulations,  Chemical Engineering Science 199 (2019) 371–380. doi:10.1016/j.ces.2019.01.009.
  * Spencer et al. [2021] R. Spencer, P. Gkinis, E. Koronaki, D. Gerogiorgis, S. Bordas, A. Boudouvis,  Investigation of the chemical vapor deposition of Cu from copper amidinate through data driven efficient CFD modelling,  Computers & Chemical Engineering 149 (2021) 107289. doi:10.1016/j.compchemeng.2021.107289.
  * Martin-Linares et al. [2023] C. P. Martin-Linares, Y. M. Psarellis, G. Karapetsas, E. D. Koronaki, I. G. Kevrekidis,  Physics-agnostic and physics-infused machine learning for thin films flows: Modelling, and predictions from small data,  Journal of Fluid Mechanics 975 (2023) A41. doi:10.1017/jfm.2023.868.
  * Shapley [1952] L. S. Shapley, A Value for N-Person Games, Technical Report, RAND Corporation, 1952.
  * Lundberg and Lee [2017] S. M. Lundberg, S.-I. Lee,  A Unified Approach to Interpreting Model Predictions,  in: Advances in Neural Information Processing Systems, volume 30, Curran Associates, Inc., 2017.
  * Barredo Arrieta et al. [2020] A. Barredo Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins, R. Chatila, F. Herrera,  Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,  Information Fusion 58 (2020) 82–115. doi:10.1016/j.inffus.2019.12.012.
  * Sundararajan and Najmi [2020] M. Sundararajan, A. Najmi,  The Many Shapley Values for Model Explanation,  in: Proceedings of the 37th International Conference on Machine Learning, PMLR, 2020, pp. 9269–9278.
  * Chong and Jun [2005] I.-G. Chong, C.-H. Jun,  Performance of some variable selection methods when multicollinearity is present,  Chemometrics and Intelligent Laboratory Systems 78 (2005) 103–112. doi:10.1016/j.chemolab.2004.12.011.
  * Lu et al. [2014] B. Lu, I. Castillo, L. Chiang, T. F. Edgar,  Industrial PLS model variable selection using moving window variable importance in projection,  Chemometrics and Intelligent Laboratory Systems 135 (2014) 90–109. doi:10.1016/j.chemolab.2014.03.020.
  * Garthwaite [1994] P. H. Garthwaite,  An Interpretation of Partial Least Squares,  Journal of the American Statistical Association 89 (1994) 122–127. doi:10.1080/01621459.1994.10476452.
  * Kumar [2021] K. Kumar,  Partial Least Square (PLS) Analysis: Most Favorite Tool in Chemometrics to Build a Calibration Model,  Reson 26 (2021) 429–442. doi:10.1007/s12045-021-1140-1.
  * Heinze et al. [2018] G. Heinze, C. Wallisch, D. Dunkler,  Variable selection – A review and recommendations for the practicing statistician,  Biometrical Journal 60 (2018) 431–449. doi:10.1002/bimj.201700067.
  * Koronaki et al. [2020] E. D. Koronaki, A. M. Nikas, A. G. Boudouvis,  A data-driven reduced-order model of nonlinear processes based on diffusion maps and artificial neural networks,  Chemical Engineering Journal 397 (2020) 125475. doi:10.1016/j.cej.2020.125475.
  * Koronaki et al. [2023] E. D. Koronaki, N. Evangelou, Y. M. Psarellis, A. G. Boudouvis, I. G. Kevrekidis,  From partial data to out-of-sample parameter and observation estimation with diffusion maps and geometric harmonics,  Computers & Chemical Engineering (2023) 108357. doi:10.1016/j.compchemeng.2023.108357.
  * Brouwer and Eisenberg [2018] A. F. Brouwer, M. C. Eisenberg, The underlying connections between identifiability, active subspaces, and parameter space dimension reduction, 2018. doi:10.48550/arXiv.1802.05641. arXiv:1802.05641.
  * Evangelou et al. [2022] N. Evangelou, N. J. Wichrowski, G. A. Kevrekidis, F. Dietrich, M. Kooshkbaghi, S. McFann, I. G. Kevrekidis,  On the parameter combinations that matter and on those that do not: Data-driven studies of parameter (non)identifiability,  PNAS Nexus 1 (2022) pgac154. doi:10.1093/pnasnexus/pgac154.
  * Hochauer et al. [2012] D. Hochauer, C. Mitterer, M. Penoy, S. Puchner, C. Michotte, H. Martinz, H. Hutter, M. Kathrein,  Carbon doped $\alpha$-Al2O3 coatings grown by chemical vapor deposition,  Surface and Coatings Technology 206 (2012) 4771–4777. doi:10.1016/j.surfcoat.2012.03.059.
  * Bar-Hen and Etsion [2017] M. Bar-Hen, I. Etsion,  Experimental study of the effect of coating thickness and substrate roughness on tool wear during turning,  Tribology International 110 (2017) 341–347. doi:10.1016/j.triboint.2016.11.011.
  * Łępicka and Grądzka-Dahlke [2019] M. Łępicka, M. Grądzka-Dahlke,  The initial evaluation of performance of hard anti-wear coatings deposited on metallic substrates: Thickness, mechanical properties and adhesion measurements – a brief review,  REVIEWS ON ADVANCED MATERIALS SCIENCE 58 (2019) 50–65. doi:10.1515/rams-2019-0003.
  * Papavasileiou et al. [2022] P. Papavasileiou, E. D. Koronaki, G. Pozzetti, M. Kathrein, C. Czettl, A. G. Boudouvis, T. J. Mountziaris, S. P. A. Bordas,  An efficient chemistry-enhanced CFD model for the investigation of the rate-limiting mechanisms in industrial Chemical Vapor Deposition reactors,  Chemical Engineering Research and Design 186 (2022) 314–325. doi:10.1016/j.cherd.2022.08.005.
  * Hastie et al. [2009] T. Hastie, R. Tibshirani, J. Friedman,  Unsupervised Learning,  in: T. Hastie, R. Tibshirani, J. Friedman (Eds.), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Series in Statistics, Springer, New York, NY, 2009, pp. 485–585. doi:10.1007/978-0-387-84858-7_14.
  * Wang et al. [2016] Y. Wang, H. Yao, S. Zhao,  Auto-encoder based dimensionality reduction,  Neurocomputing 184 (2016) 232–242. doi:10.1016/j.neucom.2015.08.104.
  * James et al. [2021] G. James, D. Witten, T. Hastie, R. Tibshirani,  Unsupervised Learning,  in: G. James, D. Witten, T. Hastie, R. Tibshirani (Eds.), An Introduction to Statistical Learning: With Applications in R, Springer Texts in Statistics, Springer US, New York, NY, 2021, pp. 497–552. doi:10.1007/978-1-0716-1418-1_12.
  * MacQueen [1967] J. MacQueen,  Some methods for classification and analysis of multivariate observations,  in: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, Oakland, CA, USA, 1967, pp. 281–297.
  * Ankerst et al. [1999] M. Ankerst, M. M. Breunig, H.-P. Kriegel, J. Sander,  OPTICS: Ordering points to identify the clustering structure,  SIGMOD Rec. 28 (1999) 49–60. doi:10.1145/304181.304187.
  * Ester et al. [1996] M. Ester, H.-P. Kriegel, J. Sander, X. Xu,  A density-based algorithm for discovering clusters in large spatial databases with noise,  in: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, AAAI Press, Portland, Oregon, 1996, pp. 226–231.
  * Schubert et al. [2017] E. Schubert, J. Sander, M. Ester, H. P. Kriegel, X. Xu,  DBSCAN Revisited, Revisited: Why and How You Should (Still) Use DBSCAN,  ACM Trans. Database Syst. 42 (2017) 19:1–19:21. doi:10.1145/3068335.
  * Murtagh and Contreras [2012] F. Murtagh, P. Contreras,  Algorithms for hierarchical clustering: An overview,  WIREs Data Mining and Knowledge Discovery 2 (2012) 86–97. doi:10.1002/widm.53.
  * Vijaya et al. [2019] Vijaya, S. Sharma, N. Batra,  Comparative Study of Single Linkage, Complete Linkage, and Ward Method of Agglomerative Clustering,  in: 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), IEEE, Faridabad, India, 2019, pp. 568–573. doi:10.1109/COMITCon.2019.8862232.
  * Fraley and Raftery [2002] C. Fraley, A. E. Raftery,  Model-Based Clustering, Discriminant Analysis, and Density Estimation,  Journal of the American Statistical Association 97 (2002) 611–631. doi:10.1198/016214502760047131.
  * Jia et al. [2014] H. Jia, S. Ding, X. Xu, R. Nie,  The latest research progress on spectral clustering,  Neural Comput & Applic 24 (2014) 1477–1486. doi:10.1007/s00521-013-1439-2.
  * Ward [1963] J. H. Ward,  Hierarchical Grouping to Optimize an Objective Function,  Journal of the American Statistical Association 58 (1963) 236–244. doi:10.1080/01621459.1963.10500845.
  * James et al. [2021] G. James, D. Witten, T. Hastie, R. Tibshirani, Statistical Learning, Springer US, New York, NY, 2021, pp. 15–57. doi:10.1007/978-1-0716-1418-1_2.
  * Tibshirani [1996] R. Tibshirani,  Regression shrinkage and selection via the lasso,  Journal of the Royal Statistical Society. Series B (Methodological) 58 (1996) 267–288. doi:10.1111/j.2517-6161.1996.tb02080.x. arXiv:2346178.
  * Hoerl and Kennard [1970] A. E. Hoerl, R. W. Kennard,  Ridge Regression: Biased Estimation for Nonorthogonal Problems,  Technometrics 12 (1970) 55–67. doi:10.1080/00401706.1970.10488634.
  * Cortes and Vapnik [1995] C. Cortes, V. Vapnik,  Support-vector networks,  Mach Learn 20 (1995) 273–297. doi:10.1007/BF00994018.
  * Breiman et al. [1984] L. Breiman, J. Friedman, R. A. Olshen, C. J. Stone, Classification and Regression Trees, Chapman and Hall/CRC, New York, 1984. doi:10.1201/9781315139470.
  * Breiman [2001] L. Breiman,  Random Forests,  Machine Learning 45 (2001) 5–32. doi:10.1023/A:1010933404324.
  * Friedman [2001] J. H. Friedman,  Greedy Function Approximation: A Gradient Boosting Machine,  The Annals of Statistics 29 (2001) 1189–1232. arXiv:2699986.
  * Geurts et al. [2006] P. Geurts, D. Ernst, L. Wehenkel,  Extremely randomized trees,  Mach Learn 63 (2006) 3–42. doi:10.1007/s10994-006-6226-1.
  * Chen and Guestrin [2016] T. Chen, C. Guestrin,  XGBoost: A Scalable Tree Boosting System,  in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, San Francisco California USA, 2016, pp. 785–794. doi:10.1145/2939672.2939785.
  * Hastie et al. [2009] T. Hastie, R. Tibshirani, J. Friedman,  Ensemble Learning,  in: T. Hastie, R. Tibshirani, J. Friedman (Eds.), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York, NY, 2009, pp. 605–624. doi:10.1007/978-0-387-84858-7_16.
  * Aggarwal [2018] C. C. Aggarwal, Neural Networks and Deep Learning: A Textbook, Springer International Publishing, Cham, 2018. doi:10.1007/978-3-319-94463-0.
  * Lundberg et al. [2020] S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, S.-I. Lee,  From local explanations to global understanding with explainable AI for trees,  Nat Mach Intell 2 (2020) 56–67. doi:10.1038/s42256-019-0138-9.
  * Zhang and Zhou [2014] M.-L. Zhang, Z.-H. Zhou,  A Review on Multi-Label Learning Algorithms,  IEEE Trans. Knowl. Data Eng. 26 (2014) 1819–1837. doi:10.1109/TKDE.2013.39.