# Meta-analysis of literature data in metal additive manufacturing: What can
we (and the machine) learn from reported data?

###### Abstract

Obtaining in-depth understanding of the relationships between the additive
manufacturing process, microstructure and mechanical properties is crucial to
overcome barriers in additive manufacturing (AM). Over the past decades, there
have been significant studies in AM, providing considerable amount of data
available for examination of such relationships broadly across many literature
studies. In this study, database of metal AM was created thanks to a large
number of literature studies. Subsequently meta-analyses on the data was
undertaken to provide insights into whether such relationships are well
reflected in the literature data. The analyses help reveal the bias and what
the data tells us, and to what extent machine learning can learn from the
data. The first major bias is associated with common practices in identifying
the process based on optimizing the consolidation. Most data reports were for
consolidation while data on microstructure and mechanical properties was
significantly less. In addition, only high consolidation was only provided.
Machine learning trained on the data was not able to learn the process -
consolidation relationship in the medium and low end ranges of consolidation.
The common identification of process maps based on only consolidation also
poses another bias because mechanical properties that ultimately govern the
quality of an AM build are controlled not only by the consolidation, but also
microstructure. However, the number of studies on quantifying the
microstructure was extremely low, limiting the learning of the microstructure
- mechanical properties relationships. Meta-analysis of the literature data
also shows weak correlation between input (i.e. process parameters) with
output (i.e. consolidation and mechanical properties). This weak correlation
is attributed to the stated biases and the highly non-monotonic and non-linear
relationships between the process and quality variables. Fortunately, machine
learning models trained on the data capture well the interactions between
process parameters are influential in the output, and predicts accurately the
yield stress, suggesting that the correlation between process, microstructure
and yield strength is well reflected in the data. Last but not least, due to
the current limitation in the process map identification, we propose to
identify the process map on the basis of not only the consolidation, but also
mechanical properties. Such a identification shows that 316L and Inconel have
much larger process map (i.e. highly printable) in comparison to the Ti6Al4V,
Hastelloy-X and Inconel 625.

###### keywords:

Data analytics, Machine learning, Processing maps, Additive manufacturing,
Alloys

Raymond Wonga,$\ast$, Anh Tranb, Bogdan Dovgyyc, Claudia Santos Maldonadoa,
Minh-Son Phama,$\ast$

a Department of Materials, Imperial College London, London, SW7 2AZ, UK

b Scientific Machine Learning, Sandia National Laboratories, Albuquerque, NM
87123, USA

c International Additive Manufacturing Group, Domaniewska 3, 05-800 Pruszkow,
Poland

$\ast$ Corresponding authors.
Email Addresses<EMAIL_ADDRESS>and<EMAIL_ADDRESS>
##  1 Introduction

Additive manufacturing (AM) has a potential to revolutionize the manufacturing
industry by offering an efficient and cost-effective method for fabricating
complex structures, providing advantages over other manufacturing techniques
[1]. Despite its advantages, metal AM presents major challenges due to the
formation of defects and undesirable microstructure, affecting the mechanical
performance and reliability of final products in applications [2, 3]. In
particular, extreme interactions between the energy beam and materials and
associated complex thermal conditions in AM cause difficulties in
understanding of the underlying relationship between alloy composition,
microstructure and properties of additively manufactured alloys [4, 5, 6, 7].
Overcoming such challenges requires fundamental understanding of the process-
microstructure-property relationship that will assist the development of (1)
forward engineering (predicting the mechanical properties of a given alloy for
a specific set of process parameters), and (2) reverse engineering
(identifying the alloy composition and corresponding process parameters for a
given set of properties).

Over the past decade, there has been a considerable number of studies
reporting the relationships between process, microstructure and properties
(PMP). Meta-analysing a considerable large data reported by many research
groups has potential to unravel such important PMP relationships and identify
the good and not good practices in studying metal additive manufacturing. Such
meta-analysis will also allow us to identify the bias and associated
implications in our learning of the PMP relationships. Meta-analysis has
becomes powerful in providing invaluable insights that may not be immediately
apparent thanks to advances in data analytics (DA) and machine learning (ML)
[8, 9, 10].

Despite the significant use of DA (and ML) in many fields, such as healthcare
and online shopping, the use of data analytics for AM is still in its early
stages [11]. Although some successes is shown in these data-driven studies
(for example, mechanical properties based on simulated microstructure or
process parameters [12, 13, 14, 15], melt pool dimensions for a given
composition or process parameters [16, 17, 18, 19, 20, 21, 22], and porosity
based on process parameters used or images of the build [23, 24, 25, 26, 27,
28, 29], such studies were only based on limited sets of conditions. While
comprehensive material databases such as Materials Project, AFLOWLIB and OQMD
are available for functional material properties, there has been little effort
to develop similar database targeted for AM [30, 31, 32]. Notably, NIST and
the now discontinued Citrination have placed efforts to address the lack of a
comprehensive database for AM. Nevertheless, it is still inadequate to be used
for big data analysis at the scale of other fields [33, 34]. Fortunately,
there are a considerable amount of data reported in literature studies for AM
in the past decades [33, 35, 36, 37, 38, 39, 40]. Such literature studies
contain valuable data reflecting some key aspects of the PMP relationships in
AM alloys. However, there had not been collection and structuring of rich data
available in literature. This study firstly creates considerable database by
collecting, subsequently structuring and organising data available in
literature with focus on the laser powder bed fusion (LPBF) that is currently
the most used AM method in fabricating metallic alloys. Secondly, the study
carries out in-depth examination of the data to identify biases in the
reported data that may hinder the learning of the PMP relationship for AM
alloys. This will be achieved by conducting an analysis of the correlation,
principal component and sensitivity analyses. Furthermore, ML algorithms will
be used to test the current performance of the obtained dataset and provide
bases to analyse the sensitivity of process parameters on the predicted
consolidation and mechanical properties of the trained ML models. Last but not
least, the study will identify process window maps that are optimized on the
basis of not only the consolidation, but also key mechanical properties that
are crucial for structural applications. Consequently, the use of DA and ML on
a significantly large number of studies can provide invaluable insights
relating to the PMP relationships in particular processing parameters and
their effects on the printed product, assisting the AM users to optimize
processing parameters. Such knowledge will also assist the AM users in
assessing the printability of existing alloys and accelerating the search for
new printable alloys with desired properties [41, 42, 43, 44, 45, 46].

##  2 Methods

An extensive literature search has been undertaken to create significant
datasets from published data reported in literature. Over 2000 data entries
has been obtained for commonly printed alloys such as Ti6Al4V, Inconel 718,
Inconel 625, Hastelloy X and 316L usig the laser powder bed fusion. The data
collection from literature reports included processing parameters,
consolidation and mechanical properties of as-built conditions and post
processing information if available (see Figure 2). Powder bed fusion (PBF) is
currently the most commonly used AM technique and has most comprehensive
published data in comparison to other variations such as DED. Therefore, the
data collection was done for PBF with focus on LPBF. The knowledge gained from
this analysis will be applicable to electron beam powder bed fusion.

The work focuses on exploratory data analysis (EDA) to establish relations
between different variables in the dataset to understand the characteristics
(including biases and limitations of current data reporting practices) and
underlying process - consolidation - mechanical properties relationships
hidden in the obtained datasets. This includes analysis of the reported data
and uncovering relations between input parameters and output properties
(including consolidation, yield stress (YS), ultimate tensile stress (UTS) and
elongation). Following this, analysis of optimized process windows has been
undertaken to gain insights into the relation between process parameters,
material and properties of the selected alloys. Finally, trained supervised ML
models have been used to investigate the impact of current limits of the
reported data (including common practices) in the learning by machine.

###  2.1 Explanatory Data Analysis

Correlation techniques have been conducted to investigate the underlying
correlation, issues and characteristics within the dataset in order to uncover
deeper relationships between data features. The most commonly used techniques
are Pearson’s correlation coefficient and Spearman’s correlation coefficient.
Pearson’s is used to capture linearity correlation between two variables and
is defined as

$r_{p}=\frac{\sum(x-\overline{x})(y-\overline{y})}{\sqrt{\sum(x-\overline{x})^{2}\sum(y-\overline{y})^{2}}}\vskip
12.0pt plus 4.0pt minus 4.0pt$ (1)

where $x$ and $y$ are the individual values of the two variables,
$\overline{x}$ and $\overline{y}$ denote the mean of the two variables in the
dataset [47, 48]. Spearman’s correlation is used to capture the strength of a
monotonic relation between two variables and is given by

$r_{s}=1-6\sum\frac{d^{2}}{N(N^{2}-1)}\vskip 12.0pt plus 4.0pt minus 4.0pt$
(2)

where $d$ denotes the difference between the two variables ranks, and $N$
denotes the sample size [47, 48]. The two correlations are typically used in
conjunction with each other to capture both types of correlation which the
other would not be able to capture.

The established relationship between the variation of processing parameters
and the resulting defects suggests that the processing parameters are subject
to multicollinearity. An example of the correlation has been demonstrated by
Gordon et al. when optimizing laser power and speed, correlation is shown due
to the underlying dependencies of the objective when optimizing print quality
by balancing the two parameters [49]. Multicollinearity among the processing
parameters (predictors) may have a negative impact on the performance of ML
models, as changes in one process parameter will inherently influence the
values of one or more other process parameters. Hence, individual changes in
process parameters and its effect on the dependent variables will be difficult
to differentiate between the effect of other process parameters [50]. To
address this, variance inflation factor (VIF) has been used to quantify the
degree of multicollinearity. VIF measures the degree of collinearity of each
predictor by forming a regression of one predictor to all the other
predictors, calculated as follows

$VIF_{n}=\frac{1}{1-R_{n}^{2}}\vskip 12.0pt plus 4.0pt minus 4.0pt$ (3)

where $R_{n}^{2}$ is the coefficient of determination of the auxiliary
regression for the $n$th predictor. A VIF of 1 indicates no collinearity;
whereas, VIF exceeding 5 or 10 indicates high levels of multicollinearity as a
standard practice.

###  2.2 Process Window Identification

To better understand the influence of multiple process parameters on the print
quality, a window (i.e. heat map) of process parameters optimized for a
separate output (or a combined output that consolidates all considered
separate output variables) have been generated. This approach analyzes the raw
data without the influence of ML and can provide users with predictions of
output qualities purely based on reported data. Because laser power and laser
speed are the most varied input parameters in literature (Figure 3(b)),
heatmap was constructed on the basis of these two separate input variables.
Two types of heatmaps with have been generated, $Hmap_{1}$ and $Hmap_{2}$.
$Hmap_{1}$ is based on an individual quality (i.e. output), either YS, average
work-hardening, elongation or consolidation ($z$-axis) for a given laser speed
($x$-axis) and laser power ($y$-axis). Such that four $Hmap_{1}$ will be
generated for each output variable. As discussed earlier that consolidation
(or any single quality variable) is not sufficient to reflect the quality.
Therefore, $Hmap_{2}$ is constructed to identify a process window that can
achieve a combined variable that includes all the considered individual four
output variables.

The generation of $Hmap_{1}$ consists of interpolating the data points within
its convex hull to obtain $z$ values at the point of interest $P$ within this
region. As the data points are not uniformly distributed, Barycentric
interpolation was chosen over other methods as it accounts for the distances
between neighboring data points, leading to more accurate and smoother
estimates of the function value. In particular, the method employs a series of
interpolations over small regions of parameterized triangles formed between
data points [51, 52, 53]. The interpolation of $P$ uses Barycentric
coordinates ($\lambda$) of the parameterized triangles, given by

$\lambda_{n}=\frac{A_{n}}{A_{total}}\vskip 12.0pt plus 4.0pt minus 4.0pt$ (4)

such that $\lambda$ represents the proportional size of the interior triangles
formed by connecting point $P$ with each vertex of the parameterized triangle.
Where $A_{n}$ denotes the area of an interior triangle and $A_{total}$
represents the total area of the parameterized triangles. An illustration of
the Barycentric coordinates is shown in Figure 1. Using these coordinates, $P$
is determined as a weighted average dependent on the distance of the
neighboring $z$ values as follows

$P=\sum_{n=1}^{3}\lambda_{n}z_{n}\vskip 12.0pt plus 4.0pt minus 4.0pt$ (5)

To generate $Hmap_{2}$, all interpolated values for $Hmap_{1}$ of the same
material has been normalized with a min-max scaler and summed with equal
weighting to create a map which shows the optimized process parameter regions.
Thus, the map considers process - consolidation - mechanical properties
relationships as opposed to single properties like consolidation. The min-max
scaler ensures all values for the map ranges from 0 to 1, such that the
scaling of output qualities will contribute equally to the summarized map
[54]. However, if any output quality is considered to be more important, a
higher weighting factor can be added to allow more contribution from that
individual output to the combined quality.

Figure 1: Illustration of Barycentric interpolation to obtain value at point
$P$, given three known $Z$ values, corresponding to YS, average work-
hardening, elongation or consolidation.

###  2.3 Machine Learning

Principal component analysis (PCA) was used to reduce the dimensionality of
the input data. The efficacy of the dimensionality reduction has been
investigated to study the use of volumetric energy density (VED) that is
commonly used metric to optimize print quality. Furthermore, multiple
supervised ML techniques have been employed to examine the accuracy and
effects of data biases and limitations on the ML performance. Machine learning
models employed include decision tree, random forest, support vector machine,
neural network, XGBoost, LightGBM and CatBoost. Based on the observations of
non-Gaussian distributed data and sensitivity analyses shown in Figure 3 and
Table 1 respectively, the selection of these models focuses on methods which
doesn’t assume Gaussian distribution and is less susceptible to
multicollinearity [55, 56, 57]. To employing ML models on the dataset, it is
essential to pre-process the data by converting it into a machine-readable
format. Steps taken includes standardizing, dropping a set of data if the
entry contains any missing values and applying One Hot Encoding the material
and treatment data. One Hot Encoding creates a new category in a binary format
for unique inputs within the material and treatment information dataset. This
is necessary as the machine is incapable of taking string inputs, thereby
converting the data to numerical inputs allows the machine to consider
material type and treatment information. Furthermore, the binary format of the
categorical values avoids asserting false linear relationships which label
encoding may induce as the ML models may misinterpret the numerical labelling
as rankings [54]. Following this, the training, validation and test data has
been split into a 80%/10%/10% ratio to evaluate the final performance of the
model.

About 80% of more than 2000 data entries obtained from literature was dropped
due to the incompleteness for example missing mechanical properties data.
Efforts to impute the missing data has been undertaken; however, the
significant amount of incomplete data resulted in poor performance. Therefore,
the model has been trained on over 300 points (of full data) after
preprocessing the data To produce an effective model, all aforementioned
models were hyperparameter-tuned with 500 trials and a RMSE loss function; the
selection of this loss function is to reduce the effect of large outliers
during the prediction and training phase by penalizing larger errors due to
outliers [58].

###  2.4 Sensitivity Analysis

It is common to simply trust a ML model because of high accuracy predictions
and omit the interpretation of what the model has learnt. However, it is
difficult to interpret what the machine has learned from the data. Sensitivity
analysis enables interpretability of ML models by revealing the influence of
model inputs on the model outputs. This analysis will allow quantification of
the influence of process parameters and how this varies across different
models. Subsequently, the resulting values can be used to determine whether a
ML model is able to reflect the underlying science between process and
properties [59, 60]. Sobol sensitivity analysis is a variance-based approach
to quantify how the uncertainty in individual model inputs contributes to the
uncertainty of model outputs; the method offers two ways of sensitivity
analysis. The main effect sensitivity index determines the individual effects
of a single process parameter input, without considering the interaction of
this parameter with other process parameters. This can be obtained by

$S_{i}=\frac{{\mathbb{V}[\mathbb{E}(Z|x_{i})]}}{{\mathbb{V}(Z)}}\vskip 12.0pt
plus 4.0pt minus 4.0pt$ (6)

such that $S_{i}$ measures the variance of the conditional expectation
$\mathbb{V}[\mathbb{E}(Y|x_{i})]$, relative to the total variance
$\mathbb{V}(Z)$ for output $Z$, given an input $x_{i}$ [61, 62]. Total effect
sensitivity index ($S_{T_{i}}$) determines the combined effects of a single
process parameter input, considering their interactions with other process
parameters. This can be calculated

$S_{T_{i}}=\frac{{\mathbb{E}[\mathbb{V}(Z|x_{\sim
i})]}}{{\mathbb{V}(Z)}}\vskip 12.0pt plus 4.0pt minus 4.0pt$ (7)

where $\mathbb{V}(Z|x_{\sim i})$ denotes the variance of conditional
expectation for output $Z$ for all inputs except the $i$-th element [61, 62].
This can also be expressed as the sum of the first ($S_{i}$) and higher order
interactions with other model inputs. For example, for laser power, this is
expressed as

$S_{T_{P}}=S_{\\{P\\}}+S_{\\{P,v\\}}+S_{\\{P,h\\}}+S_{\\{P,t\\}}+S_{\\{P,v,h,t\\}}\vskip
12.0pt plus 4.0pt minus 4.0pt$ (8)

where $P$, $v$, $h$, and $t$ denotes laser power, laser speed, hatch spacing
and layer thickness, respectively. Such that $S_{\\{P,v\\}}$ is the measure of
the effect from the co-variance of both laser power and laser speed. The
calculation of $S_{i}$ and $S_{T_{i}}$ typically requires approximation by
Monte-Carlo sampling. This will be achieved by generating 100,000 model inputs
using Saltelli sampling scheme [62]. Following this, the trained ML models
will use the generated model inputs to predict the interested properties. The
generated set of model inputs will provide a diverse and suitable set of
process parameter combinations ensuring that the analysis encompasses a broad
range of parameter values and interactions. By exploring the parameter space
more comprehensively, this enhances the reliability of the calculated indices
of the sensitivity analysis [62, 63, 59, 60].

##  3 Results and Discussion

###  3.1 Explanatory Data Analysis

The distribution of the compiled dataset is summarized in Figure 2. The
histogram emphasizes a bias in data reporting, most of studies reported data
concerning the consolidation, but much fewer on mechanical property data that
are, in fact, key indicators (much more than the consolidation) of the build
quality for structural applications. The histogram shows that if a machine
learning algorithm was trained on the literature reports, it might understate
the importance of mechanical properties, even though these qualities accounts
for the print quality and is significant in practice. In addition, although
there is increasing a number of studies reporting the microstructure
information, most of microstructure data is qualitative and quantitative data
of microstructure remains very rare. Consequently, due to insufficient data
reported for the microstructure and mechanical properties, ML would not able
to learn the PMP relationship and would fail to provide accurate prediction of
the mechanical data for given process parameters. Even on the consolidation
data, there are limitations in method used to measure the consolidation. Hence
the consolidation data contains considerable biases. Consolidation has been
often measured by quantifying the density of defects such as porosity. While
the optical (or electron microscopy) method can observe small defects, it is
not effective in quantifying the 3D spatial distribution. By contrast, the
X-ray tomography has limitations in observing fine defects. Therefore,
reported data of porosity do not fully reflect the consolidation of an AM
build. Also, different measurement methods can yield different results of
porosity [64], resulting in inconsistent results. Moreover, even if the
consolidation is accurately measured, it is known that the mechanical
properties and performance of any mechanical/structural component are greatly
dependent on other governing factors such as microstructure. Thus, the use of
consolidation alone does not reflect well the quality of an AM build.
Consequently, qualities that govern the mechanical performance such as YS,
UTS, elongation, toughness, fatigue and creep that are important indicators of
the load-bearing capacity and performance in structural applications should be
included in the qualification of final products. Unfortunately, there have
been insufficient data regarding toughness, fatigue and creep for meaningful
analyses Figure 2. Therefore, this study only includes YS, UTS and elongation
in consideration. To improve our consideration of the build quality, we
proposed to introduce an additional output variable that is the average work-
hardening. Work-hardening is an important parameter reflects the energy
absorption capacity of a metal according to the Considère hypothesis [65]. The
variable also indicates the tendency of a metal against the localisation that
is one of the main mechanisms responsible for crack initiation [66]. Average
work-hardening has been calculated by computing the difference between UTS and
YS, divided by elongation. Together with the use of the YS, the introduction
of the average work-hardening makes the UTS redundant and no longer needed.

Figure 2: Histogram of reported data from LPBF studies for the collected
dataset.

Analysis of the collected data showed a significant bias for high
consolidation values, with over 80% of studies only reporting results with
consolidation values above 95%. The highly skewed distribution is displayed in
Figure 3(a). The reporting of process parameters for high consolidation data
only (all is above 70% with most of reports on above the 98%) without
sufficient data for low consolidation creates a major bias. It will be shown
later in the correlation analyses that this bias causes the data not to
reflect the relationship between the process parameters and the consolidation
well. Therefore, this bias could limit the machine in learning of the full
relationship between the process parameters and consolidation. Training the
machine using ML models on this skewed data causes the machine not to be able
to accurately predict the process parameters for low consolidation, negatively
affecting the performance of ML models. It is, therefore, calling the AM
community to publish the process parameters that produce low consolidation
alongside with currently reporting of high consolidation values. This is an
essential step in formulating an extensive dataset as a ML model will produce
biased results if trained with a dataset only containing “good” data [67, 9,
68].

(a)

(b)

Figure 3: (a) Histogram of reported output properties from the collected LPBF
dataset. (b) Histogram of reported processing parameters from the collected
LPBF dataset.

Spearman’s rank and Pearson’s correlation coefficient between the process
parameters and relevant outputs are shown in Figure 4(a) and Figure 4(b),
respectively. Spearman’s rank has been used as it is more appropriate for
heavy-tailed distributions and can be used to uncover monotonic relationships
[47, 48]. Thus, Spearman’s rank is suitable for the obtained dataset, where
the majority of distributions for both input parameters (Figure 3(b)) and
output properties (Figure 3(a)) are both heavy-tailed. Whereas Pearson’s is
suitable to examine the linearity between variables. It is highly unlikely
that the underlying physics of AM is linear, hence the Pearson’s correlation
coefficient reflects the non-linearity between two given variables [47, 48].

(a)

(b)

Figure 4: (a) Heatmap of Spearman’s rank correlation coefficient between
processing parameters and output properties, denoting the strength of
monotonic relationship between subsequent variable pairs. (b) Heatmap of
Pearsons’s rank correlation coefficient between processing parameters and
output properties, denoting the strength of linearity between subsequent
variable pairs.

Volumetric energy density (VED) is a commonly used metric that consolidates
the key process parameters such as power, beam speed, layer thickness, hatch
spacing into a unified parameter and use it to optimize the print quality.
Therefore, analysis of the correlation between VED with output variables such
as consolidation and mechanical properties (reflecting the build quality) was
also included in this study to discuss the effectiveness of the use of the
VED. The colorbar besides the heatmap represents the correlation between the
corresponding variables, where 1 denotes a perfect positive monotonic/linear
relationship and -1 denotes a perfect negative monotonic/linear relationship,
thus a value of 0 signifies no correlation. The closer the correlation is to
$\pm 1$, the more precise the association between two variables can be
explained by their corresponding monotonic/linear relationship [69, 70].
However, the range limits on the bar has been scaled to [-0.4, 0.4] for a
clearer presentation. Both Spearman’s (Figure 4(a)) and Pearson’s correlation
coefficients (Figure 4(b)) indicate that overall laser power has the strongest
correlation to the four quality variables, with VED showing similar
coefficients but slightly weaker scores. Where the highest Spearman’s
correlation coefficient is 0.31 for laser power and elongation, while the
highest Pearson’s correlation coefficient is 0.20 for the correlation between
layer thickness and VED with elongation. Nevertheless, with no coefficients
exceeding $\pm$0.5, the low magnitude of the calculated coefficients suggests
weak correlations between the processing parameters and the output properties.
This is clearly not correct and highlights detrimental implication of biases
within the data and/or the non-monotonicity in the correlationship. One major
bias is as highlighted earlier regarding the availability of full spectrum of
data in literature (Figure 3(a) \- Figure 3(b)). Majority of literature data
only reported the consolidation and mechanical properties corresponding to
optimized print parameters. Authors only published (or get their studies
published) data of high quality builds while the data corresponding to low
quality were not published. This bias is particularly shown in the
consolidation reports: over 80% of reported consolidation values are over 95%,
Figure 3(a). All available data of print parameters only correspond to a
narrow range of values (limited to high consolidation and optimal mechanical
properties). The lack of data that show strong effects of process parameters
on build quality outside of the narrow range of optimized values causes the
available data to fail at reflecting a strong correlation between the print
parameter variables and output variables, explaining the low values for the
Spearman’s and Pearson’s coefficients. Such a bias in only reporting optimized
parameters makes the use of reported data fail to accurately capture the full
spectrum of the process - property relationships [49]. Another reason is that
the complex physics governing the melting and solidification of LPBF process
can’t be captured with a single one-to-one monotonic correlation between
individual variables. This suggests a metric (e.g. VED) involving all
parameters simultaneously should be capable of capturing the correlation of
input - output variables. Surprisingly, the correlations between VED and the
quality variables were quite similar to those of the laser power. It is
important to note that the correlation values calculated in Figure 4 may be
affected by the presence of multicollinearity among the processing parameters.
This is true for AM processes in which multiple parameters are tuned to
achieve optimized quality. The multicollinearity is also reflected in a fact
that the Pearson’s correlation values were quite low [69, 70].

The variance inflation factor (VIF) is presented in Table 1 to examine the
degree of multicollinearity. VIF suggests that all considered parameters are
highly multicollinear with the layer thickness and hatch spacing has the
highest VIF, suggesting that the value choice of one (or both) of these two
process parameters is dependent on the choice of other parameters. It is
likely that the collinear dependence is extrinsic and engineered by printer
users in optimizing the build quality. Laser power, speed and other parameters
are often tied to one another to optimize the print quality [49]. To examine
the correlation of an individual input parameter with output variables, it is
necessary to have clean data in which only an input parameter is varied while
the other input parameters are fixed. Unfortunately, such clean data is not
publicly available in published literature. Most common method to address
multicollinearity involves removing features with high VIF. However, as the
objective is to understand how process parameters affect the build quality,
removal of any features is avoided. Thus, algorithms immune to
multicollinearity will be used to train ML models, such as neural networks and
tree-based algorithms. Where neural networks are not affected due to the
overparameterization of coefficients or weights at each layer of the network,
rendering the inflated regression coefficients redundant [71]. Whereas tree-
based algorithms selects single features at a time when splitting the tree in
a forward-stagewise manner, improving the model as demonstrated by Hastie et
al. [57].

Variable |  VIF
---|---
Laser power (W) |  4.78
Laser speed (mm/s) |  2.92
Layer thickness ($\mu$m) |  8.32
Hatch spacing ($\mu$m) |  8.56
Table 1: Multicollinearity detection of processing parameters with variance
inflation factor (VIF).

###  3.2 Principal Component Analysis (PCA)

VED is often used in literature to reducing the dimensionality in the
relationship between the quality of a build and process parameters. To further
evaluate the efficacy of reducing the dimensionality in the data (in
particular, the use of VED as a parameter to optimize print quality), PCA has
been undertaken to reduce the dimensionality of all the process parameters
into two variables for visualization. Where VED is, in effect, also a form a
dimensionality reduction, thus a comparison would show the efficacy between
both VED and the generated principal components ability to capture the
correlation between process parameters and material properties. The PCA
transformation used for this study was performed as follows

$\centering\begin{bmatrix}0.5238&-0.0316&0.5966&0.6072\\\
0.4325&0.8685&-0.1185&-0.2114\\\
\end{bmatrix}\times\begin{bmatrix}P_{1}&P_{2}&\cdots&P_{n}\\\
V_{1}&V_{2}&\cdots&V_{n}\\\ T_{1}&T_{2}&\cdots&T_{n}\\\
S_{1}&S_{2}&\cdots&S_{n}\end{bmatrix}=\begin{bmatrix}PC_{1,1}&PC_{1,2}&\cdots&PC_{1,n}\\\
PC_{2,1}&PC_{2,2}&\cdots&PC_{2,n}\\\ \end{bmatrix}\@add@centering$ (9)

such that the product of the eigenvectors and P, V, T and S (power, scanning
speed, layer thickness and hatch spacing respectively) is used to calculate
the principal components $PC_{1}$ and $PC_{2}$. The eigenvector is generated
by performing eigen decomposition on the covariance matrix of the dataset.
Only two eigenvectors with the highest eigenvalues, $PC_{1}$ and $PC_{2}$ are
retained, as these correspond to the two eigenvectors which capture the
highest amount of variance in the data. The transformed space of both PC is
represented in the biplot shown in Figure 5. The length of the arrow depicts
the strength of an individual process parameter with respect to its PC
direction, whereas the angle represents the contribution of the process
parameter to a PC: e.g., if a parameter is parallel to $PC_{1}$, it
contributes only to this component [67, 72, 73]. Thus, the transformed space
suggests that layer thickness and hatch spacing contributes mostly to
$PC_{1}$, whereas laser speed mostly contributes to $PC_{2}$, with laser power
contributing to both equally. The angle of the arrows shows that the most
influential variables for the construction of $PC_{1}$ are layer thickness,
hatch spacing and the laser power, whereas laser speed primarily contributes
to $PC_{2}$. Although the arrow lengths suggests laser speed has the greatest
influence in generating a principal component, the biplot shows laser power
correlates to all other process parameters studied. It should note that a
principal component analysis can capture the correlation between constituent
variables. The layer thickness and hatch spacing are very similar in creating
the PC1, this suggests the two parameters are highly correlated with each
other, where laser power is equally correlated to the laser speed, and the
hatch spacing/layer thickness. Such correlation is consistent with the VIF
values Table 1 in which the VIFs of layer thickness and hatch spacing are
almost the same while the VIF of the laser power is in between that of laser
speed and those of layer thickness and hatch spacing. This implies that laser
power is used as the main parameter to balance with the adjustment of the
other parameters on optimizing the quality build.

Figure 5: PCA biplot representing the contribution and relationship between
the processing parameters and both dimensionally-reduced principal components.
The scatter plot showcases the projection of original data onto the reduced
dimensional space.

Figure 6(a) displays the Spearman’s rank heatmap for the two PCs. The
generated map shows that the $PC_{1}$ has stronger monotonic correlation with
the quality variables (apart from the work hardening) than VED, hence is more
effective at capturing the relationship between process parameters and
quality. However, $PC_{2}$ performs a lot worse than VED, suggesting that
$PC_{1}$ is capable of capturing the majority of the correlation between the
process and properties. This is due to the process of PCA transformation: To
maximise the captured variance in one direction, the other is reduced in the
process [67, 72, 73]. The Spearman’s and Pearson’s rank heatmaps (Figure 6(a)
\- 6(b)) show the PC’s better ability in capturing the correlation in
comparison to VED. In particular, $PC_{1}$ outperforms the VED at capturing
the correlation of YS, elongation and consolidation. Nevertheless, despite
popular belief, the results of the correlation for $PC_{1}$ questions the
suitability of VED as a metric to optimize the print quality. However, all the
correlation values of PC1 are still relatively low, all being less than 0.5.
This may imply potential issues in ML training as the machine may struggle
with uncovering underlying patterns and correlation between processing
parameters and properties [67].

(a)

(b)

Figure 6: (a) Heatmap of Spearman’s rank correlation coefficient between
principal component’s of processing parameters after dimensionality reduction
and output properties, denoting the strength of monotonic relationship between
subsequent variable pairs. (b) Heatmap of Pearson’s rank correlation
coefficient, denoting the strength of linear relationship between subsequent
variable pairs.

###  3.3 Process Parameter Optimization

The common practice of identifying the process window is based on the
consolidation. While achieving high consolidation is important, consolidation
is not a single indicator of the material performance in structural
application in which more than 90% of failures is due to mechanical
performance, in particular fatigue [74]. Thus, the aim was to include multiple
variables that better reflect the print quality and by analyzing the data
across many different groups, the results will yield in a better
identification of process window for high quality including consolidation.
Therefore, selected processing parameters (laser power and speed) have been
optimized with respect to not only the consolidation, but also YS, work-
hardening and elongation following the method stated in Section 2.2. However,
as the data on fatigue (and creep) is largely missing, these properties have
not been considered for the identification of these windows.

The generated optimized maps ($Hmap_{2}$) for commonly used AM alloys are
displayed in Figure LABEL:fig:Non-treated_individual_heatmaps. The process
maps ($Hmap_{1}$) for individual quality variable are provided in Supporting
Information Figure S2 \- S6. The red cross data points depict the collected
data, whereas the color of the maps depicts the degree of quality with 1
denoting the highest quality, whereas 0 denoting the lowest quality. The
generated heatmaps considers each alloy individually, as the identification of
process parameters is highly dependent on the materials properties. Hence, a
single map for all alloys would be obsolete as it wouldn’t be an accurate
representation of the true processing windows for an individual alloy;
nevertheless, the map considering all materials is provided in Supporting
Information Figure S1 for reference.

The overall printability of an alloy can be evaluated by the area of the
process map for high quality. Such that an alloy with a larger process map can
be printed with the high quality and with wider ranges of process parameters,
i.e. more printable than another alloy with a smaller process map area. Figure
LABEL:fig:Non-treated_individual_heatmaps suggests 316L and IN718 are most
printable amongst all considered alloys. Furthermore, the maps suggests that
IN625 and Hastelloy X have slightly better printability than Ti6Al4V. The low
printability of Ti6Al4V is likely due to the loss of element (up to 0.9wt%
loss of Al) and low ductility because of martensite and high dislocation
densities [75, 76].

###  3.4 Machine Learning Performance

Following the preprocessing and training procedures stated in Section 2.3, the
top 20 highest performing hyperparameters were used to train the 7 ML
algorithms. The performance of all 140 trained models have been summarized
with RMSE as the metric to evaluate the accuracy, Figure 7. The RMSE on the
boxplot shows CatBoost and random forest exhibited the highest variability in
performance, whereas XGBoost, LightGBM and neural networks demonstrated the
least variability. Such that these three algorithms displayed consistent
performance among the 20 trained models for each algorithm. However, on
average, XGBoost emerged as the best-performing algorithm, surpassing CatBoost
by a small margin. Overall, the performance of all trained models aligns well
with existing literature, such that boosting algorithms typically have the
best performance, followed by random forest, neural networks, support vector
machine and decision tree [77]. Notably, the performance of neural network did
not agree with this result, likely due to the limited size of the training
data, as neural networks typically require large volumes of data to achieve
optimal performance [78].

Figure 7: Boxplot comparing RMSE performance metric of all trained machine
learning models, where the median of the samples is displayed at the top.

To assess the ML performance, the testing dataset has been fed to the best
performing XGBoost model to give predictions of YS, average work-hardening,
elongation and consolidation. Figure 8 shows the comparison between the true
values obtained from literature and the output predictions from the trained
XGBoost model. The predictions for YS and elongation shows great agreement
with the true values, the observation is reinforced by the calculated
coefficient of determination ($R^{2}$) which were 0.91 and 0.83, respectively.
The $R^{2}$ values suggests that 91% and 83% of the variance can be explained
by the model, for YS and elongation respectively. Notably, the $R^{2}$ value
for YS was the highest in all trained ML models. This high performance of ML
on the YS is likely because of the high correlation between YS and the primary
dendritic (or cellular) spacing. It is found that the YS is inversely related
to the cellular dendritic spacing that is inversely proportional to the
cooling rate, which is, in turn, controlled by the process parameters [79,
80]. This implies that there is a well defined correlation between the process
parameters and YS reflected by the ML good performance, suggesting that ML is
capable of reflecting the underlying science of such relationship and such a
capability should be improved if there were more studies reporting the
cellular spacing. The XGBoost model provides a reasonable prediction of the
work hardening. As the work hardening was calculated based on YS, UTS and
elongation, thus the accumulation of errors in predictions is expected to the
reduce accuracy of work hardening predictions. However, the high accuracy
predictions for YS and elongation suggests the ML’s low performance on the
hardening was mainly due to the accuracy of predicting UTS. The ML model
performed worst regarding the consolidation with 67% of the variance not
predicted by the model. The low performance of consolidation (Figure 8c2)
highlights the detrimental consequence of the bias in reporting the
consolidation values as discussed earlier: The majority of consolidation
reports was $>$98% (Figure 3(a)). Such a bias negatively affected the learning
of the ML models. Furthermore, because the majority of consolidation data lies
within the 90% - 100% range, the ML’s prediction on the consolidation was
highly weighted by the training of ML on the known data of this range. The
performance of ML worsens for predicting consolidation further away from the
known range (Figure 8c1). Moreover, the low performance of ML may be related
to the difficulty in capturing the stochastic nature of porosity formation in
the melting and cooling of AM process. Last but not least, as the property is
influenced by the build location/direction due to the build up of residual
heat and scanning strategy which were not included in the training of ML
models; lowering the prediction accuracy of ML. To address this limitation,
these relevant process parameters should ideally be incorporated into the
training of ML model. However, this was not implemented due to insufficient
data availability concerning these factors.

Although the model considers whether the sample has been heat treated or HIP
by means of One Hot Encoding, the results suggests no clear distinction in
affecting the accuracy of predictions between treated and non treated samples;
however, over time with more collected data, it is likely that the these
observations may differ. Nevertheless, the results shown in Figure 8 suggests
that the trained ML model has some credit in characterizing the relationship
between process parameters and the output properties considered, in particular
for YS and elongation. Furthermore, the difference in performance between
simple and complex ML models such as non-boosted and boosted algorithms only
yielded in a minor increase in performance, suggesting that the biggest
underlying factor is the quality of data. Therefore, getting high quality data
with minimal biases would significantly improve the performance of ML models.

|
---|---

(a)

(b)

|
---|---

(c1)

(c2)

---

(d)

Figure 8: Results of XGBoost model for predicted output properties against
experimentally measured values collected in literature for all alloys. The
predicted output properties considered are (a) yield stress, (b) average work
hardening, (c1) consolidation, (c2) zoom in view of consolidation and (d)
elongation. The red line depicts a perfect 1 to 1 fit, whereas the black line
represents the line of regression obtained from the XGBoost model.

###  3.5 Sensitivity Analysis

Sensitivity analysis has been used to examine the trained ML models ability to
account for important influence of AM process parameters in the quality. The
best performing XGBoost and CatBoost model have been used for the analysis.
The Sobol indices for the predicted properties by the XGBoost model is
displayed in Figure 9. The values have been normalised such that the sum of
the main effect sensitivity index is equal to 1, as does the sum of the total
effect sensitivity index. The results shows that overall, laser power and
speed are the most influential process parameters for all the investigated
quality variables. However the results show that XGBoost was not able to
reflect the underlying science of the process - mechanical property relations.
For example, while the XGBoost model shows the laser power was seen highly
influential in consolidation, the influence of laser speed was very weak. This
is not consistent with a fact that both the laser power and speed are often
used to optimize the consolidation [49]. In addition, the ML model shows a
large influence of laser speed in YS, but not the laser power despite the two
key parameter are used in controlling the thermal condition, in particular the
cooling rate that governs the spacing of primary dendrites or cells [80, 81,
82, 83, 84].

Figure 9: Bar plots comparing the main effect sensitivity index and the total
effect sensitivity index for the XGBoost model. The main effect sensitivity
index measures the influence of an individual process parameter on a given
output property, without considering its interactions with other process
parameters. Whereas total effect sensitivity index measures the overall
influence of an individual process parameter on a given output property,
including both the main effect sensitivity and effects through its
interactions with other process parameters.

Interestingly, the Sobol indices calculated by the CatBoost model (Figure 10)
reflects the known underlying science well. This can be seen with the Sobol
indices for the yield stress, as laser speed and power is shown to be the two
most influential process parameters. This aligns well with the correlation
between YS with speed and power with the cooling rate, hence YS as discussed
earlier. However, hatch spacing is also shown to have high impact, especially
with combined interactions as indicated by the total effect index. Although
the ML’s prediction for consolidation was suboptimal (Figure 8), the
calculated Sobol indices reflect the roles of process parameters in the main
mechanisms of porosity formation. As process induced pores, such as keyhole
pores resulting from excessive power density, and lack of fusion pores caused
by insufficient molten metal due to inadequate energy density, i.e. dependent
on melt pool geometry, layer thickness and hatch spacing [85, 80, 49]. Hence,
it is anticipated that the influence from processing parameters on
consolidation are expected to be approximately equal - this is reflected well
in (Figure 10). The mechanism of work hardening relies on interactions of
dislocations between themselves and with other crystallographic features. In
additive manufacturing conditions, alloys often consist of high dislocation
density regions at the cellular (or dendritic) boundaries. Consequently, finer
cells (or dendrites) lead to increased interactions between mobile
dislocations and immobile dislocations at the dislocation-rich regions. Thus,
the primary factors influencing this property are laser power and speed, as
they dictate the cooling rate, hence the cell (dendrite) spacing [86, 87].
Whereas elongation is dependent on both consolidation and work hardening. As
such, it is expected to exhibit a similar influence of process parameters as
the as the combined effect of these two properties. The analysis also revealed
that interactions between process parameters can significantly affect the
studied quality variables, indicating the presence of synergistic effects.
This can be seen with hatch spacing as the total effect index is always
greater than the main effect index. This suggests that although hatch spacing
alone has weak influence on properties, the interaction of hatch spacing with
other process parameters was found to have substantial impact on properties.

Figure 10: Bar plots comparing the main effect sensitivity index and the total
effect sensitivity index for the CatBoost model. The main effect sensitivity
index measures the influence of an individual process parameter on a given
output property, without considering its interactions with other process
parameters. Whereas total effect sensitivity index measures the overall
influence of an individual process parameter on a given output property,
including both the main effect sensitivity and effects through its
interactions with other process parameters.

##  4 Conclusion

A considerably large literature data for metal additive manufacturing (AM) was
created in this study. A comprehensive and in-depth examination of the data
highlights major biases and limitation of the literature data, limiting the
understanding of the process, microstructure, mechanical property (PMP)
relationship: (1) Most studies only reported consolidation, with almost three
times more reports than mechanical properties such as yield stress and
elongation. (2) Most literature data obtained were in optimized or near
optimized conditions, with 84% of the consolidation data reported above values
of 95%. Correlation analysis shows that this bias limits the literature data
in revealing the strong correlation between process parameters and quality
variables such as the consolidation and mechanical properties. (3)
significantly lack of quantitative data on microstructure such as the spacing
of primary dendrites or cells.

Meta-analyses of the collected were done, showing weak correlation between the
process parameters (i.e. input) including the volumetric energy density (VED),
with the consolidation and mechanical properties (i.e. output). Such weak
correlation is likely due to (1) the stated biases and the (2) the correlation
between input process parameters and quality variables are non-monotonic with
high multicollinearity. The hatch spacing and layer thickness are found to be
most collinear, reflecting a common practice in AM process identification: the
values of these two parameters are often balanced by tuning the beam power and
beam speed. While the correlation analysis and study of ML performances
demonstrate potential for data-driven approaches for metal additive
manufacturing, the quality of the dataset hinders the results due to current
reporting biases and practices.

The bias (1) reflects another common practice (in AM publications): the
identification of process maps is commonly based only on consolidation. Such
bias poises a serious limitation on the optimization of parameters because the
quality of an AM build is ultimately governed by mechanical properties such as
yield strength, elongation and hardening. The significant data enables us to
identify the process map on the basis of not only consolidation, but also
yield stress, elongation and work hardening. The process map identification
results show that amongst all the alloys considered in this study, 316L and
Inconel 718 are the most printable alloys followed by Hastelloy X and Inconel
625, where the least printable alloy is Ti6Al4V.

The present study also investigates dimensionality reduction of processing
parameters using principal component analysis. The dimensionality reduction
was compared to VED, a common metric that consolidates multiple processing
parameters for process optimization. Two principal components show much
stronger correlation between the PC with the output, suggesting an alternate
way (in comparison to the VED) to reduce dimension in optimizing the build
consolidation and quality.

The bias (2) seriously limits the use of machine learning in learning the full
spectrum of the process parameter - consolidation relationship, hence
negatively affecting the ML performance in predicting the consolidation. This
effect results in a low accuracy of ML predictions for consolidation, and most
evidently in the low value range of consolidation. Furthermore, the minor
increase in accuracy by using boosting algorithms compared to non-boosting
algorithms further suggests the quality of the obtained dataset is the most
significant factor in improving the ML models’ performance. We are, therefore,
calling the AM community to publicly share data of wide spectrums, in
particular process parameters producing low and intermediate ranges of
consolidation and mechanical properties beyond consolidation or density.
Furthermore, increase data availability through open-access and standardize
reporting formats to provide easily accessible data. To aid in this effort, an
online template for users to contribute data or use the training dataset and
code associated with this work are available in the open-source Github
repository at
https://github.com/RaymondWKWong/MetaAnalysis$\textunderscore$MetalAM.

Last but not least, due to insufficient data reported for the microstructure
and mechanical properties, ML would not able to learn the PMP relationship.
Given the inherent correlation between microstructure and mechanical
properties (in particular for the long term performance such as fatigue), next
significant efforts should be given to generating microstructure data and
fatigue.

Acknowledgements

The authors would like to thank the EPSRC for supporting the research [grant
number EP/K503733/1]. R. Wong and M.S. Pham would like to thank Jalal Al-Lami
for providing part of Inconel 718 literature data used for this study. M.S.
Pham, R. Wong and C.S. Maldonado thank the Imperial College London’s support
via a Imperial-Nanyang Technological University seed fund.

The views expressed in the article do not necessarily represent the views of
the U.S. Department of Energy or the United States Government. Sandia National
Laboratories is a multimission laboratory managed and operated by National
Technology and Engineering Solutions of Sandia, LLC., a wholly owned
subsidiary of Honeywell International, Inc., for the U.S. Department of
Energy’s National Nuclear Security Administration under contract DE-
NA-0003525.

## Supporting Information

Figure S1: $Hmap_{2}$, optimized processing window maps for the non-treated
dataset, where the map considers all materials, optimizing YS, average work-
hardening, elongation and consolidation. The brighter regions outlines the
regions of laser power and laser speed which users are advised to use to
obtain better overall print quality. Figure S2: Non-treated 316L $Hmap_{1}$,
individual processing window maps for yield strength, average work-hardening,
elongation and consolidation. Figure S3: Non-treated Ti6Al4V $Hmap_{1}$,
individual processing window maps for yield strength, average work-hardening,
elongation and consolidation. Figure S4: Non-treated IN625 $Hmap_{1}$,
individual processing window maps for yield strength, average work-hardening,
elongation and consolidation. Figure S5: Non-treated IN718 $Hmap_{1}$,
individual processing window maps for yield strength, average work-hardening,
elongation and consolidation. Figure S6: Non-treated Hastelloy X $Hmap_{1}$,
individual processing window maps for yield strength, average work-hardening,
elongation and consolidation.

## References

  * [1] T. DebRoy, H. Wei, J. Zuback, T. Mukherjee, J. Elmer, J. Milewski, A. Beese, A. Wilson-Heid, A. De, W. Zhang,  _Progress in Materials Science_ 2018, _92_ 112\.
  * [2] J. J. Lewandowski, M. Seifi,  _Annual Review of Materials Research_ 2016, _46_ , 1 151.
  * [3] D. D. Gu, W. Meiners, K. Wissenbach, R. Poprawe,  _International Materials Reviews_ 2012, _57_ , 3 133\.
  * [4] J. Schmelzle, E. V. Kline, C. J. Dickman, E. W. Reutzel, G. Jones, T. W. Simpson,  _Journal of Mechanical Design, Transactions of the ASME_ 2015, _137_ , 11.
  * [5] W. E. King, A. T. Anderson, R. M. Ferencz, N. E. Hodge, C. Kamath, S. A. Khairallah, A. M. Rubenchik,  _Applied Physics Reviews_ 2015, _2_ , 4 041304.
  * [6] X. Zhang, E. Liang,  In _IOP Conference Series: Materials Science and Engineering_ , volume 493. Institute of Physics Publishing,  ISSN 1757899X, 2019 .
  * [7] E. Maleki, S. Bagherifard, M. Bandini, M. Guagliano,  Surface post-treatments for metal additive manufacturing: Progress, challenges, and opportunities, 2021.
  * [8] T. Segaran, J. Hammerbacher,  _Beautiful data_ ,  O’Reilly, 2009.
  * [9] T. A. Runkler,  Data Analytics Models and Algorithms for Intelligent Data Analysis 3\. Edition,  Technical report.
  * [10] R. Rasmussen, H. Gulati, C. Joseph, C. Stanier, O. Umegbolu,  _Data Analyst_ ,  BCS, The Chartered Institute for IT, 2019.
  * [11] E. Horvitz, D. Mulligan,  _Science_ 2015, _349_ , 6245 253.
  * [12] A. Garg, K. Tai, C. H. Lee, M. M. Savalani,  _Journal of Intelligent Manufacturing_ 2014, _25_ , 6 1349.
  * [13] J. Zhang, P. Wang, R. X. Gao,  _Computers in Industry_ 2019, _107_ 11\.
  * [14] O. Bayraktar, G. Uzun, R. Çakiroğlu, A. Guldas,  _Polymers for Advanced Technologies_ 2017, _28_ , 8 1044\.
  * [15] A. Garg, K. Tai, M. M. Savalani,  _Rapid Prototyping Journal_ 2014, _20_ , 2 164.
  * [16] S. Lee, J. Peng, D. Shin, Y. S. Choi,  _Science and Technology of Advanced Materials_ 2019, _20_ , 1 972.
  * [17] C. Kamath,  _International Journal of Advanced Manufacturing Technology_ 2016, _86_ , 5-8 1659.
  * [18] Z. Yang, Y. Lu, H. Yeung, S. Krishnamurty,  _Journal of Computing and Information Science in Engineering_ 2020, _20_ , 5.
  * [19] G. Tapia, S. Khairallah, M. Matthews, W. E. King, A. Elwany,  _International Journal of Advanced Manufacturing Technology_ 2018, _94_ , 9-12 3591.
  * [20] Z. Yang, Y. Lu, S. Li, J. Li, Y. Ndiaye, H. Yang, S. Krishnamurty, B. H. School, S. Spring,  In-Process Data Fusion for Process Monitoring and Control of Metal Additive Manufacturing,  Technical report, 2021.
  * [21] L. Meng, J. Zhang,  _JOM_ 2020, _72_ , 1 420.
  * [22] Z. Wang, C. Jiang, P. Liu, W. Yang, Y. Zhao, M. F. Horstemeyer, L. Q. Chen, Z. Hu, L. Chen,  _npj Computational Materials_ 2020, _6_ , 1.
  * [23] B. Rankouhi, S. Jahani, F. E. Pfefferkorn, D. J. Thoma,  _Additive Manufacturing_ 2021, _38_.
  * [24] F. Imani, R. Chen, E. Diewald, E. Reutzel, H. Yang,  _Journal of Manufacturing Science and Engineering, Transactions of the ASME_ 2019, _141_ , 11.
  * [25] Q. Liu, H. Wu, M. J. Paul, P. He, Z. Peng, B. Gludovatz, J. J. Kruzic, C. H. Wang, X. Li,  _Acta Materialia_ 2020, _201_ 316\.
  * [26] A. Garg, J. S. L. Lam,  _Measurement: Journal of the International Measurement Confederation_ 2015, _75_ 210\.
  * [27] D. S. Shin, C. H. Lee, U. Kühn, S. C. Lee, S. J. Park, H. Schwab, S. Scudino, K. Kosiba,  _Journal of Alloys and Compounds_ 2021, _862_.
  * [28] G. Tapia, A. H. Elwany, H. Sang,  _Additive Manufacturing_ 2016, _12_ 282\.
  * [29] Z. Wang, X. Wang, W. Yang, Y. Xiao, Y. Liu, L. Chen,  yNet: a multi-input convolutional network for ultra-fast simulation of field evolvement,  Technical report.
  * [30] R. H. Taylor, F. Rose, C. Toher, O. Levy, K. Yang, M. Buongiorno Nardelli, S. Curtarolo,  _Computational Materials Science_ 2014, _93_ 178\.
  * [31] A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder, K. A. Persson,  Commentary: The materials project: A materials genome approach to accelerating materials innovation, 2013.
  * [32] J. E. Saal, S. Kirklin, M. Aykol, B. Meredig, C. Wolverton,  _JOM_ 2013, _65_ , 11 1501.
  * [33] Nist additive manufacturing material database,  URL https://ammd.nist.gov/.
  * [34] Citrination,  URL https://citrination.com/.
  * [35] Z. Wang, W. Yang, Q. Liu, Y. Zhao, P. Liu, D. Wu, M. Banu, L. Chen,  _Journal of Manufacturing Processes_ 2022, _77_ , January 13.
  * [36] E. Popova, T. M. Rodgers, X. Gong, A. Cecen, J. D. Madison, S. R. Kalidindi,  _Integrating Materials and Manufacturing Innovation_ 2017, _6_ , 1 54.
  * [37] C. Wang, X. P. Tan, S. B. Tor, C. S. Lim,  _Additive Manufacturing_ 2020, _36_ , January 101538\.
  * [38] S. S. Razvi, S. Feng, A. Narayanan, Y. T. T. Lee, P. Witherell,  _Proceedings of the ASME Design Engineering Technical Conference_ 2019, _1_ 1\.
  * [39] B. Zhang, P. Jaiswal, R. Rai, P. Guerrier, G. Baggs,  _Rapid Prototyping Journal_ 2019, _25_ , 3 530.
  * [40] J. Lyu, J. Akhavan, S. Manoochehri,  _Data in Brief_ 2022, _41_ 107852\.
  * [41] B. Dovgyy, M. Simonelli, M. S. Pham,  _Materials Research Letters_ 2021, _9_ , 8 350.
  * [42] Y. T. Tang, C. Panwisawas, J. N. Ghoussoub, Y. Gong, J. W. Clark, A. A. Németh, D. G. McCartney, R. C. Reed,  _Acta Materialia_ 2021, _202_ 417\.
  * [43] S. Thapliyal, M. Komarasamy, S. Shukla, L. Zhou, H. Hyer, S. Park, Y. Sohn, R. S. Mishra,  _Materialia_ 2020, _9_.
  * [44] W. Mycroft, M. Katzman, S. Tammas-Williams, E. Hernandez-Nava, G. Panoutsos, I. Todd, V. Kadirkamanathan,  _Journal of Intelligent Manufacturing_ 2020, _31_ , 7 1769.
  * [45] W. Yan, S. Lin, O. L. Kafka, Y. Lian, C. Yu, Z. Liu, J. Yan, S. Wolff, H. Wu, E. Ndip-Agbor, M. Mozaffar, K. Ehmann, J. Cao, G. J. Wagner, W. K. Liu,  _Computational Mechanics_ 2018, _61_ , 5 521.
  * [46] E. Popova, T. M. Rodgers, X. Gong, A. Cecen, J. D. Madison, S. R. Kalidindi,  _Integrating Materials and Manufacturing Innovation_ 2017, _6_ , 1 54.
  * [47] G. G. V. Douglas C. Montgomery, Elizabeth A. Peck,  Introduction to Linear Regression Analysis,  Technical report, 2012.
  * [48] Hogg, McKean, Craig,  Introduction to Mathematical Statistics,  Technical report, 2018.
  * [49] J. V. Gordon, S. P. Narra, R. W. Cunningham, H. Liu, H. Chen, R. M. Suter, J. L. Beuth, A. D. Rollett,  _Additive Manufacturing_ 2020, _36_.
  * [50] J. Y.-l. Chan, S. Mun, H. Leow, K. T. Bea, W. K. Cheng, S. W. Phoong, Z.-w. Hong, Y.-l. Chen,  _Mathematics_ 2022, _10_ , 8 1283.
  * [51] P. Virtanen, R. Gommers, et al.,  _Nature Methods_ 2020, _17_ , 3 261.
  * [52] C. B. Barber, D. P. Dobkin,  The Quickhull Algorithm for Convex Hulls,  Technical report, 1996.
  * [53] J. P. Berrut, L. N. Trefethen,  _SIAM Review_ 2004, _46_ , 3 501.
  * [54] C. Albon,  _Machine learning with Python cookbook : practical solutions from preprocessing to deep learning_ ,  2018.
  * [55] J. Brownlee,  _Machine Learning Mastery_ 2017, _91_ 399\.
  * [56] A. Géron,  _Hands-on Machine Learning whith Scikit-Learing, Keras and Tensorfow_ ,  2019.
  * [57] T. Hastie, R. Tibshirani, J. Friedman,  _The Elements of Statistical Learning_ ,  Springer Series in Statistics. Springer New York Inc., New York, NY, USA, 2001.
  * [58] T. Chai, T. Chai, R. R. Draxler,  _Geosci. Model Dev_ 2014, _7_ 1247\.
  * [59] A. E. Bilali, T. Abdeslam, N. Ayoub, H. Lamane, M. A. Ezzaouini, A. Elbeltagi,  _Journal of Environmental Management_ 2023, _327_ 116890\.
  * [60] B. Kapusuzoglu, S. Mahadevan,  _Reliability Engineering & System Safety_ 2021, _214_ 107712\.
  * [61] I. Sobol,  _Mathematics and Computers in Simulation_ 2001, _55_ , 1 271, the Second IMACS Seminar on Monte Carlo Methods.
  * [62] A. Saltelli, P. Annoni, I. Azzini, F. Campolongo, M. Ratto, S. Tarantola,  _Computer Physics Communications_ 2010, _181_ , 2 259\.
  * [63] A. Tran, T. Wildey, H. Lim,  _Frontiers in Materials_ 2022, _9_.
  * [64] A. B. Spierings, M. Schneider, R. Eggenberger,  _Rapid Prototyping Journal_ 2011, _17_ , 5 380.
  * [65] D. of Materials Science, M. U. of Cambridge,  URL https://www.doitpoms.ac.uk/tlplib/mechanical_testing_metals/necking.php.
  * [66] J. Al-Lami, P. Hoang, C. Davies, T. Pirzada, M.-S. Pham,  _Materials Characterization_ 2023, _199_ 112815\.
  * [67] G. James, D. Witten, T. Hastie, R. Tibshirani,  An Introduction to Statistical Learning with Applications in R Second Edition,  Technical report, 2021.
  * [68] J. Grus,  Data Science from Scratch,  Technical report.
  * [69] J. Hauke, T. Kossowski,  _Quaestiones Geographicae_ 2011, _30_ , 2 87.
  * [70] J. C. de Winter, S. D. Gosling, J. Potter,  _Psychological Methods_ 2016.
  * [71] R. D. De Veaux, L. H. Ungar 1994, 393–402.
  * [72] H. Abdi, L. J. Williams,  Principal component analysis, 2010.
  * [73] I. T. Jollife, J. Cadima,  Principal component analysis: A review and recent developments, 2016.
  * [74] S. Suresh,  _Fatigue of Materials_ ,  Cambridge University Press, 2 edition, 1998.
  * [75] J. Lertthanasarn, C. Liu, M. Pham,  _Materials Science and Engineering: A_ 2021, _818_ 141436\.
  * [76] T. Mukherjee, J. S. Zuback, A. De, T. DebRoy,  _Scientific Reports_ 2016, _6_.
  * [77] R. Caruana, A. Niculescu-Mizil,  _ACM International Conference Proceeding Series_ 2006, _148_ 161\.
  * [78] Q. Guo, S. Jin, M. Li, Q. Yang, K. Xu, Y. Ju, J. Zhang, J. Xuan, J. Liu, Y. Su, Q. Xu, Y. Liu,  _Science China Earth Sciences_ 2020, _63_ , 10 1457\.
  * [79] A. Piglione, B. Attard, V. Vieira Rielli, C.-T. Santos Maldonado, M. M. Attallah, S. Primig, M.-S. Pham,  _Additive Manufacturing_ 2021, _47_ 102347\.
  * [80] M.-S. Pham, B. Dovgyy, P. A. Hooper, C. M. Gourlay, A. Piglione,  _Nature Communications_ 2020, _11_ , 1.
  * [81] H. Bikas, P. Stavropoulos, G. Chryssolouris,  _International Journal of Advanced Manufacturing Technology_ 2016, _83_ , 1-4 389.
  * [82] H. R. Kotadia, G. Gibbons, A. Das, P. D. Howes,  A review of Laser Powder Bed Fusion Additive Manufacturing of aluminium alloys: Microstructure and properties, 2021.
  * [83] Z. Liu, D. Zhao, P. Wang, M. Yan, C. Yang, Z. Chen, J. Lu, Z. Lu,  Additive manufacturing of metals: Microstructure evolution and multistage control, 2022.
  * [84] P. Bajaj, A. Hariharan, A. Kini, P. Kürnsteiner, D. Raabe, E. A. Jägle,  _Materials Science and Engineering A_ 2020, _772_.
  * [85] W. J. Sames, F. A. List, S. Pannala, R. R. Dehoff, S. S. Babu,  _International Materials Reviews_ 2016, _61_ , 5 315\.
  * [86] G. Dieter, D. Bacon, D. Bacon,  _Mechanical Metallurgy_ ,  Materials Science and Engineering Series. McGraw-Hill, 1988.
  * [87] M. Pham, B. Dovgyy, P. Hooper,  _Materials Science and Engineering: A_ 2017, _704_ 102\.