rmkRemark pfProof potProof of Theorem LABEL:thm2 mode=titleIntegrating AI in
NDE: Techniques, Trends, and Further Directions Conceptualization of this
study, Methodology, Software 1]organization=Fraunhofer-Institut für
Zerstörungsfreie Prüfverfahren IZFP, addressline=Campus E3 1, city=66123
Saarbrücken, country=Germany 2]organization=Technische Universität Ilmenau,
addressline=Ehrenbergstraße 29, city=98693 Ilmenau, country=Germany
3]organization=Universität des Saarlandes, addressline=Campus, city=66123
Saarbrücken, country=Germany 4]organization=Hochschule für Technik und
Wissenschaft des Saarlandes, addressline=Goebenstrasse 40, city=66117
Saarbrücken, country=Germany

# Integrating AI in NDE: Techniques, Trends, and Further Directions

Eduardo Pérez  Cemil Emre Ardic  Ozan Çakıroğlu  Kevin Jacob  Sayako Kodera
Luca Pompa  Mohamad Rachid  Han Wang  Yiming Zhou  Cyril Zimmer  Florian Römer
Ahmad Osman  [  [  [  [

###### Abstract

The digital transformation is fundamentally changing our industries, affecting
planning, execution as well as monitoring of production processes in a wide
range of application fields. With product line-ups becoming more and more
versatile and diverse, the necessary inspection and monitoring sparks
significant novel requirements on the corresponding Nondestructive Evaluation
(NDE) systems. The establishment of increasingly powerful approaches to
incorporate Artificial Intelligence (AI) may provide just the needed
innovation to solve some of these challenges.

In this paper we provide a comprehensive survey about the usage of AI methods
in NDE in light of the recent innovations towards NDE 4.0. Since we cannot
discuss each NDE modality in one paper, we limit our attention to magnetic
methods, ultrasound, thermography, as well as optical inspection. In addition
to reviewing recent AI developments in each field, we draw common connections
by pointing out NDE-related tasks that have a common underlying mathematical
problem and categorizing the state of the art according to the corresponding
sub-tasks. In so doing, interdisciplinary connections are drawn that provide a
more complete overall picture.

###### keywords:

Artificial intelligence Machine Learning NDE 4.0 ECT 3MA UT TT VT

##  1 Introduction

Nondestructive Evaluation (NDE) is one of the key ingredients for ensuring
safety, reliability, and efficiency of products as well as infrastructure we
use and depend on everyday. In light of challenges such as increasing
technological complexity of our industries, aging infrastructure, or the
general need to reduce resource consumption, its significance for industry and
society is expected to grow dramatically in the near future. Like industry
itself, the field of NDE has undergone several transformations, adopting
innovations in sensor technology, networking and computing power and more
recently, AI methods to cope with the requirements of increasingly complex
industries. At the intersection between NDE and Industry 4.0 technologies, the
field of "NDE 4.0" has emerged as a confluence of Industry 4.0 digital
technologies, physical inspection methods, and business models. It embraces
the digital transformation, including digital twins of products and inspection
systems, Industrial Internet of Things (IIoT) technologies for seamless
networked integration, semantic interoperability through standardized
interfaces, common data formats and appropriate meta descriptions,
heterogeneous compute power through cloud and edge technologies, and other
recent developments. By many, NDE 4.0 is also seen as a potential step to
increase automation of current NDE systems, as its growing complexity allows
it to handle increasingly sophisticated tasks autonomously.

The present paper provides a survey on the state of the art in the usage of AI
methods for NDE, in light of the promises of the NDE 4.0 developments. We
discuss to what extent certain levels of automation can now be handled through
AI-powered NDE approaches, which advances have been demonstrated in that
regard recently, and which limitations we are still facing. It is illustrative
to categorize the degree of NDE automation according to NDE automation levels
as suggested in [1], which we later decompose into specific tasks, as well as
the common NDE operator levels according to DIN EN ISO 9712 [2]. A level 1
operator is mostly responsible for conducting measurements under the
supervision of a higher level personnel and thus, the tasks are easier to
automate. On the other hand, the tasks of a level 2 operator (result
interpretation and decision making) or even a level 3 operator (modality
selection, inspection design and full responsibility of the entire chain)
require much higher degree of sophistication.

Since the entire discussion on NDE automation is highly dependent on the
considered modality, we treat a selection of NDE modalities in different
sections of this paper. As the entire field of NDE is too wide to provide a
thorough review in just one paper, we limit our scope to selected,
representative modalities: magnetic methods (eddy current testing and
micromagnetic methods), Ultrasound, thermography, and at the end also touching
on optical inspection techniques.

As a second contribution, the paper attempts to provide connections between
the AI-powered NDE tasks and their underlying mathematical problems that are
often similar. The common patterns help seeing intersections between related
fields that can benefit from interdisciplinary work and give rise to powerful
cross-domain AI approaches.

The paper is organized as follows: section 2 reviews magnetic methods,
Ultrasound is treated in section 3, in section 4 we discuss thermography, and
section 5 we consider optical inspection. The common patterns between AI-
powered NDE tasks are elaborated on in section 6 before drawing conclusions in
section 7.

Throughout this work, quantities in bold lowercase $\bm{v}$ are vectors, while
those in bold uppercase $\bm{M}$ are matrices. Scalar elements of vectors and
matrices are denoted as $\bm{v}(i)$ and $\bm{M}(i,j)$, respectively. The
symbol $(\cdot)^{\text{H}}$ represents the Hermitian or complex conjugate
transpose, and $(\cdot)^{\dagger}$ is the pseudo-inverse. Calligraphic symbols
$\mathcal{S}$ denote sets. Further symbols are introduced throughout the work
as needed.

##  2 Magnetic methods

Several NDE methods rely on the magnetic interaction of a probe with the test
specimen. These include, amongst others, eddy current testing of electrically
conductive materials. During testing, an alternating magnetic field is
generated by a coil, which induces eddy currents in the material to be tested.
The eddy current density then generates the secondary magnetic field which is
measured with a sensor. Typically, the measurement sensor also contains the
excitation coil, enabling to infer the eddy current density. The measured
parameters are the amplitude and the phase shift to the excitation signal. In
the case of ferromagnetic materials, some magnetic properties, which correlate
with the mechanical properties of the material, can also be detected with
other techniques. For example, the Barkhausen effect, which is caused by the
movement of the magnetic domain walls of the sample upon magnetic excitation,
can be analyzed. A combination of those two methods together with the analysis
of the harmonics of the tangential magnetic field and the incremental
permeability [3] is possible, and among such mixed modalities we highlight the
Micromagnetic Multiparameter Microstructure and stress Analysis (3MA) II
system. The 3MA-X8 system is a further development, focusing on a more robust
and less complex device and sensor design [4]. This system also combines eddy
current analysis, incremental permeability analysis and harmonic analysis,
omitting the Barkhausen noise analysis [4]. These methods find wide
application in the field of material characterization as well as defect
detection.

###  2.1 Eddy current testing

Generally, eddy current probe impedance is recorded as a function of probe
position and/or excitation frequency, which is used for detecting flaws such
as cracks in materials. Inferring those parameters from the impedance
measurements is an inverse problem and widely recognized as a complex
theoretical problem. Already in 1997 Rekanos et al. [5] suggested the use of
Neural Networks (NNs) to tackle the inversion problem. Since then, a number of
studies have focused on the detection and evaluation of flaws with the help of
Artificial Neural Networks (ANNs). Of particular interest here are defects
such as cracks, which could paralyse operations and endanger safety.

Thus, Helifa et al. [6], Harzallah et al. [7], Demachi et al. [8] and Bernieri
et al. [9] dealt with the detection and assessment of cracks. While Helifa et
al. and Harzallah et al. made use of AI supported inversion for the detection
and characterization of surface cracks, Bernieri et al. discussed crack shape
reconstruction and compared the results using an ANN and Support Vector
Regression (SVR) and Demachi et al. studied the estimation of crack depths in
non-magnetic material using a Convolutional Neural Network (CNN) with two
convolution layers, two pooling layers and a fully connected layer. They were
able to estimate the crack depth within an error of
$0.05\text{\,}\mathrm{m}\mathrm{m}$. For their research, Helifa et al. used a
Multilayer Perceptron (MLP) with 21 input neurons corresponding to 21
different sensor positions, a hidden layer with 80 neurons and hyperbolic
tangent activation function and an output layer with a linear activation
function. The network was trained on simulated data and achieved relative
errors of $2.5\text{\,}\%$ for the depth and $2\text{\,}\%$ for the length of
the cracks. Harzallah et al. took a similar approach and used a MLP with 1
input layer, 1 hidden layer and 1 output layer and trained it on simulated
signals. They stopped the training after 30 epochs at an Mean Squared Error
(MSE) of about $5\text{\times}{10}^{-7}\text{\,}\mathrm{mm}$2 for the train
and validation data, while the test error was about
$1\text{\times}{10}^{-2}\text{\,}\mathrm{mm}$2 for length and depth of the
crack. On the other hand, Bernieri et al. discussed crack shape reconstruction
and compared the results using an ANN and SVR. The database for the training
was constructed by simulation. A $200\text{\,}\mathrm{mm}$ ×
$200\text{\,}\mathrm{m}\mathrm{m}$ × $2\text{\,}\mathrm{m}\mathrm{m}$ aluminum
plate was simulated, where the defect region was divided into a regular grid
of 20 × 4 elements, imposing defect depth $z_{0}$ and height $a$ steps of
$0.5\text{\,}\mathrm{m}\mathrm{m}$ and length $l$ step of
$1\text{\,}\mathrm{m}\mathrm{m}$. This way, a data set of 200 simulated
magnetic field maps was obtained. They found that the SVR outperformed the
three layered MLP in terms of Mean Absolute Error (MAE) as shown in table 1.

Table 1: Crack shape reconstruction results of Bernieri et al. [9] |  MAE $l$ |  MAE $a$ |  MAE $z_{0}$
---|---|---|---
SVR |  0.043 |  0.12 |  0.12
ANN |  0.14 |  0.24 |  0.24

If we move into the area of nuclear reactors, not only cracks but even minor
flaws are of great interest and must be reliably recognised for safe
operation. Thus, Song et al. [10] as well as Yusa et al. [11] have dealt with
the investigation of flaws in steam reactor pipes. In order to train a NN on
an synthetic dataset generated by simulation to determine the type and size
parameters of flaws in steam generator tubes, Song et al. [10] built a
synthetic database with 400 ECT signals generated from 200 axisymmetric
machined grooves in four types with two test frequencies per flaw. They used a
probabilistic NN for the flaw classification and a back propagation network
for the flaw sizing. This resulted in an overall accuracy of $91\text{\,}\%$
in the determination of the flaw type while the correlation coefficients
between actual and estimated size parameters exceeded 0.97. For the
reconstruction of natural cracks that occurred in steam generator tubes, Yusa
et al. [11] modeled cracks as a pixel map and simulated eddy current signals
from that. NNs were trained using a database that contained 400 cases of
simulated data. Subsequently, genuine natural cracks that were found in a
steam generator tube of a nuclear power plant were measured and reconstructed
using the trained network, demonstrating good agreement with the results of
the destructive inspection carried out afterwards. Wrzuszczak [12] detected
flaws in conducting layers of aluminum and copper as well as in ferrous tubes
by using NNs. They tried MLP s and Radial Basis Function (RBF) networks and
showed, that the RBF outperformed the MLP on this task, resulting in a lower
training time.

Many other Eddy Current Testing (ECT) tasks also can be improved by the aid of
AI methods. For instance, Glorieux et al. [13] studied the electrical
conductivity profiles using NN inversion of multi-frequency eddy current data.
A MLP consisting of three sigmoid input neurons and one linear output neuron
was trained on 2000 simulated conductivity profiles which was obtained using
piecewise constant functions (20 segments). Afterwards the network was tested
on real data in form of artificially stacked samples and showed good results.
On the other hand, Rao et al. [14] used NN s to test austenitic stainless
steel welds in order to detect and characterize longitudinal and transverse
surface-breaking notches. On the basis of multi-frequency eddy current data
they evaluated the depth of the surface-breaking notches in the welds with a
maximum deviation of $0.08\text{\,}\mathrm{m}\mathrm{m}$. Kuzmin et al. [15]
applied NN s for recognizing rail structural elements in magnetic and eddy
current defectograms. They investigated three classes of structural elements
of a railroad track: (1) a bolted joint with a straight or beveled rail
connection, (2) a flash butt rail weld, and (3) an aluminothermic rail weld.
Patterns that couldn’t be assigned to these three classes were considered as
defects and attributed to a separate fourth class. The used NN contained 780
input neurons, a hidden layer with 800 neurons and four output neurons. The
training set contained 16287 real and 8323 generated samples of structural
elements of three classes and 11417 real samples of conditional defects. The
validation set contained 4098 generated samples of structural elements of
three classes and 1903 real samples of conditional defects, whereas the test
set consisted of 4098 and 1902 samples of the same kinds, respectively. The
achieved accuracy was $99.83\text{\,}\%$ on training data, $99.45\text{\,}\%$
on validation data and $99.23\text{\,}\%$ on test data. In order to identify
broken wires within a wire rope, Cao et al. [16] used a RBF network (3 input
neurons, 2 output neurons, one hidden layer) and were able to not only detect
if there was a broken wire, but also estimate how many wires were broken.
Further Ali et al. [17] gave a review on system development in ECT and
technique for defect classification and characterization as well as an
overview of AI methods used in defect measuring, while AbdAlla et al. [18]
gave a review of challenges in improving the performance of ECT as well as of
AI methods used in defect measuring.

###  2.2 Barkhausen effect

The Barkhausen effect was discovered in 1919 by the German physicist Heinrich
Barkhausen. Barkhausen noise consists of discontinuous changes in the
magnetization of ferromagnetic materials in a changing external magnetic
field. This is explained by the fact that the shifting Bloch walls are
temporarily "trapped" by the lattice perturbations and only break free from
their anchoring abruptly when the driving force is increased (Barkhausen
event). A further increase of the field strength causes the magnetization in
the domains to rotate in the direction of the external magnetic field. When
applying an external alternating magnetic field to a ferromagnetic specimen,
the resulting Barkhausen noise can be recorded with a Hall probe. Since the
Barkhausen events are tightly connected to the lattice defects in the
material, the characteristics of the recorded Barkhausen noise correlate with
the mechanical properties of the material, which are also linked to lattice
defects.

Consequently this enables the evaluation of magnetic properties of the
material under investigation, like done by Maciusowicz et al. [19], but also
the mechanical properties of the material, like Wang et al. [20] and Zhang et
al. [21] show. Consequently, this enables the evaluation of magnetic
properties of the material under investigation as well as the mechanical
properties of the material. Aiming to identify the magnetic porperties of the
test materials, Maciusowicz et al. [19] evaluated the grade and the magnetic
directions of conventional and high grain oriented electrical sheets subjected
to selected surface engineering methods. Several Machine Learning (ML)
techniques (decision trees, discriminant analysis, Support Vector Machine
(SVM), naïve Bayes, $k$-Nearest Neighbour ($k$NN), ensemble classifiers and
ANN) are compared. The achieved results generally indicated an approximately
$10\text{\,}\%$ advantage of the deep learning model over the classical ones
in terms of accuracy in each of the considered cases. For determining the
mechanical properties of the test objects, Wang et al. applied a NN for the
steel stress detection based on Barkhausen noise in order to achieve robust
results against temperature fluctuations, which have an impact on the stress
as well as on the received signal. Zhang et al. predicted yield strength,
ultimate tensile strength and elongation of cold rolled strips by the use of a
fully connected NN with two hidden layers (five input nodes, three output
nodes, five nodes in first hidden layer and six nodes in second hidden layer).
In the test results the largest relative errors of yield strength, tensile
strength and elongation are $5.66\text{\,}\%$, $2.99\text{\,}\%$ and
$3.62\text{\,}\%$, respectively.

###  2.3 3MA II and further combinations of methods

By combining several methods into one framework, the robustness, reliability
and precision with regard to the determination of mechanical properties can be
increased. An example of such a combination of methods is given by the 3MA II
device, which not only combines eddy current and Barkhausen noise
measurements, but further also measures the harmonic distortion of a pure
sinusoidal exitation as well as the incremental permeability of the material.
Thus 3MA or similar devices are limited to ferromagnetic materials, but can be
used in a wide range of applications, including but not limited to the
estimation of mechanical properties, as Jedamski et al. [22], Wang et al.
[23], Xing et al. [24] as well as Sheng et al. [25] have done in their work,
whereby Sheng et al. further combined the 3MA II with Electro Magnetic
Acoustic Transducer (EMAT).

In their work, Jedamski et al. [22] determined the hardness and the case
hardening depth using linear regression as well as NNs and compared the
results. Their sample set consists of 54 discs made from one batch of steel
AISI 4820 (DIN 18CrNiMo7-6) with a diameter of
$68\text{\,}\mathrm{m}\mathrm{m}$ and a thickness of
$20\text{\,}\mathrm{m}\mathrm{m}$. The samples were gas carburized and oil
quenched in 27 variations. The best results achieved are shown in table 2.

Table 2: Results of the estimation of hardness and carburization depth from Jedamski et al. [22], comparing the linear regression and the neural network approach |  |  RMSE Hardness (HV1) |  RMSE Carburization Depth (mm)
---|---|---|---
Training score |  linear regression |  12.5 |  0.074
neural network |  7.1 |  0.03
Test score |  linear regression |  17.9 |  0.13
neural network |  16.4 |  0.103

Both Wang et al. [23] and Xing et al. [24], predicted the surface hardness in
ferromagnetic Materials by using a device quite similar to 3MA II. Wang et al.
[23] studied the prediction of surface hardness in carbon steels, referred as
Cr12MoV steel, 45 steel, and martensitic stainless steel of 3Cr13 in Chinese
standards. For each material, a total of 27 specimens
($200\text{\,}\mathrm{m}\mathrm{m}$ × $60\text{\,}\mathrm{m}\mathrm{m}$ ×
$3\text{\,}\mathrm{m}\mathrm{m}$) were cut from a steel plate along its
rolling direction. Afterwards, quenching, followed by tempering, was
performed. The NN used to predict the surface hardness consisted of 31 input
neurons for selected magnetic measurands, one hidden layer with 50 neurons and
one output neuron. The achieved Root Mean Squared Error (RMSE) of the
predictions of surface hardness was
$7.52\text{\,}\mathrm{H}\mathrm{V}\mathrm{3}\mathrm{0}$ for the Cr12MoV steel,
$4.27\text{\,}\mathrm{H}\mathrm{V}\mathrm{3}\mathrm{0}$ for the 3Cr13 steel
and $7.44\text{\,}\mathrm{H}\mathrm{V}\mathrm{3}\mathrm{0}$ for the 45 steel.
Xing et al. [24] on the other hand, evaluated the surface hardness only on
Cr12MoV steel. For that purpose, a batch of 8 plates with dimensions of
$250\text{\,}\mathrm{m}\mathrm{m}$ x $60\text{\,}\mathrm{m}\mathrm{m}$ x
$3\text{\,}\mathrm{m}\mathrm{m}$ were cut from the cold work die steel plates
and were quenched. Afterwards, hardness (HV30) was tested on 3 positions per
plate. A simple feed forward network with an input layer for the micromagnetic
measurands, two hidden layers with 10 nodes and an output layer with one node
for the hardness was used. The MAE and Standard Deviation (SD) in surface
hardness prediction were approximately $1.02\text{\,}\mathrm{H}\mathrm{V}$ and
$1.04\text{\,}\mathrm{H}\mathrm{V}$, respectively.

In order to to predict the Tensile Strength (Rm) and surface hardness (HA) in
pipeline steel, Sheng et al. [25] further combined 3MA II with EMAT. Training
and validation data were acquired on seventeen pipeline steel samples
($300\text{\,}\mathrm{m}\mathrm{m}$ × $750\text{\,}\mathrm{m}\mathrm{m}$,
thickness > $12\text{\,}\mathrm{m}\mathrm{m}$) of various lots and grades. For
the test data an X65 pipeline steel plate with a size of
$4\text{\,}\mathrm{m}$ × $12\text{\,}\mathrm{m}$ with a tensile strength of
$670\text{\,}\mathrm{M}\mathrm{P}\mathrm{a}$ was prepared. The tensile
strength was assumed to be constant over the whole plate, hardness was tested
at 65 points distributed over the plate. The prediction results show that,
compared to MLP, Random Forest (RF), and Deep Neural Network (DNN), the
approach based on the extreme Gradient Boosting (XGBoost) was found to yield
the highest prediction accuracy in terms of RMSE and SD for the test data set,
as shown in table 3.

Table 3: Comparision of the results from different approaches to estimate tensile strength and hardness from Sheng et al. [25] |  Rm |  HA
---|---|---
|  SD (MPa) |  RMSE (HV1)
XGBoost |  17.69 |  11.85
MLP |  31.9 |  25.45
RF |  19.47 |  15.49
DNN |  18.47 |  14.22

Another example of AI aided estimation of mechanical properties based on the
combination of several techniques is given by Grönroos et al. [26], who
estimated the embrittlement of a nuclear reactor pressure vessel, which is
measured as the Ductile-Brittle Transition Temperature (DBTT). For that
purpose, they combined 3MA II with Micromagnetic Inductive Response and
Barkhausen Emission Method, Magnetic Adaptive Testing, Direct Current Reversal
Potential Drop, Thermoelectric Power Measuring Method and Ultrasonic Testing
(UT) to collect data on standard Charpy V-notch samples made out of six
different steel alloys that had been treated in different irradiation
conditions. Their data set consists of 29 nondestructively measured parameters
and ductile-brittle transition temperature data for 157 samples. Several
machine learning methods were compared: linear Huber Loss Regression (HLR),
SVR and an ANN consisting of 19 input neurons, one output neuron, 2 hidden
layers with 16 and 8 neurons respectively and two dropout layers in between.
The results in terms of RMSE, MAE and Coefficient of Determination (R2) are
shown in table 4.

Table 4: Comparision of the results from different approaches to estimate the DBTT from Grönroos et al. [26] |  | HLR | SVR | ANN
---|---|---|---|---
10-fold cross-validation score |  MAE (°C) | 13.67 | 13.58 | 12.7
Training score |  MAE (°C) | 9.32 | 8.68 | 9.21
RMSE (°C) | 14.49 | 13.63 | 12.57
R2 | 0.97 | 0.97 | 0.98
Test score |  MAE (°C) | 17.1 | 17.77 | 16.04
RMSE (°C) | 18.67 | 20.4 | 22.08
R2 | 0.95 | 0.94 | 0.93

###  2.4 3MA-X8

In comparison to the 3MA II, omitting the Hall sensor, as well as the
Barkhausen noise and eddy current coils simplified the probe to a voltage-
controlled electromagnet [4]. By exciting the electromagnet with a low- and
high frequency component and measuring the voltage and current of the probe,
features of the three methods of measurement, eddy current analysis,
incremental permeability analysis and harmonic analysis, are extracted [4].
This simplification leads to a more robust system better suited for higher
measurement speeds and sensor arrays [4]. The implementation of ML-Methods
considerably enhanced precision and robustness of the 3MA-X8 system even
further. New approaches improved the application fields of process
optimization in steel mills and reactor safety research in particular [27,
28].

Youssef et al. [29] describes initial studies on improving the accuracy by
extending the feature space. In the feature space extension, the available
time signals are analyzed in more detail by using Fourier transform, a
qualitative hysteresis reconstruction and a multidimensional curve tracing
approach, where mathematical characteristics such as high points, zero points
and axis intercepts are included as features. The underlying data was acquired
in a tensile test on three different normalized materials, 100Cr6, C45 and
S235 where each material was stressed in increments of $5\text{\,}\%$ up to
$50\text{\,}\%$ of its Yield Strength (Re). To classify the stress levels
independently of the material, a Linear Discriminant Analysis (LDA) reduces
the feature space down to two dimensions before a $k$NN-Classifier with k = 10
predicts the class. Regarding the classifiability a $k$-Fold Cross Validation
($k$FCV) with a randomized train-test-split and a k = 10 was used to determine
the accuracy before and after feature space extension. Considering the
quantifiability of the stress levels, a class based validation assigns every
second stress level to the test data set. Both, the classifiability and the
quantifiability were improved significantly, as shown in table 5 below using
the accuracy. To further describe the quantifiability, a Multiple Linear
Regression (MLR) predicts the stress level also based on a class based
validation assigns every second stress level to the test data set. The
quantifiability was also improved significantly, as shown in table 5.

Table 5: Classifiability and quantifiability comparison of the 3MA-X8 feature spaces on tensile test data [29] | LDA & $k$NN |  |
---|---|---|---
| accuracy (%) | MLR
Feature space |  Random $k$FCV |  Class based
validation |  RMSE
(% Re) |  R2
(%)
21-dimensional |  64.05 |  63.08 |  4 |  93.6
Extended |  95.14 |  94.21 |  1.96 |  98.46

Youssef et al. [30] further investigates different ML methods based on the
same experimental setup as in the previous study [29], while also comparing
the performances of the feature spaces. Specifically, it was compared whether
and with what effort the prediction quality of MLR can be achieved under
similar conditions with SVR and MLP-Regression with a single hidden layer and
linear activation functions, which are frequently used in regression problems.
Under variation of the regularization parameter c in the SVR and variation of
the number of neurons in the hidden layer of MLP-Regression, the best
performing model was chosen. The train data RMSE and R2 of these models are
shown in table 6. The extended feature space has performed significantly
better overall. However, neither the SVR nor the MLP-Regression produced
significant improvements under the conditions explained.

Table 6: Different ML methods of the 3MA-X8 feature space on tensile test data [30] Model |  Feature space |  |  RMSE (% Re) |  R2
(%)
---|---|---|---|---
MLR |  21-dimensional |  Train |  7.99 |  78.07
Test |  7.83 |  69.35
MLR |  Extended |  Train |  2.3 |  98.19
Test |  4.99 |  87.54
SVR |  21-dimensional |  Train |  8.32 |  69.85
Test |  7.62 |  66.69
SVR |  Extended |  Train |  4.11 |  93.96
Test |  4.49 |  90.33
MLP-Regression |  21-dimensional |  Train |  10.33 |  63.42
Test |  9.48 |  55.08
MLP-Regression |  Extended |  Train |  5.54 |  89.49
Test |  5.69 |  83.82

Youssef et al. [27] further published the application of ML methods with the
3MA-X8 in process optimization in steel mills, proving the successful
application of the prior investigated methodologies [30, 29] in an
industrially relevant environment. The focus was on the detection of local
hardness inhomogeneities in heavy plates for the pipeline industry, where the
task was to detect hardness differences of +/- 30 HV 10 at a minimum diameter
of $10\text{\,}\mathrm{mm}$. Supervised ML methods sufficiently suppressed
various interferences coming from residual fields and scale layers.
Unsupervised ML algorithms suppressed the lift-off influence based on a series
of measurements with a sensor lift-off variation of $0\text{\,}\mathrm{mm}$ to
$3\text{\,}\mathrm{mm}$ on sample heavy plate segments. Based on these
findings, the product "PLAMAT-MM-32M" was developed for the use in steel
mills.

Zimmer et al. [28] described the application on Reactor Pressure Vessel
(RPV)-steel, where the objective is to differentiate between stress and
microstructural influences. The materials used are 20MnMoNi5-5 and 22NiMoCr3-7
with further variation levels introduced by Low-cycle fatigue (LCF) tests and
tensile tests which leads to a more complex dataset in comparison to prior
studies [30, 29]. Regarding the features of the dataset, the extended feature
space was further extended by the raw signal samples as features themselves.
To illustrate the ambiguities between microstructure and stress influences,
fig. 1 shows a single feature of the 21-dimensional feature space. For stress-
independent microstructure characterization, a train-test-split randomly
divided the dataset into $80\text{\,}\%$ train data and $20\text{\,}\%$ test
data. Based on this dataset, an LDA performed a dimensional reduction to four
dimensions. The first two dimensions shown in fig. 2 indicate a much clearer
separation of individual material variants illustrated by the the point
clouds. Subsequently, a $k$NN-Classifier with k = 5 neighbors and uniform
weighting was trained. The extended feature space significantly improves the
model with $0.77\text{\,}\%$ misclassified data points of the test data
compared to $3.89\text{\,}\%$ of the 21-dimensional feature space. For
microstructure-independent stress determination, a MLR predicted the stress in
MPa. For the train-test split, the dataset was divided into seven segments
along the target variable, alternating between train and test data to detect
overfitting and overgeneralization. A hierarchical approach was performed,
dividing the dataset in different subgroups prior to fitting the MLR. This
approach was able to improve the models accuracy significantly in comparison
to the global models as shown in table 7.

Figure 1: 20MnMoNi5-5 stress-dependent scatterplot of feature DZmax (maximum value of incremental permeability) which is not sufficient to differentiate the material variants considered in [28] Figure 2: First two canonical components transformed by LDA of the extended feature space [28], demonstrating a clearer separation of individual material variants compared to fig. 1. "M1" refers to the material 20MnMoNi5-5 and "M2" refers to the material 22NiMoCr3-7 Table 7: Hierarchical modeling of the 3MA-X8 feature space on RPV-steel data [28] Feature Space |  Subgroup |  Train RMSE (MPa) |  Test RMSE (MPa)
---|---|---|---
21-dimensional |  Global |  43.91 |  40.10
Extended |  Global |  13.98 |  14.66
Extended |  Tensile tests |  9.94 |  11.01
Extended |  LCF tests |  9.36 |  10.87
Extended |  20MnMoNi5-5 |  8.14 |  8.93
Extended |  22NiMoCr3-7 |  10.91 |  13.95

Youssef et al. [4] systematically examined the previous approaches and
combined them into a global methodology. In order to map nonlinear
relationships between the feature space and the target variable, a polynomial
extension linearizes the feature space iteratively, until the model quality
reaches an optimum. This systematic approach was applied on the RPV-steel
materials and experimental setup of the prior paper [28]. To determine the
stress in % Re, a MLR was conducted. Regarding the train-test split, a segment
based approach was used, dividing the dataset into segments along the target
variable alternating between train and test data. Before the main model was
trained, the number of segments was determined by iteratively increasing the
number of segments, until no significant improvement in the MLR occurs. As a
result, the train-test split divided the data into 21 train segments and 20
test segments. To show the improvement of the iterative polynomial extension,
different iterations and feature spaces were examined as shown in table 8.
Comparing the eleventh polynomial grade of the 21-dimensional feature space to
the fourth polynomial grade of the extended feature space it is clear, that in
this example the use of the extended feature space yields a significantly
better result with fewer iterations.

Table 8: Iterative polynomial extension of the 3MA-X8 feature space [4] Feature Space |  Polynomial Grade |  |  RMSE (% Re) |  R2 (%)
---|---|---|---|---
21-dimensional |  1 |  Train |  6.721 |  81.0
Test |  6.567 |  78.2
21-dimensional |  11 |  Train |  3.014 |  96.2
Test |  2.863 |  95.9
Extended |  4 |  Train |  0.830 |  99.7
Test |  0.974 |  99.5

##  3 Ultrasound

Ultrasound signals, characterized by frequencies from 20 kHz to the gigahertz
range and beyond human hearing, are pivotal in computational imaging,
especially in NDE. A key technique in this domain is pulse-echo imaging, which
involves the capture and utilization of scattered echoes arising from
inhomogeneities within the measurement area.

In its simplest form, ultrasound data can be modeled via the wave equation
[31], so that the measured pressure field $p$ obeys

$\displaystyle\nabla^{2}p-\frac{1}{c^{2}}\frac{\partial^{2}p}{\partial
t^{2}}=-s.$ (1)

In (1), $p=p(t,\bm{x})$ is a pressure field that depends on the time $t$ and
position $\bm{x}\in^{d}$ for a $d$-dimensional space. The speed of sound
$c=c(\bm{x})$ may in general vary over space due to changes in density and
compressibility, and the term $s=s(t,\bm{x})$ represents sources in the
medium. Alternatively, when working with a monochromatic field or after
separation of variables, the frequency domain representation of the field
obeys the Helmholtz equation, meaning

$\displaystyle\nabla^{2}\tilde{p}+k^{2}\tilde{p}=-\tilde{s},$ (2)

where quantities with a tilde $\tilde{(\cdot)}$ are frequency domain
counterparts to the corresponding quantities without the hat. As a result,
$\tilde{p}=\tilde{p}(\omega,\bm{x})$ and $\tilde{s}=\tilde{s}(\omega,\bm{x})$
depend on the angular frequency $\omega$, and all quantities including the
wavenumber $k=k(\omega,\bm{x})$ can vary over space.

A common practice is to obtain approximate solutions to eqs. 1 and 2 by
discretizing them into systems of linear equations. In the time domain, the
problem can be formulated as

$\displaystyle\bm{p}=\bm{G\gamma},$ (3)

where $\bm{p}$ is obtained by discretizing $p$, e.g. along the temporal and
spatial axes. The matrix $\bm{G}$ is referred to as the _model_ or
_measurement_ matrix, and its formulation depends on the task. Similarly, the
vector $\bm{\gamma}$ encodes the sought-after parameters, and its exact
definition depends on the application.

Many ultrasound-based NDE techniques revolve around estimating the speed of
sound $c$, the wavenumber $k$, or the underlying physical parameters that
determine them. Quantitative and qualitative estimates of these parameters are
employed in detection, localization, imaging, classification, and more.
Additionally, the amount of information about these parameters depends on the
way in which the data is measured. As a result, the optimization of
measurement parameters is another well-researched task. We discuss recent
trends and open opportunities in the application of artificial intelligence to
ultrasound NDE tasks next.

###  3.1 Ultrasound Imaging

Conventional techniques like Plane Wave Imaging (PWI) and Full Matrix Capture
(FMC) transmit several incident waves from different angles and/or locations
and collect the scattered echoes with one or more receivers [32]. These
traditional techniques mostly depend on augmenting the number of transmitted
signals to enhance the accuracy and resolution of the reconstruction or
classification.

A mathematical model for the received signal in the ultrasound imaging can be
defined in the form of matrix-vector multiplication as

$\displaystyle{\bm{\tilde{p}}}_{\mathrm{rec}}={\bm{G}}{\bm{\gamma}}+{\bm{\tilde{\eta}}}.$
(4)

where ${\bm{\tilde{p}}}_{\mathrm{rec}}$ is the received or measured frequency
domain signal vector carrying distinct samples, ${\bm{\gamma}}$ is the
parameter of interest, and ${\bm{\tilde{\eta}}}$ represents complex additive
noise. A common choice for the construction of ${\bm{G}}$ is to employ the
first-order Born approximation and Green’s function for a homogeneous,
isotropic medium, as shown in [33]. Following this model, the inverse
scattering problem in which $\bm{\gamma}$ is estimated from the measurement
data ${\bm{\tilde{p}}}_{\mathrm{rec}}$ can be reformulated as an optimization
problem of the form

$\displaystyle\hat{\bm{\gamma}}=\operatorname*{argmin}_{\bm{\gamma}}\lambda
R(\bm{\gamma})+\left\|\tilde{\bm{p}}_{\mathrm{rec}}-\bm{G\gamma}\right\|_{2}^{2},$
(5)

where $R(\cdot)$ is a regularizer and $\lambda$ regulates its impact on the
solution.

One example for the solution of eq. 5 is the case where
$\bm{\tilde{p}}_{\text{rec}}$ is represented sparsely by ${\bm{G}}$. In that
case, $R(\cdot)=\|\cdot\|_{1}$ and $\bm{\gamma}$ can be estimated by
harnessing a reduced amount of measurement data without reducing performance;
in some cases, even by exploiting just a single acquisition from each element
of the ultrasound transducer array with optimal excitation [33, 34, 35, 36].
There are several conventional sparse data recovery algorithms, i.e. the
Iterative Shrinkage-Thresholding Algorithm (ISTA) [37] based on proximal
gradient descent, its accelerated version Fast ISTA (FISTA) [38] and the
Alternating Direction Method of Multipliers (ADMM) [39] which splits the
problem into simpler subproblems which separately focus on addressing the
reconstruction fidelity and promoting sparsity through a penalty function.
However, all of these algorithms work iteratively and come with a significant
computational cost and time expense. Hence, novel methodologies based on
neural networks have emerged that lower the reconstruction time without
compromising the reconstruction accuracy.

In the next sections, we explore different artificial intelligence approaches
to the inverse problems involved in imaging. One approach focuses on designing
conventional neural network architectures to tackle the inverse problem, while
the other involves reconstruction based on the model of the data and well-
known classical optimization algorithms.

####  3.1.1 Ultrasound Imaging with Conventional Deep Learning Architectures

A self-supervised auto-encoder network that minimizes the MSE between
$\bm{{p}}_{\mathrm{rec}}$ and $\hat{\bm{p}}_{\mathrm{rec}}$ is employed in
[40]. This network processes the projection of measurement data in the image
domain, i.e. the intermediate quantity
${\bm{\bar{\gamma}}}=\bm{G}^{\text{H}}\bm{p}_{\mathrm{rec}}$ which resembles
Synthetic Aperture Focusing Technique (SAFT). After the autoencoder network
yields the reconstruction $\bm{\hat{\gamma}}$ as training output, its
projection to the measurement domain is calculated this time as
${\bm{\hat{p}}}_{\mathrm{rec}}=\bm{G}{\bm{\hat{\gamma}}}$. The results show
that the autoencoder gives a similar Contrast-to-Noise Ratio (CNR) and the
same axial Full-Width at Half Maximum (FWHM) as conventional Delay and Sum
(DAS) beamforming using 75 Plane Waves (PWs), as well as a lower FWHM in
lateral direction, in spite of exploiting only one PW. Similarly, based on
Deep Residual Learning (DRL) [41], a method for removing the vanishing
gradient problems in the neural networks is proposed in [42]. Firstly, the
measured signal is linearly mapped to the image domain using a fully connected
network as ${\bm{\bar{\gamma}}}=\bm{W}\bm{p}_{\mathrm{rec}}$ and the elements
of the matrix $\bm{W}$ are learned using the conventional Stochastic Gradient
Descent (SGD) training algorithm. After obtaining the initial estimation of
the reconstruction data, a second algorithm employing DRL is used to obtain
$\bm{\hat{\gamma}}$ from the initial estimate $\bm{\bar{\gamma}}$. The authors
perform a comparison of Peak Signal-to-Noise Ratio (PSNR) with another
residual sparse data reconstruction network called ReconNet [43]. The
algorithm demonstrates improved PSNR and faster reconstruction than ReconNet.

Figure 3: Inversion through unfolding of classical algorithms (left) or direct
replacement via neural networks (right)

Artifact removal by means of Generative Adversarial Neural Networks (GANs) is
studied in [44]. GANs consist of two different network structures: one
generates data and the other discriminates whether the data belongs to the
labeled data or was transferred from the generator network. This process
continues until the discriminator network fails to differentiate between the
two groups. In [44], the input data fed to the generator network is obtained
as ${\bm{\bar{\gamma}}}=\bm{G}^{\dagger}\bm{p}_{\mathrm{rec}}$ where
${\dagger}$ represents the pseudo-inverse. A DRL [41] architecture is employed
as a generator network, while the discriminator uses a CNN architecture to
classify between two groups of images. Additionally, between the generator and
discriminator networks, another block is located which tunes the output of the
generator network in order to generate a feasible set of images instead of
overemphasizing the noise values of the pixels.

####  3.1.2 Ultrasonic Imaging with Model-Based Deep Learning

In ultrasound signal processing, integrating prior knowledge, such as
structure and physics, is crucial [45]. Model-based algorithms with domain
knowledge, such as message-passing techniques in ultrasound imaging [46],
offer interpretability but depend on specific expertise. Conversely, data-
driven methods, particularly Deep Learning (DL), adapt to diverse scenarios
without domain-specific knowledge [47, 48], but require extensive data and
lack explainability due to their opaque nature. Recently, _model-based deep
learning_ has been proposed to combine the advantages of both approaches [49].

Model-based deep learning, which merges prior models with DNNs, builds on
ideas from over a decade ago. An early example, the Learned ISTA (LISTA),
incorporated neural network layers into ISTA for sparse signal recovery [50].
A recent survey by Shlezinger explores this field, outlining concepts, design
strategies, and applications. The survey categorizes model-based deep learning
into two main types: _model-aided networks_ , which integrate complete DNNs
with standard model-based algorithms, such as _deep unfolding_ or _algorithm
unrolling_ [51]; and _DNN-aided inference_, which replaces parts of the
original approach with DNNs. As an example of DNN-aided inference, [52]
leverages generative models to map a lower-dimensional vector to an
approximation of the sparse vector $\bm{\gamma}$, aiding subsequent
optimization processes.

The integration of model-based deep learning into beamforming represents a
significant development in ultrasound imaging. Sloun advanced this field by
integrating deep learning methods into various aspects of ultrasound imaging,
including scanning modes and digital receive beamforming techniques [53].
Concurrently, Luijten explored the combination of traditional ultrasound
signal processing with model-based deep learning, transforming beamforming
into parameter estimation problems [54]. A model-based DNN based on Minimum
Variance (MV) beamforming was developed, achieving high-quality image
reconstruction with reduced computational demands [55, 56]. The resulting
neural network architecture, dubbed Adaptive Beamforming by Deep LEarning
(ABLE), was shown to outperform DAS and compete with MV beamforming with
regards to CNR, FWHM, and MAE at a lower computational cost.

Deep unfolding, recognized for its transparency, reliability, and simplicity,
is a prominent method in model-based deep learning, as highlighted in [57].
Originating from the Aperture Domain Model Image REconstruction (ADMIRE), this
technique has proven effective in improving image quality and suppressing
acoustic clutter, as demonstrated in studies by Khan [58]. In the context of
Contrast-Enhanced Ultrasound (CEUS), the novel model-based DNN named
Convolutional Robust Principal Component Analysis (CORONA) has been introduced
[59]. CORONA unfolds an iterative algorithm into a fixed-length architecture,
enhancing convergence and image quality.

Moreover, Compressed Sensing (CS) with model-based deep learning has
significantly advanced optimal subsampling pattern exploration. By assuming a
finite rate of innovation (FRI) structure in beamformed signals from
frequency-domain beamforming [60] or compressed beamforming by Convolutional
Beamforming Algorithms (COBA) [61], an efficient model-based DNN utilizing a
modified LISTA has been developed. This approach, as shown in [62],
considerably reduces array elements, sampling rates, and computational times
while preserving imaging quality. Furthermore, Huijben applied the Gumbel-
Softmax reparameterization trick with LISTA, introducing the Deep
Probabilistic Subsampling (DPS) approach [63] which turns the combinatorial
problem of choosing $k$ out of $n$ samples optimally into one that can be
solved with gradient-based methods. In contrast, traditional approaches such
as the ones explored in [64] rely on greedy methods and search for suboptimal
solutions in order to remain tractable in large scenarios.

Within the DPS framework, an optimal spatial subsampling design for single-
channel synthetic aperture imaging has been developed, focusing on maximizing
Fisher information [65] or minimizing reconstruction error [66]. In
multichannel imaging, the design of subsampling matrices based on information
theory-based and task-based targets was studied within the DPS framework [67].
Additionally, Joint DPS (J-DPS) has been introduced in [68] for the structured
selection of transmitters, receivers, and Fourier coefficients during
measurements. The parallel network architecture, illustrated in Figure 4,
significantly reduces trainable parameters, enhancing the practicality of the
CS pattern in hardware applications.

Figure 4: Structure of Joint DPS Algorithm

###  3.2  Computed Tomography and Full Waveform Inversion

In Ultrasound Computed Tomography (UCT), internal cross-sections of an object
are derived from ultrasound propagation models and measurements collected from
sensors on the object’s surface [69] The cross-sections are quantitative in
nature, accurately describing the physical parameters of the object under
test. Traditional UCT methods use the first-order Born approximation [70] and
linear ray models, yielding algorithms akin to X-ray imaging. For instance,
Algebraic Reconstruction Technique (ART), typical in X-ray, is applied for
slowness maps in masonry pillars, aiding in computing Young’s modulus [71].
Similarly, [72] employs Simultaneous Iterative Reconstruction Technique (SIRT)
for slowness map generation. Nonetheless, these approaches overlook critical
phenomena like refraction and multiple scattering, thereby constraining
resolution and quantitative interpretability [73].

The ultrasound community has recently adopted techniques from geophysics and
seismology, notably Full Waveform Inversion (FWI). This method emphasizes the
accurate solution of eqs. 1 and 2 to simulate realistic wave propagation [74].
It is typically framed as the nonlinear least squares problem

$\displaystyle\operatorname*{argmin}_{\bm{\gamma}}\left\|G(\bm{\gamma})-\bm{p}_{\mathrm{rec}}\right\|_{2}^{2},$
(6)

where $G$ represents a nonlinear function mapping parameters $\bm{\gamma}$ to
simulated data, typically via a differential equation solver. Alternatively,
the problem can be reformulated in the frequency domain using
$\tilde{\bm{p}}_{\mathrm{rec}}$.

FWI has shown promise in medical ultrasound, particularly in high-resolution
brain imaging [75]. Yet, its implementation is demanding, requiring
significant computational resources for precise ultrasound simulation.
Moreover, the ill-posed nature of the problem in eq. 6 necessitates effective
regularization.

Neural networks are increasingly utilized in tomography with and without FWI,
either as a component substitute or a complete replacement for traditional
methods. For instance, in [76], neural networks aid in detecting inclusions
within reactor tanks. Here, $\bm{\gamma}$ denotes the presence of inclusions.
A preprocessing step transforms the measurement data $\bm{p}_{\mathrm{rec}}$
into a time-of-arrival vector $\bm{\tau}$ which is then mapped by a
classification CNN $\Psi$ to an estimated $\hat{\bm{\gamma}}$. Similarly, [77]
employs a neural network for inversion in material parameter estimation. They
use a modified Galerkin method for simulation and a CNN for inversion,
focusing on a task-based approach with a cost function that minimizes the
difference between predicted and reference material parameters $\bm{\gamma}$.
These studies, along with others reviewed in [1], highlight the growing role
of CNNs in ultrasonic computed tomography (UCT) and non-destructive testing
(NDE).

UCT is more common in medical imaging than in NDE, yet it remains nascent in
medical applications. Nevertheless, progress in integrating UCT with
artificial intelligence is more pronounced in medical imaging. A prime example
is [78], where authors reconstruct a speed of sound map $\bm{\gamma}$ using a
small number of sensors. Differing from previous studies, they apply a network
in the image domain: a low-resolution image $\bm{\bar{\gamma}}$ obtained using
a bent ray model is refined using a U-Net [79], trained to match high-
resolution maps obtained with extensive sensor data. The training focuses on
optimizing the reconstruction fidelity
$\|\Psi(\bm{\bar{\gamma}})-\bm{\gamma}\|_{2}^{2}$. Similar approaches are seen
in NDE, such as in [80], where a CNN computes $\bm{\gamma}$ from limited plane
wave data and forward modeling is done with Matlab’s k-wave toolboox. The
rising use of CNNs in medical imaging underscores their significance in
advancing UCT applications.

A separate approach to modeling and inversion is provided by Physics-Informed
Neural Networks (PINNs) [81, 82, 83]. PINNs break away from the previous
patterns of using classical methods of forward modeling. Instead, the focus is
shifted onto the representation of solutions to the governing differential
equations. To this end, a neural network
$u_{\bm{\theta}}:(\omega,\bm{x})\mapsto{p}(\omega,\bm{x})$ with internal
parameters $\bm{\theta}$ is employed, noting that a similar formulation is
possible in the frequency domain.

The power of this approach comes from using a simple neural network, usually a
shallow Dense Network (DN) that does not explicitly depend on $\gamma$,
endowed with desirable properties through the judicious choice of a cost
function. The cost should incorporate the governing equations and all
available initial and boundary conditions. As an example in the frequency
domain, consider the Helmholtz equation. Based on eq. 2, a term of the form

$\displaystyle\left\|(\nabla^{2}+k^{2})(\tilde{u}_{\bm{\theta}})+\tilde{s}\right\|^{2}$
(7)

can be included in the cost function. Significantly, eq. 7 is expressed in a
continuous format, underscoring that $\tilde{u}_{\bm{\theta}}$ is a solution
to the differential equation everywhere in the computational domain
$(\omega,\bm{x})$. In practice, a small set of representative points is chosen
to evaluate eq. 7 at. This acts as a regularizer for the data fidelity term
$\|\tilde{u}_{\bm{\theta}}(\omega,\bm{x})-\tilde{p}_{\text{rec}}(\omega,\bm{x})\|^{2}$.
Similar terms can be formulated for the initial and boundary conditions, or
these can be learned implicitly if enough data is available.

In the case of inverse problems, the parameter $\gamma=k$ can be replaced with
a neural network $\Psi_{\bm{\phi}}:(\omega,\bm{x})\mapsto k(\omega,\bm{x})$
with internal parameters $\bm{\phi}$. After an initial training stage for
$\tilde{u}_{\bm{\theta}}$ with data from a known reference medium, one can
then train $\tilde{u}_{\bm{\theta}}$ and $\Psi_{\bm{\phi}}$ simultaneously
from measurement data of the object of interest by considering the problem

$\displaystyle\operatorname*{argmin}_{\bm{\theta},\bm{\phi}}\left\|(\nabla^{2}+\Psi_{\bm{\phi}}^{2})(\tilde{u}_{\bm{\theta}})+\tilde{s}\right\|^{2}+\lambda\left\|\tilde{u}_{\bm{\theta}}-\tilde{p}\right\|^{2}$
(8)

with possible additional terms accounting for boundary and initial conditions.
An example of this approach can be found in [84], where PINNs are employed in
the estimation of speed of sound maps to locate cracks with high accuracy
based on time domain data.

The tomography and FWI approaches discussed in this section can roughly be
categorized as shown in table 9.

Table 9: Classification of ML-aided UCT and FWI approaches Approach |  Forward method |  Inverse method
---|---|---
Direct |  classical |  CNN
Two-step |  classical |  preprocessing + CNN
PINN |  DN |  DN

###  3.3 Ultrasound Data Classification

In the NDE 4.0 era, the fusion of DL and ML with ultrasound defect
classification has markedly enhanced the performance and efficiency of
industrial applications, resulting in algorithms that are smarter, more robust
and accurate, and more cost-effective. As with many data-driven methods, the
success of classification methods in UT hinges on the availability and quality
of training data [85]; the topic of data augmentation and synthesis is
addressed in section 3.5. This section aims to provide an overview of recent
advancements in ultrasound classification within NDE, highlighting works that
integrate feature extraction with classification techniques. Feature
extraction is critical in ultrasound, where sound data can be represented in
various ways. This step quantifies key data characteristics, aiding
classification algorithms in accurately distinguishing between classes. The
effectiveness of combining signal processing methods, such as time-frequency-
scale signal representations and dimensionality reduction, with DL networks
and ML classifiers, is demonstrated in the examples provided.

The Short-Time Fourier Transform (STFT) and wavelet transform, providing
simultaneous time and frequency domain information of signals, are integral in
ultrasound classification for feature extraction [86]. Despite STFT being a
traditional technique, its recent applications in NDE, specifically in
structural health monitoring of railway tracks using CNNs, are notable [87].
Regarding the identification of rail defects, the wavelet transform has also
proved to be a valuable tool for feature extraction. In [88], wavelet
coefficients, energy, and local entropy are utilized for extraction of the
representative features of the data alongside Kernel Principal Component
Analysis (KPCA) for the dimensionality reduction algorithm. The authors of
[88] propose utilizing SVM in the classification task. In [89], three ML
classification techniques, in particular adaptive boosting [90], XGBoost [91]
and SVM [92], are compared by accompanying wavelet energy features for
classifying the defects with laser ultrasound signals.

Addressing another crucial NDE subsector, in [93], Discrete Wavelet Transform
(DWT) is applied in a pipeline weld inspection using EMAT. The high-level
features from the time-frequency representation of A-Scan signals are used to
classify the welding defects in a gas pipeline. The wavelet features are then
given into a CNN-SVM classifier. Different solutions to the problem of
classifying ultrasound signals in welding applications are proposed in a
similar manner by employing the wavelet coefficients-based extracted features
in [94] with Artificial Bee Colony (ABC)-SVM and in [95] with CNNs. Another
efficient implementation combining the extracted wavelet coefficients and CNNs
is presented in [96] and [97] for identifying and monitoring the cracks in
concrete, respectively.

The impact of the wavelet decomposition on classification accuracy is
demonstrated in [98] by comparing two methods: one using Long Short-Term
Memory (LSTM) [99] directly on the ultrasonic signals, and the other one
applying LSTM after the wavelet feature extraction carried out. Yet another
strength of the wavelet transform is extracting definitive features so that
they can be used with unsupervised machine learning algorithms such as
Gaussian Mixture Modeling, Mean Shift Clustering, and K-means Clustering
[100]. Furthermore, apart from taking the wavelet coefficient-based values
directly as features, the usage of wavelets in reliable feature extraction can
be utilized as a preprocessing step as well. In [101], the wavelet transform
is carried out to denoise data in the context of detecting the possible
damages in wind turbine blades.

Wavelet-based feature extraction is diversified through methods like Empirical
Wavelet Transform (EWT), which adaptively decomposes signals into a finite
number of components without requiring prior time-frequency information [102].
The authors illustrate the combination of EWT with CNNs for classifying phased
array ultrasonic test signals. EWT draws inspiration from Empirical Mode
Decomposition (EMD), a method for empirically deconstructing data into
intrinsic oscillatory modes, capturing local time-scale signal information
[103]. EMD was applied alongside DWT and other features, like entropy and
crest factor, for acoustic emission signal-based wood damage and fracture
classification, using LDA for classification [104]. Additionally,Variational
Mode Decomposition (VMD) distinguishes itself by decomposing ultrasound
signals through a non-recursive, constrained variational problem-solving
approach [105]. VMD has been applied in ultrasonic measurement classification,
such as in wood hole-damage [106] and for signal denoising prior to feature
extraction [107].

Ultrasound signals, when represented sparsely using spatial sparsity and
dictionary matrices [33], show promise in classification accuracy. The authors
demonstrate this by replacing traditional iterative sparsity-promoting
algorithms with a CNN for classifying low-quality, sparse reconstruction data
[108]. This transformation involves a simple projection from the measurement
to the image domain. The CNNs, configured in various architectures like
LeNet-5 [109] and CaffeNet [110], outperforms the Smashed Filters algorithm
[111], which uses Maximum Likelihood Estimator (MLE) for manifold
classification, in terms of accuracy.

Shifting from signal processing to purely data-driven methods, a study [112]
applies a Fully Convolutional Neural Network (FCN) and a Gated Recurrent Unit
(GRU) for feature extraction in ultrasound signals, followed by a softmax
layer for classification. The authors of [113] employ an autoencoder to
discern defect-free signal characteristics and identify anomalies upon defect
presence. In contexts where ultrasound data is sparsely represented, the
authors demonstrate that classifying compressed or subsampled data using soft-
margin SVMs, yields results as accurate as those obtained using the original
fully sampled data [114]. Notably, this approach obviates the need for data
reconstruction, significantly reducing computational complexity, particularly
when the original data size substantially exceeds the subsampled one.

Last but not least, a comprehensive analysis of the recent enhancements in NDE
with ultrasound data using numerous DL techniques utilized as feature
extraction methods, such as autoencoders, GRUs, Recurrent Neural Networks
(RNNs), CNNs is discussed in [115].

###  3.4 Dictionary Learning

In ultrasound imaging inverse problems, selecting an appropriate dictionary
for the input data is crucial in finding low dimensional representations of
data, as well as in inverse problems such as imaging and denoising.
Dictionaries can be designed by reducing the mutual coherence of the
dictionary matrix, through standard optimization, or via learning techniques.

The $k$-SVD [116], a prominent method for dictionary optimization, is based on
Singular Value Decomposition (SVD). It employs an alternating minimization
approach, updating one dictionary matrix column and one data matrix row
sequentially, while keeping other variables fixed. Notably, the $k$-SVD
employs an approximate error matrix for updating the column of the dictionary
matrix, rather than the actual one. In the study [117], this limitation is
addressed in the context of sparsifying ultrasound elastography images by
using the first principal component from the actual error matrix’s Principle
Component Analysis (PCA), in contrast to the $k$-SVD’s use of the approximate
error matrix’s SVD.

Wavelet-based representation images is common in UCT and Magnetic Resonance
Imaging (MRI), but the search for suitable dictionaries is ongoing since
images synthesized from wavelets are not necessarily in the category of
natural images. In [118], authors explore a better dictionary representation
for ultrasound tomography and MRI. They employ an optimization method based on
[119], which is an iterative alternating minimization method based on the loss
function of the basis pursuit denoising problem. This method alternates
between fixing the dictionary matrix and the image data, resulting in a 2dB
improvement in PSNR level.

###  3.5 Synthetic ultrasound data

The generation of synthetic data increasingly attracts interest in NDE
applications such as UT. On one hand, the validation of inspection procedures
through sizing and classification of defects as well as Probability of
Detection (PoD) requires large amounts of measurement data representing
various types of inspection modalities and defects [120]. On the other, the
training of ML models requires volumes of labeled data pertaining to a
particular application which in practice are difficult to obtain. This section
discusses the augmentation and synthesis of measurement data using AI
techniques.

Pyle explored uncertainty quantification in DL for ultrasonic crack
characterization, employing a CNN to assess surface-breaking defects in PWI
images [121]. Two uncertainty quantification methods were applied: deep
ensembles and Monte Carlo dropout. The network, trained on 14,343 simulated
PWI images of surface-breaking defects generated via a hybrid Finite Element
(FE)/ray-based model, was further calibrated using a mix of simulated and
experimental images. Their findings indicated that while Monte Carlo dropout
showed limited effectiveness in uncertainty quantification, deep ensembles
demonstrated superior calibration and anomaly detection capabilities.
Enhancements to deep ensembles, including spectral normalization and residual
connections, further improved calibration and the detection of out-of-
distribution samples. Additionally, Gantala explored crack detection using six
synthetic Total Focusing Method (TFM) imaging datasets, created through a
combination of Finite Element Method (FEM) simulation, GAN-based methods, and
realistic noise extracted through the sliding kernel approach, to enhance an
Automated Defect-Recognition (ADR) in butt-welds [122]. Utilizing a fine-tuned
pre-trained YOLOv4 network [123], they achieved a $100\text{\,}\%$ PoD in
defect detection and less than $10\text{\,}\%$ false-calls on the hybrid
dataset without noise, and $90\text{\,}\%$ PoD with less than $17\text{\,}\%$
false-calls on the noisy dataset.

Posilovic applied GANs to create ultrasonic images that mimic real ones, using
a dataset of scanned steel blocks with artificially induced defects [124].
This dataset consists of 3825 images, comprising a total of 6238 annotations
and 4283 defect patches after manual filtering. They explored three synthetic
image generation methods: two deep learning-based GAN approaches and a
traditional copy/paste technique. The first GAN method, detectionGAN, is a
U-Net generator integrated with two PatchGAN discriminators [125] and a
pretrained YOLOv3 [126] object detector. The second method, SPADE GAN [127],
modifies the pix2pixHD [128] architecture by incorporating a pretrained YOLOv3
discriminator. Their methods were assessed through tests conducted by highly
trained human experts.

Furthermore, Singh [129] and Ni [130] presented novel applications of
generative AI in ultrasound imaging and elasticity modulus identification,
respectively. Singh developed a DNN framework for real-time super-resolution
mapping in ultrasonic non-destructive evaluation, using full aperture and
pitch-catch transducer configurations to reconstruct crystallographic
orientation maps. They employed a GAN with a U-Net generator and PatchGAN
discriminator, achieving a fourfold increase in resolution and up to
$50\text{\,}\%$ improvement in structural similarity. Ni focused on solving
the inverse problem in elasticity, utilizing a conditional generative
adversarial net to create shear modulus distributions from strain images.
Their approach involved a U-Net architecture to implement a conditional GAN
generator [131], and a PatchGAN classifier for discrimination.

Alternatively, since ultrasound propagation obeys physical laws, solving the
wave equation (1) is another approach for UT data synthesization. Given a
reference speed of sound distribution $c(\bm{x})$ and proper boundary
conditions, the solution to the wave equation corresponds to the data that
would be measured from the corresponding specimen. Conventionally, Partial
Differential Equations (PDEs) including wave equations are solved using
classical numerical methods such as finite elements or spectral approaches
[122, 121]. Due to their robustness and sophisticated algorithms, such
numerical methods play crucial roles when it comes to data synthesization.
Yet, their high computational cost may become prohibitive especially when
dealing with a large 3D region. Since the object surface needs to be
specified, dealing with complicated geometry is also challenging with such
numerical methods.

Outside of their applications in inverse problems such as FWI and UCT, PINNs
have also shown significant promise purely in forward modeling, i.e. in
enhancing numerical methods for solving the wave equation [81]. Moseley
demonstrated that a simple 10-layer shallow MLP can effectively simulate 2D
wave propagation in inhomogeneous media [82]. Song tackled the challenge of
balancing physics and boundary loss in PINNs by integrating perfectly matched
layers into the Helmholtz equation, achieving impressive results in inferring
2D wave fields with anisotropic wave speeds in inhomogenous media [132]. Offen
explored a variational approach to PINNs, successfully inferring 1D wave
propagation in a homogeneous medium with reflective boundaries [133]. This
approach reformulates a second-order PDE as a first-order differential
equation, providing an effective alternative for time-domain wave equation
solutions. For a comprehensive overview of PINNs, the review by Karniadakis is
recommended [134].

##  4 Thermography

Infrared thermography has established itself as a valuable technique in non-
destructive evaluation due to its ability to provide prompt, swift, and cost-
effective information. We will explore both state-of-the-art conventional
methodologies and their AI-based counterparts in this field. Leveraging AI,
thermography effectively eliminates blurring and reconstructs initial
temperature profiles. A pivotal focus is defect detection using DL
architectures, discussed comprehensively in this section.

###  4.1 Pulsed Phase Thermography

Pulsed Thermography (PT) [135] is one of the popular stimulus methods of
Thermographic NDE (TNDE). The specimen undergoes a pulse of external thermal
stimulus, and the resulting temperature decay is captured by an Infrared (IR)
camera. This pulse of energy can be generated using high-power amplifiers,
flash lamps, laser beams and water jets. The setup can be deployed in two
modes: reflection and transmission. In reflection mode, both the source and
the detector are positioned on the same side of the sample. In contrast, in
transmission mode, the source and detector are located on opposite sides of
the sample to be inspected. The effectiveness of this method varies based on
the pulse duration, ranging from a few milliseconds for materials with high
thermal conductivity, such as metals, to a few seconds for materials with low
thermal conductivity, like wood. This allows for the inspection of different
materials using the TNDE approach.

In a qualitative assessment, when a pulse is applied to the specimen, its
temperature rises during the application of the pulse. Subsequently, the
temperature decreases as a thermal front propagates beneath the surface,
following the principles outlined in the Fourier diffusion equation, which is
given by

$\frac{\partial T}{\partial t}=\alpha\nabla^{2}T.$ (9)

Here, $\frac{\partial T}{\partial t}$ is the is the rate of change of
temperature with respect to time, $\alpha$ is the thermal diffusivity of the
material and $\nabla^{2}T$ represents the Laplacian of the temperature field,
indicating how temperature varies spatially. The presence of subsurface
defects modifies the diffusion rate, resulting in temperature variations above
these defects when compared to the surrounding undamaged areas. Consequently,
thermal differences can be detected in thermal images above defect areas.
Moreover, the diffusion rate is time-dependent, leading to delayed and less
pronounced manifestations of deeper defects. The expressions

$\begin{split}t\propto\frac{z^{2}}{\alpha}\\\
c\propto\frac{1}{z^{3}}\end{split}$ (10)

establish the relationship among the observation time $t$, the thermal
contrast $c$, the defect depth $z$, and the thermal diffusivity $\alpha$.

In Pulsed Phase Thermography (PPT) the sample is stimulated as in PT technique
whereas the analysis is similar to Modulation Thermography (MT) technique
[136]. In this scenario, a brief rectangular heat pulse is applied to the
sample, and its reaction is observed through an infrared camera. This pulse
has a wide range of frequencies, which can be separated using the Fourier
transform. The data analysis involves choosing a specific frequency, akin to
synchronizing the output signal with the modulated input signal, similar to
the process in MT. The Fourier transform of every pixel (i, j) in the imaging
array is then computed along time domain axis.

The energy of the stimulating pulse is primarily concentrated in the lower
frequency range of the spectrum. According to eq. 10, lower frequencies can
penetrate deeper, while higher frequencies are confined closer to the surface.
Hence, it is appropriate to select lower frequencies for analysis.
Additionally, it has been observed that the phase image remains unaffected by
the surface properties of the material and the non-uniformity of the source,
unlike the amplitude image. However, in PPT analysis, time information is lost
while retrieving frequency data. Combining time information with frequency
data can be immensely valuable for depth estimation analysis. Advanced
techniques such as wavelet analysis, neural networks, synthetic data, and
statistical methods can be employed for such analyses.

In the current literature, investigations have been conducted to detect and
evaluate defects by modifying the PPT algorithm. In [137] two modified
algorithms were introduced. The first algorithm integrates a frequency-
dependent Gaussian window function, while the second algorithm utilizes a
rectangular window function. The algorithms underwent testing using synthetic
signals that modeled a subsurface cylindrical defect in a plate, and
experimental testing was conducted through PT on steel and a polymer sample.
The outcomes revealed a notably improved contrast-to-noise ratio for defects,
particularly at higher analysis frequencies.

In the general context, PPT necessitates the assessment of multiple-phase
images at various frequencies for a comprehensive sample evaluation. However,
the selection and subsequent evaluation of these images are encumbered by
certain challenges and are not always straightforward. In efforts to
ameliorate these limitations, [138] introduced an Adaptive Spectral Band
Integration (ASBI) processing technique. This approach surpasses the
capabilities of PPT, yielding an enhanced signal-to-noise ratio even for
defects that are barely visible. Furthermore, it produces a single index map
of defects for a given sequence, simplifying the interpretation process.

###  4.2 Principal Component Thermography

PCA is a widely used multivariate statistical technique designed to reduce the
complexity of data. Through linear transformations, PCA identifies the key
features within high-dimensional data and maps them onto a lower-dimensional
space. This approach offers several advantages, including straightforward
implementation, feature extraction, and noise reduction.

In the realm of Infrared Thermography (IRT), Principal Component Thermography
(PCT) [139, 140, 141] employs a method involving the singular value
decomposition of the covariance matrix derived from thermograms. This process
effectively reduces the dimensions of thermographic data and captures
significant data changes. Simultaneously, by projecting the original data onto
the directions of maximum variability, PCT efficiently diminishes noise
present in the thermographic data.

Replying solely on PCT may not yield satisfactory results. Consequently,
numerous defect inspection methods based on deep learning have been proposed
in recent times. One such advancement is the Spectral Normalized GAN (SNGAN)
algorithm [142]. The complexity of the generator (G) and discriminator (D) in
traditional GANs often leads to challenges like vanishing or exploding
gradients during training. The SNGAN tackles these issues by incorporating
spectral normalization, a technique that normalizes the weight matrix in the
discriminator [143]. This approach confines the spectral radius of the weight
matrix within a fixed range, effectively preventing issues like gradient
disappearance or explosion. By implementing this normalization, the training
stability of GANs is significantly enhanced, leading to the generation of
higher-quality images. The architecture of the SNGAN-based generative
principal component thermography (GPCT) model is depicted in fig. 5. In this
proposed approach, the traditional PCT method involves combining the generated
images with the original thermal images, followed by PCA data analysis.

This integration enhances the overall defect detection process and ensures
more accurate and reliable results.

Figure 5: Framework of the GPCT model.

###  4.3 Thermographic Signal Reconstruction

Originally developed for pulse thermography, the Thermographic Signal
Reconstruction (TSR) processing technique [144] consists in the fitting of the
experimental log-log plot thermogram by a logarithmic polynomial of degree
$n$. Using $\Delta T=\Delta T(t)$ to represent the temperature increase as a
function of time (thermogram) for a single pixel $(i,j)$, the temperature
increase is modelled as

$\log_{10}(\Delta
T)=a_{0}+a_{1}\log_{10}(t)+a_{2}\log_{10}(t)^{2}+\ldots+a_{n}\log_{10}(t)^{n}.$
(11)

This fitting method replaces the entire sequence of temperature rise images
$\Delta\bm{T}(t)$ with a series of $(n+1)$ images $\bm{A}_{m}$, $0\leq m\leq
n$, representing the polynomial coefficients, so that $\bm{A}_{m}(i,j)$ is the
$m$th coefficient at pixel $(i,j)$. These coefficients facilitate the
reconstruction of an entire thermographic sequence. The computation of the
initial and subsequent logarithmic derivatives of the thermograms is directly
conducted on the polynomial, leading to minimal temporal noise augmentation.

The procedures for fitting and deriving thermograms depend on the specific
temporal domain being analyzed. It’s essential to define a precise time window
to isolate the segment of thermograms affected solely by the particular
physical phenomena under examination. The traditional use of TSR involves
selecting optimal derivative images corresponding to distinct depth ranges.
Empirically, from [145] it was determined that a degree of $n=11$ adequately
represents delamination-like defects.

In a few recent studies, Schager et al. [146] investigated an expanded TSR
method aimed at automatically categorizing defects found within the undamaged
domain. The approach creates defect maps by utilizing the signal attributes of
flaws. The results of the research suggest that the defect map offers an
estimate of the defect’s depth and is computationally efficient.

Feng et al. [147] introduced a hybrid approach that combines TSR with the
Automatic Seeded Region Growing (ASRG) algorithm for thermal image signal
processing. The study emphasizes that the selection of defective regions is
sensitive to high image resolution, which can enhance detectability. The study
showcased enhanced detection resolution and proved valuable for automatically
selecting regions suspected of harboring defects. Ratsakou et al. [148]
proposed a novel approach for defect characterization, blending TSR with the
Canny shape-reconstruction algorithms. The method begins by fitting the raw
thermographic images to a low-degree polynomial in the log-log representation
of the time axis. Subsequently, the Canny algorithm is employed to reconstruct
the original signals. By harnessing the capabilities of the TSR algorithm for
signal denoising and compression, the method can improve the efficiency of the
reconstruction process.

###  4.4 Thermographic Image Reconstruction

The techniques mentioned in the preceding sections do not exploit spatial
information and physical knowledge of the problem simultaneously, resulting in
limited spatial resolution. In the case of TSR, each pixel is traditionally
processed separately. In PCT, spatial information is considered only in the
context of forming low dimensional vector spaces. An alternative approach is
to perform thermographic image reconstruction by directly addressing the heat
equation in (9) and solving an Inverse Heat Conduction Problem (IHCP) to find
the initial temperature distribution in the object under test. Such an
approach is comparable to FWI in ultrasound imaging and comes with similar
challenges, e.g. considerable computational cost and ill-posedness [149].

A recent advancement in thermographic imaging is the transformation of the
heat conduction problem into a wave propagation one by means of virtual waves
[150], after which the usage of the reconstruction techniques covered in
section 3 becomes possible. In particular, synthetic aperture techniques
address both the limited spatial resolution of TSR and PCT, as well as the
ill-posedness of IHCPs: the incorporation of a simplified model of wave
propagation provides robustness and improves to spatial resolution. The
inverse mapping from the measured temperature profile $T(t,\bm{x})$,
$\bm{x}\in^{d}$, to the desired initial temperature distribution $T(0,\bm{x})$
is performed through an intermediate virtual wave field $p(t,\bm{x})$.
Classical and AI-based methods can be employed at any step of the process,
resulting in increased flexibility. The authors of [151] exemplify this
flexibility by comparing three approaches to thermographic imaging: virtual
waves with classical ultrasound imaging methods, an end-to-end U-net
$\Psi:T(t,\bm{x})\mapsto T(0,\bm{x})$, and a two-step approach in which the
virtual wave mapping $T(t,\bm{x})\mapsto p(t,\bm{x})$ is done in preprocessing
and later fed to a neural network $\Psi:p(t,\bm{x})\mapsto T(0,\bm{x})$. The
authors show that both AI-based approaches outperform the classical one, and
that the usage of virtual waves in preprocessing increases reconstruction
accuracy in exchange for increased inference cost.

###  4.5 Defect Shape Detection

Neural networks for semantic segmentation prove valuable across diverse tasks,
such as autonomous vehicles, medical image diagnostics, and other combined
applications[152, 153]. In the case of thermographic imaging, semantic
segmentation allows the detection and localization of abnormalities, e.g.
tumors in medical imaging [154] and defects in NDE [155].

In [156], semantic segmentation is performed on induction PPT images of forged
steel parts by using a U-net. The network is trained by using binary masks
denoting the presence and location of cracks. The authors note that, since the
cracks are comparatively small in the Region of Interest (ROI), imbalances in
the classes must be addressed through weighting in the loss function. As is
common in the NDE context, augmentation through rotations, contrast
modifications, and elastic transformations was necessary.

##  5 Optical Camera

A significant fraction of NDE tasks can be solved through optical inspection
methods, i.e., based on optical sensors using image processing methods. Such
optical systems can consist of cameras, lasers, scanners, or a combination of
the above. However, machine vision with a camera is the most common. Figure 6
shows the typical workflow of a camera system [157].

Test SpecimenCamera & IlluminationComputerFeature
ExtractionClassificationControllerImage Acquisition Figure 6: Setup of a
camera system

A camera and a suitable (active or passive) illumination are used to take one
or multiple images of the specimen, which are subsequently transferred to the
computer. The image matrix $\bm{P}$ for a grayscale image can be represented
as $m\times n$ matrix where each entry $\bm{P}(i,j)$ represents the brightness
value of the pixel located at the $i$-th row and $j$-th column of the image.
For color images, each color plane is described like this and written into a
three dimensional array. Afterwards, feature extraction is applied. Feature
extraction in an image can be mathematically represented as a function
$f(\bm{P})$ that transforms the original image into a set of features or
vectors:

$f(\bm{P})=\\{\bm{f}_{1},\bm{f}_{2},\ldots,\bm{f}_{k}\\}$ (12)

Here, $\bm{f}_{1},\bm{f}_{2},\ldots,\bm{f}_{k}$ represent the extracted
features. These features can be of various types, depending on the type of
analysis. In NDE, relevant features could be e.g. the edges of a scratch,
certain color characteristics indicative of corrosion, etc. Then the data can
be classified. The classification process can be mathematically represented as
a function $c(f(\bm{P}))$ that maps the features to a specific class or label:

$c(f(\bm{P}))\in\mathcal{C}$ (13)

Here, $\mathcal{C}$ is the set of all classes under consideration, and $c$
yields a class or label for the image or region based on the extracted
features. According to the above examples, the question can be clarified
whether, for example, the edges found have the characteristics of a scratch,
or whether the color features found correspond to those of corrosion. After
the data evaluation, the process chain ends with a controller, which makes a
certain decision depending on the class.

Nowadays, thanks to the advanced development of neural networks, many of these
problems can be solved via Deep Learning. A simple scheme is shown in fig. 7.

Input LayerInput Image Hidden LayerConvolutional Pooling Fully Connected
Output LayerPredicted Classes  Figure 7: Convolutional Neural Network (CNN)
Architecture

In the example, the input image goes through many different convolution
operations, a common approach in image processing to establish shift
invariance of detected objects across the image. During this process,
corresponding feature maps are created. The feature maps are compressed by the
pooling layer. The fully connected layer connects the different maps so that
the desired classification is obtained in the end.

###  5.1 Image Preprocessing

Image preprocessing means the intention to achieve image enhancement in
conjunction with formatting the image if necessary. Such operations could
include smoothing the image, adjusting the contrast, resizing the image,
formatting in a different color range, eliminating the background and more.
The various operations depend very much on the actual application. This
section briefly introduces the most common operations in general. For deep
learning, image preprocessing is essential for technical, qualitative and
performance reasons.

Smoothing filters [158] are used to suppress noise and individual disturbing
pixels. A distinction is made between linear and non-linear filters. Linear
smoothing filters [158] can be described as

$\bm{G}(x,y)=\frac{1}{s}\sum_{r=0}^{R-1}\sum_{c=0}^{C-1}f_{cr}\bm{P}(x-d_{x}+c,y-d_{y}+r),$
(14)

where $\bm{G}(x,y)$ is the filtered image at position $(x,y)$,
$\bm{P}(x-d_{x}+c,y-d_{y}+r)$ is the pixel value of the input image at
position $(x-d_{x}+c,y-d_{y}+r)$, which is weighted by the filter coefficient
$f_{cr}$. The quantity $s$ is a normalization constant and $d_{x}$ and $d_{y}$
are parameters which control the size of the filter. $R$ is the number of rows
and $C$ the number of columns in the image. The row and column indices are
represented by $r$ and $c$, respectively.

Non-linear smoothing filters [158] are ’Rank-Order Filters’. Here, a certain
number of neighboring pixels are sorted into a row depending on their
intensity. A specific pixel is then selected. A common Rank-Order Filter is
the so-called Median filter [158], which always selects the median intensity
and thus makes disturbing pixels disappear completely.

Regarding image enhancement, there are many different ways to improve the
contrast of images. A general solution for this is the so-called histogram
normalization which can be described via

$\bm{G}(x,y)=\frac{L-1}{MN}\sum_{i=0}^{L-1}h(i)$ (15)

where $\bm{G}(x,y)$ is the enhanched image, $L$ is the number of intensity
levels, $M$ is the image width and $N$ the height. Additionally, $h(i)$ is the
count of the $i$-th intensity in the original image. For color images, these
procedures are carried out for each individual color plane.

Moreover, for deep learning, image resizing is very important, as neural
networks need the input images in to be in a homogeneous format. By a scaling
factor $s$ the dimensions of the resized image can be calculated based on the
original image. Then by applying an interpolation technique the old pixel
coordinates can be mapped into the new image coordinate system.

As fully connected layers in convolutional neural networks require the same
format for all images, also a color transformation may be necessary. In
general we can describe it as follows

$\bm{G}(x,y)=\bm{T}[\bm{P}(x,y)]$ (16)

$\bm{T}$ is the transformation function which applies a transformation to the
color values of each pixel. The transformation function can be defined based
on the requirements of the wished color format.

###  5.2 Feature Extraction and Classification of Optical Images

The first step in feature extraction is to define a region of interest in
which to search for the desired features. This can be achieved, for example,
using threshold methods, edge detection techniques or cluster algorithms.
Representative features that characterize the image content are then selected.
These can be color information, edges, shapes, textures or specific patterns.
The extracted features are then described as a vector as in eq. 12. The
vectors are then filtered again to reduce the amount of data or to discard
irrelevant features. Finally, the vectors may also need to be normalized to
make them comparable. Then the extracted features can be analyzed so that
based on the application the images can be classified.

Figure 8 exemplifies feature extraction in optical images by considering a
section of the head of a screw. A line is plotted here and the adjacent
graphic shows the corresponding line profile. The corresponding drop in
intensity from the line function represents the edge of the object and so a
corresponding feature can be extracted. If the entire screw is scanned in this
way, the contour of the object can be determined and the object can be
classified as a screw accordingly. In the context of Deep Learning, CNNs can
be trained to register such patterns automatically. In the following
subsection, we review the application of DL to optical images in detail.

(a) head of screw

(b) line profile

Figure 8: Example of feature extraction

###  5.3 Deep Learning based Feature Extraction and Classification

Deep learning was revolutionary in image processing. By training neural
networks, they can acquire an intuition that is similar to humans’ and thus
solve highly complex tasks. Image files are usually processed with
convolutional neural networks. These networks are predestined for extracting
features from images and then classifying them. Tasks such as object
recognition, segmentation, finding key points and much more can be easily
solved using convolutional neural networks.

There are many different architectures for this, Region Based Convolutional
Neural Networks (R-CNNs) [159] among them. A Region Proposal Network (RPN) is
used to find regions that are likely to contain objects. A CNN architecture is
then used to classify and fine-tune the bounding boxes. Ren et. al [160]
introduces the Faster R-CNN. They use as CNN architecture the VGG-16 [161] for
the classification. They tested it on the PASCAL VOC 2007 and 2012 [162]
dataset. The best results were achieved when the model was trained with the
COCO dataset [163] and the PASCASL VOC 2007 and 2012 dataset. For the VOC 2007
test data a mAP of 78.8 % were achieved and for the VOC 2012 test data the mAP
amounted to 75.9 %.

Another architecture is Single Shot MultiBox Detector (SSD) [164]. This model
uses the VGG-16 architecture as base, but disposes of the fully connected
layers. Instead of these, further convolutional layers are added so that
several feature maps with different resolutions can be generated. All these
maps are taken into account for classification. This not only makes it
possible to reliably find and classify the objects, but also to match the
object shape when generating the bounding boxes due to the large number of
feature maps at different levels. Liu et. al [164] introduced this model. In
their paper they differentiate between SSD300 and SSD512, where the number
indicates the input size of the images (300x300 vs 512 x 512). Here too, the
models were tested with the PASCAL VOC 2007 and 2012 dataset. Again the best
results were achieved if the models were trained with the VOC 2007, VOC 2012
and COCO dataset. The SSD300 achieved a mAP of 79.6 % for the VOC 2007 testset
and a mAP of 77.5 % for the VOC 2012 testset. The SSD512 achieved better
results due to the higher resolution of the input images. For the VOC 2007
data the mAP amounts to 81.6 % and for the VOC 2012 testset it is 80.0 %.

A widely used method for real-time object recognition is YOLO [165]. Here, the
image is divided into grids and a prediction is made for each grid. YOLO is
based on CNNs (see Figure 7). In 2015, Redmon et al. [166] introduced YOLO for
the first time. They trained the model with the VOC 2007 and 2012 dataset. For
the VOC 2007 data it then achieved a mAP of 63.4 %- and for the VOC 2012
dataset the YOLO scored 57.9 % mAP. Over the years, new and improved versions
of YOLO were published. YOLOv8 [167] is the latest release. The largest
version of the model (YOLOv8x) achieved the best results. It was trained with
the COCO 2017 dataset and achieved a score of 53.9 %. Moreover YOLOv8 also
provides models which perform only classification (without bounding boxes),
instance segmentation and pose estimation.

In addition to object recognition, image segmentation is also often required.
This can be achieved by extending the object recognition models. An example of
this is Mask R-CNN. Mask R-CNN [168] [159] is an approach to Instance
Segmentation. It uses the Faster R-CNN [160] [159] architecture for object
detection and a FCN [169] for semantic segmentation. He et al. trained and
tested a Mask R-CNN model with the COCO dataset. The model reached a score of
37.1 AP. In comparasion to that the YOLOv8x-seg model achieved a mAP score of
43.4. This model was trained and tested with the COCO 2017 dataset.

All these applications are extremely valuable in NDE tasks. For instance in
Katsamenis et. al [170], corrosion detection is carried out. Among other
models, Mask R-CNN is used here. Transfer learning was used to train the Mask
R-CNN model. The Mask R-CNN model is based on Inception V2 [171] and was pre-
trained on the COCO dataset. Then a dataset consisting of 116 images was used
to train and test the model. The images were captured using various devices
and have different resolutions. Furthermore, the images contained different
types of corrosion. The precision achieved was just under 0.7 and the F1-Score
achieved was just over 0.7. However, the results were improved by combining
them with color segmentation to apply boundary refinement. This shows the
importance of classical techniques such as those in Section 5.2 in order to
combine AI results with such procedures if necessary.

##  6 Common Patterns in Algorithms and ML Models

So far we presented both classical and ML based approaches separately for each
inspection modality. In the course of investigation, however, we repeatedly
encounter some algorithms and/or ML architectures regardless of the inspection
modalities or even application fields. For instance, CNNs are ubiquitously
employed for computer vision based decision making tasks in many inspection
modalities including the ones that are not covered in this paper: defects
and/or substance classification in UT [87, 93] and terahertz inspection [172],
edge detection in X-ray [173] or semantic segmentation in thermography [155],
feature extraction and classification in optical images [174] and X-ray
tomography [175, 176]. Often integrated as a part of feature extraction, CNNs
can also be used for imaging tasks such as for reconstructing the raw
terahertz measurement data [177, 178] or enhancing the resolution of UT [179,
180] and X-ray reconstructions [177, 178]. Such ubiquity can also be found for
classical algorithms as well. An example is correlation based imaging, which
is however called differently depending on the applications: SAFT and TFM for
UT, Synthetic Aperture Radar for radar and DAS beamforming for medical
ultrasound.

All these examples make it evident that there are common patterns in how to
process the data depending on the tasks instead of inspection modalities or
application fields. One can find such patterns, when these tasks deal with the
intrinsically same mathematical problems. As a result, these tasks can be
formulated mathematically in the similar, if not the same, manner. For
example, reconstructing the inner structure of a test object from UT signals
can be formulated in a very similar manner as that of reconstructing X-ray
Computed Tomography (CT) data or image deblurring [181]. This is because all
of these tasks aim to recover the quantities which cannot be observed directly
from the observations that are affected by these unknown quantities. To be
more specific, these unknown quantities can be the location of defects in a
test object for UT or X-ray reconstruction, and the unknown clean image for
deblurring tasks. This realization motivates us to make high level connections
of the references in terms of their intrinsic natures. For this purpose, we
identified four intrinsic mathematical problems we commonly encounter in NDE
applications, which are forward problems, inverse problems, optimal design and
pattern recognition.

In the following subsections, each problem is described to help identify the
intrinsic nature of a specific engineering task. The emphasis is placed on the
common patterns one can see when dealing with these tasks. In table 10, some
recommended references are provided as a summary of this section. The left
most column shows the mathematical problems these references intrinsically
solve. The specific NDE related tasks are listed in the second column. The
third column of the table provides the list of the sections which discuss how
these problems are solved for a specific inspection modality. The fourth
column shows common methods to solve these tasks, and the fifth column is the
list of references for the corresponding methods. Furthermore, these tasks are
highlighted with four different colors to illustrate how they relate to actual
NDE inspections. Inspired by the NDE automation levels propsed in [1], we
decompose a NDE inspection chain into three steps: measurement operation,
enhancement of raw measurement data and decision making. In addition to that,
we consider data synthesization as an essential part of the future of NDE
inspections, since there is a pressing need for high quality, well-labeled
data for reliable AI training. Admittedly, this is by no means an exhaustive
collection, yet it will give a brief overview of how one can interpret and
possibly solve these tasks regardless of applications. Note that the
conceptual connections we made here is just one possibility, and they can be
described and grouped differently.

The benefits of table 10 are twofold. First, acknowledging such conceptual
patterns lets us understand the similarities in the tasks and try a method in
other modalities or application fields. In the era of NDE 4.0, where high
level problems need to be solved by machines instead of human operators, we
will increasingly encounter the problems for which no state of the art exist.
However, once the nature of the tasks is identified, such common patterns can
help us determine which measure, algorithm and/or ML architecture is suitable.
Second, the number of listed methods in table 10 also indicates which tasks
are well investigated and which are still in its infancy. This can be useful
for determining the direction of future research.

###  6.1 Forward Problems

For the purposes of this work, we refer to general problems of the form
$\bm{y}=f(\bm{x})$, which shows a relation between the causal factors $\bm{x}$
and their effects $\bm{y}$ using an operator $f$. For the sake of simplicity,
in this work we represent both $\bm{x}$ and $\bm{y}$ as vectors, however they
can be arrays as well.

Forward problems seek to obtain the effects $\bm{y}$ from the known causal
factor $\bm{x}$. In the context of NDE, $f$ usually represents a measurement
process, yielding a synthesized version of measurement data $\bm{y}$.
Especially for ML-based approaches, where the lack of quality data has been
frequently identified as a major bottleneck regardless of their tasks [1],
solving forward problem is beneficial as it can enlarge the training data set.
For most cases with NDE applications, $\bm{x}$ and $\bm{y}$ are related
through the underlying physical phenomenon described by a PDE, such as
Maxwell’s equations, wave or heat equations. Since solving these PDEs is
typically very challenging, the true causal relation is generally
approximated. As a result, there are various possibilities to select the
operator $f$. For accurately computing $\bm{y}$, one can aim to find $f$ such
that it approximates the underlying PDE. Another approach is to simplify the
PDE by making assumptions on the underlying physics, enabling to explicitly
compute $f$ to compute $\bm{y}$.

###  6.2 Inverse Problems

Given the causal relation $\bm{y}=f(\bm{x})$ we considered in the previous
section, inverse problems aim to find the parameter $\bm{x}$ from a set of
observations $\bm{y}$ collected through a (measurement) process $f$. For
example, when $f$ adds noise to $\bm{x}$ so that $\bm{y}$ is a noisy version
of $\bm{x}$, the inverse problem is a denoising problem. If $\bm{y}$ is an
image and one seeks to increase its resolution, $f$ can be interpreted as
_subsampling_ operation, and recovering $\bm{x}$ becomes super-resolution
task. In the case of imaging, we consider $\bm{x}$ to be an _image_ in the
sense that it describes a parameter that varies in space, and the raw
measurement data $\bm{y}$ is defined in a different domain and/or cannot be
interpreted as an image. As a concrete example, eddy current testing deals
with the measurement of magnetic field intensity and phase obtained with
varying excitation frequencies. In the case of ultrasound, raw data often
consists of time domain acoustic pressure signals collected with different
measurement channels. For both, the data is processed to obtain an image
portraying defect locations. In the case of inverse problems, NNs facilitate
the computation of $\bm{x}$ by replacing parts of classical algorithms to
invert $f$, or replacing them entirely, i.e. $\bm{x}=\Psi(\bm{y})$, where
$\Psi$ is the NN. Typically, the quality of the inversion is measured through
a cost function $h(\bm{x},\Psi(\bm{y}))$ which we want to minimize, such as
eq. 5.

###  6.3 Optimal Design

Optimal design deals with the tuning of experimental parameters which can be
freely chosen, and which impact the information content of measurement data
with regards to parameters of interest. Recall the inverse problem from
section 6.2. In reality, the measurement data $\bm{y}$ depends not only on
$\bm{x}$, but also on the experimental parameters (e.g. the location, number,
and type of sensors employed) $\bm{\phi}$. When solving an inverse problem,
the achievable cost $h$ is affected by the experimental design. We can tune
the parameters $\bm{\phi}$ so that the cost $h$ is minimal, i.e.
$\min\limits_{\bm{\phi}}h(f(\bm{x},\bm{\phi})).$

###  6.4 Pattern Recognition

Pattern recognition can be found ubiquitously in many data analysis and
decision making tasks. In general, given the input data $\bm{x}$, we would
like to assign it a category or class $y$. There exists a correspondence
between $\bm{x}$ and $y$ given by an unknown mapping $g$ that we wish to find.
Whether a category is assigned to $\bm{x}$ as a whole, by regions, or by
sample, provides the nuance among classification, detection, and segmentation,
respectively. Oftentimes, the data $\bm{x}$ is preprocessed through a
judiciously chosen function $f$ which highlights informative _features_. This
leads to the overall pattern recognition task $y=g(f(\bm{x}))$ which renders
itself to a problem of finding an appropriate classifier $g$, preprocessing
scheme $f$, or both. NNs can be used to replace the classifier $g$ in
combination with classical feature extraction methods such as PCA, or they can
replace both the feature extraction and classification process altogether.
Since the learning is typically based on the available samples, the quality of
the samples determines the performance.

ccccl[hvlines] CarnationPink!252-24-5 Emerald!155-215-5 Apricot!3516-219-5
Emerald!1520-221-5 CornflowerBlue!2522-237-5 Intrinsic problem & Task Section
Method References
3-1Forward problems 3-1Data
synthesizaiton 3-13.5, 4.2 numerical solver [122], [121]
PINN [81], [82], [132], [133]
GAN [122], [130], [124], [142]
11-1Inverse problems 8-1Imaging 8-1 2.1, 2.2,
2.3, 2.4,
3.1.1, 3.1.2,
3.2, 4.3 classical [37], [38], [39], [71], [72], [74], [75], [182]
MLP [5], [13], [6], [7], [21]
SVR [26]
HLR [26]
XGBoost [25]
CNN [80]
Autoencoder [40], [183]
PINN [134], [81], [84]
3-1Super-resolution 3-13.1.1, 3.1.2,
3.2, 3.5 CNN [80]
U-Net [78], [79]
GAN [44], [129], [151]
4-1Optimal
design 2-1Optimal
excitation 2-13.1 classical random ([33], [34]), optimized ([35])
DL [36]
2-1Subsampling 2-13.1.2 classical [64]
DPS [63]
18-1Pattern
recognition 2-1Feature
extraction 2-12.1, 2.4,
4.2, 5 unsupervised PCA ([139])
supervised LDA ([29], [28]), SVM ([114]), SVR ([9])
6-1Classification 6-12.1, 2.4,
3.3, 5.3 classical [28], [111]
$k$NN [29], [28]
MLP [15]
Probabilistic NN [10]
CNN [108], [87], [93], [97]
YOLOv8-cls [167]
6-1Detection 6-12.1, 2.4,
5.3 MLP [14], [27]
RBF [12], [16]
YOLOv4 [123]
YOLOv8 [167]
Faster R-CNN [160]
SSD [164]
4-1Segmentation 4-12.1, 5.3 classical [184]
CNN [156], [154] [155], [169]
YOLOv8-seg [167]
Mask R-CNN [168]

Table 10:  Summary of the methods used for common NDE tasks. References are
categorized according to the intrinsic mathematical problem they are dealing
with as discussed in section 6. Each color in the table is associated to a NDE
inspection process: pink for data synthesization, green for data enhancement,
orange for measurement operation and blue for decision making. Note that end-
to-end ML methods often perform feature extraction implicitly. These methods
are not shown as feature extraction, instead categorized into their targeted
decision making tasks.

##  7 Summary and Conclusions

In this paper we discuss the state of the art in a variety of NDE techniques.
In light of the challenges brought about by the recent advances in the
corresponding industries, significant technological advances in NDE are
required. The discussion of the different NDE modalities has emphasized some
of the intricacies we are facing. More often than not, a measurement device
that is sensitive for a desired physical effect will also react to unwanted
effects such as environmental operating conditions (e.g., temperature) or
coupling effects (e.g., sensor lift-off). The complexity of the signals
requires significant experience for their interpretation in order to detect
subtle effects and recognize artefacts. Training is often hindered by lack of
available labeled data, which is costly to generate. Finally, since each
modality is very different, generic solutions are often not available or lack
in efficiency due to insufficient incorporation of the physics of the problem
and the sensors.

The paper has outlined a number of AI approaches for different NDE modalities,
demonstrating examples where AI is already helpful today. In particular, we
have shown that AI can assist us in detection (e.g., cracks, voids) and
classification (e.g., material, defect type) as well as segmentation problems.
Moreover, AI methods have been successfully applied in the estimation of
parameters (e.g., crack depth/size, material parameters). In addition, AI
methods can be a powerful tool for solving complex inverse problems for
beamforming and imaging, even in complex situations such as Ultrasound
tomography where full waveform inversion methods are required. Of course, they
can also be used for image quality improvement (e.g., deblurring, denoising,
super resolution).

Often, such methods can be tailored to be robust against calibration issues
such as variations in environmental operating conditions (e.g., temperature)
or sensor coupling (e.g., lift-off). The lack of training data can sometimes
be alleviated through synthetic data generation for which dedicated AI methods
have been devised. An interesting thread of research is to use AI methods even
very early, e.g., to assist in devising the optimal sensor placement and
related design of experiment questions.

Overall, it is evident that AI methods have become indispensable tools for
advancing the NDE field further. We are certain that we are just witnessing
the beginning of a research field that will give rise to even more powerful
semi-automated NDE systems which are required to keep up with the digital
transformation of our industries.

### CRediT authorship contribution statement

Eduardo Pérez: Conceptualization, Writing - Original Draft, Writing - Review &
Editing, Supervision. Cemil Emre Ardic: Writing - Original Draft. Ozan
Çakıroğlu: Writing - Original Draft, Visualization. Kevin Jacob: Writing -
Original Draft. Sayako Kodera: Conceptualization, Writing - Original Draft.
Luca Pompa: Writing - Original Draft. Mohamad Rachid: Writing - Original
Draft, Visualization, Writing - Review & Editing. Han Wang: Writing - Original
Draft, Visualization. Yiming Zhou: Writing - Original Draft, Visualization.
Cyril Zimmer: Writing - Original Draft, Visualization. Florian Römer: Writing
- Original Draft, Writing - Review & Editing, Project administration, Funding
acquisition. Ahmad Osman: Project administration, Funding acquisition.

### Conflicts of Interest

The authors declare no conflict of interest.

### Acknowledgements

This work was partially supported by the Fraunhofer Internal Programs under
the grant Attract 025-601128, the Federal Ministry of Education and Research
(BMBF) through the VISiMOS project with grant number 03VP10900, and the
Thuringian Ministry of Economic Affairs, Science and Digital Society (TMWWDG).

## Glossary

## Acronyms

3MA

    Micromagnetic Multiparameter Microstructure and stress Analysis
ABC

    Artificial Bee Colony
ABLE

    Adaptive Beamforming by Deep LEarning
ADMIRE

    Aperture Domain Model Image REconstruction
ADMM

    Alternating Direction Method of Multipliers
ADR

    Automated Defect-Recognition
AI

    Artificial Intelligence
ANN

    Artificial Neural Network
ART

    Algebraic Reconstruction Technique
ASBI

    Adaptive Spectral Band Integration
ASRG

    Automatic Seeded Region Growing
CEUS

    Contrast-Enhanced Ultrasound
CNN

    Convolutional Neural Network
CNR

    Contrast-to-Noise Ratio
COBA

    Convolutional Beamforming Algorithms
CORONA

    Convolutional Robust Principal Component Analysis
CS

    Compressed Sensing
CT

    Computed Tomography
DAS

    Delay and Sum
DBTT

    Ductile-Brittle Transition Temperature
DL

    Deep Learning
DN

    Dense Network
DNN

    Deep Neural Network
DPS

    Deep Probabilistic Subsampling
DRL

    Deep Residual Learning
DWT

    Discrete Wavelet Transform
ECT

    Eddy Current Testing
EMAT

    Electro Magnetic Acoustic Transducer
EMD

    Empirical Mode Decomposition
EWT

    Empirical Wavelet Transform
FCN

    Fully Convolutional Neural Network
FE

    Finite Element
FEM

    Finite Element Method
FISTA

    Fast ISTA
FMC

    Full Matrix Capture
FWHM

    Full-Width at Half Maximum
FWI

    Full Waveform Inversion
GAN

    Generative Adversarial Neural Network
GPCT

    SNGAN-based generative principal component thermography
GRU

    Gated Recurrent Unit
HLR

    Huber Loss Regression
IHCP

    Inverse Heat Conduction Problem
IIoT

    Industrial Internet of Things
IR

    Infrared
IRT

    Infrared Thermography
ISTA

    Iterative Shrinkage-Thresholding Algorithm
J-DPS

    Joint DPS
$k$FCV

     $k$-Fold Cross Validation
$k$NN

     $k$-Nearest Neighbour
KPCA

    Kernel Principal Component Analysis
LCF

    Low-cycle fatigue
LDA

    Linear Discriminant Analysis
LISTA

    Learned ISTA
LSTM

    Long Short-Term Memory
MAE

    Mean Absolute Error
ML

    Machine Learning
MLE

    Maximum Likelihood Estimator
MLP

    Multilayer Perceptron
MLR

    Multiple Linear Regression
MRI

    Magnetic Resonance Imaging
MSE

    Mean Squared Error
MT

    Modulation Thermography
MV

    Minimum Variance
NDE

    Nondestructive Evaluation
NDE

    Nondestructive Evaluation
NN

    Neural Network
PCA

    Principle Component Analysis
PCT

    Principal Component Thermography
PDE

    Partial Differential Equation
PINN

    Physics-Informed Neural Network
PoD

    Probability of Detection
PPT

    Pulsed Phase Thermography
PSNR

    Peak Signal-to-Noise Ratio
PT

    Pulsed Thermography
PW

    Plane Wave
PWI

    Plane Wave Imaging
R2

    Coefficient of Determination
RBF

    Radial Basis Function
R-CNN

    Region Based Convolutional Neural Network
Re

    Yield Strength
RF

    Random Forest
Rm

    Tensile Strength
RMSE

    Root Mean Squared Error
RNN

    Recurrent Neural Network
ROI

    Region of Interest
RPN

    Region Proposal Network
RPV

    Reactor Pressure Vessel
SAFT

    Synthetic Aperture Focusing Technique
SD

    Standard Deviation
SGD

    Stochastic Gradient Descent
SIRT

    Simultaneous Iterative Reconstruction Technique
SNGAN

    Spectral Normalized GAN
SSD

    Single Shot MultiBox Detector
STFT

    Short-Time Fourier Transform
SVD

    Singular Value Decomposition
SVM

    Support Vector Machine
SVR

    Support Vector Regression
TFM

    Total Focusing Method
TNDE

    Thermographic NDE
TSR

    Thermographic Signal Reconstruction
UCT

    Ultrasound Computed Tomography
UT

    Ultrasonic Testing
VMD

    Variational Mode Decomposition
XGBoost

    extreme Gradient Boosting

## References

  * [1] S. Cantero-Chinchilla, P. D. Wilcox, and A. J. Croxford.  Deep learning in automated ultrasonic NDE–Developments, axioms and opportunities.  NDT & E International, page 102703, 2022.
  * [2] ISO 9712:2021(en) Non-destructive testing - Qualification and certification of NDT personnel, 2021.
  * [3] DIN 1324 Teil 2 - Elektromagnetisches Feld - Materialgrößen, May 1988.
  * [4] S. Youssef.  Einsatz maschineller Lernalgorithmen zur mikromagnetischen Materialcharakterisierung.  PhD thesis, Universität des Saarlandes, 2021.
  * [5] I. Rekanos, T. Theodoulidis, S. Panas, and T. Tsiboukis.  Impedance inversion in eddy current testing of layered planar structures via neural networks.  NDT & E International, 30(2):69–74, April 1997.
  * [6] B. Helifa, M. Féliachi, I. Lefkaier, F. Boubenider, A. Zaoui, and N. Lagraa.  Characterization of surface cracks using eddy current ndt simulation by 3d-fem and inversion by neural network.  The Applied Computational Electromagnetics Society Journal (ACES), pages 187–194, 2016.
  * [7] S. Harzallah, R. Rebhi, M. Chabaat, and A. Rabehi.  Eddy current modelling using multi-layer perceptron neural networks for detecting surface cracks.  Frattura ed Integrità Strutturale, 12(45):147–155, June 2018.
  * [8] K. Demachi, T. Hori, and S. Perrin.  Crack depth estimation of non-magnetic material by convolutional neural network analysis of eddy current testing signal.  Journal of Nuclear Science and Technology, 57(4):401–407, November 2019.
  * [9] A. Bernieri, L. Ferrigno, M. Laracca, and M. Molinara.  Crack shape reconstruction in eddy current testing using machine learning systems for regression.  IEEE Transactions on Instrumentation and Measurement, 57(9):1958–1968, September 2008.
  * [10] S.-J. Song and Y.-K. Shin.  Eddy current flaw characterization in tubes by neural networks and finite element modeling.  NDT & E International, 33(4):233–243, June 2000.
  * [11] N. Yusa, W. Cheng, Z. Chen, and K. Miya.  Generalized neural network approach to eddy current inversion for real cracks.  NDT & E International, 35(8):609–614, December 2002.
  * [12] M. Wrzuszczak and J. Wrzuszczak.  Eddy current flaw detection with neural network applications.  Measurement, 38(2):132–136, September 2005.
  * [13] C. Glorieux, J. Moulder, J. Basart, and J. Thoen.  The determination of electrical conductivity profiles using neural network inversion of multi-frequency eddy-current data.  Journal of Physics D: Applied Physics, 32(5):616–622, January 1999.
  * [14] B. Rao, B. Raj, T. Jayakumar, and P. Kalyanasundaram.  An artificial neural network for eddy current testing of austenitic stainless steel welds.  NDT & E International, 35(6):393–398, September 2002.
  * [15] E. V. Kuzmin, O. E. Gorbunov, P. O. Plotnikov, V. A. Tyukin, and V. A. Bashkin.  Application of neural networks for recognizing rail structural elements in magnetic and eddy current defectograms.  Automatic Control and Computer Sciences, 53(7):628–637, December 2019.
  * [16] Q. Cao, D. Liu, Y. He, J. Zhou, and J. Codrington.  Nondestructive and quantitative evaluation of wire rope based on radial basis function neural network using eddy current inspection.  NDT & E International, 46:7–13, March 2012.
  * [17] K. B. Ali, A. N. Abdalla, D. Rifai, and M. A. Faraj.  Review on system development in eddy current testing and technique for defect classification and characterization.  IET Circuits, Devices & Systems, 11(4):338–351, March 2017.
  * [18] A. N. AbdAlla, M. A. Faraj, F. Samsuri, D. Rifai, K. Ali, and Y. Al-Douri.  Challenges in improving the performance of eddy current testing: Review.  Measurement and Control, 52(1–2):46–64, November 2018.
  * [19] M. Maciusowicz and G. Psuj.  Classification of grain-oriented electrical steel sheets by magnetic barkhausen noise using time-frequency analysis and selected machine learning algorithms.  Applied Sciences, 12(23):12469, December 2022.
  * [20] P. Wang, L. Zhu, Q. Zhu, X. Ji, H. Wang, G. Tian, and E. Yao.  An application of back propagation neural network for the steel stress detection based on barkhausen noise theory.  NDT & E International, 55:9–14, April 2013.
  * [21] Y. Zhang, W. Liu, K. Li, P. Wang, C. Hang, Y. Chen, X. Han, and W. Gao.  Application of a back-propagation neural network for mechanical properties prediction of ferromagnetic materials by magnetic barkhausen noise technique.  Insight - Non-Destructive Testing and Condition Monitoring, 61(2):95–99, February 2019.
  * [22] R. Jedamski and J. Epp.  Non-destructive micromagnetic determination of hardness and case hardening depth using linear regression analysis and artificial neural networks.  Metals, 11(1):18, December 2020.
  * [23] X. Wang, C. He, P. Li, X. Liu, Z. Xing, and Z. Yan.  Micromagnetic and quantitative prediction of surface hardness in carbon steels based on a joint classification-regression method.  Journal of Nondestructive Evaluation, 41(3), September 2022.
  * [24] Z. Xing, X. Wang, M. Ning, C. He, and X. Liu.  Micromagnetic and robust evaluation of surface hardness in cr12mov steel considering repeatability of the instrument.  Sensors, 23(3):1273, January 2023.
  * [25] H. Sheng and P. Wang.  Evaluation of pipeline steel mechanical property distribution based on multimicromagnetic ndt method.  IEEE Transactions on Instrumentation and Measurement, 72:1–15, 2023.
  * [26] S. Grönroos, J. Rinta-aho, T. Koskinen, and G. Sorger.  Combination of non-destructive methods in estimating irradiation-induced reactor pressure vessel steel alloy embrittlement with machine learning.  SSRN Electronic Journal, 2021.
  * [27] S. Youssef, M. Schuppmann, S. Pushkarev, P. Stopp, and O. Stawicki.  KI-basierte Prozessoptimierung im Stahlwerk – Multimodales, intelligentes und autoadaptives Sensorsystem zur Hardspotdetektion an Grobblechen.  In ZfP-Zeitung 175, page 38–44. DGZfP e.V, June 2021.
  * [28] C. Zimmer, Y. N. Rallabandi, K. Szielasko, C. Eichheimer, M. Luke, and S. Youssef.  Micromagnetic Microstructure- and Stress-Independent Materials Characterization in Reactor Safety Research.  Materials, 14(18), September 2021.
  * [29] S. Youssef, C. Zimmer, K. Szielasko, and A. Schütze.  Bewertung subjektiver und automatisierter Merkmalsextraktion periodischer Zeitsignale am Beispiel des 3MA-X8-verfahrens.  tm - Technisches Messen, 86(5):267–277, 2019.
  * [30] S. Youssef, C. Zimmer, K. Szielasko, Z. K. Suri, and A. Schütze.  Vergleich subjektiver und automatisierter Merkmalsextraktion sowie Einsatz maschineller Lernalgorithmen zur mikromagnetischen Materialcharakterisierung.  In 4.3 Machine Learning und Signalverarbeitung, pages 347–354. 20. GMA/ITG-Fachtagung Sensoren und Messsysteme 2019, June 2019.
  * [31] M. D. Verweij, B. E. Treeby, K. W. A. van Dongen, and L. Demi.  Simulation of Ultrasound Fields.  Comprehensive Biomedical Physics, pages 465–499, 2014.
  * [32] L. Le Jeune, S. Robert, E. L. Villaverde, and C. Prada.  Plane wave imaging for ultrasonic non-destructive testing: Generalization to multimodal imaging.  Ultrasonics, 64:128–138, 2016.
  * [33] M. F. Schiffner.  Random incident waves for fast compressed pulse-echo ultrasound imaging, 2019.
  * [34] M. F. Schiffner and G. Schmitz.  Fast compressive pulse-echo ultrasound imaging using random incident sound fields.  The Journal of the Acoustical Society of America, 141:3611–3611, 2017.
  * [35] O. Çakıroğlu, E. Pérez, F. Roemer, and M. Schiffner.  Optimization of transmission parameters in fast pulse-echo ultrasound imaging using sparse recovery.  In 2023 31st European Signal Processing Conference (EUSIPCO), pages 441–445, 2023.
  * [36] O. Çakıroğlu, E. Pérez, F. Roemer, and M. Schiffner.  Autoencoder-based Learning of Transmission Parameters in Fast Pulse-Echo Ultrasound Imaging employing Sparse Recovery.  In 2023 IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP). IEEE, 2023.  In Press.
  * [37] I. Daubechies, M. Defrise, and C. De Mol.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.  Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 57(11):1413–1457, 2004.
  * [38] A. Beck and M. Teboulle.  A fast iterative shrinkage-thresholding algorithm with application to wavelet-based image deblurring.  In 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 693–696, 2009.
  * [39] S. Boyd, N. Parikh, E. Chu, B. Peleato, J. Eckstein, et al.  Distributed optimization and statistical learning via the alternating direction method of multipliers.  Foundations and Trends® in Machine learning, 3(1):1–122, 2011.
  * [40] J. Zhang, Q. He, Y. Xiao, H. Zheng, C. Wang, and J. Luo.  Self-supervised learning of a deep neural network for ultrafast ultrasound imaging as an inverse problem.  In 2020 IEEE International Ultrasonics Symposium (IUS), pages 1–4. IEEE, 2020.
  * [41] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [42] H. Yao, F. Dai, S. Zhang, Y. Zhang, Q. Tian, and C. Xu.  Dr2-net: Deep residual reconstruction network for image compressive sensing.  Neurocomputing, 359:483–493, 2019.
  * [43] K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, and A. Ashok.  Reconnet: Non-iterative reconstruction of images from compressively sensed measurements.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 449–458, 2016.
  * [44] M. Mardani, E. Gong, J. Y. Cheng, S. S. Vasanawala, G. Zaharchuk, L. Xing, and J. M. Pauly.  Deep generative adversarial neural networks for compressive sensing mri.  IEEE Transactions on Medical Imaging, 38(1):167–179, 2019.
  * [45] B. W. Drinkwater and P. D. Wilcox.  Ultrasonic arrays for non-destructive evaluation: A review.  NDT & e International, 39(7):525–541, 2006.
  * [46] J.-H. Kim, A. Basarab, P. R. Hill, D. R. Bull, D. Kouamé, and A. Achim.  Ultrasound image reconstruction from compressed measurements using approximate message passing.  In 2016 24th European Signal Processing Conference (EUSIPCO), pages 557–561. IEEE, 2016.
  * [47] S. Liu, Y. Wang, X. Yang, B. Lei, L. Liu, S. X. Li, D. Ni, and T. Wang.  Deep learning in medical ultrasound analysis: a review.  Engineering, 5(2):261–275, 2019.
  * [48] R. J. Van Sloun, R. Cohen, and Y. C. Eldar.  Deep learning in ultrasound imaging.  Proceedings of the IEEE, 108(1):11–29, 2019.
  * [49] N. Shlezinger, Y. C. Eldar, and S. P. Boyd.  Model-based deep learning: On the intersection of deep learning and optimization.  IEEE Access, 10:115384–115398, 2022.
  * [50] K. Gregor and Y. LeCun.  Learning fast approximations of sparse coding.  In Proceedings of the 27th international conference on international conference on machine learning, pages 399–406, 2010.
  * [51] V. Monga, Y. Li, and Y. C. Eldar.  Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing.  IEEE Signal Processing Magazine, 38(2):18–44, 2021.
  * [52] A. Bora, A. Jalal, E. Price, and A. G. Dimakis.  Compressed sensing using generative models.  In International conference on machine learning, pages 537–546. PMLR, 2017.
  * [53] R. J. van Sloun, J. C. Ye, and Y. C. Eldar.  Deep learning for ultrasound beamforming.  arXiv preprint arXiv:2109.11431, 2021.
  * [54] B. Luijten, N. Chennakeshava, Y. C. Eldar, M. Mischi, and R. J. van Sloun.  Ultrasound signal processing: from models to deep learning.  Ultrasound in Medicine & Biology, 2023.
  * [55] B. Luijten, R. Cohen, F. J. De Bruijn, H. A. Schmeitz, M. Mischi, Y. C. Eldar, and R. J. Van Sloun.  Deep learning for fast adaptive beamforming.  In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1333–1337. IEEE, 2019.
  * [56] B. Luijten, R. Cohen, F. J. De Bruijn, H. A. Schmeitz, M. Mischi, Y. C. Eldar, and R. J. Van Sloun.  Adaptive ultrasound beamforming using deep learning.  IEEE Transactions on Medical Imaging, 39(12):3967–3978, 2020.
  * [57] Y. Li, O. Bar-Shira, V. Monga, and Y. C. Eldar.  Deep algorithm unrolling for biomedical imaging.  arXiv preprint arXiv:2108.06637, 2021.
  * [58] C. Khan, R. J. van Sloun, and B. Byram.  Unfolding model-based beamforming for high quality ultrasound imaging.  In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8682–8686. IEEE, 2022.
  * [59] O. Solomon, R. Cohen, Y. Zhang, Y. Yang, Q. He, J. Luo, R. J. van Sloun, and Y. C. Eldar.  Deep unfolded robust pca with application to clutter suppression in ultrasound.  IEEE transactions on medical imaging, 39(4):1051–1063, 2019.
  * [60] T. Chernyakova and Y. C. Eldar.  Fourier-domain beamforming: the path to compressed ultrasound imaging.  IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 61(8):1252–1267, 2014.
  * [61] R. Cohen and Y. C. Eldar.  Sparse convolutional beamforming for ultrasound imaging.  IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 65(12):2390–2406, 2018.
  * [62] A. Mamistvalov and Y. C. Eldar.  Deep unfolded recovery of sub-nyquist sampled ultrasound images.  IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 68(12):3484–3496, 2021.
  * [63] I. A. Huijben, B. S. Veeling, K. Janse, M. Mischi, and R. J. van Sloun.  Learning sub-sampling and signal recovery with applications in ultrasound imaging.  IEEE Transactions on Medical Imaging, 39(12):3955–3966, 2020.
  * [64] E. Pérez, J. Kirchhof, F. Krieg, and F. Römer.  Subsampling approaches for compressed sensing with ultrasound arrays in non-destructive testing.  Sensors, 20(23):6734, 2020.
  * [65] H. Wang, E. Pérez, and F. Römer.  Deep Learning-Assisted optimal sensor placement in ultrasound NDT.  In Proceedings of the AMA Sensor and Measurement Science International (SMSI) Conference 2023, Nuremberg, Germany, May 2023.