# Infrastructure Crack Segmentation: Boundary Guidance Method and Benchmark
Dataset

Zhili He1,2 Wang Chen1 Jian Zhang1111Corresponding authors: Jian Zhang and Yu-
Hsing Wang. Preprint submitted to Elsevier.  Yu-Hsing Wang2111Corresponding
authors: Jian Zhang and Yu-Hsing Wang. Preprint submitted to Elsevier.
1 School of Civil Engineering, Southeast University
2 Department of Civil and Environmental Engineering, The Hong Kong University
of Science and Technology
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Cracks provide an essential indicator of infrastructure performance
degradation, and achieving high-precision pixel-level crack segmentation is an
issue of concern. Unlike the common research paradigms that adopt novel
artificial intelligence (AI) methods directly, this paper examines the
inherent characteristics of cracks so as to introduce boundary features into
crack identification and then builds a boundary guidance crack segmentation
model (BGCrack) with targeted structures and modules, including a high
frequency module, global information modeling module, joint optimization
module, etc. Extensive experimental results verify the feasibility of the
proposed designs and the effectiveness of the edge information in improving
segmentation results. In addition, considering that notable open-source
datasets mainly consist of asphalt pavement cracks because of ease of access,
there is no standard and widely recognized dataset yet for steel structures,
one of the primary structural forms in civil infrastructure. This paper
provides a steel crack dataset that establishes a unified and fair benchmark
for the identification of steel cracks. BGCrack is available at
https://github.com/hzlbbfrog/BGCrack.

##  1 Introduction

Defects inevitably occur, accumulate, and expand during the service life of
civil infrastructure due to the complex work environment, thereby posing
safety risks to structures and threatening their service life [19]. Cracks are
among the principal manifestations of multiple structural damage and are
characterized by their length, width, and density, and are essential
indicators of structural performance degradation as well [11]. Consequently,
cracks pose significant safety hazards to infrastructure that cannot be
overlooked.

Regular infrastructure inspection is a fundamental approach to detecting
damage and evaluating security and service conditions, among which the crack
inspection is a crucial item [31]. However, manual onsite visual detection
still occupies the exclusive position in crack inspection, requiring qualified
engineers to mark and measure the characteristics of cracks manually [23].
Such a working pipeline has several issues, including high-security risk, high
quantitative uncertainty due to human subjectivity, excessive labor cost,
exorbitant time overhead, etc. Recently, researchers and practitioners have
shown extensive interest in rapidly evolving AI and computer vision (CV)
technologies [55] and have introduced these technologies into the field of
structural defect identification to try to alleviate the aforementioned issues
[1, 8, 9]. These novel technologies also offer the potential to achieve
automated and intelligent inspection, which would thoroughly address the
challenges associated with manual inspection.

The prevailing research paradigm for AI and CV-based crack inspection involves
training deep learning models on crack image datasets and then deploying these
models to make inferences from images outside training datasets. The core of
this pipeline consists of two elements: deep learning models and crack
datasets, both of which are covered in the contributions of this study.

Figure 1: Decomposition of images. (a) Cracks can be decomposed into the crack
edge and internal area. (b) Edge features help us distinguish and locate the
Panda’s head.

Further, researchers have conducted extensive research to develop deep-
learning models that can segment cracks accurately. From the perspective of
network structure, these studies can generally be divided into three
categories: CNN-based methods [7, 30], pure Transformer-based methods [16],
and CNN-Transformer hybrid methods [53]. These methods pay more attention to
the direct application of cutting-edge deep learning methods but neglect the
importance of designing targeted methods based on crack intrinsic
characteristics. Obviously, in most cases, cracks can be decoupled into two
parts: the crack boundary and the internal area (see Fig. 1 (a)). In the human
vision system, as shown in Fig. 1 (a) and (b), edge information is always
adopted inadvertently to help us distinguish, identify and locate objects
[24]. Some researchers have also leveraged edge features in the salient object
detection field and substantiated that integrating boundary features can
improve models’ perception accuracy and generate more accurate segmentation
images [44, 33, 54]. On the other hand, fusing edge information is helpful in
getting clear and accurate crack shapes, and Yang et al. [49] and Dang et al.
[10] have demonstrated that it is directly related to the quantitative
representation of cracks. Inspired by these findings, this paper designs the
BGCrack, which draws boundary information into the field of crack
identification and explores and models the relationships between cracks and
their edges in an explicit manner.

Accordingly, the main contributions of this study are summarized as follows:

  * •

We lead crack boundary features into crack detection and model the
relationships between the crack body and crack edge. A novel boundary guidance
crack segmentation model named BGCrack is designed.

  * •

We create and make available a steel crack dataset and establish a unified and
fair benchmark. By doing so, we want to promote standardization and
normalization in the field of crack inspection.

##  2 Related Works

Deep learning in the crack inspection. Deep learning models can be classified
into three categories based on differences in basic computer vision tasks,
namely, image classification, object detection, and semantic segmentation. As
for the image classification problem, the objective is to classify defects in
given pictures. For instance, Xu et al. [46] proposed an attribute-based
structural damage identification framework based on meta-learning, and Guo et
al. [17] developed a convolutional neural network (CNN) classifier using semi-
supervised learning. In terms of the object detection task, the intention is
to obtain box-level detection results [2]. For example, Xu et al. [47] trained
a Faster R-CNN model to identify seismic damage, and Yu et al. [52]
established a YOLOv4-FPM network to achieve real-time crack detection.
Semantic segmentation aims to achieve pixel-level feature representations from
input images. Obviously, this approach can determine not only the presence of
cracks in an image but also the location and properties of cracks and requires
less post-processing than classification. Moreover, for object detection
tasks, the highly complex characteristics of cracks make it challenging to
determine if predicted bounding boxes are appropriate and accurate for
locating cracks. There is almost no unified standard for this assessment.
Therefore, semantic segmentation is the most appropriate framework for
representing cracks among the three basic computer vision tasks and is also
the research focus of this paper.

Crack dataset. In addition to conducting research on deep learning models,
scholars have also opened sources of some crack image datasets to promote the
development of the field. This is because the performance of deep learning
models is highly dependent on data as a data-driven technology. Some notable
datasets include CrackTree206 [58], DeepCrackA [25], DeepCrackB [59], CFD
[39], and CRACK500 [48]. Unfortunately, almost all the images in these
datasets are of asphalt pavement cracks because of ease of access. Steel
structures are one of the most common structural forms in civil
infrastructure. However, there are few influential open-source datasets
covering steel cracks. Although many papers have researched the pixel-wise
segmentation problem of these cracks, they prefer to leverage their own
private datasets because of some engineering disputes or other reasons. To
promote standardization and normalization in this field, this paper presents
an open-source steel crack dataset. The aim is to establish a unified and
impartial benchmark for the identification of steel cracks. The datasets would
be available at https://github.com/hzlbbfrog/Civil-dataset.

##  3 Methodology

###  3.1 Motivation and Overall Network Architecture

For an input image $\mathbf{I}$ with cracks, the final objective for almost
all the crack identification models is to obtain the global features of cracks
$\mathbf{I}_{body}$ from $\mathbf{I}$. Based on previous analysis, different
from other models, we want BGCrack to learn the representation ability of
crack boundary $\mathbf{I}_{edge}$ and $\mathbf{I}_{body}$, simultaneously and
further to be able to make $\mathbf{I}_{edge}$ optimize $\mathbf{I}_{body}$.
Intuitively, for this purpose, BGCrack is designed in four stages: Stage 1 is
the backbone, which is adopted to extract and abstract features, Stage 2 is
boundary feature modeling, Stage 3 is global feature modeling, and Stage 4 is
the feature optimization. Based on the characteristics of features and
purposes of stages, 4 stages are contrapuntally designed in different
structures, and the overall framework of BGCrack is illustrated in Fig. 2. The
details are expounded in the following content.

Figure 2:  An overview of the proposed framework.

###  3.2 Stage 1: Backbone

The function of the backbone network is to abstract and extract features. The
backbone includes two parts, the stem cell, and the hierarchical processing
part. The stem part consists of a 3 × 3 convolution with stride 2 and a depth-
wise convolution followed by a max pooling operation with a 2 × 2 kernel size
and stride of 2. The input image is downsampled 4× to reduce the redundancy
inherent in natural images, total parameters, and computation overhead by the
stem cell. The hierarchical processing part is composed of 4 stack convolution
blocks $(\rm{Conv\\_B})$, and each of them includes a convolution layer with
the kernel size of 3 × 3 and stride 1 and a depth-wise convolution. A simple
non-parametric operator, max pooling, is plugged between two blocks to
downsample feature maps similar to U-Net [37]. ConvNeXt [27], the latest
research of CNN, empirically demonstrates that (1) fewer activation functions
and normalization layers can improve network performance to a certain extent,
and (2) large kernel size such as 7 × 7 is more effective than the most widely
used size of 3 × 3. Inspired by ConvNeXt, the normalization (Norm) layers and
activation functions are re-arranged. Taking the two stack convolution layers
as an example, they are structured to “Conv→Norm→Conv→Non-linearity” rather
than the most classic structure, “Conv→Norm→Non-linearity→Conv→Norm→Non-
linearity”. It is important to note that the nonlinear activation function
adopted in BGCrack is SiLU, which is a smoothing function and has more
significant gradients than Sigmoid. The batch normalization (BN) is utilized
as the normalization layer unless noted specified. Besides, the kernel size of
depth-wise convolution layers is all set to 7 × 7. Finally, four-level
features can be extracted by Stage 1, _i.e_., ${\rm{Conv\\_B}}_{k},\
k=1,2,3,4$.

###  3.3 Stage 2: Boundary Feature Modeling

The purpose of Stage 2 is to strip boundary features $\mathbf{I}_{edge}$ from
the features modeled by Stage 1. Based on the spectral bias [35] of deep
neural networks (DNNs), we know that DNNs have a learning preference for low-
frequency information, and they learn low frequencies faster than high-
frequency information. Because the crack edge features are typical high-
frequency features, they may not be fully modeled by DNNs, and it is
inevitable to lose key information during the training process. A high-
frequency information enhancement (HFIE) module is designed to alleviate this
problem and better extract high-frequency edge features. Further, because low-
level features of CNNs contain more fine-grained detail information
represented by edges [57], the HFIE module is just embedded into the low-level
features output by Stage 1, _i.e_., ${\rm{Conv\\_B}}_{1}$ and
${\rm{Conv\\_B}}_{2}$, considering the efficiency of the algorithm. After
features pass through HFIE, they are fed into the edge (EDGE) module to
conduct the feature embedding to get 2 low-level features, ${\rm EDGE}_{1}$
and ${\rm EDGE}_{2}$. For the high-level features, they are sent to the EDGE
module, and 2 high-level features can be obtained correspondingly, _i.e_.,
${\rm EDGE}_{3}$ and ${\rm EDGE}_{4}$.

Figure 3:  An overview of the proposed framework.

HFIE Module. The design idea of the HFIE module comes from the FcaNet [34], a
frequency channel attention network, which firstly proves that attention
modules represented by SENet [21] usually discard potentially helpful
information of high-frequency components except the lowest one because of the
global average pooling, a common choice of the deep learning community to
compress channel information. Then the authors propose the multi-spectral
framework, FcaNet, to extract multi-frequency information to boost network
performance using the discrete cosine transform (DCT). However, we find the
FcaNet has two drawbacks: (1) It divides the channels of feature maps into
different blocks first, then allocates different frequency components to
different blocks, which draws into unreasonable prior information; that is,
different channels correspond to different frequencies. (2) It just considers
channel-wise frequency and ignores spatial-wise frequency. Many research
papers have demonstrated that a combination of two-dimensional (2D)
information is superior to considering only channel-wise information [43].
Therefore, the HFIE module is designed to mitigate the aforementioned issues
to enhance the representation power of networks, which is illustrated in Fig.
3.

Precisely, the HFIE module consists of two parallel sub-modules. The first is
the spatial-wise frequency enhancement sub-module to convey the
transformation,
$\mathcal{F}_{1}:\mathbf{X}\rightarrow\widetilde{\mathbf{O}}\in\mathbb{R}^{C\times
H\times W}$. The other one is the channel-wise frequency enhancement sub-
module to convey the transformation,
$\mathcal{F}_{2}:\mathbf{X}\rightarrow\hat{\mathbf{O}}\in\mathbb{R}^{C\times
H\times W}$. Here $\mathbf{X}\in\mathbb{R}^{C\times H\times W}$ represents the
input feature map, and $C$, $H$, and $W$ denote the channel, height, and
width, respectively. Combining the $\widetilde{\mathbf{O}}$ and
$\hat{\mathbf{O}}$, the output of HFIE $\mathbf{O}$ can be obtained:

$\mathbf{O}=w_{1}\widetilde{\mathbf{O}}\oplus
w_{2}\widehat{\mathbf{O}}=w_{1}\mathcal{F}_{1}(\mathbf{X})\oplus
w_{2}\mathcal{F}_{2}(\mathbf{X}),$ (1)

where $\oplus$ denotes the element-wise addition, $w_{1}$ and $w_{2}$ are the
learnable parameters.

The spatial-wise frequency enhancement sub-module will enhance frequency
related to the spatial dimension. First, $k_{1}$ one-dimensional (1D)
frequency components
$\left[f_{i}\right],i\in\left\\{1,2,\ldots,k_{1}\right\\}$ are picked. Then,
$k_{1}$ 1D DCT basis functions
$\left[\mathbf{a}^{f_{i}}\right],i\in\left\\{1,2,\ldots,k_{1}\right\\}$ can be
computed, where $\mathbf{a}^{f_{i}}\in\mathbb{R}^{C}$ means the basis function
vector of frequency $f_{i}$. Next, $k_{1}$ feature maps with different
frequencies can be computed:

$\mathbf{X}_{i}^{S}=\mathbf{X}\odot\mathbf{a}^{f_{i}},i\in\left\\{1,2,\ldots,k_{1}\right\\},$
(2)

where $\mathbf{X}_{i}^{S}\in\mathbb{R}^{C\times H\times W}$ is the feature
tensor with the basis frequency $f_{i}$ and $\odot$ represents the channel-
wise multiplication. Afterward, a channel-wise max pooling operation
$\mathcal{F}_{CMP}$, and a channel-wise sum $\mathcal{F}_{CS}$ are conducted
to exact the frequency relationship between channels. Next, a spatial
frequency adjustment tensor $\mathbf{O}_{SFA}\in\mathbb{R}^{1\times H\times
W}$ can be obtained by concatenating these $2\times k_{1}$ plane tensor,
conducting information interaction by a 3 × 3 convolution layer, and mapping
values to (0,1) using a Sigmoid function $\sigma$. Finally,
$\widetilde{\mathbf{O}}$ can be computed by the following formula:

$\mathcal{F}_{1}(\mathbf{X})=\widetilde{\mathbf{O}}=\mathbf{O}_{SFA}\circledast\mathbf{X},$
(3)

here $\circledast$ represents the spatial-wise multiplication.

The channel-wise frequency enhancement sub-module wants to enhance frequency
related to the channel dimension. Firstly, $k_{2}$ 2D frequency components
$\left[\left(f_{i}^{H},f_{i}^{W}\right)\right],i\in\left\\{1,2,\ldots,k_{2}\right\\}$
are selected. Based on these, $k_{2}$ 2D DCT basis functions
$\left[\mathbf{A}^{\left(f_{i}^{H},f_{i}^{W}\right)}\right],i\in\left\\{1,2,\ldots,k_{2}\right\\}$
can be gotten, here
$\mathbf{A}^{\left(f_{i}^{H},f_{i}^{W}\right)}\in\mathbb{R}^{H\times W}$
denotes the basis function plane array of spatial frequency
$\left(f_{i}^{H},f_{i}^{W}\right)$. Next, $k_{2}$ feature maps with different
frequencies can be computed:

$\mathbf{X}_{i}^{C}=\mathbf{X}\circledast\mathbf{A}^{\left(f_{i}^{H},f_{i}^{W}\right)},i\in\left\\{1,2,\ldots,k_{2}\right\\},$
(4)

where $\mathbf{X}_{i}^{C}\in\mathbb{R}^{C\times H\times W}$ is the feature
tensor. Then $k_{2}$ point-wise convolution layers are applied to fuse channel
contextual information and compress the number of channels to ${C/k_{2}}$:

$\mathbf{X}_{i}^{C/k_{2}}=\mathrm{PWConv}_{i}\left(\mathbf{X}_{i}^{C}\right),i\in\left\\{1,2,\ldots,k_{2}\right\\},$
(5)

here $\mathbf{X}_{i}^{C/k_{2}}\in\mathbb{R}^{\left(C/k_{2}\right)\times
H\times W}$ and $\mathrm{PW\ {Conv}}_{i}$ is $i$-th point-wise convolution.
Afterwards, concatenating $k_{2}$ $\mathbf{X}_{i}^{C/k_{2}}$, the number of
channels can be recovered, and $\mathbf{X}^{C}\in\mathbb{R}^{C\times H\times
W}$ with different frequency components can be obtained. Then, a global max
pooling operation $\mathcal{F}_{GMP}$, and a space-wise sum operation
$\mathcal{F}_{SS}$ are conducted to exact the frequency relationship along
spatial directions, and 2 vectors $\bm{max}\in\mathbb{R}^{C\times 1\times 1}$
and $\bm{sum}\in\mathbb{R}^{C\times 1\times 1}$ can be achieved.

Later, two multi-layer perceptrons (MLPs) are plugged as channel context
semantic aggregators to achieve point-wise-channel interactions with a
reduction ratio $r$, which is set to 8 like [19]. Similar to the previous sub-
module, a Sigmoid function is adopted to map the values to $\left(0,1\right)$.
Further, two trainable parameters $w_{1}$ and $w_{2}$ are added to adjust the
weights adaptively:

$\mathbf{O}_{CFA}=\sigma\left[w_{1}\times\left(\mathrm{MLP}_{1}(\bm{max})\right)+w_{2}\times\left(\mathrm{MLP}_{2}(\bm{sum})\right)\right].$
(6)

Eventually, the output $\hat{\mathbf{O}}$ can be computed by the following
equation:

$\mathcal{F}_{2}(\mathbf{X})=\widehat{\mathbf{O}}=\mathbf{O}_{CFA}\odot\mathbf{X}.$
(7)

It is important to note that $k_{1}$ and $k_{2}$ are set to 8, 1D frequency
components are set to
$\left[0\times\left\lfloor\frac{C}{8}\right\rfloor,\ldots,7\times\left\lfloor\frac{C}{8}\right\rfloor\right]$,
and 2D frequency components are set
to$\left[\left(0\times\left\lfloor\frac{H}{8}\right\rfloor,0\times\left\lfloor\frac{W}{8}\right\rfloor\right),\ldots,\left(7\times\left\lfloor\frac{H}{8}\right\rfloor,7\times\left\lfloor\frac{W}{8}\right\rfloor\right)\right]$
simply in the all the experiments. Obviously, the choice of the fundamental
frequency is not unique. FcaNet has conducted a very complex parameter
selection to achieve the best results for the specific dataset based on the
selected frequency set, but the frequency set is not universally
representative. HFIE takes the most simple and fundamental set to show the
generalization and effectiveness of the algorithm. It is obvious that the
performance can be improved further after parameter optimization like FcaNet.

###  3.4 Stage 3: Global Feature Modeling

The purpose of Stage 3 is to model global semantic features
$\mathbf{I}_{body}$ from the features extracted by Stage 1. Similar to Stage
2, 4 modules named the body module (BODY) are designed to embed features from
Stage 1, and the corresponding multi-level features are ${\rm BODY}_{k},\
k=1,2,3,4$. For design simplicity and consistency, the architecture of the
BODY module is the same as that of the EDGE module.

However, unlike Stage 2, the global feature modeling of cracks needs to pay
attention to the long-term context information, which can be roughly explained
by the recognition process of the human visual system. As shown in Fig. 4 (a),
it is difficult to judge which one is the real crack just based on the short-
range features. However, we can quickly know that the top one is the crack and
the below one is an artificial marker once taking the more context semantic
information and the long-term dependency relationship of pixels into
consideration (see Fig. 4 (b)), namely, we can model the global features of
cracks more accurately. To achieve this, a novel global information perception
(GIP) module is designed to capture and model long-distance semantic
information better and enhance the perception capability of networks for
global information. Additionally, because high-level features of CNNs have a
larger receptive field, which brings more redundant long-range context
information [57, 51], embedding GIP into high-level features is more
effective. Therefore, ${\rm BODY}_{3}$ and ${\rm BODY}_{4}$ are appended by a
GIP module, respectively. On the other hand, high-level features have a
smaller spatial resolution, so this plugging scheme can also bring less
calculation quantity.

Figure 4:  Comparison between local information and long-range semantic
information. (a) Local information. (b) Long-range semantic information.

GIP module. Motivation. The inherent locality of convolution operations causes
CNNs to be experts in modeling local information but have congenital
limitations and yield subpar performances in capturing long-term context
features [3], such as cracks’ global semantic information that shows massive
changes with respect to modality and size. Therefore, how to improve the
global perception capability of networks has become a hotspot issue and has
drawn extensive attention from academic and industrial circles.

An effectual method is ushering the ingredients with global awareness into
native CNNs, such as the Fast Fourier Transform (FFT) algorithm, which can
model long-range context information from a view of the frequency domain. Some
influential models have been proposed based on FFT, represented by LaMa [40],
a compelling image inpainting model using Fast Fourier Convolution (FFC) [6],
and DeepRFT [28], which is an advanced image deblurring framework, utilizing
residual convolution blocks with FFT. Inspired by the success of FFT in CNNs,
it is also drawn into the GIP module to model global contexts in the frequency
domain.

Additionally, another way is to substitute CNNs with other units, among which
the most representative framework is Transformer [42], which can capture long-
term dependencies because the self-attention mechanism in it can bring
intrinsically global perception without the complex hierarchical stacking
structures like CNNs. Transformer has achieved tremendous and widespread
success in multiple fields, such as natural language processing [42], machine
translation [13], and computer vision [14]. Specifically, it has been regarded
as one of the most significant research progress in the field of computer
vision, and some frameworks based on Transformer have also made an enormous
impact, such as ViT [14], DPT [36], and Swin Transformer [26]. Therefore,
Transformer is also led into the GIP module to learn long-range relations in
the pixel domain, aka the time domain.

Then, the overall structure of the GIP module can be acquired, which is
resented in Fig. 5 and consists of two branches: The first one is named the
frequency domain branch, in which the GIP module performs Discrete Fourier
Transform (DFT) on a feature map by the FFT algorithm to provide the global
perception. The second one is the time domain branch, in which Transformer is
plugged to extract long-distance context dependency. Then a fusion model is
designed to integrate the input and two outputs. To the authors’ best
knowledge, the GIP module takes the lead in integrating global information
from the frequency domain and time domain organically, which can enhance the
representation power for global features of cracks effectively to improve
identification accuracy. The detailed design is illustrated below.

Figure 5:  The architecture of the global information perception (GIP) module.
LN_VT is the layer normalization (LN) method used in most vision transformer
frameworks. LN_PyTorch is the default LN in PyTorch. The difference is that
the affine transformation parameters of LN_VT are channel-wise, and the
parameters of LN_PyTorch are element-wise.

DFT branch. Let the input feature map be $\mathbf{M}\in\mathbb{R}^{C\times
H\times W}$. (1) Performing 2D DFT on $\mathbf{M}$ by the 2D real FFT
algorithm to get the complex spectrum:

$\mathcal{F}(\mathbf{M})=\operatorname{RFFT}(\mathbf{M}),$ (8)

where $\mathcal{F}\left(\mathbf{M}\right)\in\mathbb{C}^{C\times
H\times\left(\left\lfloor{W/2}\right\rfloor+1\right)}$ is the complex
spectrum, here the conjugate symmetry of spectrums is adopted to compress the
width dimension, and RFFT represents the 2D real FFT algorithm. (2) Decoupling
the real and imaginary parts of the complex spectrum to gain
$\mathcal{R}\left(\mathcal{F}\left(\mathbf{M}\right)\right)\in\mathbb{R}^{C\times
H\times\left(\left\lfloor{W/2}\right\rfloor+1\right)}$, and
$\mathcal{I}\left(\mathcal{F}\left(\mathbf{M}\right)\right)\in\mathbb{R}^{C\times
H\times\left(\left\lfloor{W/2}\right\rfloor+1\right)}$, respectively. (3)
Concatenating $\mathcal{R}\left(\mathcal{F}\left(\mathbf{M}\right)\right)$ and
$\mathcal{I}\left(\mathcal{F}\left(\mathbf{M}\right)\right)$ along the channel
direction and using a block to model the new compound features. The design of
the block refers to the MetaFormer [50]. It can be simply described as
“BN→$\rm{DW\ Conv}$→BN→$\rm{PW\ {Conv}}_{1}$→SiLU→$\rm{PW\ {Conv}}_{2}$”,
where $\rm{DWConv}$ denotes the depth-wise convolution with the 7 × 7 kernel
size, $\rm{PW\ {Conv}}_{1}$ compresses the number of channels to a quarter of
its previous size, and the function of $\rm{PW\ {Conv}}_{2}$ is to restore the
number of channels. Next, transforming the real features into the complex
field to get a new complex spectrum
$\widetilde{\mathcal{F}}\left(\mathbf{M}\right)\in\mathbb{C}^{C\times
H\times\left(\left\lfloor{W/2}\right\rfloor+1\right)}$. (4) Performing inverse
2D real FFT to convert the frequency domain into the time domain. Then, the
output feature map $\mathbf{O}_{d}\in\mathbb{R}^{C\times H\times W}$ can be
acquired after a convolution block with 2 3 × 3 convolutions.

Transformer branch. Although Transformer has many advantages compared with
CNNs, every coin has two sides. Transformer’s design has an inherent defect
that lacks the spatial inductive bias, the intrinsic property of CNNs [45],
which causes that: (1) Transformer needs more parameters to lean visual
representations and is heavy-wight [29], _e.g_., ViT-L/16 vs. MobileNetv3
[20]: 307M vs. 7.5 M parameters, and DPT-Large vs. DeepLabv3 [4]: 343M vs. 59
M parameters. (2) It needs to be trained on sufficient amounts of data to
generalize well [14], _e.g_., ViT is trained on the JFT-300M dataset
containing over 300M images, and DeiT [41] relies on extensive data
augmentation tricks. These drawbacks make Transformer need rich computing
resources and unfriendly for tasks without rich data, such as the crack
identification task involved in this paper. Therefore, some researchers
propose CNN-Transformer hybrid models meriting both the local perception and
spatial inductive bias of CNN and the global modeling ability of Transformer
to apply to downstream tasks successfully, _e.g_., TransUNet [3]for medical
image segmentation and MSHViT [18] for defect detection. Essentially, BGCrack
is also a hybrid solution, and Transformer is embedded in a CNN. On the other
hand, some authors try to modify the Transformer and usher the spatial
inductive bias into the vanilla Transformer. A representative method is
MobileViT [29], which implicitly incorporates convolution-like characteristics
into Transformer to make it learn global representations with the spatial bias
to achieve the goals of lightweight and better generalization ability [29].
Inspired by MobileViT, a similar lightweight Transformer block is exploited to
substitute the vanilla Transformer.

As shown in Fig. 5, let the input feature map be
$\mathbf{M}\in\mathbb{R}^{C\times H\times W}$ as well. (1) a point-wise
convolution is utilized to project the input features into a high-dimensional
space to get a d-dimensional tensor $\mathbf{M}_{1}\in\mathbb{R}^{d\times
H\times W}$, where $d>C$. (2) A convolution block containing two layer
normalization (LN) layers, a $n\times n$ depth-wise convolution layer, and a
point-wise convolution layer is appended to get the local representation
$\mathbf{M}_{2}\in\mathbb{R}^{d\times H\times W}$. (3) In order to make this
module to be able to earn global information with the spatial inductive bias,
$\mathbf{M}_{2}$ is unfolded into N non-overlapping flattened patches in the
spatial dimension to acquire $\mathbf{M}_{U}\in\mathbb{R}^{N\times P\times
d}$, where $N=\frac{H\times W}{P}$ represents the number of patches,
$P=h_{P}\times w_{P}$ denotes the area of patches, and $h_{P}$ and $w_{P}$
means the height and width of patches, respectively. The following Transformer
would calculate the attention among patches to learn global representations.
However, it doesn’t encode the local information in each patch, which induces
there are “holes” in the modeled global attention. Therefore, with an eye to
extracting full long-range contexts, the local receptive field brought by the
depth-wise convolution in (1) needs to cover patches, _i.e_., $h_{P}\leq n$
and $w_{P}\leq n$. (4) For every vector $\mathbf{v}_{p}\in\mathbb{R}^{1\times
1\times d},p\in\left\\{1,2,\ldots,P\right\\}$ within each patch, the inter-
patch relationships are modeled by Transformer, and $P$ global correlation
feature maps $\mathbf{M}_{T}^{p}\in\mathbb{R}^{N\times 1\times d}$ are gained:

$\mathbf{M}_{T}^{p}=\text{ Transformer }\left(\mathbf{M}_{U}^{p}\right),$ (9)

where $\mathbf{M}_{U}^{p}\in\mathbb{R}^{N\times 1\times d}$ denotes the
combination of all the $p$-th vector $\mathbf{v}_{p}$ in $N$ patches.
Concatenating all the $\mathbf{M}_{T}^{p}$ along the patch dimension, the
feature map after the Transformer can be acquired, let it be
$\mathbf{M}_{T}\in\mathbb{R}^{N\times P\times d}$. Because the spatial
locations of pixels in each patch and inter-patch spatial relationships are
retained, it keeps spatial inductive bias compared with the conventional
Transformer. (5) Folding $\mathbf{M}_{T}$ to recover the shape of the feature
map before unfolding to achieve $\mathbf{M}_{F}\in\mathbb{R}^{d\times H\times
W}$. (6) A point-wise convolution is adopted to embed $\mathbf{M}_{F}$ into
$C$-dimensional to get the output features
$\mathbf{O}_{t}\in\mathbb{R}^{C\times H\times W}$.

Fusion. A fusion block is designed to fuse an input $\mathbf{M}$ and two
outputs $\mathbf{O}_{d}$ and $\mathbf{O}_{t}$. First, three learnable
parameters $w_{1}$, $w_{2}$, and $w_{3}$ are added to adjust the weights:

$\mathbf{M}_{\text{Fuse
}}=w_{1}\times\mathbf{O}_{d}+w_{2}\times\mathbf{M}+w_{3}\times\mathbf{O}_{t}.$
(10)

Then a simple convolution block with the structure of “3 × 3 Conv→BN→3 × 3
Conv→SiLU” is applied to model the fused feature map $\mathbf{M}_{Fuse}$ to
obtain the final output of the GIP module
$\mathbf{O}_{G}\in\mathbb{R}^{C\times H\times W}$.

Figure 6:  The structures of the proposed modules. (a) Self-fusion module
(SFM). (b) Cross optimization module (COM).

###  3.5 Stage 4: Feature Joint Optimization

This stage is the fusion and joint optimization stage of the boundary features
$\mathbf{I}_{edge}$ output by Stage 2 and the global features
$\mathbf{I}_{body}$ obtained by Stage 3. To simplify notations, let the 8
output features of two stages be $E_{j}^{0},\ j=1,2,3,4$ and $B_{j}^{0},\
j=1,2,3,4$, respectively. Here $E_{j}^{i}$ denotes the $j$-th boundary feature
map in the $i$-th calculation phase, and $i=0,1,2$ (see Fig. 2 above). For the
same reason, we have $B_{j}^{i}$. Besides, the dense connection design is
incorporated into this stage to improve the information flow, strengthen the
feature propagation and maintain network gradients. Each set of tensors is
directly added to all the corresponding tensors before and then input to the
next module (see Fig. 2).

In this stage, first of all, a self-fusion module (SFM) is designed to promote
information interaction and fusion of multi-level and multi-scale features
inside the boundary feature set and global feature set to get $E_{j}^{1},\
j=1,2,3,4$ and $B_{j}^{1},\ j=1,2,3,4$. Afterward, because the boundary
features are helpful to improve the body features, and the body features are
also beneficial for refining the boundary features, a cross optimization
module (COM) is further developed to promote the integration between the two
kinds of features to gain $E_{j}^{2},\ j=1,2,3,4$ and $B_{j}^{2},\ j=1,2,3,4$.
After that, two feature fusion modules (FFM), _i.e_., ${\rm FFM}_{E}$ for edge
features, and ${\rm FFM}_{B}$ for body features, are proposed to align the
spatial dimensions, namely, recover the size from inputting size, $H\times W$,
and the corresponding outputs $\mathbf{O}_{E}\in\mathbb{R}^{1\times H\times
W}$ and $\mathbf{O}_{B}\in\mathbb{R}^{1\times H\times W}$ are achieved.

The detailed interpretation of the architectures of the above-mentioned
modules, _i.e_., SFM, COM, and FFM, can be seen in the next subsections.

SFM. The structure of the SFM is presented in Fig. 6 (a). The fusion of the
boundary features is the same as that of the global features, so, for
simplicity, we introduce the fusion of the boundary features. The fusion
includes two branches, namely, an upsampling branch and a downsampling branch.
In the upsampling branch, the spatial size of the feature map is gradually
enlarged, and the low-level feature map is fused with the current features
simultaneously. By this branch, the high-level and large-scale information can
spread to low-level and small-scale feature maps, and new tensors are gained,
$E_{1,u}^{1}$, $E_{2,u}^{1}$, and $E_{3,u}^{1}$. The purpose and process of
the downsampling branch are exactly the opposite of the upsampling branch. In
the downsampling branch, the low-level and small-scale information is
integrated into the high-level and large-scale feature maps, and
correspondingly, $E_{2,d}^{1}$, $E_{3,d}^{1}$, and $E_{4,d}^{1}$ are gotten.
The final output of the SFM is the addition of the upsampling features and
downsampling features simply: $E_{1}^{1}=E_{1,u}^{1}+E_{1}^{0}$,
$E_{2}^{1}=E_{2,u}^{1}+E_{2,d}^{1}$, $E_{3}^{1}=E_{3,u}^{1}+E_{3,d}^{1}$, and
$E_{4}^{1}=E_{4}^{0}+E_{4,d}^{1}$.

COM. Before designing the COM, the relationship between $\mathbf{I}_{body}$
and $\mathbf{I}_{edge}$ needs to be clarified in advance. As introduced
before, a crack can be decoupled into two parts, namely, the crack boundary
$\mathbf{I}_{edge}$ and internal area $\mathbf{I}_{inner}$. Therefore, the
logical interrelations of crack features can be obtained:

$\mathbf{I}_{body}=\mathbf{I}_{body}\cup\mathbf{I}_{\text{edge }}.$ (11)

Further, we have

$\mathbf{I}_{\text{edge }}=\mathbf{I}_{\text{edge }}\cap\mathbf{I}_{\text{body
}}.$ (12)

Based on the logical interrelations, it is clear that on the one hand,
$\mathbf{I}_{edge}$ can improve the accuracy of $\mathbf{I}_{body}$, and on
the other hand, $\mathbf{I}_{body}$ is able to refine $\mathbf{I}_{edge}$
[21], which is also the design motivation of COM. Because Eq. (12) are
difficult to represent by a CNN, they are further converted to the following
equation equivalently:

$\mathbf{I}_{\text{edge }}=\mathbf{I}_{\text{edge
}}\otimes\mathbf{I}_{\text{body }},$ (13)

where $\otimes$ denotes element-wise multiplication. Based on the equations,
the COM is designed, and the details are shown in Fig. 6 (b).

Figure 7:  The structures of the proposed modules. (a) Self-fusion module
(SFM). (b) Cross optimization module (COM).

Specifically, the COM can be defined as

$\displaystyle\mathrm{E}_{j}^{2}=\mathcal{E}\left(\mathrm{E}_{j}^{1},\left\\{\mathrm{~{}B}_{k}^{1}\right\\}_{k=j}^{4}\right),$
(14a)
$\displaystyle\mathrm{B}_{j}^{2}=\mathcal{B}\left(\mathrm{B}_{j}^{1},\left\\{\mathrm{E}_{k}^{1}\right\\}_{k=j}^{4}\right),$
(14b)

where $\mathcal{E}$ and $\mathcal{B}$ represent the boundary feature
optimization function and global feature optimization function, respectively.
Because the lower-level features contain more distractors and background
information, and the higher-level features can extract semantic and
discriminative information with few distractors [44], the current features are
optimized by the same level or higher-level features, _e.g_.,
$\left\\{B_{k}^{1}\right\\}_{k=j}^{4}$ are adopted to fine $E_{j}^{2}$,
instead of the lower-level $\left\\{B_{k}^{1}\right\\}_{k=1}^{j-1}$.
Therefore, the Eq. (14a) can be expanded as

$\mathrm{E}_{j}^{2}=we_{0}^{j}\mathrm{E}_{j}^{1}+we_{1}^{j}\widehat{\mathrm{E}}_{j}^{1}+we_{2}^{j}\widehat{\mathrm{E}}_{j+1}^{1}+\cdots+we_{4-j+1}^{j}\widehat{\mathrm{E}}_{4}^{1},$
(15)

where
$\mathbf{we}=\left({we}_{0}^{j},{we}_{1}^{j},\ldots,{we}_{4-j+1}^{j}\right)$
is a learnable weight parameter vector. The Eq. (14b) can also be expanded as

$\mathrm{B}_{j}^{2}=wb_{0}^{j}\mathrm{~{}B}_{j}^{1}+wb_{1}^{j}\hat{\mathrm{B}}_{j}^{1}+wb_{2}^{j}\hat{\mathrm{B}}_{j+1}^{1}+\cdots+wb_{4-j+1}^{j}\hat{\mathrm{B}}_{4}^{1},$
(16)

where
$\mathbf{wb}=\left({wb}_{0}^{j},{wb}_{1}^{j},\ldots,{wb}_{4-j+1}^{j}\right)$
is another learnable weight parameter vector.

FFM. To put it simply, FFM is a hierarchical and multi-level feature fusion
module whose designs are inspired by U-Net. For integrating the boundary and
global features, two similar blocks, _i.e_., ${\rm FFM}_{E}$ and ${\rm
FFM}_{B}$, are developed (see Fig. 7 above). Next, taking the ${\rm FFM}_{B}$
as an example, introduce the FFM’s structure. Because global features are the
final purpose, ${\rm FFM}_{B}$ not only fuse 4 global sub-features,
$B_{j}^{2},\ j=1,2,3,4$, but also integrate 4 edge sub-features, $E_{j}^{2},\
j=1,2,3,4$, to refine the global output further. Firstly, it starts from the
highest-level features $B_{4}^{2}$ and $E_{4}^{2}$, and concatenates them
along the channel dimension. Then, a depth-wise convolution and a standard 3 ×
3 convolution are appended to extract information. Next, a transposed
convolution is adopted to enlarge the combined feature map to twice its
original spatial size. The above process is repeated until the lowest-level
features $B_{1}^{2}$ and $E_{1}^{2}$ are fused. After that, $X_{S}$, output
from the stem part, is integrated further to supplement details. Finally, a
point-wise convolution layer is utilized to depress the number of channels to
1. For the convenience of differentiation, we let the outputs be
$\mathbf{P}_{E}\in\mathbb{R}^{1\times H\times W}$ and
$\hat{\mathbf{P}}_{B}\in\mathbb{R}^{1\times H\times W}$, respectively.
Further, $\mathbf{P}_{E}$ is integrated into the $\hat{\mathbf{P}}_{B}$ to
refine the global features to get the final global feature output
${\mathbf{P}}_{B}$, as shown in Fig. 2.

###  3.6 Loss Function

First of all, the most classic binary cross-entropy (BCE) loss is applied to
the similarity assessment between the prediction and ground-truth. The BCE
loss between the global output and the corresponding ground-truth is

$\displaystyle\mathcal{L}_{\mathrm{BCE}}^{B}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\left\\{-\frac{1}{HW}\sum_{(h,w)}\left[\mathbf{G}_{B}^{i}(h,w)\times\log\left({\mathbf{P}}_{B}^{i}(h,w)\right)\right]\right.$
(17)
$\displaystyle-\frac{1}{HW}\left.\sum_{(h,w)}\left[\left(1-\mathbf{G}_{B}^{i}(h,w)\right)\times\log\left(1-{\mathbf{P}}_{B}^{i}(h,w)\right)\right]\right\\},$

Figure 8:  Representative image samples in the benchmark dataset.

where $N$ is the batch size, ${{\mathbf{P}}}_{B}^{i}$ denotes $i$-th
prediction in a mini-batch, and
$\mathbf{G}_{B}^{i}\in\left\\{0,1\right\\}^{1\times H\times W}$ is the
corresponding true label. Moreover, the BCE loss between the edge output and
the corresponding pixel-level label can be formulated as follows:

$\displaystyle\mathcal{L}_{\mathrm{BCE}}^{E}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}\left\\{-\frac{1}{HW}\sum_{(h,w)}\left[\mathbf{G}_{E}^{i}(h,w)\times\log\left({\mathbf{P}}_{E}^{i}(h,w)\right)\right]\right.$
(18)
$\displaystyle-\frac{1}{HW}\left.\sum_{(h,w)}\left[\left(1-\mathbf{G}_{E}^{i}(h,w)\right)\times\log\left(1-{\mathbf{P}}_{E}^{i}(h,w)\right)\right]\right\\},$

where $\mathbf{P}_{E}^{i}$ denotes $i$-th boundary prediction in a mini-batch,
and $\mathbf{G}_{E}^{i}\in\left\\{0,1\right\\}^{1\times H\times W}$ represents
the corresponding ground-truth.

On the other hand, the number of crack pixels in photographs is significantly
lower than the number of non-crack pixels since most cracks are fine and
unnoticeable. Therefore, the crack/non-crack pixels are of a highly imbalanced
distribution, and fundamentally, the crack prediction belongs to a class
imbalance problem. Dice loss [12] concentrates on measuring the overlap and
similarity between the prediction and ground-truth. It can thereby alleviate
the class-imbalance issue because it is insensitive to the amount of
foreground and background pixels [12]. Consequently, the Dice loss is also
adopted to train networks and can be defined by

$\mathcal{L}_{\text{Dice
}}^{B}=\frac{1}{N}\sum_{i=1}^{N}\left\\{1-\frac{2\sum_{(h,w)}\left[{\mathbf{P}}_{B}^{i}\times\mathbf{G}_{B}^{i}\right]+\epsilon}{\sum_{(h,w)}\left[{\mathbf{P}}_{B}^{i}\right]+\sum_{(h,w)}\left[\mathbf{G}_{B}^{i}\right]+\epsilon}\right\\},$
(19)

where $\epsilon$ is a Laplace smoothing item to avoid zero division and is set
to ${10}^{-6}$. Furthermore, the identification of crack edges is also a class
imbalance problem, so the Dice loss function can also be employed as a metric
to represent the overlap between $\mathbf{P}_{E}^{i}$ and
$\mathbf{G}_{E}^{i}$:

$\mathcal{L}_{\text{Dice
}}^{E}=\frac{1}{N}\sum_{i=1}^{N}\left\\{1-\frac{2\sum_{(h,w)}\left[{\mathbf{P}}_{E}^{i}\times\mathbf{G}_{E}^{i}\right]+\epsilon}{\sum_{(h,w)}\left[{\mathbf{P}}_{E}^{i}\right]+\sum_{(h,w)}\left[\mathbf{G}_{E}^{i}\right]+\epsilon}\right\\},$
(20)

Further, the gradients of images can represent the pixel changes in images,
and the gradients in the crack edge areas are greater than those in the
interior portions. In order to improve the representation power of the global
output to crack edges and enhance the prediction accuracy, a Charbonnier loss
function [22] is applied to assess the gradient similarity:

$\mathcal{L}_{\mathrm{Grad}}=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{HW}\sum_{(h,w)}\left[\sqrt{\left(\mathbf{P}_{G}^{i}-\mathbf{G}_{G}^{i}\right)^{2}+\varepsilon^{2}}\right],$
(21)

where $\mathbf{P}_{G}^{i}$ is the $i$-th gradient image and obtained by
conducting the Scharr operator on the predicted output
${{\mathbf{P}}}_{B}^{i}$, and $\mathbf{G}_{G}^{i}$ denotes the $i$-th ground-
truth gradient image and achieved by conducting the Scharr operator on the
ground-truth label $\mathbf{G}_{B}^{i}$ similarly. $\varepsilon$ is set to
${10}^{-3}$ following [22].

Ultimately, the overall loss $\mathcal{L}_{Total}$ is a mixed loss function
combing aforementioned loss functions:

$\mathcal{L}_{\text{Total
}}=\alpha_{1}\mathcal{L}_{\mathrm{BCE}}^{B}+\alpha_{2}\mathcal{L}_{\mathrm{BCE}}^{E}+\alpha_{3}\mathcal{L}_{\text{Dice
}}^{B}+\alpha_{4}\mathcal{L}_{\text{Dice
}}^{E}+\alpha_{5}\mathcal{L}_{\mathrm{Grad}},$ (22)

where $\alpha_{1}$, $\alpha_{2}$,$\alpha_{3}$, $\alpha_{4}$ and $\alpha_{5}$
are five weighting hyperparameters to regulate the impact of different sub-
loss functions and are set to 1 simply.

##  4 Steel Crack Dataset

The proposed benchmark dataset composed of steel crack images is introduced in
this section, which is named Steelcrack for convenience of expression.

To enhance the practicality of datasets, all the images in these datasets are
directly captured from multifarious practical engineering structure projects
in diverse scenarios, such as Nanjing Second Yangtze River Bridge, Humen
Bridge, etc. Consequently, these images have different backgrounds and crack
characteristics regarding length, width, and color. Moreover, defect images
are typically captured by workers or inspection equipment in actual
engineering, so poor imaging quality is inevitable. In order to adapt to these
real-world shooting conditions, images with varying levels of blurring are
included in the dataset. Some typical and representative images coming from
the dataset are shown in Fig. 8.

After obtaining defect images, flip and rotation tricks are adopted for data
augmentation. Then, small patches are cropped from the original images to form
the Steelcrack, following the settings of other open-source datasets.
Specifically, all the images have a fixed pixel resolution, 512 × 512, and the
numbers of images of the training set, validation set, and test set are 3300,
525, and 530, respectively. Details are listed in Table 1 above as well.

Table 1: Details of the proposed datasets.

Dataset name |  Image Size |  Number of images of the training set |  Number of images of the validation set |  Number of images of the test set
---|---|---|---|---
Steelcrack | 512 × 512 | 3300 | 525 | 530

##  5 Implementation Details

###  5.1 Training and Inference Environment

The hardware environment of training and testing algorithms is as follows:
Intel Xeon E5-2697 @ 2.70 GHz CPU, 3 $\times$ Nvidia GeForce RTX 2080Ti GPUs,
and 256 GB RAM. The main software environment: Ubuntu 18.04 operating system,
CUDA 11.6.2, CUDNN 8.6.0, and Python 3.8.15. The deep learning framework
employs the PyTorch developed by Meta, and the version is 1.12.1.

###  5.2 Training Policy

The Adam optimizer is utilized for training networks, and the hyper-parameters
are set to the default values that are $\beta=\ \left(0.9,\ 0.999\right)$,
$\varepsilon={10}^{-8}$ and $weight\ decay=0$. The batch size is set as 9, and
the learning rate is fixed to $6\times{10}^{-3}$ unless specified otherwise.
Networks are all trained for 70 epochs before inference. For simplicity, no
additional data augmentation tricks and learning rate schedulers are adopted
in the training stage. It is noteworthy that all the models are implemented in
this unified training pipeline for equitable and standardized comparisons.

###  5.3 Evaluation Metrics

In order to validate and evaluate the performance of proposed models, two
types of metrics are adopted, namely, effect and efficiency metrics.

Effect metrics. In pixel-level segmentation tasks, class-wise metrics, the
mean intersection over union (mIoU) and Dice coefficient (Dice), are the two
most widely used metrics. However, for the crack detection task, engineers pay
more attention to the volume of defects in a single image, so the performance
of algorithms on each image is more worthy of attention. Therefore, mIoU and
Dice are modified to image-wise metrics and named the mean image-wise
Intersection over Union (mi IoU) and mean image-wise Dice coefficient (mi
Dice), respectively. The calculation process of them is introduced as follows:

mi IoU. Let the $n$-th predicted results of a model be
${\widetilde{\mathbf{P}}}^{n}\in\left(0,1\right)^{1\times H\times
W},n\in\left[1,N\right]$, here $N$ denotes the number of images to be
evaluated. As a binary classification problem, pixels are firstly divided into
positive and negative pixels with the threshold segmentation method, and 0.5
is usually used as the classification threshold:

$\text{ Class
}_{\widetilde{\mathbf{P}}_{i,j}^{n}}=\left\\{\begin{array}[]{cl}0\text{ means
Negative }&\tilde{\mathbf{P}}_{i,j}^{n}\in(0,0.5)\\\ 1\text{ means Postive
}&\widetilde{\mathbf{P}}_{i,j}^{n}\in[0.5,1)\end{array},\right.$ (23)

where
${\widetilde{\mathbf{P}}}_{i,j}^{n}\left(i\in\left[1,H\right],j\in\left[1,W\right]\right)$
means the output value in spatial position $\left(i,j\right)$. On the other
hand, the ground-truth of the $n$-th image be
$\mathbf{G}^{n}\in\left\\{0,1\right\\}^{1\times H\times W}$. By comparing
${\rm Class}_{{\widetilde{\mathbf{P}}}_{i,j}^{n}}$ and
$\mathbf{G}^{n}$,${TP}^{n}$, ${FN}^{n}$ and ${FP}^{n}$ can be obtained, where
${TP}^{n}$ represents the number of true positive pixels, ${FN}^{n}$ denotes
the number of false negative pixels, and ${FP}^{n}$ is the number of false
positive pixels. Then, mi IoU be obtained:

$\operatorname{mi}\mathrm{IoU}=\frac{1}{N}\sum_{n=1}^{N}\frac{TP^{n}+\varepsilon}{FN^{n}+FP^{n}+TP^{n}+\varepsilon},$
(24)

here $\varepsilon$ is a constant with a small value to prevent the denominator
from being zero and is set to ${10}^{-6}$.

mi Dice. mi Dice be defined by the following formula:

$\text{ mi Dice
}=\frac{1}{N}\sum_{n=1}^{N}\frac{2\left|\tilde{\mathbf{P}}^{n}\otimes\mathbf{G}^{n}\right|+\varepsilon}{\left|\tilde{\mathbf{P}}^{n}\right|+\left|\mathbf{G}^{n}\right|+\varepsilon}.$
(25)

Obviously, mi IoU and mi Dice are a discrete evaluation metric and a
continuous evaluation metric, respectively, and both of them have a positive
correlation with the models’ performance. Additionally, mi Dice is more
interested in the similarity at the image level, whereas mi IoU is inclined to
explore the pixel-level performance of predictions. Consequently, combining
both of them will result in a more accurate and thorough evaluation of model
performance.

Efficiency metrics. Two measures are presented in this research to assess the
models’ computational consumption. One is ”#Param.” which is used to gauge the
size of a deep learning model and the value of #Param. is equal to the number
of parameters in a model. Multiply-accumulate operations (MACs) are another
indicator used to assess networks’ computational expense. Ptflops, an easy-to-
use open-source tool for PyTorch, is the only tool to be used to calculate the
values of these two efficiency indexes in this paper to maintain consistency
in evaluation criteria.

##  6 Experiment

###  6.1 Comparison with Representative Segmentation Networks

BGCrack is compared with some influential segmentation algorithms, such as
U-Net and its classical variants, DeepLabv3+ [5], SCRN [44], and TransUNet
[3], from the perspectives of quantitative and qualitative analysis. The
corresponding results are listed in Table 2 and summarized in Fig. 9,
respectively.

Quantitative comparison. Table 2 shows the detailed comparison results and the
highest scores are bolded. It is clear that BGCrack, while using fewer
parameters and calculations, is still able to outperform all other models in
terms of all assessment measures.

Table 2: Quantitative evaluation of different methods.

Method | mi IoU (%) | mi Dice (%) | #Param. (M) | MACs (G)
---|---|---|---|---
U-Net [37] | 68.49 | 75.13 | 7.77 | 55.01
U-Net (large) [37] | 69.81 | 76.85 | 31.04 | 219.01
U-Net++ [56] | 72.23 | 78.37 | 9.16 | 138.63
Attention U-Net [32] | 71.25 | 77.54 | 34.88 | 266.54
CE-Net[15] | 76.00 | 81.54 | 29.00 | 35.60
DeepLabv3+ MobileNetv2[5] | 68.22 | 71.07 | 5.81 | 29.13
DeepLabv3+ xception[5] | 67.40 | 71.48 | 54.70 | 83.14
DeepLabv3+ ResNet-101[5] | 69.04 | 49.45 | 59.34 | 88.84
SCRN [44] | 73.23 | 78.91 | 25.23 | 31.29
TransUNet [3] | 64.34 | 72.55 | 67.87 | 129.96
BGCrack | 77.16 | 85.33 | 2.32 | 15.76

Qualitative visualization results. As shown in Fig. 9 below, BGCrack can
accurately and consistently identify cracks with different backgrounds,
features, and artificial markers compared to other deep-learning models.
Besides, the predicted cracks of BGCrack are more transparent and have less
noise than that of other methods. The shape of the predicted cracks is also
more in line with the ground truth. This directly shows the gain effect of
boundary information on recognition results. In general, the visualization
results reflect BGCrack’s strong robustness and generalization ability.

Figure 9:  Visualization results of different models.

###  6.2 Ablation Study

Ablation experiments are conducted to validate the rationality of the design.

Ablation of the boundary feature modeling. One of the main contributions of
this paper is to introduce crack boundary features to crack identification. An
ablation experiment is conducted to explore the effectiveness of this idea.
Specifically, parts and modules related to the edge feature modeling are
removed from the BGCrack, and the remaining parts are composed of a new sub-
network. Then, it is trained using the total loss function removing
$\mathcal{L}_{BCE}^{E}$ and $\mathcal{L}_{Dice}^{E}$. The corresponding
evaluation results are shown in Table 3. Obviously, considering edge features
is helpful to segment cracks accurately based on the comparison in Table 3. We
hope this study can inspire researchers to conduct broader and deeper research
in this direction.

Table 3: Ablation studies of the boundary feature modeling.

Edge | mi IoU (%) | mi Dice (%) | #Param. (M) | MACs (G)
---|---|---|---|---
1F5F4 | 73.57 | 82.57 | 1.94 | 8.17
1F5F8 | 77.16 | 85.33 | 2.32 | 15.76

Effectiveness of the gradient loss function. The effectiveness of the gradient
loss function $\mathcal{L}_{Grad}$ is evaluated here. We take the other 4 loss
functions keeping the same weighting hyperparameters to train the BGCrack and
record the corresponding results in Table 4 above. Based on the data in Table
4, the performance of the model is reduced if the gradient loss function is
removed from the total loss, which verifies the availability of the gradient
loss.

Table 4: Effectiveness validation of the gradient loss function.

Loss | mi IoU (%) | mi Dice (%) | #Param. (M) | MACs (G)
---|---|---|---|---
1F5F4 | 76.53 | 84.85 | 2.32 | 15.76
1F5F8 | 77.16 | 85.33 | 2.32 | 15.76

Ablation experiments of the HFIE and GIP modules. The HFIE and GIP modules are
adopted to enhance high-frequency information and global perception,
respectively. They are both lightweight and plug-and-play modules. In this
subsection, ablation experiments are conducted to validate the effectiveness
of the two modules. As shown in Table 5, HFIE and GIP can both enrich
representation ability with negligible computational overheads.

Table 5: Effectiveness of the HFIE and GIP.

Model | mi IoU (%) | mi Dice (%) | #Param. (M) | MACs (G)
---|---|---|---|---
-HFIE-GIP | 74.06 | 82.93 | 2.16 | 15.54
-GIP | 74.37 | 83.27 | 2.19 | 15.68
-HFIE | 75.89 | 84.21 | 2.28 | 15.62
BGCrack | 77.16 | 85.33 | 2.32 | 15.76

###  6.3 Visualization Analysis

To delve into if BGCrack can pay attention to defects areas correctly, the
feature maps are visualized by Grad-CAM [38], which is one of the most
influential visual explanation tools and can reveal key evidence and reasons
that models use for reference. The heat maps of global and boundary features
are presented in Fig. 10. Obviously, BGCrack focuses on the crack locations
correctly, which provides a visual explanation for the high recognition
accuracy of the model.

Figure 10:  Visualization feature maps generated by Grad-CAM.

##  7 Conclusion

Cracks are one of the critical signs of structural performance degradation,
and improving the accuracy of crack identification is an issue that has
attracted extensive attention. The conventional processing method uses
advanced deep learning models directly, such as Transformer, but neglects the
inherent characteristics of this problem. Therefore, this paper tries to
combine the characteristics with deep learning techniques and design a
targeted algorithm named BGCrack. The main contributions of this paper are as
follows:

(1) BGCrack decouples cracks into two parts: the crack boundary and internal
area, and integrates boundary information into segmentation to generate more
accurate and clear predictions. Targeted modules, such as HFIE, GIP, and COM,
are designed to enhance the model’s representation power. Extensive
experiments have been conducted to validate the effectiveness of the proposed
design.

(2) Because steel structures are one of the main structural forms in civil
engineering, fatigue cracks in steel structures are very common during service
and are important safety hazards. This paper provides a steel crack dataset to
promote the research on the identification of steel cracks.

The current pipeline still has several limitations; hence it is suggested that
further research might be conducted in the following directions: (1) Improving
the performance of inspection algorithms further. Although BGCrack outperforms
other advanced segmentation frameworks, there is still room for improvement
with respect to inspection accuracy. The following research plan is to develop
more robust detectors with better generalization capability using cutting-edge
AI technologies. (2) The existing framework focuses on intelligent pixel-level
segmentation, which is just the first step of defect detection. Defect
quantification tools will be integrated into this framework to build a more
complete and thorough inspection procedure. (3) In recent days, large language
models (LLM) represented by Chat-GPT and GPT-4 have drawn extensive attention
from all over the world and pushed the research of AI to a peak. It is a
promising direction for developing Civil-GPT, integrating intelligent
planning, structural design, construction, and maintenance to assist civil
engineers in making scientific decisions and reduce engineers’ time overhead.

Declaration of competing interest. The authors declare that they have no known
competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Acknowledgements. The research presented was financially supported by the
National Key R&D Program of China [grant number 2022YFC3801700], the
Innovation Technology Fund, Midstream Research Programme for Universities
[grant number MRP/003/21X], and the Hong Kong Research Grants Council [grant
number 16205021]. The authors would like to thank Prof. Yang Xu and Prof. Hui
Li of Harbin Institute of Technology for providing part of the crack images.
Finally, contributions by the anonymous reviewers are also highly appreciated.

## References

  * [1] Young-Jin Cha, Wooram Choi, and Oral Büyüköztürk.  Deep learning-based crack damage detection using convolutional neural networks.  Computer-Aided Civil and Infrastructure Engineering, 32(5):361–378, 2017.
  * [2] Young-Jin Cha, Wooram Choi, Gahyun Suh, Sadegh Mahmoudkhani, and Oral Büyüköztürk.  Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types.  Computer-Aided Civil and Infrastructure Engineering, 33(9):731–747, 2018.
  * [3] Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L. Yuille, and Yuyin Zhou.  Transunet: Transformers make strong encoders for medical image segmentation, 2021.
  * [4] Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam.  Rethinking atrous convolution for semantic image segmentation, 2017.
  * [5] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In Computer Vision – ECCV 2018, pages 833–851, Cham, 2018. Springer International Publishing.
  * [6] Lu Chi, Borui Jiang, and Yadong Mu.  Fast fourier convolution.  In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS’20, Red Hook, NY, USA, 2020. Curran Associates Inc.
  * [7] Wooram Choi and Young-Jin Cha.  Sddnet: Real-time crack segmentation.  IEEE Transactions on Industrial Electronics, 67(9):8016–8025, 2020\.
  * [8] J.K. Chow, Z. Su, J. Wu, P.S. Tan, X. Mao, and Y.H. Wang.  Anomaly detection of defects on concrete structures with the convolutional autoencoder.  Advanced Engineering Informatics, 45:101105, 2020.
  * [9] Jun Kang Chow, Zhaoyu Su, Jimmy Wu, Zhaofeng Li, Pin Siang Tan, Kuan fu Liu, Xin Mao, and Yu-Hsing Wang.  Artificial intelligence-empowered pipeline for image-based inspection of concrete structures.  Automation in Construction, 120:103372, 2020.
  * [10] L. Minh Dang, Hanxiang Wang, Yanfen Li, Yesul Park, Chanmi Oh, Tan N. Nguyen, and Hyeonjoon Moon.  Automatic tunnel lining crack evaluation and measurement using deep learning.  Tunnelling and Underground Space Technology, 124:104472, 2022.
  * [11] Jianghua Deng, Amardeep Singh, Yiyi Zhou, Ye Lu, and Vincent Cheng-Siong Lee.  Review on computer vision-based crack detection and quantification methodologies for civil structures.  Construction and Building Materials, 356:129238, 2022.
  * [12] Ruoxi Deng, Chunhua Shen, Shengjun Liu, Huibing Wang, and Xinru Liu.  Learning to predict crisp boundaries.  In Computer Vision – ECCV 2018, pages 570–586, Cham, 2018. Springer International Publishing.
  * [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  * [14] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In ICLR, 2021.
  * [15] Zaiwang Gu, Jun Cheng, Huazhu Fu, Kang Zhou, Huaying Hao, Yitian Zhao, Tianyang Zhang, Shenghua Gao, and Jiang Liu.  Ce-net: Context encoder network for 2d medical image segmentation.  IEEE Transactions on Medical Imaging, 38(10):2281–2292, 2019.
  * [16] Feng Guo, Yu Qian, Jian Liu, and Huayang Yu.  Pavement crack detection based on transformer network.  Automation in Construction, 145:104646, 2023.
  * [17] Jingjing Guo, Qian Wang, and Yiting Li.  Semi-supervised learning based on convolutional neural network and uncertainty filter for façade defects classification.  Computer-Aided Civil and Infrastructure Engineering, 36(3):302–317, 2021.
  * [18] Joakim Bruslund Haurum, Meysam Madadi, Sergio Escalera, and Thomas B. Moeslund.  Multi-scale hybrid vision transformer and sinkhorn tokenizer for sewer defect classification.  Automation in Construction, 144:104614, 2022.
  * [19] Zhili He, Shang Jiang, Jian Zhang, and Gang Wu.  Automatic damage detection using anchor-free method and unmanned surface vessel.  Automation in Construction, 133:104017, 2022.
  * [20] Andrew Howard, Mark Sandler, Bo Chen, Weijun Wang, Liang-Chieh Chen, Mingxing Tan, Grace Chu, Vijay Vasudevan, Yukun Zhu, Ruoming Pang, Hartwig Adam, and Quoc Le.  Searching for mobilenetv3.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1314–1324, 2019.
  * [21] Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu.  Squeeze-and-excitation networks.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(8):2011–2023, 2020.
  * [22] Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang.  Deep laplacian pyramid networks for fast and accurate super-resolution.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5835–5843, 2017.
  * [23] Gang Li, Shuanhai He, Yongfeng Ju, and Kai Du.  Long-distance precision inspection method for bridge cracks with image processing.  Automation in Construction, 41:83–95, 2014.
  * [24] Xiangtai Li, Xia Li, Li Zhang, Guangliang Cheng, Jianping Shi, Zhouchen Lin, Shaohua Tan, and Yunhai Tong.  Improving semantic segmentation via decoupled body and edge supervision.  In Computer Vision – ECCV 2020, pages 435–452, Cham, 2020. Springer International Publishing.
  * [25] Yahui Liu, Jian Yao, Xiaohu Lu, Renping Xie, and Li Li.  Deepcrack: A deep hierarchical feature learning architecture for crack segmentation.  Neurocomputing, 338:139–153, 2019.
  * [26] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 9992–10002, 2021.
  * [27] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11966–11976, 2022.
  * [28] Xintian Mao, Yiming Liu, Fengze Liu, Qingli Li, Wei Shen, and Yan Wang.  Intriguing findings of frequency selection for image deblurring, 2022\.
  * [29] Sachin Mehta and Mohammad Rastegari.  Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer.  In International Conference on Learning Representations, 2022.
  * [30] Futao Ni, Zhili He, Shang Jiang, Weiguo Wang, and Jian Zhang.  A generative adversarial learning strategy for enhanced lightweight crack delineation networks.  Advanced Engineering Informatics, 52:101575, 2022.
  * [31] FuTao Ni, Jian Zhang, and ZhiQiang Chen.  Zernike-moment measurement of thin-crack width in images enabled by dual-scale deep learning.  Computer-Aided Civil and Infrastructure Engineering, 34(5):367–384, 2019.
  * [32] Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, Ben Glocker, and Daniel Rueckert.  Attention u-net: Learning where to look for the pancreas, 2018.
  * [33] Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand.  Basnet: Boundary-aware salient object detection.  In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7471–7481, 2019.
  * [34] Zequn Qin, Pengyi Zhang, Fei Wu, and Xi Li.  Fcanet: Frequency channel attention networks.  In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 763–772, 2021.
  * [35] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville.  On the spectral bias of neural networks.  In Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5301–5310. PMLR, 09–15 Jun 2019.
  * [36] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.  Vision transformers for dense prediction.  In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 12159–12168, 2021.
  * [37] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015. Springer International Publishing.
  * [38] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In 2017 IEEE International Conference on Computer Vision (ICCV), pages 618–626, 2017.
  * [39] Yong Shi, Limeng Cui, Zhiquan Qi, Fan Meng, and Zhensong Chen.  Automatic road crack detection using random structured forests.  IEEE Transactions on Intelligent Transportation Systems, 17(12):3434–3445, 2016.
  * [40] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky.  Resolution-robust large mask inpainting with fourier convolutions.  In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3172–3182, 2022.
  * [41] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herve Jegou.  Training data-efficient image transformers & distillation through attention.  In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10347–10357. PMLR, 18–24 Jul 2021.
  * [42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In NeurIPS, 2017.
  * [43] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon.  Cbam: Convolutional block attention module.  In Computer Vision – ECCV 2018, pages 3–19, Cham, 2018. Springer International Publishing.
  * [44] Zhe Wu, Li Su, and Qingming Huang.  Stacked cross refinement network for edge-aware salient object detection.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 7263–7272, 2019.
  * [45] Tete Xiao, Mannat Singh, Eric Mintun, Trevor Darrell, Piotr Dollár, and Ross Girshick.  Early convolutions help transformers see better, 2021.
  * [46] Yang Xu, Yuequan Bao, Yufeng Zhang, and Hui Li.  Attribute-based structural damage identification by few-shot meta learning with inter-class knowledge transfer.  Structural Health Monitoring, 20(4):1494–1517, 2021.
  * [47] Yang Xu, Shiyin Wei, Yuequan Bao, and Hui Li.  Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network.  Structural Control and Health Monitoring, 26(3):e2313, 2019.  e2313 STC-18-0081.R1.
  * [48] Fan Yang, Lei Zhang, Sijia Yu, Danil Prokhorov, Xue Mei, and Haibin Ling.  Feature pyramid and hierarchical boosting network for pavement crack detection.  IEEE Transactions on Intelligent Transportation Systems, 21(4):1525–1535, 2020.
  * [49] Xincong Yang, Heng Li, Yantao Yu, Xiaochun Luo, Ting Huang, and Xu Yang.  Automatic pixel-level crack detection and measurement using fully convolutional network.  Computer-Aided Civil and Infrastructure Engineering, 33(12):1090–1109, 2018.
  * [50] Weihao Yu, Mi Luo, Pan Zhou, Chenyang Si, Yichen Zhou, Xinchao Wang, Jiashi Feng, and Shuicheng Yan.  Metaformer is actually what you need for vision.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10809–10819, 2022.
  * [51] Wei Yu, Kuiyuan Yang, Hongxun Yao, Xiaoshuai Sun, and Pengfei Xu.  Exploiting the complementary strengths of multi-layer cnn features for image retrieval.  Neurocomputing, 237:235–241, 2017.
  * [52] Zhenwei Yu, Yonggang Shen, and Chenkai Shen.  A real-time detection approach for bridge cracks based on yolov4-fpm.  Automation in Construction, 122:103514, 2021.
  * [53] Erhu Zhang, Linhao Shao, and Yang Wang.  Unifying transformer and convolution for dam crack detection.  Automation in Construction, 147:104712, 2023.
  * [54] Jiaxing Zhao, Jiang-Jiang Liu, Deng-Ping Fan, Yang Cao, Jufeng Yang, and Ming-Ming Cheng.  Egnet: Edge guidance network for salient object detection.  In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8778–8787, 2019.
  * [55] Shanglian Zhou, Carlos Canchila, and Wei Song.  Deep learning-based crack segmentation for civil infrastructure: data types, architectures, and benchmarked performance.  Automation in Construction, 146:104678, 2023.
  * [56] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang.  Unet++: Redesigning skip connections to exploit multiscale features in image segmentation.  IEEE Transactions on Medical Imaging, 39(6):1856–1867, 2020.
  * [57] Lanyun Zhu, Deyi Ji, Shiping Zhu, Weihao Gan, Wei Wu, and Junjie Yan.  Learning statistical texture for semantic segmentation.  In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12532–12541, 2021.
  * [58] Qin Zou, Yu Cao, Qingquan Li, Qingzhou Mao, and Song Wang.  Cracktree: Automatic crack detection from pavement images.  Pattern Recognition Letters, 33(3):227–238, 2012.
  * [59] Qin Zou, Zheng Zhang, Qingquan Li, Xianbiao Qi, Qian Wang, and Song Wang.  Deepcrack: Learning hierarchical convolutional features for crack detection.  IEEE Transactions on Image Processing, 28(3):1498–1512, 2019.