# OBSERVATION OF SINGLE TOP-QUARK + PHOTON PRODUCTION WITH THE ATLAS DETECTOR

M. ALHROOB 111On behalf of the ATLAS Collaboration.
Copyright 2022 CERN for the benefit of the ATLAS Collaboration. CC-BY-4.0
license. University of Oklahoma,
Norman, Oklahoma, USA

###### Abstract

These proceedings present the first observation of single top-quark production
associated with a photon using the full Run2 proton-proton dataset collected
by the ATLAS experiment [1] at the LHC.

##  1 Introduction

Single top-quark production with an associated vector boson is a rare process
predicted by the Standard Model (SM) and fundamental for probing the top-quark
electroweak couplings. These processes play a role in constraining non-
resonant contributions of Physics Beyond the SM parametrised in the effective
field theory framework. The ATLAS and CMS collaborations have observed single
top-quark production processes with both a W boson (tW) [2, 3] and a Z boson
(tZq) [4, 5]. The latest CMS measured tZq cross-section is 87.9 fb with 12%
uncertainty [6]. The CMS collaboration reported evidence for the associated
single top-quark production with a photon ($tq\gamma$) with a 4.3 $\sigma$
significance using a partial Run2 dataset of 35.9 fb-1 of proton-proton
collisions collected at 13 TeV with a measured $tq\gamma$ cross-section of 115
fb $\pm$ 30% [6].

In these proceedings, we report the observation of the $tq\gamma$ production
process made by the ATLAS collaboration using the full Run2 13 TeV proton-
proton dataset of 139 fb-1 collected by the ATLAS detector at the LHC.

##  2 Signal definition and fiducial cross-sections

In the search for the $tq\gamma$ production process [7], the ATLAS
collaboration considered only semileptonic top-quark decays. The photon can be
radiated from the either the top quark or other hard scattered particle in the
interaction, such as an initial state quark. The photon can be also radiated
by the charged particles from the top-quark decay. Figure 1 represents an
example of one of the Feynman diagrams where a photon is radiated from the top
quark, with subsequent semileptonic top-quark decay. The signal signature
consists of a photon, an electron or muon, missing transverse momentum from
the neutrino, a $b$-tagged jet, and a forward jet due to t-channel production.

Figure 1: Feynman diagram at leading order in QCD for $tq\gamma$ production
with semi-leptonic top-quark decay [7].

The $tq\gamma$ production cross-section is measured in fiducial phase space at
the parton level, excluding photons radiated from the top-quark decay
products. A particle-level fiducial cross-section is also measured, including
photons radiated from the top-quark decay products, covering the signal phase
space. The parton-level fiducial cross-section is defined by requiring at
least an isolated photon with $p_{T}\geq$ 20 GeV, $|\eta|\leq$2.5 and $\Delta
R>$ 0.4 from final state hard-scattering charged partons. Also, a charged
lepton is required with $|\eta|\leq$2.5. The parton-level fiducial cross-
section is multiplied by the branching fraction of the top-quark decaying
semileptonically, $\sigma_{tq\gamma}\times{\cal{B}}(t\rightarrow\ell\nu b)$,
is $406^{+25}_{-32}$ fb where the uncertainty is due to varying the PDF and
renormalisation and factorisation scales.

Similarly, the fiducial cross-section is calculated at the particle level,
including photons radiated from the top-quark decay products. The photon
production from final-state particles is estimated using the traditional SM
$t$-channel process with photon production simulated through the parton
shower. The fiducial volume is defined by requiring at least one isolated
photon with $p_{T}\geq$ 20 GeV and $|\eta|<$ 2.37, one electron or muon with
$p_{T}\geq$ 25 GeV and $|\eta|<$ 2.5, at least one b-jet with $p_{T}\geq$ 25
GeV and $|\eta|<$ 2.5, at least one neutrino, excluding those coming from the
hadronic decay. Jets within $\Delta R=$ 0.4 from a photon or a lepton are
removed. Events are removed if a photon is within $\Delta R=$ 0.4 from a
lepton or a jet. The calculated fiducial particle-level cross-section
multiplied by the branching fraction of the top-quark decaying
semileptonically, $\sigma_{tq\gamma}\times{\cal{B}}(t\rightarrow\ell\nu
b)+\sigma_{(t\rightarrow\ell\nu b\gamma)q},$ is $207^{+26}_{-11}$ fb with
$20\%$ contribution estimated from events with photons radiated from top-quark
decay products. The uncertainty is due to varying the PDF and renormalisation
and factorisation scales.

##  3 Event selection

Events are selected using a single lepton ($e$ or $\mu$) trigger and are
required to have exactly one reconstructed well-isolated lepton with
$p_{T}>27$ GeV and $|\eta|<$ 2.47 for electrons and $|\eta|<$ 2.5 for muons.
At least one well-isolated photon with $p_{T}>$ 20 GeV and $|\eta|<$ 2.37 is
required. Electrons and photons are rejected if they are within 1.37
$<|\eta|<$ 1.52. Due to the presence of a neutrino, a minimum missing
transverse momentum, $E^{\rm miss}_{\rm T}$, is required to be greater than 30
GeV. To reduce the contribution of the $Z\gamma$ background due to electrons
faking photons, $|m_{e\gamma}-90|>$10 GeV criteria is required. Also, events
are required to have at least one jet with $p_{T}>$25 GeV within $|\eta|<4.5$,
in which exactly one jet is b-tagged with 70% efficiency. Events with
additional loose b-jets (tagged with 85% efficiency) are rejected. Two signal
regions (SRs) are defined based on the number of forward jets (fj)
($4.5>|\eta|>$2.5): with $\geq 1$ fj SR, which has the most number of signal
events, and with $=0$ fj SR. The dominant background processes are
$t\bar{t}\gamma$ and $W\gamma$ which are measured using two dedicated control
regions (CRs). Table 1 shows the SRs and CRs designed to extract the signal
and control the dominant background processes.

|  $\geq$ 1 fj SR | 0 fj SR |  $t\bar{t}\gamma$ CR |  $W\gamma$ CR
---|---|---|---|---
Leptons ($p_{T}\geq$ 27 GeV) | = 1 | = 1 | = 1 | = 1
$E^{{\rm miss}}_{T}$ |  $\geq$ 30 GeV |  $\geq$ 30 GeV |  $\geq$ 30 GeV |  $\geq$ 30 GeV
Photons ($p_{T}\geq$ 20 GeV) |  $\geq$ 1 |  $\geq$ 1 |  $\geq$ 1 |  $\geq$ 1
$|m_{e\gamma}-91.2|$ |  $\geq 10$ GeV |  $\geq 10$ GeV |  $\geq 10$ GeV |  $\geq 10$ GeV
forward jets ($2.5<|\eta|<4.5$ ) |  $\geq$ 1 | $=0$ | $\geq 0$ | $\geq 0$
b-jets (70% WP) | = 1 | = 1 | =1 | = 0
Additional b-jets (85% WP) | = 0 | = 0 |  $\geq$ 1 |  $\geq$ 1
Table 1: A summary of the signal and control regions. In red, the selection
that leads to the orthogonality with other regions.

The contribution of processes with electrons faking photons, mainly dilepton
$t\bar{t}$ and Z+jets, is estimated by comparing data and MC using the tag-
and-probe method $N(Z\rightarrow e(e\rightarrow\gamma))/N(Z\rightarrow
e^{+}e^{-})$. An enriched region with $e\rightarrow\gamma$ fakes is selected
by inverting $m_{e\gamma}$ or $m_{e^{+}e^{-}}$ ( e.g $|m_{e\gamma}-90|<20$
GeV), $E^{\rm miss}_{\rm T}$ cuts and by requiring no $b$-tagged jet. This
estimate is validated in a region with similar requirements but with $\geq 1$
$b$-tagged jet requirement. The contribution of processes with hadrons faking
photons, mainly lepton+jets $t\bar{t}$, is estimated using the ABCD method.
Where the ABCD regions’ definitions are based on photon identification and
isolation criteria. The correction factors are $p^{\gamma}_{T}$ and
$\eta^{\gamma}$ dependant.

##  4 Signal and background separation

A deep, fully connected feedforward neural network (NN) is trained for each SR
to enhance the signal and background separation. Each NN consists of three
hidden layers with 128/8/32 and 128/16/16 hidden nodes for the $\geq 1$ fj and
0 fj SRs, respectively. A Rectified Linear Unit (ReLU) activation function is
used for the hidden nodes, while for the output node, a sigmoid function is
used. For the $\geq 1$ fj SR, 15 well-modelled input variables are selected,
while 12 well-modelled input variables are employed for the 0 fj SR,
respectively. Figure 2(a) presents the post-fit NN distribution of the $\geq
1$ fj SR, where a clear separation between the signal and background processes
is shown. Figure 2(b) presents the post-fit NN distribution of the 0 fj SR.
The two NNs are also used to construct the NN distribution for the
$t\bar{t}\gamma$ CR as shown in 2(c), where the NN choice is based on the
number of forward jets in the event.

(a)

(b)

(c)

(d)

Figure 2: Post-fit distributions of the NN output in the $\geq 1$ fj SR (a), 0
fj SR (b), $t\bar{t}\gamma$ CR (d) and the total event yield in $W\gamma$ CR
[7].

##  5 Results

The $tq\gamma$ cross-section is extracted using a binned maximum-likelihood
fit that is performed on the four signal and control regions presented in
Figure 2. Three NN distributions are used in the two SRs and $t\bar{t}\gamma$
CR, while in the $W\gamma$ CR, a single bin is used. The fit containes three
free-floating parameters: one for the signal, one for the $t\bar{t}\gamma$
background and the last for $W\gamma$ background. Other backgrounds and
systematic uncertainties are constrained using Gaussian priors. The
significance of the observed signal is 9.1 $\sigma$ compared to the predicted
significance of 6.7 $\sigma$. The measured parton-level fiducial cross-section
multiplied by the branching fraction of the semileptonic decay of the top
quark is $\sigma_{tq\gamma}\times{\cal{B}}(t\rightarrow\ell\nu b)=580\pm
19\,{(\rm stat.)}\pm 63\,{(\rm syst.)}$ fb. While the measured particle-level
fiducial cross-section multiplied by the branching fraction of the
semileptonic decay of the top quark is
$\sigma_{tq\gamma}\times{\cal{B}}(t\rightarrow\ell\nu
b)+\sigma_{(t\rightarrow\ell\nu b\gamma)q}=287\pm 8\,{(\rm stat.)}\pm
31\,{(\rm syst.)}$ fb.

The dominant systematic uncertainties affecting the parton-level cross-section
are $t\bar{t}\gamma$ modelling with 5.6%, jets and $E^{\rm miss}_{\rm T}$ with
4%, background MC statistics with 3.5%, $tq\gamma$ MC statistics with 3.4% and
$t\bar{t}$ modelling with 3.4%. Similarly, the dominant systematic
uncertainties for the particle-level cross-section are $t\bar{t}\gamma$
modelling with 5.7%, jets and $E^{\rm miss}_{\rm T}$ with 3.9%, background MC
statistics with 3.5%, $tq\gamma$ MC statistics with 3.1% and $t\bar{t}$
modelling with 3.1%.

##  6 Conclusion

The ATLAS collaboration at the LHC has observed the $tq\gamma$ process with a
significance of 9.1 $\sigma$ using the full Run2 dataset of 139 fb-1 collected
at 13 TeV, compared to the predicted significance of 6.7 $\sigma$ [7]. The
measured fiducial cross-section at parton-level multiplied by the branching
fraction of the semileptonic decay of the top quark is 580 fb $\pm$ 11%. In
comparison, the measured particle-level fiducial cross-section multiplied by
the branching fraction of the semileptonic decay of the top quark is 287 fb
$\pm$ 11%. These measured cross-sections are within 2.5 $\sigma$ and 1.9
$\sigma$ from the SM predictions for the parton-level and particle-level
fiducial cross-sections, respectively.

## References

## References

  * [1] ATLAS Collaboration, The ATLAS Experiment at the CERN Large Hadron Collider, JINST 3, S08003 (2008).
  * [2] ATLAS Collaboration, Measurement of the production cross-section of a single top quark in association with a $W$ boson at 8 TeV with the ATLAS experiment, JHEP 01, 064 (2016).
  * [3] CMS Collaboration, Observation of the associated production of a single top quark and a $W$ boson in $pp$ collisions at $\sqrt{s}=$8 TeV, Phys. Rev. Lett. 112, 231802 (2014).
  * [4] CMS Collaboration, Observation of Single Top Quark Production in Association with a $Z$ Boson in Proton-Proton Collisions at $\sqrt{s}$ =13 TeV, Phys. Rev. Lett. 122, 132003 (2019)
  * [5] ATLAS Collaboration, Observation of the associated production of a top quark and a $Z$ boson in $pp$ collisions at $\sqrt{s}=13$ TeV with the ATLAS detector, JHEP 07, 124 (2020)
  * [6] CMS Collaboration, Inclusive and differential cross section measurements of single top quark production in association with a Z boson in proton-proton collisions at $\sqrt{s}$ = 13 TeV, JHEP 02, 107 (2022)
  * [7] ATLAS Collaboration, Observation of single-top-quark production in association with a photon at the ATLAS detector, ATLAS-CONF-2022-013, [https://cds.cern.ch/record/2805217]


# Structure-aware scale-adaptive networks for cancer segmentation in whole-
slide images

Yibao Sun1, Giussepi Lopez1, Yaqi Wang2, Xingru Huang1⋆, Huiyu Zhou3, Qianni
Zhang1 ⋆Corresponding author: Xingru Huang.1Yibao Sun, Giussepi Lopez, Xingru
Huang and Qianni Zhang are with the School of Electronic Engineering and
Computer Science, Queen Mary University of London, Mile End Road, London, E1
4NS, United Kingdom (e-mail: yibao.sun; e.g.lopezmolina; xingru.huang;
qianni.zhang@qmul.ac.uk).2Yaqi Wang is with the College of Media Engineering,
Communication University of Zhejiang, Hangzhou, China (e-mail:
wangyaqi@cuz.edu.cn).3Huiyu Zhou is with the School of Informatics, University
of Leicester, University Road, Leicester, LE1 7RH, United Kingdom (e-mail:
hz143@leicester.ac.uk).

###### Abstract

Cancer segmentation in whole-slide images is a fundamental step for viable
tumour burden estimation, which is of great value for cancer assessment.
However, factors like vague boundaries or small regions dissociated from
viable tumour areas make it a challenging task. Considering the usefulness of
multi-scale features in various vision-related tasks, we present a structure-
aware scale-adaptive feature selection method for efficient and accurate
cancer segmentation. Based on a segmentation network with a popular encoder-
decoder architecture, a scale-adaptive module is proposed for selecting more
robust features to represent the vague, non-rigid boundaries. Furthermore, a
structural similarity metric is proposed for better tissue structure awareness
to deal with small region segmentation. In addition, advanced designs
including several attention mechanisms and the selective-kernel convolutions
are applied to the baseline network for comparative study purposes. Extensive
experimental results show that the proposed structure-aware scale-adaptive
networks achieve outstanding performance on liver cancer segmentation when
compared to top ten submitted results in the challenge of PAIP 2019. Further
evaluation on colorectal cancer segmentation shows that the scale-adaptive
module improves the baseline network or outperforms the other excellent
designs of attention mechanisms when considering the tradeoff between
efficiency and accuracy. The sourcecode is publicly available at:
https://github.com/sigma10010/histo_img_seg.

###### Index Terms:

Cancer Segmentation, Structural Similarity, Deep Learning, Digital Pathology,
Computational Pathology

##  I Introduction

According to the Global Cancer Statistics 2020 of cancer incidence and
mortality produced by the International Agency for Research on Cancer, cancer
is a leading cause of death worldwide, accounting for nearly 10 million deaths
in 2020 [1]. Histopathology is considered a clinical gold standard for
diagnosis of a variety of diseases including cancers. Surgical resection is
the only treatment associated with long-term survival for patients with most
types of cancer. Samples from the resected cancerous tissue are used to
produce stained slides for further analysis. Measurements like tumour burden
are obtained and used by radiologists to evaluate the prognosis after clinic
surgery.

Image segmentation, in the context of computer vision, is defined as the
process of assigning labels to all the pixels from an image; as a result, a
mask covering the whole image is created. In the medical field, several types
of images are used to detect and diagnose cancer, e.g., computed tomographies,
magnetic resonance images (MRIs), ultrasound scans or sonograms, digitized
specimen slides; among others. Segmentation techniques have been successfully
applied to several medical tasks like pneumothorax segmentation [2], brain
tumour segmentation [3, 4], skin lesions [5], cardiac MRIs segmentation [6],
etc. The promising outcomes have highlighted the importance of using computer-
aided diagnosis (CAD) to assist the experts with the detection and
quantitative analysis of cancer.

Tumour area segmentation in stained histopathology tissue slides is a
fundamental step for cancer grading and assessment when used along with CAD
systems, especially for viable tumour burden quantification. Traditionally,
experts use a semiquantitative grading system for analyzing the degree of
tumour burden. However, manual delineation requires professional domain
knowledge and is laborious and time-consuming due to the large variations in
the shape of the tumour regions. Also, the standard of semiquantitative
grading is based on the experience of doctors, which is unavoidably subjective
and suffers from inter-observer variability, leading to uncertainty in
examination of difficult cases. Therefore, automatic solutions for accurate,
objective and reproducible tumour area segmentation in whole-slide images are
desired for many applications, such as estimating the degree of tumour burden.

Although automatic tumour area segmentation is highly demanded and many
endeavours have been poured into, there still have several challenges to deal
with. First of all, the boundary of tumour area is irregular and fuzzy due to
the infiltration of cancerous cells. Existing methods try to learn more robust
features to solve the issue [7]. In [8], multi-scale features have been
demonstrated to be potential in cancer segmentation. However, as it can be
seen from their experimental results, the model performs unsatisfactorily and
only secure the $7^{th}$ place in a challenge on hepatocellular carcinoma
segmentation. Another challenge is that there always have some small regions
similar to cancer in whole-slide images apart from the viable tumour areas.
Previous approaches like U-Net [9] cannot segment these small regions
properly, which will be demonstrated in the Experiments (Section IV). In
addition to the vague boundary and the small regions, there are some other
difficulties such as image variations between samples even when they come from
the same patient, image artifacts created when preparing and scanning the
samples (wrinkles, dust, blur created by samples with different density, among
others), colours variation when using products from different vendors, large
dimensions of WSI (they are in the gigapixel range, consequently, they cannot
be directly used as an input for a neuronal network), class imbalance
(remarked difference of samples per class) and distribution imbalance (large
difference in the areas covered by each class), etc.

To address these challenges for the purpose of efficient and accurate cancer
segmentation, we present structure-aware scale-adaptive feature selection
networks, by leveraging state-of-the-art techniques of deep learning with
tailored designs including attention mechanisms, residual learning [10],
structural similarity and multi-scale feature fusion, etc. Contributions
includes: 1) We design a scale-adaptive module for dynamic feature selection,
which is easily integrated into a segmentation network and can ensure the
network to learn more robust features around tumour for the vague boundary
problem. Instead of averagely fusing features from different scales, we
propose to learn weights for scale-adaptive feature selection; 2) A structural
similarity metric is proposed to regulate the network training procedure for
better tissue structure awareness, which is helpful for solving the issue of
small regions. We empirically validate the impact of the structural similarity
loss on regulating networks training and its improvement on tumour area
segmentation; 3) We comprehensively study and compare the proposed structure-
aware scale-adaptive networks with several advanced designs of attention
mechanisms on tumour area segmentation. All the sourcecode has been made
publicly available online.

##  II Related work

As convolutional neural networks (CNN) become the state-of-the-art in learning
visual representation, it is natural to apply CNNs to solve the image
segmentation problem for the requirement of coarse-to-fine inference. Inspired
by the process of using CNNs for image classification in which a category is
assigned to each input image, early segmentation approaches directly adopt the
classification networks to predict a category for each pixel through training
a patch-wise classifier with much smaller input size when compared to the
whole input image [11, 12]. This kind of patch-based methods always work with
a sliding window to obtain segmentation map. Typically, a patch is fed into
the trained classifier to predict a label for the central pixel of the patch.
The patch-based methods for semantic segmentation have inherent drawbacks like
low computational efficiency and blocky prediction result [13].

To address the drawbacks of patch-based methods, fully convolutional networks
(FCN) have been proposed since 2015 [14], which has driven recent advances in
applying CNN for image segmentation. Many works including U-Net [9], SegNet
[15], DeconvNet [16] and Deeplab [17] improve FCN with a better encoder-
decoder network architecture. In the field of medical imaging, an increasing
number of U-Net variants have been successfully applied to various scenarios
and image modalities [18, 19, 20], including digitised histopathological
images [21, 22, 23]. The main advantage of these FCN based methods for pixel
labelling when compared to the patch-based ones is that the networks can be
trained in an end-to-end, pixel-to-pixel way.

Due to the impressive success of CNN for natural image classification and
segmentation, there has been an increasing interest in applying this new
technology to histology image analysis [24, 25]. For instance, authors in [26]
propose an convolutional neural network to extract information from multiple
scales to classify breast cancer histology images and their approach achieves
a high sensitivity of 95.6% for carcinoma cases. Deep convolutional neural
networks are also used for classifying histology images of many other kinds of
cancer [27]. Besides histopathology image classification, there are also
strong attempts for histology image segmentation. Ciresan et al. propose a
network in a sliding-window setup to predict a class label for each pixel,
which won the electron microscopy segmentation challenge at ISBI 2012 [28].
Most early methods for cancer segmentation are empowered by patch-based
classifier with a sliding-window post processing. As FCN becomes the state-of-
the-art in image segmentation, many work adopt the encoder-decoder
architecture for end-to-end cancer segmentation [29, 8, 30]. Among them, an
ensemble model achieves the best published result on hepatocellular carcinoma
segmentation [30]. Following these researches, we also adopt the encoder-
decoder architecture in our design. A comparative analysis along with these
previous methods is given in Section IV-E.

Inspired by the important role of attention mechanism in human visual
perception, many early researches have made success in applying visual
attention on scene understanding [31, 32]. Due to the success of deep learning
[33], especially the breakthroughs of CNN in learning powerful visual
representation [34, 35, 10], an increasing amount of works try to leverage
attention mechanism for further improvement on CNN [36, 37, 38, 39, 40]. Among
them, several attention mechanisms including the squeeze-and-excitation (SE)
block [37], the bottleneck attention module (BAM) [41] and the convolutional
block attention module (CBAM) [42] gain great success in more robust feature
learning for a couple of visual tasks including image segmentation. Some works
like the attention gated networks attempt to leverage salient regions for
medical image segmentation [43, 7] and achieve the state-of-the-art results.
We are going to compare the proposed network with these advanced networks
based on attention mechanisms.

##  III Methodology

Inspired by the recent advances in structural similarity and hierarchical
feature fusion for better visual representation learning for semantic
segmentation, we propose a structure-aware scale-adaptive feature selection
network for cancer segmentation in whole-slide images. In this section, we
describe in details the proposed networks and loss functions.

###  III-A Overview of the network architecture

Figure 1: The architecture of the proposed structure-aware scale-adaptive
networks. For concise purpose, down-sampling units between each two stages in
the encoding path and up-sampling units in the decoding path are omitted.
Processing units in dashed boxes denote that attention blocks are optional.
Figure 2:  Building blocks. Left: a basic convolutional block. Right: a basic
convolutional block with shortcut. F: feature map, Conv: convolution, BN:
batch normalization [44], ReLU: rectified linear unit [45].

Beyond the patch-based methods, U-Net and its variants form a major stream in
medical image segmentation. According to the summary of the challenge on liver
cancer segmentation [46], most of the submitted top methods apply U-Net
variants with a similar encoder-decoder architecture for tumour area
segmentation. For the purpose of comparative analysis, we also adopt an
encoder-decoder architecture for the proposed networks, which is mainly
composed of an encoding path and a decoding path, as illustrated in Fig. 1. In
the encoding path, basic convolutional blocks (Fig. 2) and down-sampling units
are stacked for feature map extraction. The feature map output via the
encoding path is then gradually decoded in the decoding path which consists of
convolutional blocks and up-sampling units. The decoded feature map at the
final stage, of the same size as the input image, is then fed to a
segmentation head - a convolution with $1\times 1$ kernel - for final
segmentation prediction. Instead of using a single feature map for prediction,
multi-scale feature maps decoded at different stages are simultaneously fed to
a scale-adaptive feature selection (SAFS) module before being sent to the
segmentation head. More details of the SAFS module are given in Section III-C.

###  III-B Network variants

For the purpose of comparative studies, we instantiate several networks with
different settings in terms of skip/shortcut connections, convolutional
blocks, and attention blocks, etc. The first one is a baseline U-Net, which is
built of basic convolutional blocks without any attention block applied, and
only a single feature map is fed to the segmentation head for prediction. The
second one is based on U-Net but built of shortcut convolutional blocks, as
shown in Fig. 2, referred to as U-Net+SC to distinguish from the baseline
U-Net.

Several state-of-the-art attention mechanisms including the attention gate
(AG) [43, 7], the squeeze-and-excitation (SE) block [37], the bottleneck
attention module (BAM) [41] and the convolutional block attention module
(CBAM) [42] are integrated into the baseline U-Net, the resulting networks are
named as U-Net+AG, U-Net+SE, U-Net+BAM and U-Net+CBAM respectively. Models
trained with AG implicitly learn to suppress irrelevant regions in an input
image while highlighting useful salient features. It is added to each layer in
the decoding path of the baseline U-Net. SE applies only channel attention,
while both BAM and CBAM combine channel attention and spatial attention in a
sequential way, yet there are some slight differences on how to aggregate
information across spatial dimension between them. All of SE, BAM and CBAM are
applied to every intermediate feature map of each stage in both the encoding
path and the decoding path.

Since dependencies among tissue types normally exist in a whole-slide range,
being able to capture long-range spatial dependencies is critical for cancer
segmentation. This is, however, not possible for CNN due to its property of
local connectivity. A novel selective-kernel (SK) convolution has been
proposed for larger and adaptive receptive fields to improve CNN [47]. To
exploit the effectiveness of the adaptive receptive fields on
histopathological image segmentation, the basic convolutional blocks in the
encoding path of the baseline U-Net are substituted as the selective-kernel
convolutions to form a new network, which is named as U-Net+SK.

###  III-C The scale-adaptive feature selection module

Multi-scale features have been demonstrated very useful for many vision
related tasks such as object detection [48], MRI reconstruction [49] and
histopathological image segmentation [8]. To explore its effectiveness in
tumour area segmentation, we design a network called U-Net+MS, in which
multiple feature maps $\\{F_{1},F_{2},...,F_{n}\\}$ decoded at different
stages of different scales are sent to the segmentation head in a parallel way
for final prediction. A simple fusion is performed to acquire an averaged
feature map $\overline{F}$, i.e.,
$\overline{F}=\frac{1}{n}\sum_{i=1}^{n}F_{i}$, before prediction. The averaged
feature map can take the advantages of multi-scale features. Meanwhile, it
could degrade the overall performance when features from some specific scales
have poor performance. Inspired by the work of selective kernel for adaptive
receptive fields of CNN [47], we propose a scale-adaptive feature selection
(SAFS) module as an alternative feature fusion method, which is integrated
into the baseline U-Net. The resulting network is referred to as U-Net+SAFS in
the following descriptions.

Figure 3:  The scale-adaptive feature selection module. MLP: multi-layer
perceptron with a single hidden layer.

A two-branch case of the SAFS module is shown in Fig. 3. Given feature maps
$F_{1}$ and $F_{2}$ from different scales, which are fused together to obtain
a united feature map $F$, i.e., $F=F_{1}\oplus F_{2}$, where $\oplus$ is an
element-wise summation. The united feature map $F\in\mathbb{R}^{C\times
H\times W}$ is then fed to a global pooling unit to generate channel-wise
statistics $\mathbf{p}=\\{p_{i}|i=1,2,...,C\\}$, where $p_{i}=\frac{1}{H\times
W}\sum_{r=1}^{H}\sum_{c=1}^{W}F(r,c)$, which aggregates information across
spatial dimensions. The summation fusion procedure before the global pooling
unit ensures the information is aggregated from all the branches of different
scales. Multiple paralleled MLPs and a SoftMax activation are used to map the
statistics $\mathbf{p}$ to a concatenated channel attention weights
$\mathbf{q}$ which can be expressed as $\mathbf{q}=\mathbf{a}\cup\mathbf{b}$,
where $\mathbf{a}$ and $\mathbf{b}$ are channel attention weights for feature
maps from different scales respectively, i.e.,
$\mathbf{a}=\\{a_{i}|i=1,2,...,C\\}$, $\mathbf{b}=\\{b_{i}|i=1,2,...,C\\}$,
which subjects to $a_{i}+b_{i}=1$. Finally, a scale-adaptive feature map
$\hat{F}$ is obtained via a dynamic selection procedure, which can be
expressed as $\hat{F}=(\mathbf{a}\otimes F_{1})\oplus(\mathbf{b}\otimes
F_{2})$, where $\otimes$ is an element-wise multiplication. For the sake of
clarity, these steps are summarised in the Algorithm 1. As it can be seen, the
scale-adaptive feature is actually a weighted average of features at different
scales. Unlike an averagely fused feature, the scale-adaptive feature takes
advantages of features at different scales without performance degradation,
since the weights are learned in a self-adaptive way.

0: Feature maps $\left\\{F_{i}|i=1,2,...,n\right\\}$, s.t.
$F_{i}\in\mathbb{R}^{C\times H\times W}$

0: A scale-adaptive feature map $\hat{F}$

1: Concatenate feature map: $\tilde{F}\leftarrow Cat(F_{1},F_{2},...F_{n})$,
s.t. $\tilde{F}\in\mathbb{R}^{n\times C\times H\times W}$

2: Unite feature map: $F\leftarrow\sum_{i=1}^{n}F_{i}$, s.t.
$F\in\mathbb{R}^{C\times H\times W}$

3: Spatial information aggregation: $\mathbf{p}\leftarrow\frac{1}{H\times
W}\sum_{r=1}^{H}\sum_{c=1}^{W}F(r,c)$, s.t. $\mathbf{p}\in\mathbb{R}^{C}$

4: Reduction rate: $r\leftarrow 8$

5: Define a fully connected layer: $FC(in\\_dims:C,out\\_dims:\frac{C}{r})$

6: Dimension reduction: $\mathbf{z}\leftarrow FC(\mathbf{p})$, s.t.
$\mathbf{z}\in\mathbb{R}^{\frac{C}{r}}$

7: $\mathbf{z^{\prime}}\leftarrow\\{\\}$

8: while $n\neq 0$ do

9: Define a fully connected layer: $FC(in\\_dims:\frac{C}{r},out\\_dims:C)$

10: $\mathbf{z^{\prime}}\leftarrow\mathbf{z^{\prime}}\cup FC(\mathbf{z})$

11: $n\leftarrow n-1$

12: end while

13: $\mathbf{q}\leftarrow SoftMax(\mathbf{z^{\prime}})$, s.t.
$\mathbf{q}\in\mathbb{R}^{n\times C}$

14: Expand $\mathbf{q}$, s.t. $\mathbf{q}\in\mathbb{R}^{n\times C\times
H\times W}$

15: $\hat{F}\leftarrow Sum(\mathbf{q}\otimes\tilde{F},dim=0)$, s.t.
$\hat{F}\in\mathbb{R}^{C\times H\times W}$, $\otimes$ denotes element-wise
multiplication.

Algorithm 1 The scale-adaptive feature selection

###  III-D Loss functions

As depicted in the network architecture (Fig. 1), given an image as the input,
a neural network predicts a segmentation map of the same size as the input
image. Widely used cross-entropy loss is applied to train networks with
different configurations. Given a prediction map
$M^{(p)}\in\mathbb{R}^{H\times W}$ and the corresponding ground-truth map
$M^{(g)}\in\mathbb{R}^{H\times W}$ of size of $H\times W$ pixels, the pixel-
wise cross-entropy loss can be expressed as:

$\displaystyle\mathcal{L}_{ce}(M^{(p)},M^{(g)})=-\frac{1}{H\times
W}\sum_{r=1}^{H}\sum_{c=1}^{W}[M^{(g)}(r,c)log(M^{(p)}(r,c))$ (1)
$\displaystyle+(1-M^{(g)}(r,c))log(1-M^{(p)}(r,c))]$

where $M^{(g)}(r,c)\in\\{0,1\\}$ is the ground truth label at pixel $(r,c)$,
and $M^{(p)}(r,c)\in[0,1]$ is the predicted probability of being target
instance.

Pixels located around tumour border play a key role in tumour area
segmentation. To focus on those key pixels, structural similarity is applied
to capture local differences between a prediction map and the corresponding
ground-truth map. A structural similarity (SSIM) index has been proposed for
perceived image quality assessment, which compares luminance, contrast and
structure between two images [50]. Let
$\mathbf{x}=\\{\mathbf{x}_{i}|i=1,2,...,K^{2}\\}$ denote a patch of $K\times
K$ pixels extracted from a predicted segmentation map, i.e.,
$\mathbf{x}\subseteq M^{(p)}$, and
$\mathbf{y}=\\{\mathbf{y}_{i}|i=1,2,...,K^{2}\\}$ denote the aligned patch of
the corresponding ground-truth map $M^{(g)}$, the SSIM index of $\mathbf{x}$
and $\mathbf{y}$ can then be calculated as:

$\displaystyle
SSIM(\mathbf{x},\mathbf{y})=\frac{(2\mu_{x}\mu_{y}+C_{1})(2\sigma_{xy}+C_{2})}{(\mu_{x}^{2}+\mu_{y}^{2}+C_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+C_{2})}$
(2)

where $\mu_{x}$ and $\mu_{y}$ are the mean intensity of $\mathbf{x}$ and
$\mathbf{y}$ respectively, $\sigma_{x}$ and $\sigma_{y}$ are the standard
deviation of $\mathbf{x}$ and $\mathbf{y}$ respectively, $\sigma_{xy}$ is the
covariance between $\mathbf{x}$ and $\mathbf{y}$, and $C_{1}$, $C_{2}$ are
small constants to avoid instability when $\mu_{x}^{2}+\mu_{y}^{2}$ or
$\sigma_{x}^{2}+\sigma_{y}^{2}$ is very close to zero, which are set as
$0.01^{2}$ and $0.03^{2}$ respectively in our experiments.

With the above definitions, the SSIM loss between a prediction map
$M^{(p)}\in\mathbb{R}^{H\times W}$ and the corresponding ground-truth map
$M^{(g)}\in\mathbb{R}^{H\times W}$ is calculated as:

$\displaystyle\mathcal{L}_{ssim}$ $\displaystyle(M^{(p)},M^{(g)})=$ (3)
$\displaystyle\frac{1}{(H-K)(W-K)}\sum_{\forall\mathbf{x}\subseteq
M^{(p)},\forall\mathbf{y}\subseteq M^{(g)}}(1-SSIM(\mathbf{x},\mathbf{y}))$

where $(H-K)(W-K)$ is the total number of patch pair $(\mathbf{x},\mathbf{y})$
to calculate $SSIM$. In our experiments, we choose $K=11$ and apply a sliding
window (kernel) with a stride of 1 to extract patches. Moreover, we apply
multi-scale structural similarity [51], which is more flexible than the
single-scale one. It can be observed that the SSIM loss is calculated on small
patches extracted from prediction map and ground truth. Losses between patches
located within areas of tumour or other tissue types are small while those of
patches around tumour border are large. To minimize the SSIM loss forces the
networks to focus on pixels around tumour border, so as to improve the
segmentation results. This is demonstrated in the ablation study (Section
IV-D).

Besides the SSIM loss, a couple of metric based losses have been proposed for
image classification and segmentation, such as the IoU loss [52, 53] and the
Dice loss [54]. In order to perform comparative study, we also test the IoU
loss, which in the same context as the cross-entropy loss can be expressed as:

$\displaystyle\mathcal{L}_{iou}(M^{(p)},M^{(g)})=$ (4) $\displaystyle
1-\frac{\sum_{r=1}^{H}\sum_{c=1}^{W}M^{(g)}(r,c)M^{(p)}(r,c)}{\sum_{r=1}^{H}\sum_{c=1}^{W}[M^{(g)}(r,c)+M^{(p)}(r,c)-M^{(g)}(r,c)M^{(p)}(r,c)]}$

Different to the SSIM loss, the IoU loss is a global measure between the
segmentation map and the ground truth.

##  IV Experiments

###  IV-A Datasets

TABLE I: A summary of the datasets and the derived image patches. Dataset |  PAIP 2019 | PAIP 2020
---|---|---
WSI | Type | Liver cancer | Colorectal cancer
Amount | 50 | 47
Patch | Size | 256 $\times$ 256 | 400 $\times$ 400 | 512 $\times$ 512 | 384 $\times$ 384
Level | 2 | 2 | 2 | 3
Amount | 15,637 | 6,161 | 5,850 | 4,772
|  |  |  |  |

Two published datasets (PAIP 2019 and PAIP 2020) for liver cancer and
colorectal cancer assessment are used for evaluation. Table I gives a summary
for whole-slide images (WSIs) and patches of two datasets respectively. The
PAIP 2019 111https://paip2019.grand-challenge.org/Dataset/ dataset is released
for liver cancer segmentation and viable tumour burden estimation, which
contains 50 slides of liver cancer in total. All the WSIs are scanned by
Aperio AT2 at $20\times$ magnification (level 0). A ground-truth binary mask
to denote viable tumour area or not is provided for each slide. Due to high
resolution of whole-slide images and memory limitation, each whole-slide image
is cropped into small patches of different sizes, e.g., $400\times 400$
pixels, at the lowest magnification factor (level 2), with an overlap of 200
pixels for training purpose. The PAIP 2020 222https://paip2020.grand-
challenge.org/Dataset/ dataset is released for the automated classification of
molecular subtypes in colorectal cancer. There are 47 slides of colorectal
cancer in total with annotated binary masks. All the WSIs are scanned by
Aperio AT2 at $40\times$ magnification. In total, 4,772 small patches each of
$384\times 384$ pixels are extracted from whole-slide images at the lowest
magnification factor of $5\times$ for training. To test the proposed methods,
whole-slide images are randomly split into 5 groups for 5-fold cross
validation before they are cropped into small patches. Same split over WSIs is
performed on the cropped patches to avoid correlation.

###  IV-B Evaluation metrics

Widely used metrics including the Jaccard similarity and the Dice coefficient
are used to validate the effectiveness of the proposed method for cancer
segmentation. Given a segmentation map ($M^{(s)}$) and its corresponding
ground-truth map ($M^{(g)}$), the pixel-wise Jaccard similarity ($JS$) and
Dice coefficient ($DC$) can be calculated as follows:

$\displaystyle JS(M^{(s)},M^{(g)})=\frac{\left|M^{(s)}\cap
M^{(g)}\right|}{\left|M^{(s)}\cup M^{(g)}\right|},$ (5) $\displaystyle
DC(M^{(s)},M^{(g)})=\frac{2\times\left|M^{(s)}\cap
M^{(g)}\right|}{\left|M^{(s)}\right|+\left|M^{(g)}\right|},$ (6)

where $\left|\cdot\right|$ denotes the number of pixels labeled as target
instance. Besides the Jaccard similarity and the Dice coefficient, we also
report specificity ($SP=\frac{TN}{TN+FP}$), sensitivity/recall
($RC=\frac{TP}{TP+FN}$) and precision ($PC=\frac{TP}{TP+FP}$) for each test to
provide additional evaluation, where $TP$, $FP$, $TN$, $FN$ stands for pixel-
wise true positives, false positives, true negatives and false negatives
respectively.

Normally, a valid evaluation of a model needs to base on a sequence of images.
Therefore, given segmentation results and the corresponding ground-truth maps
for the image sequence, we calculate the accumulated value of $TP$, $FP$,
$TN$, $FN$ based on a pixel-wise confusion matrix. In cases of cancer
segmentation in histopathological images, we usually have a dataset of image
tiles and WSIs. For image tile segmentation, accumulated value of each metric
on the testing dataset of tiles is considered as model performance. For WSI
segmentation, a clipped Jaccard similarity is considered to penalize
inaccurate result, which can be calculated as:

$\displaystyle\hat{JS_{i}}=\begin{cases}\frac{\left|M_{i}^{(s)}\cap
M_{i}^{(g)}\right|}{\left|M_{i}^{(s)}\cup M_{i}^{(g)}\right|},&\text{ if
}\frac{\left|M_{i}^{(s)}\cap M_{i}^{(g)}\right|}{\left|M_{i}^{(s)}\cup
M_{i}^{(g)}\right|}\geq 0.65\\\ 0.&\text{ otherwise}\end{cases}$ (7)

where $M_{i}^{(s)}$ denotes the segmentation map for the $i^{th}$ WSI and
$M_{i}^{(g)}$ the corresponding ground-truth map. An average of $\hat{JS}$ is
considered as a final score on a testing dataset of WSIs:

$\displaystyle S_{wsi}=\frac{1}{N}\sum_{i=1}^{N}\hat{JS_{i}}$ (8)

where $N$ is the number of testing WSIs.

###  IV-C Training and inference

All the models are trained using the Adam [55] optimizer with a basic learning
rate of 1e-4 which is adjusted at last 10 training epochs with a decayed rate
of 0.1. The coefficients for computing running averages of gradient and its
square are set as 0.5 and 0.999 respectively. According to the losses
convergence observation, we stop training after 100 training epochs. The
training procedure takes around 5 hours for each model on a GPU of NVIDIA
GeForce GTX Titan X, which depends on the range of batch size from 2 to 4.

To ensure the segmentation models are more robust against variations of
histology images, data augmentation is performed during training by randomly
transforming the training images in ways of colour jitter, rotation,
horizontal flip and vertical flip with a certain probability of 0.5. For
colour jitter, the image colour are randomly adjusted in their brightness,
contrast, saturation and hue. For rotation, the images are rotated by a degree
randomly selected from {0,90,180,270}. In the inference phase, evaluation on
image patches and WSIs are performed respectively.

###  IV-D Ablation study

To acquire an optimal setting for building networks for image segmentation, we
extensively validate the effects of model scale, number of skip connection,
amount of prediction head and different loss functions on liver cancer
segmentation in terms of Dice coefficient ($DC$), Jaccard similarity ($JS$),
score on whole-slide image ($S_{wsi}$), etc.

Following [56], resolution, depth and width of networks are considered as
three factors for measuring model scale. We define the network resolution as
the size of input image ($H\times W$), depth as the last down-sampling stage
of feature map ($D$) and width as the initial channel amount ($C$), as shown
in the network architecture (Fig. 1). The performance of the baseline U-Net
tested on the PAIP 2019 dataset against different scales of model are listed
in Table II. As can it be observed, the model performs better at the middle-
level resolution of $400\times 400$ on both image patches and WSIs when
compared to the one of resolution of $256\times 256$ or $512\times 512$. For
the depth factor, the performance increase as the network gets deeper, which
is consistent with the common understanding of deep learning. Considering the
trade-off between performance and model size, we choose $D=5$ for the rest of
experiments. For the width factor, we test networks with different initial
channels (16, 32 and 48). An observation is that the network with a width of
32 performs better on image patches while the one with a greater width of 48
gets a better result on WSIs. $C=32$ is chosen for the rest of the experiments
when an additional condition, memory limitation, is considered.

TABLE II: Effect of model scale (resolution, depth and width) on the performance of the U-Net. | Model scale | $SP$ | $PC$ | $RC$ | $DC$ | $JS$ | $S_{wsi}$
---|---|---|---|---|---|---|---
Resolution [Depth 5 Width 32] | $256\times 256$ | 85.87 $\pm$ 2.99 | 84.90 $\pm$ 2.94 | 88.58 $\pm$ 7.27 | 86.59 $\pm$ 4.57 | 76.63 $\pm$ 6.83 | 31.08 $\pm$ 8.36
400 $\times$ 400 | 95.94 $\pm$ 1.08 | 91.22 $\pm$ 1.90 | 87.61 $\pm$ 7.02 | 89.24 $\pm$ 4.19 | 80.82 $\pm$ 6.68 | 73.78 $\pm$ 5.15
$512\times 512$ | 89.19 $\pm$ 3.20 | 88.12 $\pm$ 2.38 | 87.36 $\pm$ 6.85 | 87.64 $\pm$ 4.25 | 78.24 $\pm$ 6.45 | 34.34 $\pm$ 9.99
Depth [Width 32 400 $\times$ 400] | 3 | 91.28 $\pm$ 2.42 | 81.59 $\pm$ 3.50 | 79.62 $\pm$ 7.56 | 80.43 $\pm$ 4.81 | 67.54 $\pm$ 6.86 | 43.14 $\pm$ 9.63
4 | 94.36 $\pm$ 1.71 | 87.93 $\pm$ 3.44 | 85.73 $\pm$ 5.82 | 86.79 $\pm$ 4.52 | 76.94 $\pm$ 7.17 | 66.01 $\pm$ 7.39
5 | 95.94 $\pm$ 1.08 | 91.22 $\pm$ 1.90 | 87.61 $\pm$ 7.02 | 89.24 $\pm$ 4.19 | 80.82 $\pm$ 6.68 | 73.78 $\pm$ 5.15
Width [Depth 5 400 $\times$ 400] | 16 | 95.09 $\pm$ 0.86 | 89.42 $\pm$ 2.14 | 87.29 $\pm$ 5.91 | 88.29 $\pm$ 3.99 | 79.26 $\pm$ 6.30 | 69.37 $\pm$ 6.15
32 | 95.94 $\pm$ 1.08 | 91.22 $\pm$ 1.90 | 87.61 $\pm$ 7.02 | 89.24 $\pm$ 4.19 | 80.82 $\pm$ 6.68 | 73.78 $\pm$ 5.15
48 | 95.60 $\pm$ 0.93 | 90.59 $\pm$ 1.38 | 87.77 $\pm$ 7.81 | 88.98 $\pm$ 4.51 | 80.44 $\pm$ 7.02 | 77.17 $\pm$ 7.83
|  |  |  |  |  |  |

To explore the skip-connection design for neural network construction, we test
the baseline U-Net with different types of skip connection. Experimental
results are listed in Table III. All the networks with skip connection
outperform the one without skip connection on WSIs (see the $S_{wsi}$ column),
demonstrating that the skip-connection design is effective, although the
improvement on image patches are not significant. In addition, networks with
heavier skip connections are stronger than the ones with light skip
connections. For example, when all feature maps at different stages
(1/8+1/4+1/2+1/1) in the encoding path are connected and fused to the
corresponding feature maps in the decoding path, the network achieves a better
performance while compared against others.

TABLE III: Effect of different types of skip connection on the performance of the U-Net. # skip connection | $SP$ | $PC$ | $RC$ | $DC$ | $JS$ | $S_{wsi}$
---|---|---|---|---|---|---
0 (without) | 94.73 $\pm$ 0.91 | 88.87 $\pm$ 2.38 | 87.83 $\pm$ 6.39 | 88.28 $\pm$ 4.34 | 79.29 $\pm$ 6.86 | 69.22 $\pm$ 8.86
1/8 | 94.77 $\pm$ 2.06 | 89.13 $\pm$ 3.12 | 87.26 $\pm$ 7.21 | 88.04 $\pm$ 4.54 | 78.92 $\pm$ 7.11 | 70.70 $\pm$ 9.78
1/8+1/4 | 95.01 $\pm$ 1.13 | 89.51 $\pm$ 1.53 | 86.66 $\pm$ 7.74 | 87.90 $\pm$ 4.40 | 78.68 $\pm$ 6.93 | 75.92 $\pm$ 3.68
1/8+1/4+1/2 | 94.93 $\pm$ 1.45 | 89.29 $\pm$ 2.46 | 87.26 $\pm$ 5.76 | 88.20 $\pm$ 3.86 | 79.10 $\pm$ 6.12 | 72.96 $\pm$ 6.73
1/8+1/4+1/2+1/1 | 95.94 $\pm$ 1.08 | 91.22 $\pm$ 1.90 | 87.61 $\pm$ 7.02 | 89.24 $\pm$ 4.19 | 80.82 $\pm$ 6.68 | 73.78 $\pm$ 5.15
|  |  |  |  |  |

To observe the effect of different fusion schemes for multi-scale features in
prediction, feature maps at different decoding stages are fused for final
prediction. Two kinds of fusion methods are tested, i.e., average fusion and
adaptive fusion. For each fusion method, we test different settings including
the fusions of two feature maps, three feature maps and four feature maps
respectively against the case of only one single feature map for prediction.
Results are listed in Table IV. As it can be seen, for averaging fusion, the
case of three feature maps outperforms the others in terms of clipped Jaccard
similarity and Dice coefficient, although the improvement is limited when
compared to the one with a single feature map. One possible reason behind this
is that the overall performance of the averaging feature map could be degraded
when features of some specific scales have poor performance. Unlike average
fusion, a great increase on performance can be seen for cases of adaptive
fusion. For instance, the mean clipped Jaccard similarity increases from 73.12
to 77.07 when compared to the counterpart of average fusion. This demonstrates
that fusion methods play a key role in applying multi-scale features for
prediction.

TABLE IV: Effect of multi-scale/scale-adaptive features on the performance of the U-Net. # scale | $SP$ | $PC$ | $RC$ | $DC$ | $JS$ | $S_{wsi}$
---|---|---|---|---|---|---
1 (single) | 94.93 $\pm$ 1.45 | 89.29 $\pm$ 2.46 | 87.26 $\pm$ 5.76 | 88.20 $\pm$ 3.86 | 79.10 $\pm$ 6.12 | 72.96 $\pm$ 6.73
2 (multiple) | 95.32 $\pm$ 1.41 | 89.92 $\pm$ 2.26 | 85.89 $\pm$ 7.23 | 87.70 $\pm$ 4.20 | 78.35 $\pm$ 6.63 | 73.12 $\pm$ 6.52
3 (multiple) | 94.67 $\pm$ 1.68 | 88.96 $\pm$ 2.62 | 88.46 $\pm$ 6.46 | 88.61 $\pm$ 4.11 | 79.78 $\pm$ 6.46 | 75.40 $\pm$ 7.81
4 (multiple) | 94.86 $\pm$ 1.36 | 89.10 $\pm$ 2.49 | 87.33 $\pm$ 5.80 | 88.16 $\pm$ 4.05 | 79.05 $\pm$ 6.38 | 73.26 $\pm$ 8.32
2 (adaptive) | 94.87 $\pm$ 2.00 | 89.54 $\pm$ 3.10 | 89.73 $\pm$ 6.46 | 89.52 $\pm$ 4.10 | 81.28 $\pm$ 6.56 | 77.07 $\pm$ 8.62
3 (adaptive) | 94.96 $\pm$ 1.15 | 89.42 $\pm$ 2.50 | 89.22 $\pm$ 7.16 | 89.25 $\pm$ 4.81 | 80.91 $\pm$ 7.56 | 77.59 $\pm$ 8.92
|  |  |  |  |  |

To demonstrate the influence of using different loss functions, We test
networks trained by different losses including the widely used cross-entropy
loss (CE), the SSIM loss, the IoU loss and different combinations. Table V
gives a comparison among these losses in terms of segmentation performance. An
interesting observation is that either the SSIM loss or the IoU loss cannot
work properly when applied alone. It is better to combine them with the CE
loss. Improvements can be seen on the combination between CE and SSIM or
combination of all three losses. Fig. 4 shows the descent of each loss
function in comparison between the training and testing modes. The objective
function is to minimise the difference between a segmentation map and a
corresponding ground truth. Since the CE loss is calculated on the pixel
level, while SSIM is calculated on patch level and IoU is a global
measurement, the final convergent value of each loss is different.

TABLE V: Effect of different loss functions on the performance of the baseline model U-Net. Loss function | $SP$ | $PC$ | $RC$ | $DC$ | $JS$ | $S_{wsi}$
---|---|---|---|---|---|---
$\mathcal{L}_{ce}$ | 95.94 $\pm$ 1.08 | 91.22 $\pm$ 1.90 | 87.61 $\pm$ 7.02 | 89.24 $\pm$ 4.19 | 80.82 $\pm$ 6.68 | 73.78 $\pm$ 5.15
$\mathcal{L}_{iou}$ | 88.65 $\pm$ 3.99 | 79.22 $\pm$ 7.52 | 90.36 $\pm$ 4.89 | 84.36 $\pm$ 6.24 | 73.45 $\pm$ 9.24 | 56.89 $\pm$ 9.42
$\mathcal{L}_{ssim}$ | 95.56 $\pm$ 1.24 | 90.21 $\pm$ 2.58 | 85.23 $\pm$ 7.98 | 87.50 $\pm$ 5.11 | 78.14 $\pm$ 7.90 | 68.98 $\pm$ 4.97
$\mathcal{L}_{ce}$+$\mathcal{L}_{iou}$ | 95.58 $\pm$ 1.22 | 90.42 $\pm$ 2.70 | 88.00 $\pm$ 6.10 | 89.14 $\pm$ 4.32 | 80.67 $\pm$ 6.90 | 69.02 $\pm$ 8.77
$\mathcal{L}_{ce}$+$\mathcal{L}_{ssim}$ | 96.10 $\pm$ 0.74 | 91.48 $\pm$ 1.25 | 86.96 $\pm$ 6.66 | 89.04 $\pm$ 3.78 | 80.45 $\pm$ 6.07 | 75.63 $\pm$ 5.46
$\mathcal{L}_{ce}$+$\mathcal{L}_{ssim}$+$\mathcal{L}_{iou}$ | 95.64 $\pm$ 0.49 | 90.67 $\pm$ 0.89 | 88.46 $\pm$ 6.64 | 89.42 $\pm$ 3.58 | 81.06 $\pm$ 5.78 | 73.56 $\pm$ 2.97
|  |  |  |  |  |
Figure 4:  Losses against training epochs of the baseline U-Net.

###  IV-E Evaluation on liver cancer segmentation

In this section, we test and compare networks with different architectures on
liver cancer segmentation. To validate the effectiveness of the proposed
networks for cancer segmentation in whole slide images, digital slides of
liver cancer from the challenge of PAIP 2019 are used for networks training
and evaluation. Besides the structure-aware scale-adaptive networks, several
networks with superior selective-kernel convolutions and attention mechanisms
including channel and spatial attention are also tested. Comparison among
these networks are listed in Table VI. The network (U-Net+SAFS+SSIM), to which
both the SAFS module and the structural similarity (SSIM) are applied, scores
a clipped Jaccard similarity of 79.89$\pm$5.08, with a 7% improvement compared
to the baseline U-Net. The improvement can also be seen in Fig. 5, where many
false positives of the baseline U-Net are gradually eliminated when the SAFS
and the SSIM are applied, showing the effectiveness of the scale-adaptive
features and structural similarity. The elimination of false positives, most
of which are dissociated from viable tumour areas, is more obvious while the
SSIM is applied, due to the ability of SSIM to focus on differences of local
patches.

The scale-adaptive feature selection module and its combinations with the
structural similarity (SSIM), shortcut convolution (SC), and selective-kernel
convolution (SK) are tested respectively, results listed in Table VII together
with the top ten results of the PAIP 2019 challenge. Note that a testing set
of 40 extra WSIs are provided during the period of the PAIP 2019, while our
results are cross-validated using the training set. Although it is not
absolutely fair to make such a comparison and the training set we use is
smaller, the results are listed for reference. As it can be seen, the proposed
networks achieve outstanding performance. Specifically, the network with the
combination of SAFS and SSIM (U-Net+SAFS+SSIM) scores $79.89\pm 5.08$ which is
almost the same as the best published result of an ensemble model[30], while
the network with shortcut and selective-kernel convolutions (U-Net+SK+SC)
achieves the best clipped Jaccard similarity of 81.00$\pm$4.98. Although the
proposed network achieves a high overall performance, the variance is merely
acceptable. The relatively large variance reveals that our network may face
generalisation issues to some instances. We will pay special attention to
these issues in our future work.

TABLE VI: Comparison of networks with different architectures tested on the PAIP 2019 and the PAIP 2020 respectively. #P denotes the number of parameters. Architecture |  #P (million) | PAIP 2019 (liver cancer) | PAIP 2020 (colorectal cancer)
---|---|---|---
$DC$ | $JS$ | $S_{wsi}$ | $DC$ | $JS$ | $S_{wsi}$
U-Net (baseline) | 8.7680 | 88.20 $\pm$ 3.86 | 79.10 $\pm$ 6.12 | 72.96 $\pm$ 6.73 | 91.72 $\pm$ 3.15 | 84.86 $\pm$ 5.30 | 80.12 $\pm$ 7.87
U-Net+SC | 8.9713 | 88.31 $\pm$ 4.28 | 79.33 $\pm$ 6.75 | 73.05 $\pm$ 6.45 | 91.87 $\pm$ 3.19 | 85.12 $\pm$ 5.39 | 79.53 $\pm$ 6.47
U-Net+AG | 8.8826 | 88.80 $\pm$ 4.45 | 80.14 $\pm$ 7.06 | 75.77 $\pm$ 8.10 | 92.32 $\pm$ 3.09 | 85.88 $\pm$ 5.25 | 79.84 $\pm$ 8.76
U-Net+SE | 8.9922 | 89.48 $\pm$ 4.62 | 81.27 $\pm$ 7.29 | 79.02 $\pm$ 5.04 | 92.64 $\pm$ 2.99 | 86.44 $\pm$ 5.12 | 83.10 $\pm$ 8.16
U-Net+BAM | 9.3177 | 90.00 $\pm$ 3.72 | 82.01 $\pm$ 6.04 | 76.14 $\pm$ 8.95 | 93.01 $\pm$ 2.85 | 87.06 $\pm$ 4.95 | 83.22 $\pm$ 9.42
U-Net+CBAM | 8.9963 | 87.75 $\pm$ 3.90 | 78.39 $\pm$ 6.25 | 75.08 $\pm$ 5.53 | 92.78 $\pm$ 2.60 | 86.64 $\pm$ 4.52 | 83.00 $\pm$ 8.28
U-Net+SK | 19.8725 | 89.51 $\pm$ 4.48 | 81.30 $\pm$ 7.17 | 79.11 $\pm$ 6.98 | 92.84 $\pm$ 2.58 | 86.74 $\pm$ 4.49 | 82.06 $\pm$ 8.65
U-Net+MS | 8.8791 | 88.61 $\pm$ 4.11 | 79.78 $\pm$ 6.46 | 75.40 $\pm$ 7.81 | 91.87 $\pm$ 3.16 | 85.12 $\pm$ 5.36 | 80.05 $\pm$ 7.94
U-Net+SAFS | 8.8051 | 89.52 $\pm$ 4.10 | 81.28 $\pm$ 6.56 | 77.07 $\pm$ 8.62 | 92.42 $\pm$ 3.21 | 86.08 $\pm$ 5.50 | 82.97 $\pm$ 4.97
U-Net+SAFS+SSIM | 8.8051 | 90.06 $\pm$ 3.65 | 82.11 $\pm$ 5.93 | 79.89 $\pm$ 5.08 | 92.46 $\pm$ 3.18 | 86.14 $\pm$ 5.43 | 84.22 $\pm$ 5.22
|  |  |  |  |  |  |
Figure 5:  WSIs of liver cancer and segmentation maps of different networks. TABLE VII: Comparison with top ten teams participating in the challenge of PAIP 2019. Ranks are from the challenge leaderboard [46]. Rank | Team | $S_{wsi}$
---|---|---
- | U-Net+SAFS | 77.07$\pm$8.62
- | U-Net+SAFS+SC | 78.66$\pm$5.85
- | U-Net+SAFS+SSIM | 79.89$\pm$5.08
- | U-Net+SAFS+SK | 79.99$\pm$8.01
- | U-Net+SK+SC | 81.00$\pm$4.98
- | Ensemble [30] | 79.70
1 | Hyun Jung | 78.90
2 | Team Sen | 77.72
3 | Team MIRL-IITM | 75.03
4 | Team Damo AIC | 67.18
5 | Team QuIIL | 66.52
6 | Team CUHK-Med | 66.24
7 | Team DAISYlab@UKE | 65.96
8 | Team COSYPath | 63.13
9 | Ching-Wei Wang | 60.65
10 | Team LRDE | 52.99
|  |

###  IV-F Evaluation on colorectal cancer segmentation

In this section, we further test and compare the proposed networks on
colorectal cancer segmentation using WSIs of PAIP 2020 in which tiny blank
regions within viable tumour areas are also considered as cancer (see the GT
column in Fig. 6). Results are listed in Table VI. An interesting observation
is that either the scale-adaptive feature selection module or the structural
similarity can lead to improvement in colorectal cancer segmentation when
compared to the baseline, which is similar to the evaluation on liver cancer
segmentation. Especially when both methods are applied simultaneously, the
network (U-Net+SAFS+SSIM) achieves the best clipped Jaccard similarity of
84.22$\pm$5.22. We visualize some examples of whole-slide images and the
corresponding segmentation maps from networks with different settings to
demonstrate the improvement (Fig. 6). However, the improvement is not as
significant as the results on the liver cancer task, mainly because there is
less influences of small regions outside the viable tumour areas in WSIs of
colorectal cancer. Networks with superior attention mechanisms can also
outperform the baseline. For instance, U-Net+BAM scores better in Jaccard
similarity (JS) and Dice coefficient (DC) on small image patches.
Nevertheless, considering the tradeoff between the number of network
parameters and segmentation performance, we propose the combination of SAFS
and SSIM for efficient and accurate cancer segmentation.

Figure 6:  WSIs of colorectal cancer and segmentation maps of different
networks.

##  V Conclusion

We present a structure-aware scale-adaptive network to efficiently and
accurately segment tumour area in whole-slide images. This challenging image
segmentation problem is tackled by applying a SAFS module to dynamically
select feature maps from different scales for final prediction. Moreover, the
structural similarity metric is proposed to regulate the network training
procedure for better tissue structure capturing, so as to improve the
segmentation performance without increasing the number of network parameters.
The proposed network has been evaluated on two cancer segmentation benchmarks.
Significant improvements are achieved by exploiting scale-adaptive features
together with the structural similarity loss, demonstrating the effectiveness
of the dynamic and scale-adaptive feature selection approach for cancer
segmentation. Specifically, experimental results show that the proposed
network yield outstanding performance on the benchmark of PAIP 2019 for liver
cancer segmentation while compared to previous methods, and on the colorectal
cancer segmentation benchmark when compared with the baseline or other
networks with excellent attention designs. Future work will further
investigate and compare the proposed networks for cancer segmentation on WSIs
at different magnification scales.

## References

  * [1] Hyuna Sung, Jacques Ferlay, Rebecca L Siegel, Mathieu Laversanne, Isabelle Soerjomataram, Ahmedin Jemal, and Freddie Bray,  “Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries,”  CA: a cancer journal for clinicians, vol. 71, no. 3, pp. 209–249, 2021.
  * [2] Alexey Tolkachev, Ilyas Sirazitdinov, Maksym Kholiavchenko, Tamerlan Mustafaev, and Bulat Ibragimov,  “Deep learning for diagnosis and segmentation of pneumothorax: the results on the kaggle competition and validation against radiologists,”  IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 5, pp. 1660–1672, 2020.
  * [3] Haocheng Shen and Jianguo Zhang,  “Fully connected crf with data-driven prior for multi-class brain tumor segmentation,”  in 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017, pp. 1727–1731.
  * [4] Mohamed Akil, Rachida Saouli, Rostom Kachouri, et al.,  “Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy,”  Medical image analysis, vol. 63, pp. 101692, 2020.
  * [5] Baiying Lei, Zaimin Xia, Feng Jiang, Xudong Jiang, Zongyuan Ge, Yanwu Xu, Jing Qin, Siping Chen, Tianfu Wang, and Shuqiang Wang,  “Skin lesion segmentation via generative adversarial networks with dual discriminators,”  Medical Image Analysis, vol. 64, pp. 101716, 2020.
  * [6] Xenia Alba, Karim Lekadir, Marco Pereanez, Pau Medrano-Gracia, Alistair A Young, and Alejandro F Frangi,  “Automatic initialization and quality control of large-scale cardiac mri segmentations,”  Medical image analysis, vol. 43, pp. 129–141, 2018.
  * [7] Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Heinrich, Bernhard Kainz, Ben Glocker, and Daniel Rueckert,  “Attention gated networks: Learning to leverage salient regions in medical images,”  Medical image analysis, vol. 53, pp. 197–207, 2019.
  * [8] Rüdiger Schmitz, Frederic Madesta, Maximilian Nielsen, Jenny Krause, Stefan Steurer, René Werner, and Thomas Rösch,  “Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture,”  Medical image analysis, vol. 70, pp. 101996, 2021.
  * [9] Olaf Ronneberger, Philipp Fischer, and Thomas Brox,  “U-net: Convolutional networks for biomedical image segmentation,”  in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241.
  * [10] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [11] Clement Farabet, Camille Couprie, Laurent Najman, and Yann LeCun,  “Learning hierarchical features for scene labeling,”  IEEE transactions on pattern analysis and machine intelligence, vol. 35, no. 8, pp. 1915–1929, 2012.
  * [12] Carlo Gatta, Adriana Romero, and Joost van de Veijer,  “Unrolling loopy top-down semantic feedback in convolutional deep networks,”  in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2014, pp. 498–505.
  * [13] David Grangier, Léon Bottou, and Ronan Collobert,  “Deep convolutional networks for scene parsing,”  in ICML 2009 Deep Learning Workshop. Citeseer, 2009, vol. 3, p. 109\.
  * [14] Jonathan Long, Evan Shelhamer, and Trevor Darrell,  “Fully convolutional networks for semantic segmentation,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
  * [15] Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla,  “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,”  IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.
  * [16] Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han,  “Learning deconvolution network for semantic segmentation,”  in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1520–1528.
  * [17] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille,  “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,”  IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 834–848, 2017.
  * [18] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang,  “Unet++: A nested u-net architecture for medical image segmentation,”  in Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11. Springer, 2018.
  * [19] Suiyi Li, Yuxuan Chen, Su Yang, and Wuyang Luo,  “Cascade dense-unet for prostate segmentation in mr images,”  in International Conference on Intelligent Computing. Springer, 2019, pp. 481–490.
  * [20] Chen Li, Yusong Tan, Wei Chen, Xin Luo, Yuanming Gao, Xiaogang Jia, and Zhiying Wang,  “Attention unet++: A nested attention-aware u-net for liver ct image segmentation,”  in 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020, pp. 345–349.
  * [21] Yilong Li, Zhaoyang Xu, Yaqi Wang, Huiyu Zhou, and Qianni Zhang,  “Su-net and du-net fusion for tumour segmentation in histopathology images,”  in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, 2020, pp. 461–465.
  * [22] Ruiwei Feng, Xuechen Liu, Jintai Chen, Danny Z Chen, Honghao Gao, and Jian Wu,  “A deep learning approach for colonoscopy pathology wsi analysis: accurate segmentation and classification,”  IEEE Journal of Biomedical and Health Informatics, 2020.
  * [23] Nisarg A Shah, Divij Gupta, Romil Lodaya, Ujjwal Baid, and Sanjay Talbar,  “Colorectal cancer segmentation using atrous convolution and residual enhanced unet,”  arXiv preprint arXiv:2103.09289, 2021.
  * [24] Andrew Janowczyk and Anant Madabhushi,  “Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases,”  Journal of pathology informatics, vol. 7, 2016.
  * [25] Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez,  “A survey on deep learning in medical image analysis,”  Medical image analysis, vol. 42, pp. 60–88, 2017.
  * [26] Teresa Araújo, Guilherme Aresta, Eduardo Castro, José Rouco, Paulo Aguiar, Catarina Eloy, António Polónia, and Aurélio Campilho,  “Classification of breast cancer histology images using convolutional neural networks,”  PloS one, vol. 12, no. 6, pp. e0177544, 2017.
  * [27] Harshita Sharma, Norman Zerbe, Iris Klempert, Olaf Hellwich, and Peter Hufnagl,  “Deep convolutional neural networks for automatic classification of gastric carcinoma using whole slide images in digital histopathology,”  Computerized Medical Imaging and Graphics, vol. 61, pp. 2–13, 2017\.
  * [28] Dan Ciresan, Alessandro Giusti, Luca Gambardella, and Jürgen Schmidhuber,  “Deep neural networks segment neuronal membranes in electron microscopy images,”  Advances in neural information processing systems, vol. 25, pp. 2843–2851, 2012.
  * [29] Seyed Alireza Fatemi Jahromi, Ali Asghar Khani, Hatef Otroshi Shahreza, Mahdieh Soleymani Baghshah, and Hamid Behroozi,  “A deep learning framework for viable tumor burden estimation,”  in 2020 6th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS). IEEE, 2020, pp. 1–7.
  * [30] Xiyue Wang, Yuqi Fang, Sen Yang, Delong Zhu, Minghui Wang, Jing Zhang, Kai-yu Tong, and Xiao Han,  “A hybrid network for automatic hepatocellular carcinoma segmentation in h&e-stained whole slide images,”  Medical Image Analysis, vol. 68, pp. 101914, 2021.
  * [31] Laurent Itti, Christof Koch, and Ernst Niebur,  “A model of saliency-based visual attention for rapid scene analysis,”  IEEE Transactions on pattern analysis and machine intelligence, vol. 20, no. 11, pp. 1254–1259, 1998.
  * [32] Ronald A Rensink,  “The dynamic representation of scenes,”  Visual cognition, vol. 7, no. 1-3, pp. 17–42, 2000.
  * [33] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton,  “Deep learning,”  nature, vol. 521, no. 7553, pp. 436–444, 2015.
  * [34] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton,  “Imagenet classification with deep convolutional neural networks,”  Advances in neural information processing systems, vol. 25, pp. 1097–1105, 2012.
  * [35] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich,  “Going deeper with convolutions,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
  * [36] Jianlong Fu, Heliang Zheng, and Tao Mei,  “Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4438–4446.
  * [37] Jie Hu, Li Shen, and Gang Sun,  “Squeeze-and-excitation networks,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141.
  * [38] Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu,  “Dual attention network for scene segmentation,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3146–3154.
  * [39] Li Mi and Zhenzhong Chen,  “Hierarchical graph attention network for visual relationship detection,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13886–13895.
  * [40] Xudong Guo, Xun Guo, and Yan Lu,  “Ssan: Separable self-attention network for video representation learning,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12618–12627.
  * [41] Jongchan Park, Sanghyun Woo, Joon-Young Lee, and In So Kweon,  “Bam: Bottleneck attention module,”  arXiv preprint arXiv:1807.06514, 2018.
  * [42] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon,  “Cbam: Convolutional block attention module,”  in Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19.
  * [43] Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al.,  “Attention u-net: Learning where to look for the pancreas,”  arXiv preprint arXiv:1804.03999, 2018.
  * [44] Sergey Ioffe and Christian Szegedy,  “Batch normalization: Accelerating deep network training by reducing internal covariate shift,”  in International conference on machine learning. PMLR, 2015, pp. 448–456.
  * [45] Vinod Nair and Geoffrey E Hinton,  “Rectified linear units improve restricted boltzmann machines,”  in Icml, 2010.
  * [46] Yoo Jung Kim, Hyungjoon Jang, Kyoungbun Lee, Seongkeun Park, Sung-Gyu Min, Choyeon Hong, Jeong Hwan Park, Kanggeun Lee, Jisoo Kim, Wonjae Hong, et al.,  “Paip 2019: Liver cancer segmentation challenge,”  Medical Image Analysis, vol. 67, pp. 101854, 2021.
  * [47] Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang,  “Selective kernel networks,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 510–519.
  * [48] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie,  “Feature pyramid networks for object detection,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125.
  * [49] Thanh Nguyen-Duc, Tran Minh Quan, and Won-Ki Jeong,  “Frequency-splitting dynamic mri reconstruction using multi-scale 3d convolutional sparse coding and automatic parameter selection,”  Medical image analysis, vol. 53, pp. 179–196, 2019.
  * [50] Wang Zhou,  “Image quality assessment: from error measurement to structural similarity,”  IEEE transactions on image processing, vol. 13, pp. 600–613, 2004\.
  * [51] Zhou Wang, Eero P Simoncelli, and Alan C Bovik,  “Multiscale structural similarity for image quality assessment,”  in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. Ieee, 2003, vol. 2, pp. 1398–1402.
  * [52] Md Atiqur Rahman and Yang Wang,  “Optimizing intersection-over-union in deep neural networks for image segmentation,”  in International symposium on visual computing. Springer, 2016, pp. 234–244.
  * [53] Gellért Máttyus, Wenjie Luo, and Raquel Urtasun,  “Deeproadmapper: Extracting road topology from aerial images,”  in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3438–3446.
  * [54] Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso,  “Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations,”  in Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 240–248. Springer, 2017.
  * [55] Diederik P Kingma and Jimmy Ba,  “Adam: A method for stochastic optimization,”  arXiv preprint arXiv:1412.6980, 2014.
  * [56] Mingxing Tan and Quoc Le,  “Efficientnet: Rethinking model scaling for convolutional neural networks,”  in International Conference on Machine Learning. PMLR, 2019, pp. 6105–6114.