# Uncertainty-Aware Lung Nodule Segmentation with Multiple Annotations

Qiuli Wang, Han Yang, Lu Shen, Mengke Zhang Q. Wang, H. Yang and L. Shen
contribute equally to this paper.

###### Abstract

Since radiologists have different training and clinical experience, they may
provide various segmentation maps for a lung nodule. As a result, for a
specific lung nodule, some regions have a higher chance of causing
segmentation uncertainty, which brings difficulty for lung nodule segmentation
with multiple annotations. To address this problem, this paper proposes an
Uncertainty-Aware Segmentation Network (UAS-Net) based on multi-branch U-Net,
which can learn the valuable visual features from the regions that may cause
segmentation uncertainty and contribute to a better segmentation result.
Meanwhile, this network can provide a Multi-Confidence Mask (MCM)
simultaneously, pointing out regions with different segmentation uncertainty
levels. We introduce a Feature-Aware Concatenation structure for different
learning targets and let each branch have a specific learning preference.
Moreover, a joint adversarial learning process is also adopted to help learn
discriminative features of complex structures. Experimental results show that
our method can predict the reasonable regions with higher uncertainty and
improve lung nodule segmentation performance in LIDC-IDRI.

###### Index Terms:

Lung Nodule, Segmentation, Uncertainty, Semantic Feature

##  I Introduction

Lung nodule segmentation with multiple annotations is a task closely related
to the clinical diagnosis process and plays a crucial role in Computer-Aided
Diagnosis (CAD) systems for lung nodule [1]. Traditional methods usually
ignore the uncertain regions between multiple annotations and select a single
annotation as the learning target [2, 3, 4, 5]. To make the most of multiple
annotations, studies like [6, 7, 8] propose to model these uncertainties as
probabilistic distributions and produce segmentation masks with the random
variables in latent space. However, relying on random variables, these methods
cannot produce stable segmentation maps. How to make full use of multiple
annotations is a problem that remains to be solved.

Figure 1: A: Overview of High Certainty (HC) mask, Low Certainty (LC) mask,
and Multi-Confidence Mask. B: Kernel density estimation of HC and LC masks in
LIDC-IDRI. The HU in LC are mainly distributed around _HU -750_ , and the HU
in HC are mainly distributed around _HU 0_. The MCM is combined with HC and LC
masks.

In this paper, we propose that regions that cause uncertain segmentation
results are not ‘random’, and some features have higher chances that cause
uncertainty between multiple annotations. More importantly, conventional deep
learning methods also have difficulty in dealing with these regions and
features. To demonstrate our findings, we propose to fuse all annotations for
a lung nodule as the Multi-Confidence Mask (MCM). As shown in Figure 1.A, the
MCM is the combination of the multiple annotations’ union and intersection. We
define two categories for the mask pixels: High Confidence (HC) mask and Low
Confidence (LC) mask. The HC mask is the region labeled by all radiologists
(intersection of all annotations), and the LC mask is the region that the
radiologists have disagreements (the difference of union and intersection).
The question is, what causes the differences between HC and LC masks?

To reveal this phenomenon, we further calculate the HU (Hounsfield Unit)
kernal estimations in HC and LC of LIDC-IDRI [9], and show them in Figure 1.B.
As can be seen, HC and LC masks have quite different HU distributions: HU
values in HC masks are mainly distributed around _HU 0_ , while HU values in
LC masks are mainly distributed around _HU -750_. This phenomenon indicates
that the radiologists may have more diverse opinions about nodule low dense
tissues and boundaries [10]. This study’s goal is to learn the HC and LC
according to their density characteristics, ensure the segmentation
performance of HC regions, and improve the segmentation performance of LC
regions as well as possible. By doing so, we can improve the overall
segmentation performance, and predict lung nodule regions with different
segmentation uncertainties.

To achieve this goal, we introduce a Uncertainty-Aware Segmentation Network
(UAS-Net). This network is based on a multi-branch U-Net, and each branch has
a specific learning target: one for the annotations’ union, one for the
annotations’ intersection, and one for the single plausible annotation between
the union and intersection. The learning of union and intersection can be
combined as the predicted MCM, and all targets contribute to the final
segmentation result.

Figure 2: Overview of the Uncertainty-Aware Segmentation Network. The network
contains three up-sampling branches sharing the same down-sampling process.
For the learning of multiple annotations’ union and intersection, the first
two branches adopt the Feature-Aware Concatenation (FA-Cat) which can change
the learning preferences of by switching the adaptive methods during
concatenation. We use a pix2pix for the better generation of the multi-
confidence mask, and use a malignancy level discriminator for the better
learning of discriminative visual features.

The main challenge for our UAS-Net is to learn the discriminative visual
features from different learning targets, especially these regions with
different segmentation uncertainties. we address this challenge by introducing
a Feature-Aware Concatenation (FA-Cat) that can control the learning
preferences. The FA-Cat first uses the self-attention mechanism to compress
feature maps, then uses a replaceable self-adaptive method to capture specific
visual features for the concatenation. The self-adaptive method decides the
learning preferences of FA-Cat, the self-attention mechanism help to resolve
more features after feature map compression, and by switching the self-
adaptive method, we can adjust the learning preference of each branch.

Moreover, we introduce a joint adversarial process to guide the parameters’
training, which contains a lung nodule synthesis network and a malignancy
level discriminator. The synthesis network guides the learning of the
uncertain region features by transforming the MCM masks to images, since the
better mask can contribute to better synthesis results [11, 12]. The a
malignancy level discriminator is used to guide the learning of discriminative
features jointly.

Our contributions can be summarized as follows: (1) We introduce a Multi-
Confidence Mask (MCM), which can combine multiple annotations’ union and
intersection, and reflect lung nodule regions with different segmentation
confidences. (2) We propose a Uncertainty-Aware Segmentation Network (UAS-
Net), which aims to learn the valuable visual features from regions with
different uncertainty levels, improve the segmentation performance, and
provide MCM simultaneously. (3) We introduce a Feature-Aware Concatenation
(FA-Cat). By combining the self-attention mechanism and self-adaptive method,
FA-Cat can change learning preferences during concatenation.

##  II Methods

###  II-A The Network Architecture

The architecture of the Uncertainty-Aware Segmentation Network is shown in
Figure 2. The input of the network is a $n-annotation$ set
$A=\\{a_{1},a_{2},...a_{n}\\},n\leq 4$, where $n$ represents the annotation
number. Then the network has three learning targets: intersection $\cap(A)$,
union $\cup(A)$ and specific annotation $R(A)$. The $\cap(A)$ indicates the
regions that all radiologists’ consensus; the $\cup(A)$ shows the possible
largest region of lung nodules; the $\cup(A)-\cap(A)$ indicates the regions
that may cause disagreements; the $R(A)$ provides a possible result between
the consensus and disagreements. The Multi-Confidence Mask (MCM) is the
combination of $\cup(A)-\cap(A)$ and $\cap(A)$.  The selection of single
annotation is consistent with studies [13, 14]. Since the area of
$\cup(A)-\cap(A)$ is too small for learning, we use the union set $\cup(A)$
instead of $\cup(A)-\cap(A)$ as the learning target. The learned union and
intersection are combined directly as the MCM.

Sharing the same down-sampling process, our method has three up-sampling
branches. The first branch is assigned for the learning of multiple
annotations’ union $\cup(A)$, the second branch is assigned for the learning
of multiple annotations’ intersection $\cap(A)$. The last branch is a general
U-Net up-sampling branch. All branches are concatenated together for the
learning of a specific annotation $R(A^{\prime})$. Correspondingly, we have
predicted $\cup(A^{\prime})$, $\cup(A^{\prime})$, and $R(A^{\prime})$.

The first two branches have specific learning preferences by using different
adaptive methods of Feature-Aware Concatenation Block (FA-Cat), and the third
branch obtains a conventional segmentation map by using the traditional
encoding-decoding structure. The $R(A^{\prime})$ obtained by combining
features captured by different branches is the segmentation result with the
largest amount of information for each lung nodule, which is the overall
segmentation. The details of FA-Cat will be discussed in Section II-B. Then,
we fuse the $\cup(A^{\prime})$ and $\cup(A^{\prime})$ as Multi-Confidence Mask
(MCM) to show different certainty levels of each lung nodule’s different
regions by adding the $\cup(A^{\prime})$ and $\cup(A^{\prime})$ directly:

$MCM=normalize(\cup(A^{\prime})+\cup(A^{\prime}))$ (1)

Moreover, we design a joint adversarial loss to guide the network: (1) It
forces the network to generate a better MCM by transforming the mask back to
nodule image using a pix2pix [15, 16], since better mask quality contributes
to better nodule synthesis [17, 18]; (2) It guides all branches to learn
discriminative features using a malignancy level discriminator.

###  II-B Feature-Aware Concatenation

The advantage of our network is that our network can use different
concatenation strategies for a specific learning targets, which is achieved by
introducing a Feature-Aware Concatenation. The architecture of FA-Cat is shown
in Figure 3. Assuming the convolutional feature maps is
$\mathbf{X}\in\mathbb{R}^{c\times w\times h}$, it will passes the FA-Cat and
gets $\mathbf{U}\in\mathbb{R}^{(c+\frac{c}{16})\times w\times h}$. The FA-Cat
sequentially has two blocks: a Squeeze-and-Excitation (SE) Block, and a
Squeeze-and-Adapt (SA) Block.

Figure 3: Illustration for the FA-Cat. Our network contains five down-sampling
layers, and each branch has five up-sampling layers. Each down-sampling and
up-sampling layers are connected with concatenation correspondingly. The first
two layers use our FA-Cat, and the last three layers use the general
concatenation of U-Net.

The SE Block is based on Spatial Squeeze and Channel Excitation Block (cSE)
[19] and learns the channel-wise information of convolutional feature maps.
First, we apply the global average pooling on $\mathbf{X}$ and get a vector
$z\in\mathbb{R}^{c\times 1\times 1}$. Next $z$ will be fed into an encoder-
decoder operation defined as $\tilde{z}=W_{1}(W_{2}z)$, where
$W_{1}\in\mathbb{R}^{C\times\frac{C}{16}}$, and
$W_{2}\in\mathbb{R}^{\frac{C}{16}\times C}$. Then we have
$\tilde{z}\in\mathbb{R}^{c\times 1\times 1}$. Finally, we get the output
$\mathbf{U}=\mathbf{X}\times\tilde{z}$, $\mathbf{U}\in\mathbb{R}^{c\times
w\times h}$.

The Squeeze-and-Adapt (SA) Block combines the original conventional features
with the features learnt by the self-adaptive method. First, SA Block
compresses the $\mathbf{U}\in\mathbb{R}^{c\times w\times h}$ to
$\mathbf{U^{\prime}}\in\mathbb{R}^{\frac{C}{16}\times w\times h}$. Then the
self-adaptive method $\xi$ will extract visual features from
$\mathbf{U^{\prime}}$ and gets
$\xi(U^{\prime})\in\mathbb{R}^{\frac{C}{16}\times w\times h}$. The final
output of SA Block which is also the output of FA-Cat is
$\mathbf{Y}=\mathbf{U}\oplus\xi(U^{\prime})$,
$\mathbf{Y}\in\mathbb{R}^{(c+\frac{c}{16})\times w\times h}$, where
$\mathbf{U}$ is the output of SE Block, and $\xi(U^{\prime})$ provides
features learnt by the self-adaptive method. In the two FA-Cats of our
network, we select Sobel edge detection and Otsu [20] respectively as adaptive
methods to extract different features. The reason we choose these two methods
is that: (1) The learning of intersection needs to pay more attention to the
high-dense nodule regions. As mentioned in the study [11], Otsu can reserve
the nodule density information well, so we adopt Otsu in the FA-Cat for the
intersection learning; (2) The learning of union needs to pay more attention
to the low-dense nodule regions. Otsu can reserve the low-dense regions well,
but it cannot capture delicate features around the nodule, such as
spiculation. In this case, the method which can capture texture information is
better. As a result, we choose Sobel in the FA-Cat for the union learning.

###  II-C Loss Functions

First of all, we use BCE (Binary Cross-Entropy) to measure the segmentation
loss:

$\displaystyle S(A,A^{\prime})=BCE(\cap(A^{\prime}),\cap(A))+$ (2)
$\displaystyle BCE(\cup(A^{\prime}),\cup(A))+BCE(R(A^{\prime}),R(A))$

Secondly, the Multi-Confidence Mask (MCM) $M$ will be transformed back to lung
nodule CT image $X^{\prime}$ with an adversarial pix2pix, which contains a
generator and a discriminator. The loss for pix2pix generator is defined as
$\mathcal{L}_{G}$:

$\mathcal{L}_{L1}(G)=\mathbb{E}_{X,M,A}[||A-G(X,M)||_{1}]$ (3)

where $G$ is the pix2pix generator that transforms MCM $M$ to synthesis CT
image $X^{\prime}$ according to $X$. The pix2pix discriminator based on
conventional CNN is used to discriminate the input tuples: {$X^{\prime}$, $M$}
and {$X$, $M$}:

$\displaystyle\mathcal{L}_{GAN}(G,D)=$
$\displaystyle\mathbb{E}_{X,M}[logD(X,M)]+$ (4)
$\displaystyle\mathbb{E}_{X^{\prime},M}[log(1-D(X^{\prime},M)]$

Moreover, a CNN discriminator $C$ is used to discriminate the malignancy level
of input $R(A^{\prime})$, which also adopts the cross-entropy as the loss
function.

Then we have our final object:

$\displaystyle\mathop{\arg\min}_{G,S,C}\mathop{\max}_{D}\mathcal{L}_{GAN}(G,D)+\mathcal{L}_{L1}(G)+S(A,A^{\prime})+C(R(A^{\prime}))$
(5)

##  III Experiments

###  III-A Dataset and Experimental Settings

The used dataset is LIDC-IDRI [9], which has 2635 annotated lung nodules with
malignancy labels. We extract 1859 lung nodules, and calculate the
intersection and union of annotations through their multiple annotations. To
comprehensively evaluate the network, we use five-cross validation in this
study. We split the dataset into five folds and ensure each fold has the same
malignant/benign nodule distributions. Our network is built on PyTorch [21].
All experiments are run on a GPU of NVIDIA Tesla V100.

###  III-B Performance of General Lung Nodule Segmentation

We use Dice and IoU as the evaluation metrics for the lung nodule
segmentation, and compare our network with a variety of commonly used medical
image segmentation networks which are trained with single annotation. The
experimental results are shown in Table I. According to Table I, the average
dice score of our network is 85.04, which is the highest comparing to other
methods. Generally speaking, U-Net-based methods have better performances than
traditional machine learning methods. R2U-Net [22] achieves 83.59, which uses
recurrent residual block to effectively increase the depth of the network and
make the network could extract better features. The Channel U-Net [23] applies
the channel-attention unit recursively on feature maps at every layer of the
network, which is effective and it achieves 84.02. Our UAS-Net achieves 85.04,
which is the highest. At the same time, our average IoU value is 73.94, which
is better than all the above methods except the Multi-Orientation U-Net [24].
However, the Dice value of Multi-Orientation U-Net is only 83.00, so in
general, the performance of our method is still convincing.

TABLE I: Comparison between our framework and existing methods on the LIDC–IDRI dataset. JA indicates the joint adversarial loss. _Method_ | _Dice_ | _IoU_
---|---|---
Level Set [25] | 60.63 | -
Graph Cut [13] | 68.90 | -
CF-CNN [13] | 82.15 | -
FCN V-Net [26] | 79.59 | -
R2U-Net [22] | 83.59 | -
Dual-branch ResNet [14] | 82.74 | -
Multi-Orientation U-Net [24] | 83.00 | 76.00
3D CNN [27] | 83.10 | 71.85
Channel U-Net [23] | 84.02 | 73.53
Nested U-Net [28] | 83.44 | 72.72
Baseline(U-Net) [29] | 83.54 | 73.19
UAS-Net | 85.04 | 73.94

The key to performance improvements is that each branch can capture different
visual features with the help of FA-Cat. As mentioned above, our network uses
two versions of FA-Cat: one adopts the Sobel edge detection, another one
adopts the Otsu. We expect that FA-Cat (Sobel) can pay more attention to the
delicate features and help the learning of union, and FA-Cat (Otsu) can pay
more attention to the density changes and the learning of intersection. To
intuitively observe the effect of Fa-Cat, we visualize the activation heatmaps
of conventional features in the last CNN layer of FA-Cat with Figure 4. As can
be seen, the FA-Cat (Sobel) and FA-Cat (Otsu) have different learning
interests, obviously. FA-Cat (Otsu) pays more attention (red) to high dense
areas in lung nodules, while the FA-Cat (Sobel) pays more attention (yellow)
to the lung nodules’ edge. In Figure 4 we also show the 3D visualization of
the most active neurons in the CNN layers with red pillars. It is quite clear
that FA-Cat (Otsu) activates more neurons in the nodule centers while the FA-
Cat (Sobel) activates more neurons around the nodules. It means FA-Cat (Sobel)
and FA-Cat (Otsu) work as we expected.

Figure 4: Visualization of convolutional 2D and 3D activation heatmaps in FA-
Cat (Sobel) and FA-Cat (Otsu). It can be seen that FA-Cat (Sobel) pays more
attention to the edge of pulmonary nodules, while FA-Cat (Otsu) pays more
attention to the high-density areas of pulmonary nodules.

By combining the multiple branches, our method can learn the features of high
dense areas and edge areas simultaneously, contributing to better overall
segmentation performance. In order to prove the above conclusion, we compare
the segmentation results of different methods in Figure 5. As can be seen, our
method has better performance when dealing with lung nodules with complex
structures, such as low dense tissues.

Figure 5: Segmentation comparisons between U-Net, R2U-Net, and our method. It
can be seen that the our method has a better segmentation effect on nodules
with cavities, spiculation signs or low-density nodules.

###  III-C Prediction of the Multi-Confidence Mask

This section contains two parts. The first part discusses the prediction of
union and intersection of the annotation set, which is the basis of the Multi-
Confidence Mask. The second part demonstrates the prediction of the Multi-
Confidence Mask.

####  III-C1 Multiple Annotations’ Union and Intersection

The quality of the predicted union and intersection directly affects the final
estimation of Multi-Confidence Mask (MCM). To demonstrate the performance of
union and intersection prediction, we design a comparative experiment of U-Net
[29], Dual-Branch U-Net (which add a general branch to the original U-Net),
Feature-Aware Dual-Branch U-Net (which add the Feature-Aware Concatenation
Block to the Dual-Branch U-Net) and our method. The difference between our
method and Feature-Aware Dual-Branch U-Net is that our method contains a joint
adversarial learning process and one more branch for the final segmentation.
The experimental results are shown in Table II.

TABLE II: Prediction of Intersection and Union Framework | Dice
---|---
_Normal Segmentation_ |
U-Net | 0.8361
_Intersection Segmentation_ |
U-Net | 0.8337
Dual-Branch U-Net | 0.8367
Feature-Aware Dual-Branch U-Net | 0.8395
Our Method | 0.8403
_Union Segmentation_ |
U-Net | 0.8611
Dual-Branch U-Net | 0.8698
Feature-Aware Dual-Branch U-Net | 0.8535
Our Method | 0.8689

According to the Table II, the original U-Net achieves an average Dice of
0.8361 with single annotation for each lung nodule. Meanwhile, U-Net achieves
average Dice scores of 0.8337 and 0.8611 with annotation intersection and
union. Compared to the U-Net, Dual-Branch U-Net has similar performance in the
intersection prediction, the Feature-Aware concatenation achieves 0.8395, and
our network achieves the highest 0.8403. It means our Feature-Aware
concatenation and the joint adversarial process can improve the intersection
prediction. In the union prediction, Dual-Branch achieves the highest 0.8698,
the Feature-Aware Dual-Branch U-Net achieves 0.8535, which is the lowest, and
our network reaches 0.8689, which is higher than the U-Net, but slightly lower
than Dual-Branch U-Net.

####  III-C2 Multi-Confidence Mask

Figure 6 shows the segmentation of multiple annotations’ union, intersection,
and MCM. Colors are used to show the differences, and the original outputs are
grayscale.

Figure 6: Prediction of the Multi-Confidence Mask. For each nodule, we provide
the predicted union (P-Union), the predicted intersection (P-Intersection),
the predicted MCM (P-MC Mask), and the standard union (S-Union), intersection
(S-Intersection), MCM (S-MC Mask). The blue regions in the MCM
(standard/predicted) indicate the regions that have higher uncertainty.

As can be seen, our Multi-Confidence Mask (MCM) has several advantages: (1)
MCM can better show the significant cavitary features of lung nodules (Figure
6 A). On the prediction of the intersection, the cavity is obviously larger,
and we believe it is because the prediction is associated with density: the
density of the lung nodule is smaller than the nodule core. (2) MCM can better
show the spiculation sign, which is an important feature for the diagnosis of
the malignant nodule (Figure 6 B, C). The spiculation is a stellate distortion
caused by the intrusion of nodules into surrounding tissue [30], which is low-
dense and distributed at the nodule edges. The network pays more attention to
the low-dense tissues when segmenting the union of multiple annotations. As a
result, it naturally has better performance on spiculation sign segmentation.
(3) MCM can segment the part-solid nodule (Figure 6 D) better, which means the
network can take advantage of the multiple annotations since the experts tend
to have different opinions in the low-dense regions.

Since the LC and HC masks are small and Dice can not directly measure the
ability of MCM prediction, we calculate the HU distributions in HC and LC
masks, and compare them with the real ones. If our network can predict the
regions with high uncertainty well, the HU distributions of predicted HC and
LC should also fit the same distribution. As shown in this Figure 7, the
predicted and real curves are almost the same, which means our predicted
uncertain regions are convincing from the perspective of statistics.

Figure 7: Kernel density estimation for the predicted HC and LC. The dotted
lines indicate the real annotations, and the solid lines mean predicted
annotations. As can be seen, our prediction has a very similar kernel density
estimation, which means they have almost the same data distribution.

###  III-D Effect of the Joint Adversarial Process

To demonstrate the effect of the joint adversarial process, we perform two
experiments, one contains JAP (Joint Adversarial Process), and one does not.
The results of the five-fold evaluation are listed in Table III. As can be
seen, the adversarial learning process leads to lower Dice scores in all five
folds.

TABLE III: Dice of Five-Cross Validation

Models | Fold1 | Fold2 | Fold3 | Fold4 | Fold5 | Average
---|---|---|---|---|---|---
Ours (without JAP) | 0.8470 | 0.8516 | 0.8546 | 0.8548 | 0.8563 | 0.8529
Ours | 0.8432 | 0.8503 | 0.8522 | 0.8532 | 0.8533 | 0.8504

As mentioned above, our adversarial learning process contains two parts: (1) A
MCM to CT image transformation with pix2pix; (2) A malignancy level
discriminator. The transformation between MCM and CT image forces the network
to pay more attention to the low texture tissues, which are low-dense, while
the malignancy level discriminator helps the learning of spiculation. The Dice
score drops because the adversarial learning forces the network to learn more
delicate features such as cavities, spiculation, et. al.

To better demonstrate this phenomenon, we show six nodules with segmentations
in Figure 8. We can clearly see that the network can extract more details (the
red box in the figure) of spiculation, low texture level, and cavity with the
adversarial process. Nodule A, B, C, and D have cavities or low-dense tissues.
The adversarial learning makes the gaps in the segmentation larger, which will
decrease the Dice. Nodule A, D, and E have complete nodule edges such as
spiculation. Our joint adversarial learning process also improves the
detection of such features, which will also decrease the Dice. Meanwhile, a
large part of this Nodule F is low-dense. Our joint adversarial learning helps
to segment more nodule tissues. However, cases like this one are rare, and
they will not improve the average Dice.

Figure 8: Effect of joint adversarial process. Red blocks show that joint
adversarial learning can improve the segmentation of the spiculation, cavity,
and part-solid nodules. ‘JA’ means ‘Joint Adversarial Learning’.

In summary, with the joint adversarial learning, the network can capture more
minor features of nodules which aren’t marked by standard mask. Although the
segmentation of this feature will lead to a slight decrease in Dice, these
features can provide valuable information for the diagnoses of malignant lung
nodule. Therefore, we believe the joint adversarial process is of value.

##  IV Conclusions

This paper proposes a Uncertainty-Aware Segmentation Network, which can learn
from the diverse annotations for each lung nodule and improve overall
segmentation performance simultaneously. The network contains three up-
sampling branches that sharing the same down-sampling process for three
targets. By calculating the difference of the learned intersection and union,
the network show nodule regions with different certainty level with the Multi-
Confidence Mask. All three branches together produce one plausible
segmentation between the consensus and disagreements. We evaluate our network
on LIDC–IDRI. Experimental results show that our method can predict lung
nodule regions with high uncertainty levels and provide better segmentation.

## References

  * [1] H. Liu, H. Cao, E. Song, and _et al._ , “A cascaded dual-pathway residual network for lung nodule segmentation in ct images,” _PHYS MEDICA_ , vol. 63, pp. 112–121, 2019.
  * [2] W. Zhu, C. Liu, W. Fan, and _et al._ , “Deeplung: Deep 3d dual path nets for automated pulmonary nodule detection and classification,” in _Proc. IEEE WACV_ , 2018, pp. 673–681.
  * [3] H. Xie, D. Yang, N. Sun, and _et al._ , “Automated pulmonary nodule detection in ct images using deep convolutional neural networks,” _PATTERN RECOGN_ , vol. 85, pp. 109–119, 2019.
  * [4] L. Gonçalves, J. Novo, and A. Campilho, “Hessian based approaches for 3d lung nodule segmentation,” _EXPERT SYST APPL_ , vol. 61, pp. 1–15, 2016.
  * [5] D. Wu, L. Lu, J. Bi, and _et al._ , “Stratified learning of local anatomical context for lung nodules in ct images,” in _Proc. IEEE CVPR_ , 2010, pp. 2791–2798.
  * [6] S. Kohl, B. Romera-Paredes, C. Meyer, and _et al._ , “A probabilistic u-net for segmentation of ambiguous images,” in _Proc. NIPS_ , 2018, pp. 6965–6975.
  * [7] S. Hu, D. Worrall, S. Knegt, and _et al._ , “Supervised uncertainty quantification for segmentation with multiple annotations,” in _Proc. MICCAI_. Springer, 2019, pp. 137–145.
  * [8] S. A. Kohl, B. Romera-Paredes, K. H. Maier-Hein, and _et al._ , “A hierarchical probabilistic u-net for modeling multi-scale ambiguities,” _arXiv preprint arXiv:1905.13077_ , 2019.
  * [9] S. Armato III, G. McLennan, L. Bidaut, and _et al._ , “The lung image database consortium (lidc) and image database resource initiative (idri): a completed reference database of lung nodules on ct scans,” _MED PHYS_ , vol. 38, no. 2, pp. 915–931, 2011.
  * [10] M. Gao, U. Bagci, L. Lu, and _et al._ , “Holistic classification of ct attenuation patterns for interstitial lung diseases via deep convolutional neural networks,” _Comput Methods Biomech Biomed Eng Imaging Vis_ , vol. 6, no. 1, pp. 1–6, 2015.
  * [11] Q. Wang, X. Zhang, W. Zhang, and _et al._ , “Realistic lung nodule synthesis with multi-target co-guided adversarial mechanism,” _IEEE T MED IMAGING_ , 2021.
  * [12] Z. Xu, X. Wang, H. Shin, and _et al._ , “Tunable ct lung nodule synthesis conditioned on background image and semantic features,” in _Proc. MICCAI Workshops SASHIMI_. Springer, 2019, pp. 62–70.
  * [13] S. Wang, M. Zhou, Z. Liu, and _et al._ , “Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation,” _MED IMAGE ANAL_ , vol. 40, pp. 172–183, 2017.
  * [14] H. Cao, H. Liu, E. Song, and _et al._ , “Dual-branch residual network for lung nodule segmentation,” _Applied Soft Computing_ , vol. 86, p. 105934, 2020.
  * [15] P. Isola, J. Zhu, and T. Zhou, “Image-to-image translation with conditional adversarial networks,” in _Proc. IEEE CVPR_ , 2017, pp. 1125–1134.
  * [16] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” 2014.
  * [17] Q. Wang, X. Zhang, W. Chen, and _et al._ , “Class-aware multi-window adversarial lung nodule synthesis conditioned on semantic features,” in _Proc. MICCAI_. Springer, 2020, pp. 589–598.
  * [18] S. Gu, J. Bao, H. Yang, and _et al._ , “Mask-guided portrait editing with conditional gans,” in _Proc. IEEE CVPR_ , 2019, pp. 3436–3445.
  * [19] J. Hu, L. Shen, S. Albanie, and _et al._ , “Squeeze-and-excitation networks,” _IEEE T PATTERN ANAL_ , vol. PP, no. 99, 2017.
  * [20] N. Otsu, “A threshold selection method from gray-level histograms,” _IEEE T SYST MAN CY-S_ , vol. 9, no. 1, pp. 62–66, 2007.
  * [21] A. Paszke, S. Gross, F. Massa, and _et al._ , “Pytorch: An imperative style, high-performance deep learning library,” _arXiv preprint arXiv:1912.01703_ , 2019.
  * [22] M. Z. Alom, M. Hasan, C. Yakopcic, and _et al._ , “Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation,” 2018.
  * [23] B. Tolooshams, R. Giri, A. Song, and _et al._ , “Channel-attention dense u-net for multichannel speech enhancement,” in _Proc. ICASSP_ , 2020, pp. 836–840.
  * [24] P. Amorim, T. F. Moraes, J. Silva, and _et al._ , “Lung nodule segmentation based on convolutional neural networks using multi-orientation and patchwise mechanisms,” in _Proc. VipIMAGE_ , 2019, pp. 286–295.
  * [25] A. A. Farag, H. Munim, and J. H. Graham, “A novel approach for lung nodules segmentation in chest ct using level sets,” _IEEE T IMAGE PROCESS_ , vol. 22, no. 12, pp. 5202–5213, 2013.
  * [26] F. Milletari, N. Navab, and S. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in _Proc. 3DV_ , 2016, pp. 565–571.
  * [27] S. Ji, W. Xu, M. Yang, and _et al._ , “3d convolutional neural networks for human action recognition,” _IEEE T PATTERN ANAL_ , vol. 35, no. 1, pp. 221–231, 2013.
  * [28] Z. Zhou, R. Siddiquee, M. Mahfuzur, and _et al._ , “Unet++: A nested u-net architecture for medical image segmentation,” in _Proc. DLMIA Workshop_ , 2018, pp. 3–11.
  * [29] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _Proc. MICCAI_. Springer, 2015, pp. 234–241.
  * [30] S. F. Huang, R. Chang, D. Chen, and _et al._ , “Characterization of spiculation on ultrasound lesions,” _IEEE T MED IMAGING_ , vol. 23, no. 1, pp. 111–121, 2004.