# A Cascaded Approach for ultraly High Performance Lesion Detection and False
Positive Removal in Liver CT Scans

Fakai Wang, Chi-Tung Cheng, Chien-Wei Peng, Ke Yan,
Min Wu, , Le Lu, , Chien-Hung Liao, and Ling Zhang Fakai Wang and Min Wu are
with the ECE Department, University of Maryland, College Park, MD, USA.
(email:jackwangumd@gmail.com). Chi-Tung Cheng<EMAIL_ADDRESS>and
Chien-Hung Liao are with the Department of Trauma and Emergency Surgery, Chang
Gung Memorial Hospital at Linkou, Chang Gung University, Linkou, Taiwan, ROC.
Chien-Wei Peng is with Department of Gastroenterology and Hepatology, Chang
Gung Memorial Hospital, Linkou Medical Center, Linkou, Taiwan, ROC. Ke Yan, Le
Lu, and Ling Zhang are with DAMO Academy, Alibaba Group.

###### Abstract

Liver cancer has high morbidity and mortality rates in the world. Multi-phase
CT is a main medical imaging modality for detecting/identifying and diagnosing
liver tumors. Automatically detecting and classifying liver lesions in CT
images have the potential to improve the clinical workflow. This task remains
challenging due to liver lesions’ large variations in size, appearance, image
contrast, and the complexities of tumor types or subtypes. In this work, we
customize a multi-object labeling tool for multi-phase CT images, which is
used to curate a large-scale dataset containing 1,631 patients with four-phase
CT images, multi-organ masks, and multi-lesion (six major types of liver
lesions confirmed by pathology) masks. We develop a two-stage liver lesion
detection pipeline, where the high-sensitivity detecting algorithms in the
first stage discover as many lesion proposals as possible, and the lesion-
reclassification algorithms in the second stage remove as many false alarms as
possible. The multi-sensitivity lesion detection algorithm maximizes the
information utilization of the individual probability maps of segmentation,
and the lesion-shuffle augmentation effectively explores the texture contrast
between lesions and the liver. Independently tested on 331 patient cases, the
proposed model achieves high sensitivity and specificity for malignancy
classification in the multi-phase contrast-enhanced CT (99.2%, 97.1%,
diagnosis setting) and in the noncontrast CT (97.3%, 95.7%, screening
setting).

###### Index Terms:

Liver Tumor Detection, Probabilistic Segmentation, Lesion-shuffle
Augmentation.

##  I Introduction

Liver cancer ranks the 6th by incidence, but the 3rd by mortality in 2020 [1].
Similar to other cancers, it is important to detect liver cancer in the early
stage and take prompt action. Ultrasound and noncontrast computed tomography
(CT) are usually used in the screening setting, while multi-phase contrast-
enhanced CTs are used in the diagnosis setting. According to liver imaging
reporting and data system (LI-RADS) [2] guidelines, contrast-enhanced CTs or
MRIs are sufficient for identifying liver lesions in broad situations. Early
detection and accurate diagnosis of liver lesions involve many challenges,
such as low awareness, lack of examinations, missed detection of small
lesions, and misclassification of lesion types.

In the opportunistic screening setting (only the noncontrast [NC] CT
scanning), small lesions usually have poor detection performance by
radiologists, due to many factors, such as the complex contexts of the liver,
ambiguous appearances of small lesions, and low image quality. Reading NC CT
to detect potential liver lesions in the high-volume physical examination
scenario where most patients have healthy livers is a tedious task, and the
sensitivity of detection can be low. On the other hand, in the tumor diagnosis
setting, reading multi-phase contrast-enhanced CT images (NC, arterial phase
[AP], venous phase [VP], delayed phase [DP]) requires high expertise and costs
great effort to compare local image patterns to localize lesions within the
large liver organ, and then determine the lesion types. Therefore, developing
algorithms that can automatically detect and classify liver lesions in both
the screening and diagnosis setting is of great interest.

Developing highly accurate liver lesion detection and classification method
for CT images face many challenges. The first challenge is inherent complex
morphology. The liver is a large organ in the abdomen, and there are many
morphology variations in both the liver and lesions, such as location, size,
shape, intensity, and texture. The complex vessels and ducts in the liver and
surrounding organs add to the difficulty of detecting and localizing lesions.
Liver texture changes due to fat accumulation and fibrosis also create much
ambiguity in lesion detection. There are many malignant and benign lesion
types in the liver, and some types are hard to distinguish in the CT due to
appearance similarities.

The second challenge is the lack of ideal datasets; specifically, no large-
scale, high-quality (e.g., covering various liver conditions, with labels of
major tumor types, etc.) liver CT datasets are publicly available for deep
learning models to learn the comprehensive patterns. According to the LI-RADS
guideline, multi-phase contrast-enhanced liver CT images are needed for
radiologists to determine the tumor types, and in some cases, pathology
reports are required. The ideal labels are comprehensive voxel-level
annotations of related organs, vessels, and liver lesions and, importantly,
clinically-confirmed lesion types for all lesions, which are hard to obtain.
Most existing studies rely on their established in-house datasets with weak
labels, e.g., only box-level annotations of lesions and/or without labels of
lesion types. In addition, the curation of a high-quality liver lesion dataset
involves diagnosis-related privacy information and clinical practice in the
medical center, making it harder to open source.

The third challenge is how to design effective and clinically-relevant
evaluation metrics for comprehensive performance judgment of the deep learning
models. Currently, many works employ general image segmentation metrics, which
are volume- and surface-based, such as Dice, average symmetrical surface
distance (ASD), etc. Some works employ object detection metrics, such as
recall, precision, and area under the curve (AUC). However, these general
metrics do not adequately evaluate the model performance, not covering the
detecting ability of small or suspected malignant lesions, which is clinically
important in real-world applications. Therefore, we need to adopt fine-level
evaluation metrics for individual lesions with varied characteristics.

To address these challenges, we develop a customized multi-object labeling
tool for multi-phase CT images (Fig. 1), based on which we curate a large-
scale (n=1,631) CT image dataset with organs&vessels, and liver lesion masks.
We further propose a multi-sensitivity segmentation algorithm for better
lesion probabilistic inference, and introduce a lesion-shuffle training scheme
for better small-lesion detection. Our contributions lie in four folds. First,
to generate initial masks of organs&vessels and liver lesions of our private
dataset, we train a multi-object segmentation model from several public
datasets with related masks, using self-learning and cross-dataset mask
labeling. Afterward, we collaborate with hepatologists to correct lesion masks
by referring to imaging and pathology reports. Second, exploring the
confidence maps from the 3D U-Net models [3], [4], we design multi-sensitivity
algorithms to generate detection results from segmentation, maximizing
information utilization. Third, we design the lesion-shuffle training scheme,
which leads to more robust models for differentiating true lesions from hard
negatives. Fourth, we conduct both the lesion-level and patient-level
evaluations in the screening as well as the diagnosing setting, with several
new practical metrics.

Figure 1: We propose solving the liver lesion detection and diagnosis task by
a multi-class semantic segmentation. Besides segmenting lesions, we
additionally include seven related organ&vessel segmenting tasks for better
lesion performance. We customize a multi-object labeling tool (CT Labeler) and
adopt a semi-supervised learning approach to curate a large-scale multi-phase
abdominal liver CT dataset with high-quality masks. Figure 2: The proposed
workflow. The 3D CT volume goes through the $UNet^{whole}$ model to get the 3D
probability maps, and we adjust the lesion sensitivity to generate
segmentation masks at different lesion sensitivities. We design the “seg2det”
algorithm to generate precise lesion proposals and train a dedicated
reclassification module to filter out false alarms.

##  II Related work

Several organizations have proposed guidelines for liver cancer diagnosis via
CT scanning [5][6][7][8][9]. The machine models must follow these medical
principles, in order to produce clinical-relevant results. Multi-phase
contrast-enhanced CT and MRI form the cornerstone for the detection and
diagnosis of hepatocellular carcinoma (HCC), especially for patients with
cirrhosis [6][10]. The LI-RADS [2], integrated into the HCC clinical practice
guidelines by the AASLD [11], provides standardization for HCC imaging in the
contexts of screening, surveillance, diagnosis, and treatment response
assessment.

Existing studies tackle challenges from various aspects [12][13], such as the
organ contexts, lesion appearances, and data labeling. Many focus on the
multiple-stage workflow [14][15] to disentangle task complexity. Some works
address the lesion ambiguity challenge by multiple view [15], coarse to fine
classification [16], multi-phase CT images [16]. Some work addresses the data
scarcity challenge by data augmentation, generative adversarial network (GAN)
[17], and semi-supervised learning [18].

The two-stage paradigm works well in many CT-based lesion detection tasks
[19][20][21][22][23][24][25][26]. As organs pack together closely in the
abdomen, the boundaries can be vague due to the inter-organ appearance
similarity. The disproportionate volumes between organs and lesions also
impede the model training. One form of the two-stage pipeline [27] localizes
the target organ first and then uses a dedicated model to learn finer-level
lesion patterns. This coarse-to-fine approach helps reduce false positive
lesions and better distinguish lesion types. In another coarse-to-fine
pipeline [28], they employ the level-set method to further refine the tumor
segmentation. The object-based post-processing [29] can also be employed in
the second stage.

Machine models detect lesions based on pixel-level or region-level
probabilities, along with judging thresholds. However, a model optimized for
high precision may overlook malignant lesions, while a model with high recall
may produce an excessive number of false positives, leading to radiologists’
extra ruling out time and even unnecessary follow-up tests [19]. Therefore it
is critical to make the detecting threshold reasonable. The HPVD model [30]
accepts any combination of contrast-phase inputs with adjustable sensitivity
depending on the different clinical settings.

Although there have been several public CT datasets for liver tumor detection
[31][32], many shortcomings and challenges exist. The medical segmentation
decathlon (MSD) [31] collected data from several medical centers, containing
abdominal CT images covering multiple organs (pancreas, lung, prostate, liver,
colon). The AbdomenCT-1K [32] reassembles data from existing datasets such as
MSD and LiTS [33]. However, these datasets are limited in terms of scale, CT
phase availability, disease distribution, and lesion type labels. The clinical
diagnosis process follows strict routines not addressed in current public
datasets. Multi-phase CT images are required for accurate detection and
diagnosis, which is also absent.

Semi-supervised learning [34] can better utilize the knowledge in existing
labeling to create new labeling [35][36][18]. In the pseudo-labeling and self-
training settings [37], semi-supervised learning enhances the labeling
gradually.

##  III Methodology

The proposed workflow (Fig. 2) consists of three steps: whole-CT segmentation,
transforming multiple-sensitivity segmentation into detection, and lesion
reclassification. For the first step, the $UNet^{whole}$ model segments 13
classes, including six major types of liver lesions (HCC, intrahepatic
cholangiocarcinoma [ICC], metastasis [Meta], hemangioma [Hem], focal nodular
hyperplasia + implants + others [Other], and Cyst) and seven organs&vessels
(liver, hepatic vessels, portal&splenic vein, gallbladder, spleen, pancreas,
and stomach) as in Fig. 1. The second and third steps maximize the segmenting
information utilization to refine the detecting results.

From a holistic view, the system should output comprehensive information about
individual lesions in the liver, including locations, 3D extractions (masks),
volumes, densities, etc. The system output should also preserve suspicious and
ambiguous information from the original CT scanning, which may serve as a clue
for early disease alarm. From an implementation view, the system components
would include voxel classification in the 3D image space, voxel clustering for
bio-structures, and lesion extraction.

Combining holistic goals and practical implementations, the initial voxel
classification should extract reliable and detailed information, while the
following steps should effectively refine and retain information. The final
results should reflect the truthiness and comprehensiveness of the system
detections. Throughout the workflow, related information is extracted and
refined, incurring as less distortion as possible. From the information view,
it is important to select implementing operations, intermediate
representations, and evaluation metrics that preserve information and entropy.

Figure 3: Illustration of lesion probability maps and segmentation results of
varied sensitivities. The $UNet^{whole}$ generates separate confidence maps
(2) for six lesion types. The small lesion representations in each 3D map are
not stable. Decreasing or increasing the lesion sensitivity may lead to miss-
detections (3) or false alarms (4).

###  III-A Multi-Sensitivity Segmentation

We employ the 3D nnUNet [4] as the $UNet^{whole}$ backbone, which takes in one
or multiple 3D CT volumes ($\mathbf{I}\in\mathbb{R}^{Z\times Y\times X}$,
$\mathbf{I}$ is the volume) and generates probability confidence maps
$\mathbf{Prob}\in\mathbb{R}^{13\times Z\times Y\times X}$ for six liver lesion
types and seven organs&vessels,

$\mathbf{Prob}\leftarrow UNet^{whole}(\mathbf{I})$ (1)

By default, the UNet applies the argmax function on Prob for generating the
final 3D segmentation mask $\mathbf{Mask}\in\mathbb{R}^{Z\times Y\times X}$.
Specifically, for each voxel $(z,y,x)$ in the 3D space, the argmax compares
all 13 individual probability maps to find the most confident one and assigns
the corresponding label as the voxel value $\mathbf{Mask}^{pred}_{z,y,x}$ in
the segmentation mask,

$\mathbf{Mask}^{pred}_{z,y,x}\leftarrow\underset{i}{argmax}(\mathbf{Prob_{i,z,y,x}}),\
\ i\in[1,13]$ (2)

The lesions (label index: $1$–$6$) have much fewer volumes than organs
($7$–$13$), leading to imbalanced prediction tendencies in the probability
maps. Treating all 13 labels equally during segmentation map generation
results in miss-detection (i.e., false negative [FN]) of small or less obvious
lesions, as shown in Fig 3 (3). By examining the individual confidence maps
$\mathbf{Prob_{i}},\ i\in[1,13]$, we find that there is a wealth of explorable
information pertaining to lesion detection. For example, in Fig. 3 (2), the
lesion probability map has strong representations for large and middle tumors,
but the small tumor has weak confidence. By setting $f=4$ in Equation (3)
before applying the argmax in Equation (2), the high lesion sensitivity
segmentation (in Fig. 3 (4)) represents weak confidence signals for small
lesions much better. The “Prob scale” process in Fig. 2 is defined as

$\mathbf{Prob}[i,:,:,:]\leftarrow\mathbf{Prob}[i,:,:,:]\times\textit{f}\ ,\ \
i\in[1,6]$ (3)

We then develop the “seg2det” algorithm, which precisely separates individual
lesions from the $\mathbf{Mask}\in\mathbb{R}^{Z\times Y\times X}$. The lesions
of one type all have the same label in $\mathbf{Mask}$, making it difficult to
analyze individual lesions. There is no fixed shape for each lesion, and
disentangling them requires analysis of the 3D boundary. The “seg2det”
algorithm firstly separates all individual lesions in the 2D axial-view
slices, then checks the lesion overlapping between neighboring slice pairs. If
two lesions in neighboring slices have overlapping pixels in the 2D space,
then they are considered from the same lesion in the 3D space.

$\mathbf{Det}^{gt}\leftarrow seg2det(\mathbf{Mask}^{gt})$ (4)
$\mathbf{Det}^{pred}_{f=factor}\leftarrow
seg2det(\mathbf{Mask}^{pred}_{f=factor})$ (5)
$\mathbf{TP},\mathbf{FL},\mathbf{FP},\mathbf{FN}\leftarrow
match(\mathbf{Det}^{gt},\mathbf{Det}^{pred})$ (6)
$\mathbf{Det}^{pred}_{same},\mathbf{Det}^{pred}_{diff}\leftarrow
match(\mathbf{Det}^{pred}_{f=1},\mathbf{Det}^{pred}_{f=4})$ (7)

We apply the “seg2det” algorithm on both the ground truth $\mathbf{Mask}^{gt}$
and the predicted $\mathbf{Mask}^{pred}_{f=scale}$ to generate lesion
detection sets $\mathbf{Det}^{gt}$ and $\mathbf{Det}^{pred}_{f=factor}$, each
with precise 3D locations and masks. Further, we develop the “match”
algorithm, which compares two lesion detection sets and outputs the
overlapping relationship. There can be three matching results for each lesion
site between $\mathbf{Det}^{gt}$ and $\mathbf{Det}^{pred}$, which are (1)
overlap with the same label (true positive [TP]), (2) overlap with different
labels (false label [FL]), and (3) lacking correspondence (false positive [FP]
or false negative [FN]).

###  III-B Lesion Reclassification

Figure 4: The lesion shuffling process is based on the lesion mask and
detection results. The patches are randomly sampled around the lesion or in
lesion-free areas in the liver. The shuffling exchanges a lesion with normal
liver textures, creating augmentations for training or inferencing.

The drawback of high lesion sensitivity segmentation (Fig 3 (4)) is an
increased occurrence of FPs on suspicious liver nodules or normal textures. To
enhance the contrast between normal liver textures and TP/FP lesions in the
predicted lesion detection set $\mathbf{Det}^{pred}$, we employ a dedicated
segmentation model (based on nnUNet [4]), denoted as $UNet^{patch}$, which is
trained to classify voxels within liver-lesion mixture patches. All the voxels
outside the liver are removed (mask out). During inference, each lesion has
multiple randomly augmented patches and the lesion truthiness is measured by
the average classification score.

$\mathbf{Det}^{ReCls}\leftarrow reclassify(\mathbf{Det}^{pred})$ (8)

We design the “shuffle” algorithm (Equation (9)) which creates 3D augmentation
patches of the individual lesions. With the exact lesion mask and liver mask
in the 3D space, we are able to crop patches in the lesion-free liver area and
then randomly transplant the lesion into patches, as shown in Fig 4. This
context-aware augmentation process effectively utilizes possible lesion
surroundings within the same patient’s liver. By additionally shuffling
$\mathbf{Det}^{pred}_{FP}$ and $\mathbf{Det}^{pred}_{FN}$, the generated
patches serve as hard cases for training augmentation.

$\mathbf{I}_{patch},\mathbf{Mask}_{patch}\leftarrow
shuffle(\mathbf{Det},\mathbf{I})$ (9)

During inference, the “shuffle” algorithm generates $N$ original positive and
shuffled positive patches. The $UNet^{patch}$ predicts on each patch
$\mathbf{I}_{patch,n}$ to generate $\mathbf{Mask}_{patch,n}^{pred}$ in
Equation (10). The newly predicted $\mathbf{Mask}_{patch,n}^{pred}$ and the
originally predicted $\mathbf{Mask}_{patch,n}$ are binarized and compared for
lesion overlapping in Equation (11). And the overlapping ratio in all $N$
patch comparisons serves as the average score for the lesion. If the score is
below a threshold, the lesion will be discarded in the $\mathbf{Det}^{ReCls}$.
As a result, the FPs especially in high sensitivity segmentation are
effectively reduced. Such lesion reclassification process is shown in Fig 5.

Figure 5: The reclassification process. Each lesion proposal is shuffled
within the liver, to create $N$ augmented patches for the $UNet^{patch}$ to
segment. The average score is compared with a threshold to re-classify this
lesion.

$\mathbf{Mask}^{pred}_{patch,n}\leftarrow
argmax(UNet^{patch}(\mathbf{I}_{patch,n}))$ (10)

$score\leftarrow\frac{1}{N}\Sigma^{N}_{n=1}(bin(\mathbf{Mask}^{pred}_{patch,n},\mathbf{Mask}_{patch,n}))$
(11)

##  IV Experimental Methods

###  IV-A Data and Annotation

####  IV-A1 Data collection

Our dataset consists of patients collected between 2008 and 2018 at Chang Gung
Memorial Hospital, Taiwan, ROC. We follow the Helsinki declaration with
ethical permission number IRB-201800187B0 (liver tumor detection through CT
images). The received CT images are in NIfTI format with patient information
removed to protect privacy. The CT scanners include GE, SIEMENS, and TOSHIBA.
The in-plane resolution ranges from 0.6 mm $\times$ 0.6 mm to 1.0 mm $\times$
1.0 mm, and the slice thickness is 5.0 mm. Each axial slice size is 512
$\times$ 512, and the slice number varies from 35 to 78. The four-phase (NC,
AP, VP, and DP) CT images are aligned by the DEEDS [38] algorithm. The
malignant lesions are manually annotated with 3D bounding boxes by an expert
physician (C-T. C) with referring to the radiology and pathology reports [30].

Figure 6: Data curation steps. The public datasets only have Venous-Phase CTs,
and they do not differentiate lesion types. The Intelligent Editing process
utilizes the CT Labeler in Fig 1, which automatically compares lesion masks
for efficient updating in Steps $4,7,10$. Train/inference steps $5,6,8,9$
focus on segmenting six lesion types and seven organ types in the four-phase
dataset.

####  IV-A2 Curation steps

In the curation steps (Fig 6), we apply the semi-supervised learning on the
public datasets (step ①) and on our private dataset (steps ⑤, ⑥, ⑧). The
public CTs and masks are from seven segmentation tasks in the MSD challenge
[39]. The cross-inference on public datasets using pre-trained nnUNet models
[4] produces 900 CTs with seven organs&vessels masks and liver lesion masks.
The labeling knowledge (masks) is transferred to our dataset by Model A
inferencing on VP-phase to get mask 01 in step ③.

Our CT Labeler’s convenient functions, such as automatic information (image,
mask, reports) loading, automatic windowing, 3D connected-region computation,
lesion boundary visualization, and mask editing (updating/copying/deleting),
substantially increase labeling efficiency. In Fig 6, mask 00 are bounding
boxes containing pathologically-confirmed malignant tumors, and two physicians
(C-T. C and C-W. P) update mask 01 for lesions and organs in step ④. Existing
and newly discovered lesions are re-examined against the radiology and
pathology reports, divided into six sub-types. Trained on four-phase data with
corrected mask 01, Model B is much more reliable than Model A and produces
mask 02. Then we do another round of mask updating and model retraining. The
scale and quality of masks get stably improved from mask 01 to mask 03. In the
end, we get 1,631 cases, with high-quality 3D masks for seven organs&vessels
and six liver lesion types.

####  IV-A3 Data split and distribution

We divide the 1,631 cases into a train-validation set (n=1,300) and a test set
(n=331). We employ the default 5-fold training-validation split and model-
ensemble prediction in nnUNet [4]. The lesion-level statistics of the test set
are in Table I. The lesion has varied sizes/volumes, and the majority of non-
HCC lesions are smaller than 16 cm3. If multiple lesion types are present in
one patient, the patient-level main lesion (_i.e._ , patient class [PatCls])
is defined as the highest-priority lesion type or “normal”. The lesion
priority order is set as HCC $>$ ICC $>$ Meta $>$ Hem $>$ Other $>$ Cyst,
considering the malignancy degree. The “classifyPatient” function in Equation
(12) implements the above lesion classification rules.

TABLE I: The numbers of lesions stratified by volume distribution and lesion types in the test set. Lesions smaller than 0.5 $cm^{3}$ are excluded. Type \V $cm^{3}$ |  0.5$\sim$2 |  2$\sim$4 |  4$\sim$8 |  8$\sim$16 |  16$\sim$64 |  $>$64 | Total
---|---|---|---|---|---|---|---
HCC | 22 | 28 | 36 | 47 | 60 | 66 | 259
ICC | 0 | 0 | 1 | 2 | 7 | 3 | 13
Meta | 69 | 27 | 32 | 18 | 24 | 22 | 192
Hem | 1 | 2 | 1 | 0 | 3 | 5 | 12
Other | 25 | 8 | 7 | 11 | 8 | 2 | 61
Cyst | 51 | 7 | 8 | 1 | 2 | 0 | 69
All | 168 | 72 | 85 | 79 | 104 | 98 | 606
Malig | 91 | 55 | 69 | 67 | 91 | 91 | 464

$\textit{PatCls}\leftarrow\textit{classifyPatient}(\mathbf{Dets})$ (12)

###  IV-B Implementation

####  IV-B1 Configurations for lesion shuffling

In lesion reclassification (Sec. III-B), we fix the patch size (_i.e._ , input
croppings) to be $16\times 128\times 128$ voxels (approximately 8 $\times$ 10
$\times$ 10cm), enough for filtering out FP lesions. Lesions with a volume
larger than 64cm3 in $Det^{pred}$ will not go through the lesion
reclassification process.

In $UNet^{patch}$ training, to generate as many realistic input patches as
possible, we adopt four shuffling schemes in Fig. 4. The original positive
patches are randomly cropped around the lesion, and the original negative
patches are randomly sampled without containing any lesion. The shuffled
positive patch transplants a lesion into the original negative patch, and the
shuffled negative patch replaces the lesion in the original positive patch
with normal liver texture. The shuffling process generates both CT images and
ground-truth masks for individual patches. Each lesion has around $20$
augmentation patches under any shuffling scheme during $UNet^{patch}$
training.

In the reclassification inference process (Fig. 5), we generate $N=10$
augmented lesion-containing patches for each detected lesion. The binarize
function in Equation (11) calculates the overlapping volume between
$UNet^{whole}$-generated $\mathbf{Mask}_{patch,n}$ and the
$UNet^{patch}$-generated $\mathbf{Mask}^{pred}_{patch,n}$. If the average
score is smaller than a threshold of 0.5cm3, then this lesion is discarded
(_i.e._ , reclassified as FP).

####  IV-B2 Multi-sensitivity detection

We set $f=1.0$ and $f=4.0$ (Equation (3)) as lesion probability factors for
low sensitivity and high lesion sensitivity. As a result, the corresponding
detection sets after the reclassify module are $Det^{ReCls}_{f=1}$ and
$Det^{ReCls}_{f=4}$, respectively. We then apply the “match” in Equation (7)
to generate the common set $Det^{ReCls}_{same}$ and difference set
$Det^{ReCls}_{diff}$. The lesions proposals in $Det^{ReCls}_{same}$ are more
reliable, while those in $Det^{ReCls}_{diff}$ are mostly from
$Det^{ReCls}_{f=4}$ only.

###  IV-C Experiment Settings

####  IV-C1 Evaluation metrics

We measure the system performance at the lesion level and patient level. The
lesion-level detection results enable measuring the TP, FL, FP, and FN
(defined in Section III-A) for all test cases, and we can derive the overall
lesion detection recall and precision, _i.e._ , $recall=\frac{TP}{TP+FN}$, and
$prec=\frac{TP}{TP+FL+FP}$. We calculate these metrics for each lesion type.
At the patient-level, we use the classification accuracy of the patient-level
main lesion as the metric, _i.e._ ,
$acc=\frac{1}{N}\sum^{N}_{i}(PatCls^{pred}_{i}==PatCls^{gt}_{i})$ ($PatCls$ is
defined in Equation (12)).

Figure 7: Case 01 masks of different lesion sensitivities. (A) The selected
axial slice is from the venous-phase CT scanning. (B) Ground-truth mask
overlay on the CT image slice, with the arrows pointing to two HCC tumors. (C)
The automatically predicted mask by using lesion sensitivity $f=1$ on
$UNet^{whole}$ probability maps, resulting in one miss detection (FN) and one
False Labeling (FL); the Base model derives $Det^{pred}_{f=1}$ directly from
this mask. (D) The automatically predicted mask by using lesion sensitivity
$f=4$, detecting both HCC tumors; the High+ReCls model derives
$Det^{pred}_{f=4}$ from this mask. Figure 8: The patch cropping ($16\times
128\times 128$) in Fig. 7 (A) (dashed cube), containing HCC tumors while
excluding other organs. The $UNet^{patch}$ model in reclassification module
would train and test on these curated patches, and some are augmented by the
lesion shuffle process.  Figure 9:  Reclassified result for Fig. 8. The two
ground-truth HCC are in pink. The $UNet^{patch}$ segmentation mask (ReCls)
incorrectly identifies one HCC as Meta (red) due to the absence of typical HCC
characteristics described in LI-RADS.

####  IV-C2 System workflow

We illustrate the three-step workflow with intermediate results in Figs. 7–9.
In Fig. 7, Case 01 has two HCC tumors, one is small and located at the liver
boundary, and the other lacks typical HCC characteristics. These challenges
are seen in Fig. 7 (C), where the smaller one is missed and the larger one has
confusing predictions. The proposed High+ReCls (high sensitivity +
reclassification) model relies on high lesion sensitivity detections in Fig. 7
(D) and re-classifies crops in Fig. 8, to produce final ReCls results in Fig.
9.

####  IV-C3 Model variations

For the baseline (Base), we adopt the default nnUNet segmentation mask in the
first step, which uses $f=1$ in Equation (3). Base does not have the
reclassification module (the third step). So the Base detection results would
have many FNs due to the low lesion sensitivity in the first step, as well as
many FPs due to the lack of lesion filtering in the third step.

In the Base+ReCls model, we add the reclassification module. It leads to
reduced FPs correspondingly. In the High+ReCls model, we set lesion
sensitivity $f=4$ in Equation (3) for the segmentation mask generation (first
step), which leads to more detections in the second step. Therefore the
High+ReCls model has both reduced FNs and reduced FPs. Compared to Base+ReCls,
High+ReCls would have fewer FNs at the cost of a few more FPs. Most of the
time, these two models output the same PatCls for each patient case through
Equation (12). When a patient-level difference does exist, it is usually
originated from suspicious lesions, and worth human verification.

In the Base+ReCls w/o LS model, we remove lesion shuffling (LS) in the
reclassify module to investigate its effect. In this setting, the training and
testing of $UNet^{patch}$ only involve randomly cropped patches around the
lesions, and there is no shuffled positive or shuffled negative augmentation.

In the diagnosis setting, the contrast-enhanced CT scanning images
($I^{NC,AP,VP,DP}$) are aligned before being fed into the $UNet^{whole}$.
While in the screening setting, there is only the noncontrast CT image
($I^{NC}$). We run experiments of Base, Base+ReCls, Base+ReCls w/o LS, and
High+ReCls in both settings under the same experimental protocols.

##  V Results

###  V-A Lesion-Level Performances

Table II and Table III summarize the lesion-level performance of different
models on the testing set. The Malig group includes HCC, ICC, and Meta. It is
worth mentioning that our dataset contains mostly patients with liver diseases
and the performance of the screening setting may vary in real practice. In
Table I, there are $606$ lesions in total and $464$ of them are malignant. The
majority of patients with malignant lesions have large lesions (volume $>$
4cm3). The benign lesions (Hem, Other, Cyst) have less representation and are
usually of small size (volume $<$ 8cm3). In real-world screening scenarios,
most patients are free of malignant tumors, with benign lesions outnumbering
malignant ones and most lesions being small.

TABLE II: Lesion-level performance of different models in the diagnosis setting using four-phase CT (INC,AP,VP,DP, 331 testing cases). Using the “seg2det” and “match” algorithms, we precisely compare all individual lesions between the prediction and the ground truth. FN and FP are the numbers of false negative and false positive lesions, respectively, for each lesion type.  Type \Model | Base | Base+ReCls w/o LS | Base+ReCls | High+ReCls
---|---|---|---|---
FN | FP | Precision | Recall | FN | FP | Precision | Recall | FN | FP | Precision | Recall | FN | FP | Precision | Recall
HCC | 16 | 7 | 84.1% | 88.4% | 26 | 1 | 89.4% | 84.9% | 17 | 1 | 87.0% | 88.0% | 12 | 8 | 84.6% | 89.9%
ICC | 0 | 3 | 50.0% | 61.5% | 0 | 1 | 57.1% | 61.5% | 0 | 1 | 57.1% | 61.5% | 0 | 1 | 57.1% | 61.5%
Meta | 24 | 8 | 85.9% | 77.5% | 40 | 5 | 86.2% | 68.8% | 24 | 7 | 86.5% | 77.5% | 13 | 8 | 85.5% | 82.3%
Hem | 0 | 2 | 60.0% | 46.2% | 2 | 0 | 60.0% | 50.0% | 1 | 0 | 66.7% | 50.0% | 1 | 1 | 54.5% | 50.0%
Other | 6 | 14 | 52.6% | 49.2% | 19 | 2 | 66.7% | 36.7% | 11 | 3 | 67.6% | 41.7% | 9 | 8 | 62.5% | 41.7%
Cyst | 0 | 5 | 87.5% | 88.9% | 7 | 1 | 94.4% | 82.3% | 3 | 2 | 91.8% | 88.9% | 5 | 4 | 88.9% | 88.9%
All | 46 | 39 | 80.5% | 79.5% | 94 | 10 | 86.0% | 73.5% | 56 | 14 | 84.9% | 78.8% | 40 | 30 | 82.5% | 81.0%
Malig | 40 | 18 | 95.6% | 88.7% | 66 | 7 | 98.1% | 83.1% | 41 | 9 | 97.8% | 88.5% | 25 | 17 | 95.9% | 91.8%
TABLE III: Lesion-level performance of different models in the screening setting using only noncontrast CT (INC, 331 testing cases).  Type \Model | Base | Base+ReCls w/o LS | Base+ReCls | High+ReCls
---|---|---|---|---
FN | FP | Precision | Recall | FN | FP | Precision | Recall | FN | FP | Precision | Recall | FN | FP | Precision | Recall
HCC | 30 | 27 | 72.5% | 83.9% | 49 | 2 | 79.8% | 77.3% | 34 | 6 | 77.6% | 82.7% | 24 | 15 | 73.6% | 86.7%
ICC | 0 | 2 | 44.4% | 30.8% | 0 | 1 | 44.4% | 30.8% | 0 | 1 | 44.4% | 30.8% | 0 | 1 | 44.4% | 30.8%
Meta | 39 | 7 | 88.5% | 60.0% | 63 | 2 | 93.8% | 50.0% | 39 | 3 | 93.1% | 60.0% | 26 | 10 | 85.1% | 62.6%
Hem | 2 | 0 | 100% | 33.3% | 3 | 0 | 100% | 33.3% | 2 | 0 | 100% | 33.3% | 1 | 0 | 100% | 33.3%
Other | 12 | 8 | 59.0% | 39.7% | 21 | 1 | 73.9% | 29.8% | 15 | 2 | 69.0% | 34.5% | 12 | 4 | 70.0% | 35.0%
Cyst | 2 | 3 | 81.6% | 95.4% | 5 | 3 | 82.1% | 90.2% | 3 | 3 | 81.3% | 93.8% | 3 | 5 | 83.3% | 93.8%
All | 85 | 47 | 76.1% | 71.2% | 141 | 9 | 82.3% | 63.5% | 93 | 15 | 80.8% | 70.0% | 66 | 35 | 77.0% | 72.6%
Malig | 69 | 36 | 91.0% | 80.8% | 112 | 5 | 98.5% | 72.3% | 73 | 10 | 97.3% | 80.1% | 50 | 26 | 93.2% | 85.3%

####  V-A1 Base model

In the Base model, the nnUNet segmentation output ($\mathbf{Mask}^{pred}$,
lesion sensitivity $f=1$) is fed directly to the “seg2det” and “match” steps
to get the $Det^{pred}_{f=1}$, TP, FL, FP, FN results. In Table II, the Base
model has low FN and FP for the Cyst, and the recall reaches 88.9%. However,
the malignant types have high FN (40) and high FP (18), which we want to
decrease. The precision and recall for ICC, Hem, Other are relatively low, due
to unbalanced data representation and ambiguous lesion appearances. In the
screening setting of Table III, FN and FP show substantial increases (85, 47).
The precision and recall decrease substantially for all lesion types except
the Cyst (81.6%, 95.4%). Many lesions lack the characteristic appearance under
noncontrast CT scanning (INC) and are misclassified as Cyst. Compared with
Ref. [35] on the same data source (diagnosis setting) using a detection
approach, our Base model achieves a 10% higher F-score (86.2% vs. 76.3%) for
HCC, effectively supporting our semantic segmentation-for-detection approach.

####  V-A2 Base+ReCls model

For the Base+ReCls model results in Table II, the FPs and precisions for all
lesion types are improved substantially compared to the Base model, due to the
re-classification module. The cost is the slight increase of FNs. In Table
III, similar effects are observed. And the FP and precision for malignant
improve even more (72% reduction and 6.3% increase).

To investigate the influence of the Lesion Shuffling (LS) in the re-
classification module, we conduct experiments of Base+ReCls w/o LS in Table II
and Table III. The ReCls w/o LS could still effectively filter out FPs for all
lesion types, but it suppresses many true lesions leading to a large increase
of FNs. Without the context-aware LS during augmentation, the $UNet^{patch}$
model in the re-classification module could not distinguish lesions from
normal liver textures very well.

####  V-A3 High+ReCls model

The High+ReCls model sets lesion sensitivity $f=4$ in Equation (3) before
generating the $Mask^{pred}_{f=4}$ in Equation (2) and $Det^{pred}_{f=4}$ in
Equation (5). Comparing High+ReCls with Base+ReCls in Table II, the high
lesion sensitivity segmentation can effectively reduce FNs, especially for
malignant lesions, at the cost of increasing FPs. The recall rates do not
catch up comparably due to false label (lesion type prediction error)
detections among the improved portion. The high lesion sensitivity
segmentation shows a similar influence for the screening setting in Table III.

###  V-B Patient-Level Performances

In the screening setting, the patient-level result may help with patient
triage. In the diagnosis setting, when there are multiple lesions, the most
severe one draws attention. In both settings/scenes, whether the patient has a
malignant lesion or not is the primary question to answer. Therefore we
compare the patient-level malignancy classification results of different
models on the testing set in Table IV.

TABLE IV: Patient-level malignancy classification performance for different models on 331 testing cases. In both diagnosis and screening settings, sensitivity and specificity are measured for each model. The results of Base+ReCls & High+ReCls only count in 319 cases with common PatCls predictions. Model \Scene Metrics |  Diagnosis of Malig |  Screening of Malig
---|---|---
Sensitivity | Specificity | Sensitivity | Specificity
Base | 97.7% | 92.9% | 95.0% | 82.9%
Base+ReCls w/o LS | 96.2% | 98.6% | 89.3% | 98.6%
Base+ReCls | 97.3% | 98.6% | 92.3% | 95.7%
High+ReCls | 99.2% | 97.1% | 97.3% | 95.7%
Base+Recls & High+ReCls | 99.2% | 98.5% | 97.5% | 98.4%

We measure the sensitivity and specificity for malignancy classification in
Table IV for both settings. In the diagnosis setting, the specificity
increases by 5% after adding the re-classification module to the Base model.
The sensitivity of the High+ReCls model reaches 99.2%, which is nearly 2%
higher than models with a base lesion sensitivity. These changes are more
obvious in the screening setting, increasing by 12.8% and 5%, respectively.

There is a complementary relationship between Base+ReCls and High+ReCls in
their predictions. From Table II and Table III, the Base+ReCls has the
advantage of lower FPs, while the High+ReCls has the advantage of lower FNs.
Combining Base+ReCls & High+ReCls could take both advantages. The Base+ReCls &
High+ReCls performance only count in patient cases where the two models output
the same patient-level classification (PatCls), through the “classifyPatient”
function in Equation (12).

As shown in Table IV, the Base+ReCls & High+ReCls has both high sensitivity
and specificity in two different settings. A detailed prediction distribution
for the diagnosis setting is in Fig. 10. Only 3.6% (12 cases) are labeled as
uncertain by the joint model, and the rest predictions are mostly accurate. At
the lesion level in Table V, the precision and recall improve slightly
compared to High+ReCls in Table II. It has a high “rough-recall” (ignoring the
lesion type) for both the benign and malignant lesions (94.3%).

Figure 10: Patient-level malignancy classification of the Base+ReCls & High+ReCls in the diagnosis scene. When the joint model classifies a patient as malignant, benign, or normal, most of the time, it is correct. When the joint predictions differ (_i.e._ , the UNCERTAIN set), the patient probably has some liver lesions. TABLE V: Lesion-level detection of High+ReCls model on Base+ReCls & High+ReCls’s consensus portion (_i.e._ , predict the same PatCls for each patient, 319 cases) in diagnosis setting. FN, FP, FL, and TP are the false negative, false positive, false label, and true positive lesion counts for each type using the “match” function in Equation (6). Recall-rough measures the lesion recall rate ignoring the lesion type, _i.e._ , counting in both the TP and the FL.  Type\Metric | High+ReCls
---|---
FN | FP | FL | TP | Precision | Recall | Recall-rough
HCC | 11 | 6 | 13 | 223 | 85.1% | 90.3% | 95.5%
ICC | 0 | 1 | 5 | 8 | 61.5% | 61.5% | 100%
Meta | 13 | 8 | 15 | 130 | 85.5% | 82.3% | 91.8%
Hem | 1 | 1 | 5 | 6 | 60.0% | 50.0% | 91.7%
Other | 8 | 6 | 24 | 25 | 65.8% | 43.9% | 86.0%
Cyst | 5 | 4 | 2 | 56 | 88.9% | 88.9% | 92.1%
All | 38 | 26 | 64 | 448 | 83.3% | 81.5% | 93.1%
Malig | 24 | 15 | 9 | 385 | 96.3% | 92.1% | 94.3%

###  V-C Ablation

Figure 11: The influence of lesion sensitivity factor $f$ in Equation (3) on
the detection results at the lesion level (without the “reclassify” module). A
higher sensitivity leads to fewer miss detections (FN) at the cost of more
false alarms (FP). This rule applies to both the diagnosis and the screening.

####  V-C1 Multi-sensitivity joint prediction: the advantage

We conduct experiments with different sensitivities (i.e., change the scaling
factor $f$) and show the impacts on the detection results in Fig 11. The
lesion sensitivity has the opposite influence on FP and FN in both the
screening and diagnosis settings. An optimal solution would maximize reliable
information utilization while retaining essential entropy for the following
human inspection. The complementary impacts of $f$ on Base+ReCls and
High+ReCls demonstrate the necessity to harness both advantages and minimize
disadvantages in the joint prediction for the final decision.

Figure 12: The individual probability maps from $UNet^{whole}$ (scaled $f=4$,
A.1 – A.6) and $UNet^{patch}$ (B.1 – B.6) for the patch cropping in Fig. 8.
The dedicated $UNet^{patch}$ model generates lesion probability maps with
better concentration and less ambiguity.

####  V-C2 Analysis of $UNet$ probability maps

The proposed workflow consists of two 3D UNet models, $UNet^{whole}$ and
$UNet^{patch}$. We study their generated probability maps for individual
lesions in Fig. 12, to compare the modeling characteristics. The input
cropping in Fig. 8 contains two hard HCC tumors, and the multi-purpose
$UNet^{whole}$ could not generate reliable results for either one. The (A.4)
map has already been scaled by $f=4$, but it is still much weaker than lesions
in the (B.4) map. The $UNet^{whole}$ generates signals in multiple probability
maps (A.1, A.3, A.4, A.5, A.6) for the larger HCC lesion, showing poorer
differentiating ability than the $UNet^{patch}$ model.

Figure 13: False positives in $UNet^{whole}$ results. The Case 02 cropping
(A.1 – A.4) and Case 03 cropping (B.1 – B.4) are both initially classified as
HCC in $Det^{pred}_{f=4}$. They are removed in $Det^{ReCls}$ by $UNet^{patch}$
model through Equation (8).

####  V-C3 Lesion reclassification to reduce false positive detections

The $Det^{pred}_{f=4}$ contains many FPs, illustrated in Fig. 11 and Fig. 13.
Many FPs have suspicious appearances, such as venous phase hyper-enhancement
similar to HCC, or vessel concentration similar to malignant lesions. But the
liver is a large and complex organ, with many types of texture variations for
various reasons. The $UNet^{whole}$ model not only segments the liver but also
segments other related organs, which might limit its representation capability
for liver lesions. The $UNet^{patch}$ in the “reclassify” module, on the other
hand, only focuses on the liver region, with no interference from outside.
With accurate organ masks from $UNet^{whole}$ to guide the cropping process,
the training and testing patches (such as Fig. 8) contain only the liver part.

Figure 14: The Case 04 cropping is an ICC lesion, but it is initially mis-
classified as HCC in $Det^{pred}_{f=4}$ in $UNet^{whole}$ results (_i.e._ ,
false label).  Figure 15: The ground truth mask, initial prediction mask
(HCC), and the re-classified mask (ICC) for Case 04 cropping in Fig. 14.

####  V-C4 Lesion reclassification to correct false labeling detections

The $Det^{pred}$ contains some false label detections, where the model
predicted a wrong label (lesion type) for a lesion. One example is in Fig. 14,
an ICC is predicted as HCC by the $UNet^{whole}$ model. With the help of the
“reclassify” module, it is reclassified as ICC by $UNet^{patch}$ model through
Equation (8). In Fig. 15 (ReCls), a smaller portion of the lesion is still
classified as HCC due to the similarity to HCC outlooks [2]. In the end, the
“reclassify” module will produce a single lesion label from volume-based
voting. Similarly, many other FLs are corrected, leading to the precision
increase in Base+ReCls compared to Base in Table II and Fig. III.

##  VI Discussion

The workflow implementation is based on several foundations. Being familiar
with UNet [4] [40] is necessary to understand the concepts of voxel-wise
classification, individual probability maps, and segmentation mask. We develop
the “seg2det” and “match” algorithms to generate precise lesion detections
(with 3D masks) and pair-wise lesion matching, which enables subsequent
lesion-shuffle training and reclassification. We customize the CT Labeler
software for data curation, which improves labeling accuracy and efficiency.

Due to the complex conditions and various textures in the liver CT images,
some lesions may not be able to be classified into a certain type by their
image appearance. Therefore changing the lesion definition would influence the
outcome. Moreover, due to the morphable nature of lesions as biological
tissues, they may not have well-defined boundaries under certain
circumstances. Therefore, the performance of deep learning models may rely on
dataset characteristics, labeling protocols, and evaluation metrics.

Our data is from a regional liver medical center whose population is limited
to East Asian Han Chinese and the majority of records have liver-related
diseases. In real-world screening, the majority of people have no liver
tumors. Thus the performances for the diagnosis setting are more reliable than
performances for the Screening setting in this paper. Nevertheless, the
procedures of segmentation-for-detection formulation, coarse-to-fine
classification, and context-aware augmentation would generalize to all lesion
detection tasks.

##  VII Conclusion

In this paper, we propose a coarse-to-fine workflow for liver lesion detection
in CT, which achieves high performance in both diagnosis and screening
settings. By developing a customized multi-object labeling tool and adopting a
semi-supervised learning approach, we curate a large-scale (1,631 patient
cases) multi-phase CT image dataset, containing high-quality masks for seven
organs&vessels and six major types of liver lesions. Our workflow consists of
a cascade of 3D segmentation models and procedures, and we describe them with
detailed annotations and results. The proposed model could detect liver
lesions with high precision and recall rates and classify the patient-level
malignancy with high accuracy. We validate the proposed method in
comprehensive experiments, with detailed lesion-level performances, practical
patient-level analysis, and intuitive result visualization. Therefore the
proposed model holds great clinical potential in both diagnosis and screening
scenarios.

## References

  * [1] H. Rumgay and et al., “Global burden of primary liver cancer in 2020 and predictions to 2040,” _Journal of Hepatology_ , 2022.
  * [2] V. Chernyak and et al., “Liver imaging reporting and data system (li-rads) version 2018: Imaging of hepatocellular carcinoma in at-risk patients,” _Radiology_ , vol. 289, no. 3, pp. 816–830, 2018, pMID: 30251931.
  * [3] Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3d u-net: Learning dense volumetric segmentation from sparse annotation,” _ArXiv_ , vol. abs/1606.06650, 2016.
  * [4] F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation.” _Nature methods_ , 2020.
  * [5] M. Omata and et al, “Asia–pacific clinical practice guidelines on the management of hepatocellular carcinoma: a 2017 update,” _Hepatology International_ , vol. 11, no. 4, pp. 317–370, Jul. 2017.
  * [6] N. Yu, V. Chaudhari, S. Raman, C. Lassman, M. Tong, R. Busuttil, and D. Lu, “Ct and mri improve detection of hepatocellular carcinoma, compared with ultrasound alone, in patients with cirrhosis,” _Clinical gastroenterology and hepatology_ , vol. 9, pp. 161–7, 10 2010.
  * [7] C. H. Tan, S.-C. Low, and C. H. Thng, “Apasl and aasld consensus guidelines on imaging diagnosis of hepatocellular carcinoma: A review,” _International journal of hepatology_ , vol. 2011, p. 519783, 01 2011.
  * [8] L. R. Roberts, C. B. Sirlin, F. Zaiem, J. Almasri, L. J. Prokop, J. K. Heimbach, M. H. Murad, and K. Mohammed, “Imaging for the diagnosis of hepatocellular carcinoma: A systematic review and meta‐analysis,” _Hepatology_ , vol. 67, 2018.
  * [9] “Easl clinical practice guidelines: Management of hepatocellular carcinoma,” _Journal of Hepatology_ , vol. 69, no. 1, pp. 182–236, 2018.
  * [10] T. Hennedige, “Advances in computed tomography and magnetic resonance imaging of hepatocellular carcinoma,” _World Journal of Gastroenterology_ , vol. 22, p. 205, 01 2016.
  * [11] J. A. Marrero, L. M. Kulik, C. B. Sirlin, A. X. Zhu, R. S. Finn, M. M. Abecassis, L. R. Roberts, and J. K. Heimbach, “Diagnosis, staging, and management of hepatocellular carcinoma: 2018 practice guidance by the american association for the study of liver diseases,” _Hepatology_ , vol. 68, no. 2, pp. 723–750, 2018.
  * [12] K. Bousabarah and et al., “Automated detection and delineation of hepatocellular carcinoma on multiphasic contrast-enhanced mri using deep learning,” _Abdominal Radiology_ , vol. 46, 01 2021.
  * [13] Q. Jin, Z.-P. Meng, C. Sun, L. Wei, and R. Su, “Ra-unet: A hybrid deep attention-aware network to extract liver and tumor in ct scans,” _Frontiers in Bioengineering and Biotechnology_ , vol. 8, 2020.
  * [14] M. Bellver, K.-K. Maninis, J. Pont-Tuset, X. G. i Nieto, J. Torres, and L. V. Gool, “Detection-aided liver lesion segmentation using deep learning,” _ArXiv_ , vol. abs/1711.11069, 2017.
  * [15] A. A. A. Setio, F. Ciompi, G. Litjens, P. Gerke, C. Jacobs, S. J. van Riel, M. M. W. Wille, M. Naqibullah, C. I. Sánchez, and B. van Ginneken, “Pulmonary nodule detection in ct images: False positive reduction using multi-view convolutional networks,” _IEEE Transactions on Medical Imaging_ , vol. 35, no. 5, pp. 1160–1169, 2016.
  * [16] J. Zhou, W. Wang, B. Lei, W. Ge, Y. Huang, L. Zhang, Y. Yan, D. Zhou, Y. Ding, J. Wu, and W. Wang, “Automatic detection and classification of focal liver lesions based on deep convolutional neural networks: A preliminary study,” _Frontiers in Oncology_ , vol. 10, 2021.
  * [17] M. Frid-Adar, E. Klang, M. M. Amitai, J. Goldberger, and H. Greenspan, “Synthetic data augmentation using gan for improved liver lesion classification,” _2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018)_ , pp. 289–293, 2018.
  * [18] F. Shi and et al., “Semi-supervised deep transfer learning for benign-malignant diagnosis of pulmonary nodules in chest ct images,” _IEEE Transactions on Medical Imaging_ , vol. 41, no. 4, pp. 771–781, 2022\.
  * [19] O. Ozdemir, R. L. Russell, and A. A. Berlin, “A 3d probabilistic deep learning system for detection and diagnosis of lung cancer using low-dose ct scans,” _IEEE Transactions on Medical Imaging_ , vol. 39, pp. 1419–1429, 2020.
  * [20] H. Xie, D. Yang, N. Sun, Z. Chen, and Y. Zhang, “Automated pulmonary nodule detection in ct images using deep convolutional neural networks,” _Pattern Recognition_ , vol. 85, pp. 109–119, 2019.
  * [21] X. Han, “Automatic liver lesion segmentation using a deep convolutional neural network method,” _ArXiv_ , vol. abs/1704.07239, 2017.
  * [22] P. F. Christ and et al., “Automatic liver and lesion segmentation in ct using cascaded fully convolutional neural networks and 3d conditional random fields,” _ArXiv_ , vol. abs/1610.02177, 2016.
  * [23] O. I. Alirr, “Deep learning and level set approach for liver and tumor segmentation from ct scans,” _Journal of Applied Clinical Medical Physics_ , vol. 21, pp. 200 – 209, 2020.
  * [24] K. Yasaka, H. Akai, O. Abe, and S. Kiryu, “Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced ct: A preliminary study.” _Radiology_ , vol. 286 3, pp. 887–896, 2018.
  * [25] S. A. Azer, “Deep learning with convolutional neural networks for identification of liver masses and hepatocellular carcinoma: A systematic review,” _World Journal of Gastrointestinal Oncology_ , vol. 11, pp. 1218 – 1230, 2019.
  * [26] G. Chlebus, A. Schenk, J. H. Moltz, B. van Ginneken, H. K. Hahn, and H. Meine, “Automatic liver tumor segmentation in ct with fully convolutional neural networks and object-based postprocessing,” _Scientific Reports_ , vol. 8, 2018.
  * [27] L. Meng, Q. Zhang, and S. Bu, “Two-stage liver and tumor segmentation algorithm based on convolutional neural network,” _Diagnostics_ , vol. 11, no. 10, 2021.
  * [28] Y. Zhang, B. Jiang, J. Wu, D. Ji, Y. Liu, Y. Chen, E. X. Wu, and X. Tang, “Deep learning initialized and gradient enhanced level-set based segmentation for liver tumor from ct images,” _IEEE Access_ , vol. 8, pp. 76 056–76 068, 2020.
  * [29] G. Chlebus, A. Schenk, J. Moltz, B. Ginneken, H. Meine, and H. Hahn, “Automatic liver tumor segmentation in ct with fully convolutional neural networks and object-based postprocessing,” _Scientific Reports_ , vol. 8, 10 2018.
  * [30] C.-T. Cheng and et al., “A flexible three-dimensional heterophase computed tomography hepatocellular carcinoma detection algorithm for generalizable and practical screening,” _Hepatology Communications_ , vol. 6, no. 10, pp. 2901–2913, 2022.
  * [31] A. L. Simpson and et al., “A large annotated medical image dataset for the development and evaluation of segmentation algorithms,” _ArXiv_ , vol. abs/1902.09063, 2019.
  * [32] J. Ma and et al., “Abdomenct-1k: Is abdominal organ segmentation a solved problem?” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, pp. 6695–6714, 2022.
  * [33] P. F. C. Patrick Bilic and et al., “The liver tumor segmentation benchmark (lits),” _ArXiv_ , vol. abs/1901.04056, 2019.
  * [34] O. Chapelle, B. Schölkopf, and A. Zien, Eds., _Semi-Supervised Learning_. The MIT Press, 2006.
  * [35] Y. Huo and et al., “Harvesting, detecting, and characterizing liver lesions from large-scale multi-phase ct data via deep dynamic texture learning,” _ArXiv_ , vol. abs/2006.15691, 2020.
  * [36] K. Yan and et al., “Learning from multiple datasets with heterogeneous and partial labels for universal lesion detection in ct,” _IEEE Transactions on Medical Imaging_ , vol. 40, pp. 2759–2770, 2021.
  * [37] X. Yang, Z. Song, I. King, and Z. Xu, “A survey on deep semi-supervised learning,” _IEEE Transactions on Knowledge and Data Engineering_ , pp. 1–20, 2022.
  * [38] M. P. Heinrich, M. Jenkinson, B. W. Papież, S. M. Brady, and J. A. Schnabel, “Towards realtime multimodal fusion for image-guided interventions using self-similarities,” in _MICCAI 2013_. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 187–194.
  * [39] M. Antonelli, A. Reinke, S. Bakas, K. Farahani, A. Kopp-Schneider, B. A. Landman, and et al., “The medical segmentation decathlon,” _Nature Communications_ , vol. 13, no. 1, Jul 2022. [Online]. Available: http://dx.doi.org/10.1038/s41467-022-30695-9
  * [40] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015_ , N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, Eds. Cham: Springer International Publishing, 2015, pp. 234–241.