# Kidney abnormality segmentation in thorax-abdomen CT scans

Gabriel Efrain Humpire Mamani1, Nikolas Lessmann1,
Ernst Th. Scholten1, Mathias Prokop1,
Colin Jacobs1, Bram van Ginneken1,2
1Diagnostic Image Analysis Group, Radboud University Medical Center,
Nijmegen, The Netherlands
2Fraunhofer MEVIS, Bremen, Germany
<EMAIL_ADDRESS>

###### Abstract

In this study, we introduce a deep learning approach for segmenting kidney
parenchyma and kidney abnormalities to support clinicians in identifying and
quantifying renal abnormalities such as cysts, lesions, masses, metastases,
and primary tumors. Our end-to-end segmentation method was trained on 215
contrast-enhanced thoracic-abdominal CT scans, with half of these scans
containing one or more abnormalities.

We began by implementing our own version of the original 3D U-Net network and
incorporated four additional components: an end-to-end multi-resolution
approach, a set of task-specific data augmentations, a modified loss function
using top-$k$, and spatial dropout. Furthermore, we devised a tailored post-
processing strategy. Ablation studies demonstrated that each of the four
modifications enhanced kidney abnormality segmentation performance, while
three out of four improved kidney parenchyma segmentation. Subsequently, we
trained the nnUNet framework on our dataset. By ensembling the optimized 3D
U-Net and the nnUNet with our specialized post-processing, we achieved
marginally superior results.

Our best-performing model attained Dice scores of 0.965 and 0.947 for
segmenting kidney parenchyma in two test sets (20 scans without abnormalities
and 30 with abnormalities), outperforming an independent human observer who
scored 0.944 and 0.925, respectively. In segmenting kidney abnormalities
within the 30 test scans containing them, the top-performing method achieved a
Dice score of 0.585, while an independent second human observer reached a
score of 0.664, suggesting potential for further improvement in computerized
methods.

All training data is available to the research community under a CC-BY 4.0
license on https://doi.org/10.5281/zenodo.8014289.

##  1 Introduction

Kidney cancer is a significant global health issue, ranking as the 12th most
deadly cancer in the world, with an estimated 14,700 deaths in 2019 and
approximately 73,820 new cases of kidney & renal pelvis cancer worldwide [30].
With the increasing number of cases, automated tools are needed to assist
clinicians in managing this burden. For instance, by following nephrometry
scoring systems [24], automatic kidney tumor segmentation methods may help
specialists to detect and get reliable measurements of kidney tumors.

Previous research on kidney segmentation has employed a variety of
conventional methods such as region growing [25, 22], active shape models
[27], active contours [26, 31], graph cut[1, 38], level-sets[34, 2],
snakes[9], random forest[21], and watersheds[36]. However, to the best of our
knowledge, there are only a few methods that focus on segmenting kidney tumors
or cysts in the literature. Linguraru et al. [26] proposed a semi-automatic
method that combines fast marching and active geodesic contours to segment
renal tumors. Kim and Park [22] used thresholds and histograms to segment the
kidneys and applied texture analysis to the kidney parenchyma to find seeds
for a region-growing algorithm to perform kidney tumor segmentation. Chen et
al. [6] proposed a method to predict kidney tumor growth in mm2/day, manually
segmenting the kidney tumors and using a reaction-diffusion model to predict
their growth. Kaur et al. [20] proposed an iterative segmentation method for
renal lesions, which uses spatial image details and distance regularization.

In recent years, Convolutional Neural Networks (CNN) have shown to be more
effective than traditional methods based on classical computer vision
techniques and machine learning. Their ability to learn directly from raw data
has led to their widespread use in segmenting organs and structures in
different modalities. For instance, Zheng et al. [40] used an AlexNet-based
method to localize the kidneys to define a seed for an active shape model
algorithm to segment the kidneys in patients with either abdominal surgery or
kidney tumors. Sharma et al. [29] used a network that takes the first 10
layers of the VGG-16 network and upsampled them in a decoder fashion to
segment the kidneys of patients with renal insufficiency. Encoder-decoder
networks such us 2D U-Net [28] and 3D U-Net [8] proved to be robust to tackle
medical segmentation tasks in multiple medical imaging segmentation challenges
[4, 18]. Variants of these models have been extensively proposed and applied
to a wide variety of tasks, including kidney segmentation. For instance, Taha
et al. [32] segmented the artery, vein, and ureter around the kidneys using a
2D U-Net-like network that allows the deeper layers to influence more to the
final prediction. Jackson et al. [19] used a 3D U-Net-like network to segment
the kidneys. Moreover, several methods used deep learning to segment kidney
tumors [39, 37]. Yu et al. [39] proposed Crossbar-Net, a network that segments
kidney and kidney tumors and uses horizontal and vertical patches instead of
traditional squared patches. The network is divided into sets of sub-networks;
a set consists of a sub-network for vertical and another for horizontal
patches. Yang et al. [37] proposed a 3D CNN using a pyramid pooling module to
segment the kidneys and kidney tumors in abdominal CT angiographic scans.

The top competitors of the Medical Decathlon [18] and LiTS challenge [7, 14]
have achieved the highest performance using cascaded networks. These networks
divide the tasks into sub-tasks, with one network per sub-task. These networks
have different fields of view and thus complement each other, resulting in
higher performance. For instance, a first network may segment the liver and
the liver tumor as a single structure, aiming to determine the region of
interest for the second network; the second network then aims to segment the
liver tumor class only. Similarly, Blau et al. [5] used cascade networks to
segment the kidney and kidney cyst in CT scans using a 2D U-Net. Their method
used heuristics such as a distance transform and HU thresholding to select
cyst candidates within the kidney region. A second (shallow) network
classified whether a candidate represented a kidney cyst. Additionally,
Haghighi et al. [13] used a localization network for pre-processing, which
cropped the input for 3D U-Net to segment MRI images of the kidneys. In a
recent challenge on segmentation of the kidney and kidney tumors on CT [17],
nnUNet [18] was the best performing method. This method automatically adapts
its hyperparameters based on a fingerprint of the data, resulting in optimal
performance. Furthermore, it uses 5-fold cross-validation to obtain the final
prediction.

In this study, we propose an automatic method for segmenting the kidney
parenchyma and kidney abnormalities in thorax-abdomen CT scans and compare it
with the nnUNet. We trained our method on 215 thorax-abdomen CT scans and
tested on additional 50 scans; the dataset consisted of scans from patients
undergoing oncological workup. The dataset contains patients at different
stages of disease and therefore abnormalities can be present in multiple body
regions.

Figure 1: Diagram of the CT scans selection criteria for this study, with
dataset A for training and datasets B30 and B20 for testing (with and without
kidney abnormalities respectively).

##  2 Materials and Methods

###  2.1 Patient Data

The dataset used in this study was collected from the Radboud University
Medical Center, Nijmegen, the Netherlands. We randomly retrieved 1905 studies
from 929 patients referred by the oncology department in a 12 month period.
These patients did not opt-out for use of their data for research, Protected
health information was removed from the DICOM data. This retrospective study
was approved by the medical-ethical review board of the hospital. CT scanners
from two manufacturers were used to acquire the CT scans: Toshiba (Aquilion
One) and Siemens (Sensation 16, Sensation 64, and Somatom Definition AS). The
reconstruction kernels were FC09, FC09-H, B30f, B30fs, and I30f. The slice
thickness ranged from 0.5 to 3 millimeters, 90% of them between 1 and 2 mm.
Severe abnormalities throughout the body are present in this dataset resulting
from disseminated disease, surgery, chemotherapy, radiotherapy, etc.

We selected a subset to perform our experiments; the procedure is summarized
in Figure 1. We analyzed the radiology reports per study to intentionally
select potential cases that contain kidney abnormalities such as cysts,
lesions, masses, metastases, and tumors. In Dutch: ((‘cyste’ OR ‘cysten’),
(‘laesie’ OR ‘lesies’), ‘massa’, (‘metastase’ OR ‘metastasen’), and ‘tumor’).
Our selection criteria selected studies where the radiology report mentioned
in the same sentence the kidneys (’nier’ OR ’nieren’ NO ’bijnier’) and any
kidney abnormalities. Furthermore, only one clinical study per patient was
selected to get a large variety of anatomies for the segmentation task. In
case multiple studies for the same patient were found, we selected the study
with the earliest acquisition date.

We employed a radiology report analysis to curate a dataset of 138 clinical
studies from 138 patients with kidney abnormalities, including cysts, lesions,
masses, metastases, or tumors. We excluded six patients with unusual anatomy,
three patients who had received kidney transplants, two patients with kidneys
of irregular size, and one patient with a horseshoe kidney. The inclusion and
exclusion criteria gave us 132 cases for analysis, which were then balanced
with additional 133 random patient studies without kidney abnormalities, for a
total of 265 CT scans from 265 patients. The patient cohort contains 56%
males; the average age was 60 years, and the age ranged from 22 to 84. We
divided this set into 215 CT scans for training (dataset A) and 50 for testing
(dataset B). The test set was further subdivided, with 60% (30/50) containing
abnormalities (dataset B30) and the remaining 40% (20/50) devoid of
abnormalities (dataset B20). The distribution of the five types of
abnormalities (tumors, cysts, masses, lesions, and metastases) was
proportional among the 30 cases in dataset B30 (six cases per abnormality),
which were randomly selected.

In the test set, two and six patients had undergone left and right
nephrectomy, respectively, while the training set included seventeen and
eighteen patients who had undergone left and right nephrectomy, respectively.

(a) Input CT scan

(b) Annotations format 1

(c) Annotations format 2

Figure 2: Example illustrating the different annotation formats. Each
subfigure shows the same axial section, with overlays depicting the
annotations: (a) shows the axial CT section. (b) shows the annotations in
format 1: parenchyma and kidney abnormalities as a single structure (yellow
overlay). (c) shows the annotations in format 2: parenchyma (yellow overlay)
and kidney abnormalities (red overlay) as different structures. All images
have a window center of 60 HU and a window width of 360 HU.

###  2.2 Annotation procedure

Four medical students manually segmented the kidney’s parenchyma and kidney
abnormalities. They were trained by an experienced radiologist (EthS) and
consulted the radiologist whenever needed throughout the annotation process.
Adhering to a standardized protocol, the medical students annotated the kidney
parenchyma as the region composed of the renal cortex, renal medulla, and
renal pyramid. The renal hilum, collecting system, and (major and minor)
calyces were excluded as much as possible from the kidney parenchyma
annotations. We grouped cysts, lesions, masses, metastases, and tumors
connected to the kidney parenchyma as kidney abnormalities. The protocol
excluded cases with abnormalities in the collecting system.

(a) Patients without kidney abnormalities.

(b) Patients with kidney abnormalities.

Figure 3: Four examples of CT scans from the training set (dataset A) showing
coronal sections with annotations in format 2 (see Figure 2(c)) where yellow
and red overlays represent annotations of the parenchyma and kidney
abnormalities, respectively. Note that all the patients have anomalies in the
body (green arrows in the body), and both cases of (b) have only one kidney
and contain kidney abnormalities. All the slices have a window center of 60 HU
and a window width of 360 HU.

Annotators used an in-house tool based on MeVisLab [15] to fully delineate the
contours of the structures in 2D orthogonal planes. Our tool was designed to
reduce the manual annotation time by interpolating unannotated contours
between two manually delineated contours. The kidney parenchyma of the
training set was annotated using an active learning process, with medical
students correcting the kidney parenchyma predictions made by a pre-trained 3D
U-Net (it used 50 CT scans from dataset A); the kidney abnormalities were
annotated from scratch. The test set was manually annotated (i.e. the contour
interpolation option of our tool was disabled) by two medical students. One of
these was considered as the reference standard and the other one as the second
observer  . The latter was the most experienced among the medical students and
was not allowed to consult the experienced radiologist during these
annotations. The annotations of the second observer  served as a benchmark for
human performance. The annotations were initially obtained in the axial plane,
followed by corrections in coronal and sagittal planes to keep the annotation
consistent in all orthogonal directions.

This study utilized two annotation formats, format 1 and format 2, to store
the annotations. Format 1 considers the kidney parenchyma and kidney
abnormalities as a single class (see Figure 2(b)) while format 2 separates
them into two classes (see Figure 2(c)).

Samples of CT scans from patients included in this study can be seen in Figure
3. While Figure 3(a) depicts patients without kidney abnormalities, it
highlights the presence of abnormalities in other parts of the body, such as
liver tumors. Figure 3(b) shows patients with kidney abnormalities, as well as
other abnormalities in the body, such as nephrectomy and collapsed lung.

###  2.3 Segmentation network

We present an end-to-end method for segmenting renal parenchyma and
abnormalities in CT scans. We depict our architecture in Figure 4. It consists
of two segmentation networks, a multi-resolution network for kidney
segmentation (annotations in format 1, one voxel represents
4$\times$4$\times$4mm) and a high-resolution network (annotations in format 2,
one voxel represents 1$\times$1$\times$1mm). The multi-resolution network is
designed to first provide a rough localization of the kidney by processing a
low-resolution version of the CT scan. This defines an ROI for the high-
resolution network to refine the segmentation of the kidneys and kidney
abnormalities.

####  2.3.1 Pre-processing

The CT scans and annotations were resampled to 1$\times$1$\times$1$mm$ (for
high-resolution segmentation using annotations in format 2) and
4$\times$4$\times$4$mm$ (for multi-resolution segmentation using annotations
in format 1) resolutions (see Figure 4(a)). Scans and annotations were
resampled using cubic and nearest-neighbor interpolation, respectively. We
clipped the Hounsfield Units to the range [-500,400].

####  2.3.2 Multi-resolution network

We present an end-to-end cascade method for parenchyma and kidney abnormality
segmentation. Unlike traditional cascade networks, which use two separate
networks and do not allow for backpropagation, our approach uses a single
network composed of two sub-networks. The first sub-network is a 3D U-Net with
16 filters that performs multi-resolution segmentation and defines an ROI.
This network takes 3D patches of 108$\times$108$\times$108 voxels, with each
voxel representing 4$\times$4$\times$4 mm, as input using annotations in
format 1 (kidney parenchyma + kidney abnormalities) and outputs
20$\times$20$\times$20 voxels. The output is then up-sampled 4 times and
padded with zeros to match and mask out the high-resolution input image in
millimeters (108$\times$108$\times$108 mm, one voxel represents
1$\times$1$\times$1 mm). The masked-out image serves as an additional input to
the second sub-network, the high-resolution segmentation network, which uses a
3D U-Net with 32 filters and serves to fine-segment the kidneys and kidney
abnormalities (see Figure 4(b)). Figures 4(a) and Figure 4(b) illustrate our
approach and the connection between the multi-resolution and the high-
resolution segmentation network, respectively.

(a) Diagram of the proposed multi-resolution network.

(b) Input of the high-resolution segmentation.

Figure 4: (a) Diagram of the proposed network. The multi-resolution
segmentation network uses a 3D U-Net network initialized with 16 filters. It
processes blocks of 108$\times$108$\times$108 voxels and outputs the central
20$\times$20$\times$20 voxels (represented by the dashed red square). One
voxel corresponds to a resolution of 4$\times$4$\times$4mm, giving the network
a receptive field of 88$\times$88$\times$88 voxels or
352$\times$352$\times$352mm. The kidney parenchyma and kidney abnormalities
are considered a single class in the multi-resolution network (see Figure
2(b)). The high-resolution segmentation network uses a 3D U-Net architecture
initialized with 32 filters, with each voxel representing
1$\times$1$\times$1mm. Its receptive field is 88$\times$88$\times$88 mm and it
segments the parenchyma and the kidney abnormalities as different classes (see
Figure 2(c)). (b) Shows how the multi-resolution and the high-resolution
networks are connected.

####  2.3.3 Data augmentation

Data augmentation was applied randomly to 70% of the training samples using
scaling, rotation, Gaussian blurring, image intensity variation, and elastic
deformation. Up to three of these data augmentation methods were applied
randomly to each training sample, to prevent too much data distortion. When
elastic deformation was used, it was only performed in conjunction with
Gaussian blurring and image intensity variation. Interpolation methods of
cubic and nearest neighbor were used for CT scans and reference standards,
respectively. The scaling factor ranged from 0.95 to 1.05, with rotations of
up to two planes of -5∘ to 5∘ degrees. Gaussian blurring had a sigma range of
0.2 to 1.0, and image intensity variation varied between -20 and 20 HU. We
performed elastic deformation by placing ten control points in a grid,
randomly perturbed by up to 5 voxels that were used as input to cubic B-spline
interpolation.

####  2.3.4 Spatial dropout

We applied spatial dropout [33], a regularization technique that is different
from traditional dropout. Spatial dropout drops feature maps instead of
individual neurons to enforce independence among feature maps, encouraging the
network to learn more robust and generalizable features. We randomly dropped
10% of the feature maps per layer.

####  2.3.5 Loss function

The loss function determines how the network’s weights are optimized after a
forward pass. In our experiments, we used a combination of weighted
categorical cross-entropy and dice loss in the experiments.

Combined loss $\displaystyle=$
$\displaystyle\alpha*diceLoss+\gamma*TopK(weightedCrossentropy)$ (1)

where $\alpha=0.3$ and $\gamma=0.7$ were used in all the experiments. Top-$k$
[3] sorts the voxel-wise loss in descending order and keeps the top $k\%$ to
compute the final mean loss; this approach emulates an online voxel-wise hard-
mining per sample.

####  2.3.6 Post-processing

The output of the networks was post-processed to eliminate false positives.
The end-user prediction was reconstructed by stitching together the
predictions. In all the networks, the output was thresholded at 0.5 to get a
binary prediction. The predictions of the multi-resolution network were up-
sampled four times and dilated five times to mask out the predictions of the
high-resolution segmentation network. Only the kidney abnormalities that were
connected to the kidney parenchyma were kept, to ensure that there were no
spurious kidney abnormality candidates outside the kidney region.

####  2.3.7 CNN Settings

Due to the large footprint of the network, scans were divided into 3D patches
to train the 3D network. Each training sample consisted of a patch of
108$\times$108$\times$108 voxels from the CT scan and a 20$\times$20$\times$20
voxel reference standard. During training, the reference standard patches were
sampled every ten voxels in all the orthogonal planes with up to 50% overlap
among surrounding patches. During inference, the cubes do not overlap. Patches
at the border of the CT scan were mirrored to match the input network size.

The Glorot uniform algorithm [12] was used to initialize the weights of the
network. The weight-map $w$ compensated for the high-class imbalance between
the classes. The background, parenchyma, and kidney abnormality classes had
empirically defined weights of 0.05, 0.10, and 0.99, respectively. We used
Adam [23] as optimization function with learning rate$=0.00001$,
$\beta_{1}=0.9$, and $\beta_{2}=0.999$. The training stopped when the
performance on the validation set stopped improving for ten epochs, and the
model with the highest average Dice score on the validation set was selected
as the optimal model.

####  2.3.8 Implementation of the CNN

The networks were implemented using Keras and TensorFlow as backend in Python
3.6. The segmentation experiments were executed on a cluster of computers
equipped with GTX1080 and GTX1080ti graphics cards, each with 256GB of CPU
RAM.

###  2.4 Evaluation

The end-user segmentation obtained by our networks was compared to the
reference masks using the Dice score.

Dice score $\displaystyle=$ $\displaystyle\frac{2*volume(X\cap
Y)}{volume(X)+volume(Y)}$ (2)

where X is the prediction, and the Y is the reference standard.

###  2.5 Ablation study

In this section, we conducted a step-by-step evaluation of the impact of each
module (multi-resolution, data augmentation, top-k, and spatial dropout) in
our proposed network. The backbone architecture for this ablation study was
the 3D U-Net [8]. Our experiments setup started with a 3D U-Net, and
additional modules were added one by one in subsequent experiments (see the
left side of Table 1). In order to evaluate the impact of each module on the
network performance, we conducted an ablation study by adding modules to the
3D U-Net backbone architecture one by one. The baseline network, referred to
as experiment 5  , only used the 3D U-Net initialized with 32 filters and had
a single input of 108$\times$108$\times$108 voxels with 1$\times$1$\times$1 mm
per voxel, producing 20$\times$20$\times$20 voxels. The subsequent experiments
added the multi-resolution module (experiment 4  ), data augmentation module
(experiment 3  ), top-$k$ module (experiment 2  ), and spatial dropout module
(experiment 1  ) to the network. The input and output sizes and formats were
consistent across all experiments except experiment 5  ; networks receive two
inputs of 108$\times$108$\times$108 voxels each, one input of
1$\times$1$\times$1 mm and one input of 4$\times$4$\times$4 mm per voxel for
high-resolution (input of 108$\times$108$\times$108 mm using annotation format
2) and multi-resolution segmentation (input of 432$\times$432$\times$432 mm
using annotation format 1), respectively. The difference in performance
between experiment 1  (experiment with spatial dropout) and experiment 2
(experiment without spatial dropout) showed the influence of the spatial
dropout module, for example. As an initial step, we first trained the multi-
resolution module independently to reach its optimal sub-model. Afterward, we
froze the weights of the multi-resolution sub-model, except for the last three
layers to allow back-propagation from the high-resolution segmentation
network. All the experiments used 80% of dataset A for training and 20% for
validation. Each experiment was trained independently to find the optimal
model. The best model from each experiment was evaluated using test sets B20
and B30.

###  2.6 nnUNet

We conducted experiments with nnUNet [18] to compare its performance with our
methods. Unlike our approach, nnUNet processes CT scans without any
preprocessing step, while we resample the CT scans to an isotropic resolution
and clip the HU range. To gain insight about the benefits of ensemble
networks, we ensembled nnUNet with our two highest-performing methods, one at
a time. As nnUNet only uses thresholding as postprocessing, we analyzed the
impact of our dedicated postprocessing method on performance. Note that our
postprocessing eliminates false-positive kidney abnormalities that are not
attached to the parenchyma.

(a)

(b)

(c)

(d)

Figure 5: Performance comparison of our methods and the second observer  on
datasets B20 and B30 using boxplots. The red and black lines represent the
median and the mean, respectively. Boxplot (a) shows results for class
parenchyma only on the dataset B20 (twenty cases without abnormalities).
Boxplot (b) shows results for class parenchyma only on the dataset B30 (thirty
cases with abnormalities). Boxplot (c) displays the results for class
parenchyma plus abnormalities as a single structure on dataset B30 (thirty
test cases with abnormalities). Boxplot (d) shows results for Class
abnormalities only on the dataset B30 (thirty cases with abnormalities). Note
that the scale in the y-axis is different for boxplot (d). The modules for
each experiment are represented by the same color coding as in Table 1:
experiment 1  , experiment 2  , experiment 3  , experiment 4  , experiment 5
, and second observer  .

Table 1: Summary of performance of our methods and the second observer  on the test sets (dataset B20 and B30) using Dice score. The upper part of the table shows Mean and SD, while the bottom part shows 95% Confidence Intervals. We analyzed the performance per class (parenchyma and kidney abnormalities) and when both classes are combined (parenchyma + kidney abnormalities). An asterisk (∗) indicates that a value is significantly better or worse than the value of the reference system (experiment 1  ). The best Dice score values per column are in bold. Experiment | Test set: dataset B20 | Test set: dataset B30
---|---|---
Parenchyma class | Parenchyma class | Parenchyma + abnormalities class | Abnormalities class
(Fig. 5(a)) |  (Fig. 5(b)) |  (Fig. 5(c)) |  (Fig. 5(d))
Mean $\pm$ SD
Exp. 1  |  0.952$\pm$0.008 |  0.930$\pm$0.053 |  0.948$\pm$0.011 |  0.487$\pm$0.314
Exp. 2  |  ∗0.957$\pm$0.006 |  ∗0.938$\pm$0.051 |  ∗0.942$\pm$0.049 |  0.464$\pm$0.320
Exp. 3  |  0.954$\pm$0.007 |  0.930$\pm$0.064 |  0.950$\pm$0.010 |  0.440$\pm$0.310
Exp. 4  |  ∗0.956$\pm$0.007 |  0.936$\pm$0.058 |  ∗0.952$\pm$0.017 |  0.434$\pm$0.311
Exp. 5  |  0.952$\pm$0.015 |  0.931$\pm$0.057 |  0.924$\pm$0.065 |  0.390$\pm$0.315
Second observer  |  ∗0.944$\pm$0.009 |  ∗0.925$\pm$0.051 |  ∗0.941$\pm$0.017 | 0.664$\pm$0.274
nnUNet |  ∗0.960$\pm$0.015 |  ∗0.940$\pm$0.069 |  ∗0.931$\pm$0.099 |  0.521$\pm$0.303
Ens. nnUNet Exp. 1  |  ∗0.962$\pm$0.007 |  ∗0.941$\pm$0.058 |  ∗0.927$\pm$0.125 |  0.526$\pm$0.305
Ens. nnUNet Exp. 2  |  ∗0.964$\pm$0.006 |  ∗0.944$\pm$0.055 |  ∗0.928$\pm$0.129 |  0.507$\pm$0.318
nnUNet PP |  ∗0.964$\pm$0.006 |  ∗0.941$\pm$0.068 |  ∗0.955$\pm$0.042 |  0.576$\pm$0.290
Ens. nnUNet Exp. 1  PP |  ∗0.962$\pm$0.007 |  ∗0.942$\pm$0.058 |  ∗0.960$\pm$0.009 |  0.585$\pm$0.293
Ens. nnUNet Exp. 2  PP |  ∗0.965$\pm$0.006 |  ∗0.947$\pm$0.050 |  ∗0.958$\pm$0.032 |  0.566$\pm$0.309
Confidence Intervals 95%
Exp. 1  | 0.948-0.955 | 0.910-0.950 | 0.945-0.952 | 0.369-0.604
Exp. 2  | 0.954-0.960 | 0.920-0.957 | 0.924-0.961 | 0.345-0.584
Exp. 3  | 0.950-0.957 | 0.906-0.954 | 0.946-0.954 | 0.324-0.555
Exp. 4  | 0.953-0.959 | 0.914-0.957 | 0.945-0.958 | 0.318-0.550
Exp. 5  | 0.945-0.959 | 0.910-0.952 | 0.900-0.948 | 0.273-0.508
2nd observer | 0.939-0.948 | 0.906-0.944 | 0.935-0.947 | 0.561-0.766
nnUNet | 0.953-0.965 | 0.913-0.961 | 0.892-0.959 | 0.411-0.625
Ens. nnUNet Exp. 1  | 0.959-0.965 | 0.918-0.958 | 0.876-0.959 | 0.416-0.627
Ens. nnUNet Exp. 2  | 0.961-0.967 | 0.923-0.961 | 0.876-0.961 | 0.397-0.613
nnUNet PP | 0.961-0.967 | 0.915-0.962 | 0.938-0.965 | 0.469-0.673
Ens. nnUNet Exp. 1  PP | 0.959-0.966 | 0.918-0.959 | 0.957-0.963 | 0.482-0.687
Ens. nnUNet Exp. 2  PP | 0.962-0.967 | 0.926-0.963 | 0.945-0.965 | 0.454-0.672

##  3 Results

The results of the ablation study conducted on the test sets (dataset B20 and
B30) are shown in Figure 5. These results are also summarized in Table 1,
which includes asterisks (*) to indicate statistical significance (P-value
$<0.05$) between experiment 1  and other experiments, as determined by a two-
tailed Mann-Whitney U test. We evaluated the predictions of each experiment
per class to show more insights into the results of our experiments.
Furthermore, we combined the prediction of both classes (annotation format 2)
as a single structure (annotation format 1) and computed its Dice score; this
helps to make our results comparable to methods that reported kidney dice
only.

Dataset B30: The presence of kidney abnormalities characterizes the patients
in this dataset (see Figure 3(b)). The results of our experiments on dataset
B30 are displayed in Figures 5(d), 5(b), and 5(c). First, we evaluated the
performance of the methods in segmenting the kidney abnormalities class only.
The results are shown in Figure 5(d) and in the column “Dataset
B30/Abnormalities class” of Table 1. The second observer  and experiment 1
achieved the two highest scores, 0.664$\pm$0.274 and 0.487$\pm$0.314,
respectively. Experiment 5  obtained 0.390$\pm$0.315 Dice, the lowest score
when segmenting the kidney abnormalities only. Next, we evaluated the
performance of the methods in segmenting the parenchyma class only. The
results are shown in Figure 5(b) and in the column “Dataset B30/Parenchyma
class” of Table 1. The two highest scores were obtained by Experiment 2  and
experiment 4  with 0.938$\pm$0.051, 0.936$\pm$0.058, respectively, while the
second observer  obtained the lowest score with 0.925$\pm$0.051. Finally, we
evaluated the performance of the methods when segmenting both the parenchyma
and the kidney abnormalities class as a single structure (annotation format
1). The results are shown in Figure 5(c) and in column “Dataset B30/Parenchyma
+ abnormalities class” of Table 1. The two highest scores were achieved by
Experiment 4  and experiment 3  with Dice scores 0.952$\pm$0.017 and
0.950$\pm$0.010, respectively. Experiment 5  obtained the lowest Dice score
with 0.924$\pm$0.065.

Dataset B20: The patients in this dataset do not present kidney abnormalities,
but it is probable that they have other anomalies in the body (see Figure
3(a)). The results on the test set B20 are depicted in Figure 5(a) and in
Table 1 under the column “Dataset B20/Parenchyma class”. Experiment 2  and
experiment 4  obtained the highest Dice scores, 0.957$\pm$0.006 and
0.956$\pm$0.007, respectively. The second observer  obtained the lowest Dice
score with 0.944$\pm$0.009.

nnUNet: In our experiments, nnUNet obtained slightly better results in the
parenchyma class of datasets B20 and B30 compared to our experiments, a Dice
score of 0.521 $\pm$ 0.303 in the kidney abnormality class, which was higher
by +0.034 Dice than our experiment 1  . To further analyze the differences
between nnUNet and our experiments, we ensembled the predictions of nnUNet
with either experiment 1  or experiment 2  by averaging their probabilities.
The ensemble nnUNet with experiment 2  slightly improved the results of nnUNet
in the parenchyma class of both datasets but decreased in -0.014 Dice score in
the abnormality class, while the ensemble nnUNet with experiment 1  slightly
improved in +0.004 dice score compared to nnUNet in the abnormality class. The
ensemble nnUNet with experiment 2  performed slightly better than the ensemble
nnUNet with experiment 1  in all classes, except the abnormality class, where
the ensemble with experiment 1  had a Dice score of 0.526 $\pm$ 0.306, and the
ensemble with experiment 2  obtained 0.507 $\pm$ 0.318. Since nnUNet only uses
thresholding for post-processing, we applied our dedicated post-processing to
the nnUNet predictions to remove kidney abnormalities that are not attached to
the kidney, which resulted in notable improvements of +0.055, +0.059, and
+0.059 for nnUNet, ensemble nnUNet with experiment 1  , and ensemble nnUNet
with experiment 2  , respectively. As a result, the ensemble nnUNet with
experiment 1  and our dedicated post-processing was the highest-performing
experiment in the abnormality class, with a Dice score of 0.585 $\pm$ 0.293.

Table 2 compares our results with other methods published in the literature.
Some of the methods report the Dice scores for the left and right kidneys
separately, while others report a single score for both kidneys combined. To
make our results comparable to these methods, we post-processed our
predictions to obtain the Dice scores for both the left and right kidneys.

Table 2: Performance comparison between our methods (experiment 1  and experiment 2  ) and previous work of kidney segmentation and kidney tumor/abnormality segmentation using the mean Dice score as the metric. The methods listed below the line reported the presence of cases with kidney tumors/abnormalities in their datasets. The values marked with ${\dagger}$ are obtained after post-processing the prediction masks from ‘Both kidneys’ column for comparability with other methods. Method | Kidney abnor-malities | Kidneys | Num. test scans | Deep Learning | Description
---|---|---|---|---|---
Left kidney | Right kidney | Both kidneys
Methods that reported cases without kidney abnormalities or did not report
them
Jackson et al. [19] | N/A | 0.860 | 0.910 | - | 24 | Yes |
Gibson et al. [10, 11] | N/A | 0.930 | - | - | 10 | Yes |  9-folds cross-validation.
Badura and Wieclawek [2] | N/A | 0.938 | 0.944 | - | 20 | No |
Heinrich et al. [16] | N/A | 0.942 | - | - | 10 | Yes |  4-fold cross validation.
Wang et al. [35] | N/A | 0.956 | 0.954 | - | 30 | Yes |  4-fold cross-validation.
Khalifa et al. [21] | N/A | - | - | 0.973 | 60 | No |  Leave-one-out.
Exp. 1  | N/A |  0.953† |  0.951† | 0.952 | 20 | Yes |  Test set: Dataset B20, train set: Dataset A.
Exp. 2  | N/A |  0.960† | 0.955† | 0.957 | 20 | Yes |  Test set: Dataset B20, train set: Dataset A.
Second observer  | N/A |  0.943† |  0.945† | 0.944 | 20 | N/A |  Test set: Dataset B20, train set: N/A.
nnUNet | N/A |  0.956† |  0.959† | 0.960 | 20 | Yes |  Test set: Dataset B20, train set: Dataset A.
Ens. nnUNet Exp. 1  PP | N/A |  0.962† |  0.944† | 0.962 | 20 | Yes |  Test set: Dataset B20, train set: Dataset A.
Ens. nnUNet Exp. 2  PP | N/A | 0.964† |  0.946† | 0.965 | 20 | Yes |  Test set: Dataset B20, train set: Dataset A.
Methods that reported cases with kidney abnormalities
Turco et al. [34] | - | - | - | 0.800 | 55 | No | Polycystic kidneys only.
Sharma et al. [29] | - | - | - | 0.860 | 81 | Yes | Polycystic kidneys.
Skalski et al. [31] | - | - | - | 0.862 | 10 | No | Kidney cancer.
Blau et al. [5] | - | 0.870 | 0.870 | - | 46 | Yes | Cysts.
Lin et al. [25] | - | 0.873 | 0.886 | - | 30 | No | Two cases with tumor, one with a cyst.
Zheng et al. [40] | - | 0.890 | 0.920 | - | 78 | Yes | Kidney tumors.
Wieclawek [36] | - | - | - | 0.917 | 170 | No | Cysts and kidney tumors.
Yang et al. [37] | 0.802 | - | - | 0.931 | 50 | Yes | Kidney tumors.
Yu et al. [39] | 0.913 | - | - | - | 36 | Yes | Kidney tumors.
Exp. 1  | 0.488 |  0.949† |  0.951† | 0.948 | 30 | Yes |  Test set: Dataset B30, train set: Dataset A.
Exp. 2  | 0.464 |  0.956† |  0.936† | 0.942 | 30 | Yes |  Test set: Dataset B30, train set: Dataset A.
Second observer  | 0.664 |  0.939† |  0.943† | 0.941 | 30 | N/A |  Test set: Dataset B30, train set: N/A.
nnUNet | 0.521 |  0.928† |  0.951† | 0.931 | 30 | Yes |  Test set: Dataset B30, train set: Dataset A.
Ens. nnUNet Exp. 1  PP | 0.585 |  0.960† | 0.960† | 0.960 | 30 | Yes |  Test set: Dataset B30, train set: Dataset A.
Ens. nnUNet Exp. 2  PP | 0.566 | 0.963† |  0.955† | 0.958 | 30 | Yes |  Test set: Dataset B30, train set: Dataset A.

##  4 Discussion

In this paper, we presented an automatic method for the segmentation of the
(kidney) parenchyma and kidney abnormalities. We conducted experiments in an
ablation study fashion to evaluate the contribution of each module to the
performance (see Section 2.5). For instance, the comparison between experiment
5  and experiment 4  in Figure 5 shows the influence of the multi-resolution
module. Figure 5(a) shows that all of our experiments outperformed the second
observer  when segmenting the kidney parenchyma in dataset B20 (patients
without kidney abnormalities). While the presence of kidney abnormalities
affected the performance of kidney (parenchyma + abnormalities) segmentation;
see the difference of outliers between Figure 5(a) (dataset B20) and Figure
5(c) (dataset B30: patients with kidney abnormalities). One of the reasons for
this behavior may be the difficulty in defining the boundary between the
parenchyma and the kidney abnormality. When comparing the boxplots, the
interquartile range of Experiment 5  and experiment 2  obtained the largest
and the smallest interquartile range, respectively, indicating that the
combination of multi-resolution, data augmentation, and top-$k$ modules
positively impacted the segmentation of the kidneys (parenchyma +
abnormalities). Note the spatial dropout module (difference between experiment
1  and experiment 2  ) was beneficial only to the kidney abnormality class
(see Figure 5). Furthermore, Figure 5(d) shows that the mean Dice score (black
dashed line in boxplots) of our experiments gradually increases when adding
more modules (experiment 5  to experiment 1  ) when segmenting the kidney
abnormality class. This highlights the positive impact of each module in this
ablation study on the segmentation of kidney abnormalities.

Additionally, we trained nnUNet, a state-of-the-art segmentation method, on
our data and obtained results that were consistent with our previous
experiments, except for the kidney abnormality class where nnUNet achieved a
0.521 Dice score compared to 0.488 obtained by experiment 1  . To explore
further improvements, we combined nnUNet predictions with our best-performing
experiments, resulting in an ensemble nnUNet + experiment 1  that achieved
0.526 Dice score for the kidney abnormality class. Since nnUNet uses only
thresholding as postprocessing, we investigated whether postprocessing nnUNet
predictions with our dedicated postprocessing could result in better
performance. This additional postprocessing yielded a 0.585 Dice score, an
improvement of +0.064 compared to the original nnUNet with 0.521 Dice score.
While nnUNet is a state-of-the-art segmentation method, our dedicated
postprocessing method contributed to further improvement in discarding false
positive regions.

We note that the performance of the second observer  is substantially better
than any of our experiments when segmenting only the kidney abnormalities,
with an average 0.664 Dice score. Figure 5(d) shows four outliers for the
second observer  , three of these cases obtained a Dice score of zero and one
case 0.207. The volume of these four outliers is 29, 197, 282, and 5769 mm3,
three of them are below the median kidney abnormality volume in dataset B30
(1421 mm3). This demonstrates the difficulty of kidney abnormality
segmentation, even for experienced radiologists. The fact that we annotated
multiple classes of kidney abnormalities (e.g. tumors, cysts, lesions, and
masses) as a single class and the diverse patient anatomy in patients with
kidney abnormalities may have contributed to the gap in performance.

Case 1: 0.346 Dice score

Case 2: 0.266 Dice score

Case 3: 0.777 Dice score

(a) Input slice

(b) Pred. of experiment 1

(c) Contours

Figure 7: Comparison of three cases on the test set B30 between experiment 1
, the reference standard, and the second observer. (a) shows the original
slice. (b) shows the heatmaps (predictions prior to post-processing, using a
color table mapping [0,1] from transparent to green to red) of experiment 1  .
(c) shows the final predictions (red contours) of experiment 1  , the
reference standard (green contours), and the second human observer (yellow
contours). The window center and window width used for all slices were 60 HU
and 360 HU.

Table 2 compares the Dice score obtained by previous work and our methods; the
middle line separates methods that segmented kidneys without abnormalities and
kidneys with abnormalities. While some methods reported Dice score for both
kidneys as a single score as reported in this paper, others reported Dice
scores for the left and right kidneys separately; then, we postprocessed our
predictions to the same format and have a better comparison. Most of the
methods trained without kidney abnormalities achieved higher Dice scores in
the kidney parenchyma than those trained with kidney abnormalities (below the
middle line) due to the more complex task. Although the performance of
experiment 1  for kidney abnormality segmentation was the lowest (0.487) among
the previous work, the performance of the second observer  (0.664) was also
below the previous work where Yu et al. [39] obtained 0.913 and Yang et al.
[37] 0.802 Dice score. This disparity could be due to the fact that we grouped
different types of kidney abnormalities including cysts, lesions, masses,
metastases, and tumors into a single class while Yu et al. [39] and Yang et
al. [37] discarded other abnormalities different than kidney tumors. Our set
of kidney abnormalities is diverse in terms of volume, texture, image
intensity, and location in the kidney, which makes network learning difficult.

Segmenting kidney abnormalities is challenging due to the similarity between
tumors in the collecting system and kidney cysts. For instance, Figure 7 shows
three cases from dataset B30 where our method returned some false positives
due to the similarity with tumors in the collecting system. Each case shows
the kidney abnormality predictions of experiment 1  prior to post-processing
in the second row as heatmaps. While the third row shows the post-processed
segmentation, reference standard, and second observer as red, green, and
yellow contours, respectively. In all three cases, a false positive by our
method is present, indicated by an isolated red contour. In case 1, the false
positives are abnormalities in the collecting system, which have a similar
image intensity as the cysts, similarly, the second observer also segmented
one of these abnormalities in the middle region. In case 2, the false positive
appears as a small cyst-like region, while in case 3, it resembles an
irregular region in the kidney. Figure 9 shows a comparison of the final
prediction in annotation format 1 of experiment 1  , the reference standard,
and the second observer represented as red, green, and yellow contours,
respectively. This figure shows the best and median cases of datasets B20 and
B30 and the Dice score of each case computed between experiment 1  and the
reference standard.

A limitation of our study is that we excluded patients with unusual anatomy
and with abnormalities in the collecting system.

Case 1 (0.971 Dice) best from dataset $B_{20}$

Case 2 (0.950 Dice) median from dataset $B_{20}$

Case 3 (0.966 Dice) best from dataset $B_{30}$

Case 4 (0.953 Dice) median from dataset $B_{30}$

(a) Input slice

(b) Contours

Figure 9: Comparison of four cases between experiment 1  , the reference
standard, and the second observer on the test set B30 in annotation format 1.
(a) shows the original slice and (b) shows the final predictions (red
contours) of experiment 1  , the reference standard (green contours), and the
second human observer (yellow contours). All the slices have a window center
of 60 HU and a window width of 360 HU.

##  5 Conclusions

In conclusion, our ablation study and nnUNet showed that segmenting kidney
abnormalities in challenging scenarios is possible, and improved performance
can be achieved by an ensemble of different methods and dedicated
postprocessing. The results show that our method has the potential to be a
valuable tool for clinicians in detecting and monitoring kidney abnormalities.
An ablation study was conducted to better understand the impact of the
different modules of our method on its performance. Further research is needed
to optimize the performance of experiment 1  and nnUNet to test their ability
to generalize to other datasets. Overall, our work contributes to the ongoing
efforts to develop accurate and reliable computer-aided diagnosis systems for
detecting and quantifying renal abnormalities.

## References

  * Ali et al. [2007] A. M. Ali, A. A. Farag, and A. S. El-Baz.  Graph cuts framework for kidney segmentation with prior shape constraints.  In _Medical Image Computing and Computer-Assisted Intervention_ , pages 384–392. Springer, 2007.
  * Badura and Wieclawek [2016] P. Badura and W. Wieclawek.  Calibrating level set approach by granular computing in computed tomography abdominal organs segmentation.  _Applied Soft Computing_ , 49:887–900, 2016.
  * Berrada et al. [2018] L. Berrada, A. Zisserman, and M. P. Kumar.  Smooth loss functions for deep top-k classification.  _arXiv preprint arXiv:1802.07595_ , 2018.
  * Bilic et al. [2023] P. Bilic, P. Christ, H. B. Li, E. Vorontsov, A. Ben-Cohen, G. Kaissis, A. Szeskin, C. Jacobs, G. E. Humpire Mamani, G. Chartrand, F. Lohöfer, J. W. Holch, W. Sommer, F. Hofmann, A. Hostettler, N. Lev-Cohain, M. Drozdzal, M. M. Amitai, R. Vivanti, J. Sosna, I. Ezhov, A. Sekuboyina, F. Navarro, F. Kofler, J. C. Paetzold, S. Shit, X. Hu, J. Lipková, M. Rempfler, M. Piraud, J. Kirschke, B. Wiestler, Z. Zhang, C. Hülsemeyer, M. Beetz, F. Ettlinger, M. Antonelli, W. Bae, M. Bellver, L. Bi, H. Chen, G. Chlebus, E. B. Dam, Q. Dou, C.-W. Fu, B. Georgescu, X. G. i Nieto, F. Gruen, X. Han, P.-A. Heng, J. Hesser, J. H. Moltz, C. Igel, F. Isensee, P. Jäger, F. Jia, K. C. Kaluva, M. Khened, I. Kim, J.-H. Kim, S. Kim, S. Kohl, T. Konopczynski, A. Kori, G. Krishnamurthi, F. Li, H. Li, J. Li, X. Li, J. Lowengrub, J. Ma, K. Maier-Hein, K.-K. Maninis, H. Meine, D. Merhof, A. Pai, M. Perslev, J. Petersen, J. Pont-Tuset, J. Qi, X. Qi, O. Rippel, K. Roth, I. Sarasua, A. Schenk, Z. Shen, J. Torres, C. Wachinger, C. Wang, L. Weninger, J. Wu, D. Xu, X. Yang, S. C.-H. Yu, Y. Yuan, M. Yue, L. Zhang, J. Cardoso, S. Bakas, R. Braren, V. Heinemann, C. Pal, A. Tang, S. Kadoury, L. Soler, B. van Ginneken, H. Greenspan, L. Joskowicz, and B. Menze.  The Liver Tumor Segmentation Benchmark (LiTS).  _Medical Image Analysis_ , 84:102680, 2023.  ISSN 1361-8415.  doi: 10.1016/j.media.2022.102680.
  * Blau et al. [2018] N. Blau, E. Klang, N. Kiryati, M. Amitai, O. Portnoy, and A. Mayer.  Fully automatic detection of renal cysts in abdominal CT scans.  _International Journal of Computer Assisted Radiology and Surgery_ , 13(7):957–966, 2018.
  * Chen et al. [2010] X. Chen, R. Summers, and J. Yao.  FEM based 3D tumor growth prediction for kidney tumor.  In _International Workshop on Medical Imaging and Virtual Reality_ , pages 159–168. Springer, 2010.
  * Chlebus et al. [2018] G. Chlebus, A. Schenk, J. H. Moltz, B. van Ginneken, H. K. Hahn, and H. Meine.  Automatic liver tumor segmentation in CT with fully convolutional neural networks and object-based postprocessing.  _Nature Scientific Reports_ , 8:15497, Oct. 2018.  doi: 10.1038/s41598-018-33860-7.
  * Çiçek et al. [2016] Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger.  3D U-Net: Learning dense volumetric segmentation from sparse annotation.  In S. Ourselin, L. Joskowicz, M. R. Sabuncu, G. Unal, and W. Wells, editors, _Medical Image Computing and Computer-Assisted Intervention_ , pages 424–432, Cham, 2016. Springer International Publishing.  doi: 10.1007/978-3-319-46723-8_49.  URL https://doi.org/10.1007/978-3-319-46723-8_49.
  * Farmaki et al. [2010] C. Farmaki, K. Marias, V. Sakkalis, and N. Graf.  Spatially adaptive active contours: a semi-automatic tumor segmentation framework.  _International Journal of Computer Assisted Radiology and Surgery_ , 5(4):369–384, 2010.
  * Gibson et al. [2018a] E. Gibson, F. Giganti, Y. Hu, E. Bonmati, S. Bandula, K. Gurusamy, B. Davidson, S. P. Pereira, M. J. Clarkson, and D. C. Barratt.  Automatic multi-organ segmentation on abdominal CT with dense V-Networks.  _IEEE Transactions on Medical Imaging_ , 37(8):1822–1834, aug 2018a.  doi: 10.1109/tmi.2018.2806309.
  * Gibson et al. [2018b] E. Gibson, W. Li, C. Sudre, L. Fidon, D. I. Shakir, G. Wang, Z. Eaton-Rosen, R. Gray, T. Doel, Y. Hu, et al.  NiftyNet: a deep-learning platform for medical imaging.  _Computer Methods and Programs in Biomedicine_ , 158:113–122, 2018b.
  * Glorot and Bengio [2010] X. Glorot and Y. Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In _International conference on artificial intelligence and statistics_ , pages 249–256, 2010.
  * Haghighi et al. [2018] M. Haghighi, S. K. Warfield, and S. Kurugol.  Automatic renal segmentation in DCE-MRI using convolutional neural networks.  In _IEEE International Symposium on Biomedical Imaging_ , pages 1534–1537, Apr. 2018.  doi: 10.1109/ISBI.2018.8363865.
  * Han [2017] X. Han.  Automatic liver lesion segmentation using a deep convolutional neural network method.  _arXiv:1704.07239_ , 2017.  doi: 10.1002/mp.12155.
  * Heckel et al. [2009] F. Heckel, M. Schwier, and H.-O. Peitgen.  Object-oriented application development with MeVisLab and Python.  _Lecture Notes in Informatics (Informatik 2009: Im Focus das Leben)_ , 154:1338–1351, 2009.
  * Heinrich et al. [2019] M. P. Heinrich, O. Oktay, and N. Bouteldja.  OBELISK-Net: Fewer layers to solve 3D multi-organ segmentation with sparse deformable convolutions.  _Medical Image Analysis_ , 54:1–9, May 2019.  ISSN 1361-8423.  doi: 10.1016/j.media.2019.02.006.
  * Heller et al. [2021] N. Heller, F. Isensee, K. H. Maier-Hein, X. Hou, C. Xie, F. Li, Y. Nan, G. Mu, Z. Lin, M. Han, G. Yao, Y. Gao, Y. Zhang, Y. Wang, F. Hou, J. Yang, G. Xiong, J. Tian, C. Zhong, J. Ma, J. Rickman, J. Dean, B. Stai, R. Tejpaul, M. Oestreich, P. Blake, H. Kaluzniak, S. Raza, J. Rosenberg, K. Moore, E. Walczak, Z. Rengel, Z. Edgerton, R. Vasdev, M. Peterson, S. McSweeney, S. Peterson, A. Kalapara, N. Sathianathen, N. Papanikolopoulos, and C. Weight.  The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 challenge.  _Medical Image Analysis_ , 67:101821, 2021.
  * Isensee et al. [2020] F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier-Hein.  nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation.  _Nature methods_ , 2020.  ISSN 1548-7091.  doi: 10.1038/s41592-020-01008-z.
  * Jackson et al. [2018] P. Jackson, N. Hardcastle, N. Dawe, T. Kron, M. Hofman, and R. J. Hicks.  Deep learning renal segmentation for fully automated radiation dose estimation in unsealed source therapy.  _Frontiers in oncology_ , 8:215, 2018.
  * Kaur et al. [2019] R. Kaur, M. Juneja, and A. Mandal.  A hybrid edge-based technique for segmentation of renal lesions in CT images.  _Multimedia Tools and Applications_ , 78(10):12917–12937, 2019.
  * Khalifa et al. [2017] F. Khalifa, A. Soliman, A. Elmaghraby, G. Gimelfarb, and A. El-Baz.  3D kidney segmentation from abdominal images using spatial-appearance models.  _Computational and mathematical methods in medicine_ , 2017, 2017\.
  * Kim and Park [2004] D.-Y. Kim and J.-W. Park.  Computer-aided detection of kidney tumor on abdominal computed tomography scans.  _Acta Radiologica_ , 45(7):791–795, 2004.
  * Kingma and Ba [2014] D. Kingma and J. Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kutikov and Uzzo [2009] A. Kutikov and R. G. Uzzo.  The RENAL nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth.  _Journal of Urology_ , 182(3):844–853, 2009.
  * Lin et al. [2006] D.-T. Lin, C.-C. Lei, and S.-W. Hung.  Computer-aided kidney segmentation on abdominal CT images.  _IEEE Transactions on Information Technology in Biomedicine_ , 10(1):59–65, 2006.
  * Linguraru et al. [2009] M. G. Linguraru, J. Yao, R. Gautam, J. Peterson, Z. Li, W. M. Linehan, and R. M. Summers.  Renal tumor quantification and classification in contrast-enhanced abdominal CT.  _Pattern Recognition_ , 42(6):1149–1161, 2009\.
  * Lu et al. [2007] J. Lu, J. Chen, J. Zhang, and W. Yang.  Segmentation of kidney using CV model and anatomy priors.  In _Medical Imaging, Parallel Processing of Images, and Optimization Techniques_ , volume 6789 of _Proceedings of the SPIE_ , page 678911, 2007.
  * Ronneberger et al. [2015] O. Ronneberger, P. Fischer, and T. Brox.  U-Net: Convolutional networks for biomedical image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention_ , volume 9351 of _Lecture Notes in Computer Science_ , pages 234–241, 2015\.
  * Sharma et al. [2017] K. Sharma, C. Rupprecht, A. Caroli, M. C. Aparicio, A. Remuzzi, M. Baust, and N. Navab.  Automatic segmentation of kidneys using deep learning for total kidney volume quantification in autosomal dominant polycystic kidney disease.  _Nature Scientific Reports_ , 7(1):2049, 2017\.  doi: 10.1038/s41598-017-01779-0.
  * Siegel et al. [2019] R. L. Siegel, K. D. Miller, and A. Jemal.  Cancer statistics, 2019.  _CA: A Cancer Journal for Clinicians_ , 69(1):7–34, 2019.
  * Skalski et al. [2017] A. Skalski, K. Heryan, J. Jakubowski, and T. Drewniak.  Kidney segmentation in CT data using hybrid Level-Set method with ellipsoidal shape constraints.  _Metrology and Measurement Systems_ , 24(1):101–112, 2017.  doi: 10.1515/mms-2017-0006.
  * Taha et al. [2018] A. Taha, P. Lo, J. Li, and T. Zhao.  Kid-Net: convolution networks for kidney vessels segmentation from CT-volumes.  In _Medical Image Computing and Computer-Assisted Intervention_ , pages 463–471. Springer, 2018.
  * Tompson et al. [2015] J. Tompson, R. Goroshin, A. Jain, Y. LeCun, and C. Bregler.  Efficient object localization using convolutional networks.  In _Computer Vision and Pattern Recognition_ , pages 648–656, 2015\.
  * Turco et al. [2018] D. Turco, M. Valinoti, E. M. Martin, C. Tagliaferri, F. Scolari, and C. Corsi.  Fully automated segmentation of polycystic kidneys from noncontrast computed tomography: A feasibility study and preliminary results.  _Academic Radiology_ , 25(7):850–855, 2018.
  * Wang et al. [2018] Y. Wang, Y. Zhou, P. Tang, W. Shen, E. K. Fishman, and A. L. Yuille.  Training multi-organ segmentation networks with sample selection by relaxed upper confident bound.  In _Medical Image Computing and Computer-Assisted Intervention_ , pages 434–442. Springer, 2018.
  * Wieclawek [2018] W. Wieclawek.  3D marker-controlled watershed for kidney segmentation in clinical CT exams.  _Biomedical Engineering Online_ , 17(1):26, 2018\.
  * Yang et al. [2018] G. Yang, G. Li, T. Pan, Y. Kong, J. Wu, H. Shu, L. Luo, J. Dillenseger, J. Coatrieux, L. Tang, and X. Zhu.  Automatic segmentation of kidney and renal tumor in CT images based on 3D fully convolutional neural network with pyramid pooling module.  In _International Conference on Pattern Recognition_ , pages 3790–3795, Aug. 2018.  doi: 10.1109/ICPR.2018.8545143.
  * Yoruk et al. [2018] U. Yoruk, B. A. Hargreaves, and S. S. Vasanawala.  Automatic renal segmentation for MR urography using 3D-GrabCut and random forests.  _Magnetic Resonance in Medicine_ , 79(3):1696–1707, 2018.
  * Yu et al. [2019] Q. Yu, Y. Shi, J. Sun, Y. Gao, J. Zhu, and Y. Dai.  Crossbar-Net: A novel convolutional neural network for kidney tumor segmentation in CT images.  _IEEE Transactions on Image Processing_ , 28(8):4060–4074, 2019.  ISSN 1057-7149.  doi: 10.1109/TIP.2019.2905537.
  * Zheng et al. [2017] Y. Zheng, D. Liu, B. Georgescu, D. Xu, and D. Comaniciu.  Deep learning based automatic segmentation of pathological kidney in CT: Local versus global image context.  In _Advances in Computer Vision and Pattern Recognition_ , pages 241–255. Springer, Cham, 2017.  doi: 10.1007/978-3-319-42999-1_14.