# Incorporating Anatomical Awareness for Enhanced Generalizability and
Progression Prediction in Deep Learning-Based Radiographic Sacroiliitis
Detection

Felix J. Dorfner *$\dagger$ Department of Radiology, Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany
Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General
Hospital and Harvard Medical School, 149 Thirteenth St, Charlestown, MA 02129,
USA  Janis L. Vahldiek  Department of Radiology, Charité - Universitätsmedizin
Berlin corporate member of Freie Universität Berlin and Humboldt Universität
zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Leonhard Donle
Department of Radiology, Charité - Universitätsmedizin Berlin corporate member
of Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm
30, 12203 Berlin, Germany  Andrei Zhukov  Department of Gastroenterology,
Infectious Diseases and Rheumatology (incl. nutrition medicine), Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Lina
Xu  Department of Radiology, Charité - Universitätsmedizin Berlin corporate
member of Freie Universität Berlin and Humboldt Universität zu Berlin,
Hindenburgdamm 30, 12203 Berlin, Germany  Hartmut Häntze  Department of
Radiology, Charité - Universitätsmedizin Berlin corporate member of Freie
Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30,
12203 Berlin, Germany  Marcus R. Makowski  Department of Diagnostic and
Interventional Radiology, Faculty of Medicine, Technical University of Munich,
Ismaninger Strasse 22, 81675, Munich, Germany  Hugo J.W.L. Aerts  Artificial
Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical
School, 221 Longwood Avenue, Boston, MA 02115, USA  Departments of Radiation
Oncology and Radiology, Dana-Farber Cancer Institute and Brigham and Women’s
Hospital, 221 Longwood Avenue, Boston, MA 02115, USA  Radiology and Nuclear
Medicine, CARIM & GROW, Maastricht University, Minderbroedersberg 4-6, 6211 LK
Maastricht, Netherlands  Fabian Proft  Department of Gastroenterology,
Infectious Diseases and Rheumatology (incl. nutrition medicine), Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany
Valeria Rios Rodriguez  Department of Gastroenterology, Infectious Diseases
and Rheumatology (incl. nutrition medicine), Charité - Universitätsmedizin
Berlin corporate member of Freie Universität Berlin and Humboldt Universität
zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Judith Rademacher
Department of Gastroenterology, Infectious Diseases and Rheumatology (incl.
nutrition medicine), Charité - Universitätsmedizin Berlin corporate member of
Freie Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm
30, 12203 Berlin, Germany  Mikhail Protopopov  Department of Gastroenterology,
Infectious Diseases and Rheumatology (incl. nutrition medicine), Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany
Hildrun Haibel  Department of Gastroenterology, Infectious Diseases and
Rheumatology (incl. nutrition medicine), Charité - Universitätsmedizin Berlin
corporate member of Freie Universität Berlin and Humboldt Universität zu
Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Torsten Diekhoff  Department
of Radiology, Charité - Universitätsmedizin Berlin corporate member of Freie
Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30,
12203 Berlin, Germany  Murat Torgutalp  Department of Gastroenterology,
Infectious Diseases and Rheumatology (incl. nutrition medicine), Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Lisa
C. Adams  Department of Diagnostic and Interventional Radiology, Faculty of
Medicine, Technical University of Munich, Ismaninger Strasse 22, 81675,
Munich, Germany  Denis Poddubnyy$\ddagger$ Department of Gastroenterology,
Infectious Diseases and Rheumatology (incl. nutrition medicine), Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany
Department of Epidemiology, German Rheumatism Research Center, Charitéplatz 1,
10117 Berlin, Germany  Keno K. Bressem$\ddagger$ Department of
Gastroenterology, Infectious Diseases and Rheumatology (incl. nutrition
medicine), Charité - Universitätsmedizin Berlin corporate member of Freie
Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30,
12203 Berlin, Germany  Department of Diagnostic and Interventional Radiology,
Faculty of Medicine, Technical University of Munich, Ismaninger Strasse 22,
81675, Munich, Germany  Artificial Intelligence in Medicine (AIM) Program,
Mass General Brigham, Harvard Medical School, 221 Longwood Avenue, Boston, MA
02115, USA  Department of Radiology and Nuclear Medicine, German Heart Center
Munich, Munich, Germany

###### Abstract

Purpose: To examine whether incorporating anatomical awareness into a deep
learning model can improve generalizability and enable prediction of disease
progression.

Methods: This retrospective multicenter study included conventional pelvic
radiographs of 4 different patient cohorts focusing on axial spondyloarthritis
(axSpA) collected at university and community hospitals. The first cohort,
which consisted of 1483 radiographs, was split into training (n=1261) and
validation (n=222) sets. The other cohorts comprising 436, 340, and 163
patients, respectively, were used as independent test datasets. For the second
cohort, follow-up data of 311 patients was used to examine progression
prediction capabilities. Two neural networks were trained, one on images
cropped to the bounding box of the sacroiliac joints (anatomy-aware) and the
other one on full radiographs. The performance of the models was compared
using the area under the receiver operating characteristic curve (AUC),
accuracy, sensitivity, and specificity.

Results: On the three test datasets, the standard model achieved AUC scores of
0.853, 0.817, 0.947, with an accuracy of 0.770, 0.724, 0.850. Whereas the
anatomy-aware model achieved AUC scores of 0.899, 0.846, 0.957, with an
accuracy of 0.821, 0.744, 0.906, respectively. The patients who were
identified as high risk by the anatomy aware model had an odds ratio of 2.16
(95% CI: 1.19, 3.86) for having progression of radiographic sacroiliitis
within 2 years.

Conclusion: Anatomical awareness can improve the generalizability of a deep
learning model in detecting radiographic sacroiliitis. The model is published
as fully open source alongside this study.

*First author.
$\dagger$Corresponding author<EMAIL_ADDRESS>
$\ddagger$These authors contributed equally as last authors.

_Keywords_ Pelvic X-Ray, Artificial Intelligence, Axial Spondyloarthritis,
Rheumatology, Radiology

##  1 Introduction

Axial spondyloarthritis (axSpA) is a chronic, immune-mediated disease that
predominantly affects the axial skeleton, particularly the sacroiliac joints
(SIJ) and the spine. The disease spectrum encompasses both radiographic axSpA
(also referred to as ankylosing spondylitis – AS), marked by visible
structural damage to the SIJ on radiographs (normally fulfilling the
radiographic criterion of the modified New York criteria for AS [1], and non-
radiographic axSpA, where such damage is absent [2]. Detecting radiographic
sacroiliitis still plays an essential role in diagnosing and classifying
axSpA. According to the modified New York criteria, a sacroiliitis grade of
$\geq$ 2 on both SIJs or a grade of 3-4 on one SIJ serves as the criterion for
the presence of definite radiographic sacroiliitis [1]. While Magnetic
Resonance Imaging (MRI) has proven instrumental in identifying active
inflammatory and structural changes, particularly in early disease stages,
conventional radiography retains certain advantages. It is more cost-
effective, widely available, and more accessible in many regions of the world
as compared to MRI. According to international recommendations focusing on
imaging in SpA, radiography of sacroiliac joints is still recommended as the
first imaging method if axSpA is suspected [3]. A critical limitation in
detecting radiographic sacroiliitis, however, lies in the high inter- and
intraobserver variability [4], with evidence indicating that specialized
central readers outperform untrained counterparts [5]. To address these
issues, previous work has explored the utility of deep learning to detect
radiographic sacroiliitis, as such a model could learn from the expert
annotated training data and then be utilized to make consistent and accurate
diagnoses even in medical centers without specialized human readers. The
proposed convolutional neural network (CNN) achieved a performance that was
comparable to human expert readers on standard pelvic radiographs [6, 7].
Although axSpA primarily affects the SJI, a pelvic X-ray encompasses the
entire pelvic area and surrounding soft tissue, introducing extraneous
elements that reduce the signal-to-noise ratio of the image and potentially
diminish the algorithm’s effectiveness. However, extracting the SJI could
increase the AI model’s accuracy by eliminating irrelevant distractions and
only focusing on the relevant anatomical areas. This concept of anatomical
awareness is inspired by the process in which human readers assess
radiographs: initially identifying the anatomical region of interest and
subsequently assessing changes therein. In this study, we aim to evaluate the
efficacy of an anatomy-aware approach in the detection of sacroiliitis.
Specifically, we compare the performance of deep learning models trained on
cropped images, containing only the SIJ, to those trained on uncropped
radiographs. Furthermore, we investigated whether deep learning models could
detect changes in the image with a higher sensitivity than a human reader and
thereby identify patients who are at a higher risk of progression to
radiographic axSpA.

##  2 Methods

###  2.1 Cohort Description

Imaging data from four different studies was used:

  * •

Patients With Axial Spondyloarthritis: Multicountry Registry of Clinical
Characteristics (PROOF). PROOF is a continuing study carried out in clinical
settings across 29 countries. It includes 2170 adult patients who have been
diagnosed with axSpA (non-radiographic or radiographic) $\leq$ 1 year prior to
study enrolment and fulfill the ASAS classification criteria for axSpA [8].
1483 radiographs from PROOF (all of them diagnosed with axSpA, including
radiographic and non-radiographic forms) were available as training data for
our study.

  * •

German Spondyloarthritis Inception Cohort (GESPIC). GESPIC is a multicenter
inception cohort study conducted in Germany and includes 525 patients with
axSpA (non-radiographic or radiographic) [9]. In 436 patients (all of them
diagnosed with axSpA, including radiographic and non-radiographic forms),
radiographs of the sacroiliac joints were available for inference.

  * •

Identification of the Optimal Referral Strategy for Early Diagnosis of Axial
Spondyloarthritis (OptiRef). OptiRef is a clinical study that includes a total
of 361 patients who reported symptoms suggestive of axSpA [10]. From this
study, 340 images (110 diagnosed with axSpA, including radiographic and non-
radiographic forms, and 230 without SpA) were available for inference.

  * •

Diagnostic Accuracy of MRI and CT in axSpA (DAMACT). DAMACT is a combined
Dataset of the SacroIliac Magnetic Resonance Computed Tomograph (SIMACT) trial
and the Virtual Non-Calcium / Susceptibility Weighted Imaging (VNCa/SWI)
trial. The studies included a total of 178 patients with suspected axSpA [11,
12, 13]. 163 radiographs (89 diagnosed with axSpA, including radiographic and
non-radiographic forms, and 74 without SpA) of this dataset were available for
inference.

Table 1 shows further information on the patients clinical characteristics and
Figure 1 shows the patient flowchart.

Figure 1: Flowchart for the selection of images from the PROOF dataset
(training and validation) and GESPIC, DAMACT, and OptiRef datasets (testing).
r-axSpA: radiographic axSpA, nr-axSpA: non-radiographic axSpA. Table 1:
Patient Characteristics

| PROOF | GESPIC | OPTIREF | DAMACT
---|---|---|---|---
Clinical Parameter | N = 1,483 | N = 436 | N = 340 | N = 163
axSpA, n (%) | 1483 (100) | 436 (100) | 110 (32.4) | 89 (54.6)
radiographic axSpA, n (%) | 980 (66.1) | 219 (50.2) | 61 (17.9) | 47 (28.8)
non-radiographic axSpA, n (%) | 503 (33.9) | 217 (49.8) | 49 (14.4) | 42 (25.8)
Male sex, n (%) | 941 (63.5) | 235 (53.9) | 167 (49.1) | 82 (50.3)
Age, years, mean (S.D.) | 34.6 (10.6) | 35.5 (10.2) | 38.1 (10.7) | 38.2 (10.6)
Symptom duration, years, mean (S.D.) | 4.7 (6.9) | 4.1 (2.7) | 8.4 (7.7) | -
HLA-B27 positivity, n (%) | 789 (53.2) | 347 / 433 (80.1) | 153 / 334 (45.8) | 96/154 (62.3)
Family history of SpA, n (%) | 278 (18.7) | 137 (31.4) | 50 / 302 (16.6) | 22/109 (20.2)
Arthritis, n (%) | 475 (32.0) | 57 (13.1) | 65 (19.1) | 56 (34.4)
Enthesitis, n (%) | 532 (35.9) | 91 (20.9) | 59 (17.4) | -
Dactylitis, n (%) | 82 (5.5) | 9 (2.1) | 2 (0.6) | -
Uveitis, n (%) | 145 (9.8) | 75 (17.2) | 31 (9.1) | 29/152 (19.1)
Psoriasis, n (%) | 99 (6.7) | 42 (9.6) | 36 (10.6) | 22/152 (14.4)
IBD, n (%) | 38 (2.6) | 12 (2.8) | 8 (2.4) | 10/137 (7.3)
CRP, mg/L, mean (S.D.) | 15.6 (23.1) | 11.3 (18.0) | 3.1 (5.9) | 5.4 (9.6)
Elevated CRP, n (%) | 686 (46.3) | 188 / 423 (44.4) | 55 / 337 (16.3) | 20 (12.3)
ASDAS-CRP, mean (S.D.) | 2.9 (1.1) | 2.5 (1.0) | 2.4 (0.8) | -
BASDAI, mean (S.D.) | 4.5 (2.3) | 3.9 (2.1) | 4.4 (1.9) | 4.5 (1.8)
BASFI, mean (S.D.) | 3.3 (2.5) | 2.7 (2.3) | 2.6 (2.1) | -
Treatment with NSAIDs, n (%) | 1,155 (77.9) | 295 (67.7) | - | -
Treatment with systemic steroids, n (%) | 114 (7.7) | 37 (8.5) | - | -
Treatment with csDMARDs, n (%) | 464 (31.3) | 96 (22.0) | - | -
Treatment with a TNF inhibitor, n (%) | 198 (13.4) | 11 (2.5) | - | -

ASDAS, Ankylosing Spondylitis Disease Activity Score; BASDAI, Bath Ankylosing
Spondylitis Disease Activity Index; BASFI, Bath Ankylosing Spondylitis
Functional Index; CRP, C reactive protein; DMARDs, disease-modifying
antirheumatic drugs; HLA-B27, human leukocyte antigen B27; IBD, inflammatory
bowel disease; NSAID, non-steroidal anti-inflammatory drugs; S.D., standard
deviation; SpA, spondyloarthritis; TNFi, tumor necrosis factor alpha

###  2.2 Label generation

Radiographs of the sacroiliac joints were collected, anonymized, and
subsequently graded by trained and calibrated readers using the modified New
York criteria. For each image, grading was performed by multiple expert
readers. Please refer to the Appendix for a detailed description of the
grading process.

###  2.3 Datasets and pre-processing

The images used in the training and testing phases are sourced from different
datasets, as described above. The Pelvis and Sacrum were manually segmented on
images in the training dataset using 3D Slicer and MONAI Label [14, 15]. These
were used to train the Segmentation Model. In the segmentations generated by
the model the sacrum labels were dilated and the resulting intersection with
the pelvis was used to demark the SIJ, around which bounding boxes were
defined. The boxes served as the basis for creating cropped images that show
only the SIJ and the area in between. The same preprocessing was applied to
the full radiographs and the cropped images to ensure consistency in the data
that fed into the different models.

The PROOF Dataset was used to train the model and was therefore randomly split
into a training dataset (1261 images, 85%) and a validation dataset (222
images, 15%). The GESPIC, OptiRef, and DAMACT datasets were each used as
independent test datasets.

Figure 2: Illustration of the anatomy-aware processing pipeline

###  2.4 Model training

For Segmentation, we used a CNN based on the U-Net architecture [16]. For
preprocessing, all images were resized to 512x512 pixels and their intensity
was normalized. The weighted sum of the Dice loss and Cross Entropy Loss was
used as the loss function. The training was performed using the AdamW
optimizer algorithm and the one-cycle learning rate policy as described by
Smith et al. [17, 18]. The training data was augmented using random
transformations, such as cropping, rotation up to 10°, shearing, intensity and
contrast shifting, and Gaussian noise.

For Classification, we utilized a CNN based on the ResNet-50 architecture,
pre-trained on the ImageNet-1k dataset [19]. For preprocessing, all images
underwent resizing and cropping to uniform dimensions. A Contrast Limited
Adaptive Histogram Equalization (CLAHE) filter was further applied to
normalize intensity and improve contrast [20]. To augment the training data,
we introduced a multitude of random transformations - Comprising random
flipping, random rotating up to 10°, random zooming by $\pm$ 10%, and random
shearing. Cross Entropy loss with label smoothing was used as the loss
function. Again, the AdamW optimizer algorithm and the one-cycle learning rate
policy were used. Additionally, we implemented discriminative learning rates,
in which the learning rate increased for deeper layers. Mixup was further used
to reduce overfitting [21, 22]. A progressive resizing approach was
implemented, training first on images with sizes of 106 x 158 pixels and then
increasing the resolution through 208 x 314, 312 x 472 up to 416 x 628 pixels
after every 25, 25, 30, and 40 epochs of training respectively. Both full
radiographs and SIJ-cropped images were employed to train two separate but
fully comparable models. These models’ performance was then examined across
different testing datasets. A comprehensive detailing of the training regimens
is available in the appendix. To validate the model’s focus on clinically
relevant features, Gradient-weighted Class Activation Mapping (Grad-CAM) was
employed [23]. The training was performed on a dedicated Ubuntu workstation
with two Nvidia GeForce RTX 2080 Ti graphics cards and on a GPU Server of the
Scientific Computing Cluster of the Charité Universitätsmedizin Berlin using
two Nvidia A100 graphics cards. The training was performed using PyTorch
(Version 1.13.0) with PyTorch Lightning (Version 1.8.6) and MONAI (Version
1.1.0) [24, 25]. All code and the anatomy aware model are available as fully
open source at: https://github.com/FJDorfner/Anatomy-Aware-Classification-
axSpA.

###  2.5 Follow-up analysis

For the GESPIC dataset, follow-up data was available for a total of 311
patients at different time points. The most data points were available for the
2-year follow-up, which includes 251 patients. Out of these, 135 were
initially classified as not having radiographic sacroiliitis. Odds ratios were
calculated to compare the proportion of patients with a confident false
positive prediction at the baseline timepoint to the total patients without
radiographic axSpA at baseline for both models. Predictions were considered
confident for probabilities $>$ 0.7.

###  2.6 Statistical analysis

Metrics for training and validation/testing were computed using TorchMetrics
[26]. The Python packages NumPy (Version 1.22.2), pandas (Version 1.4.4),
scikit-learn (Version 0.24.2), matplotlib (Version 3.7.1), and seaborn
(Version 0.12.2) were used for data analysis and visualization [27, 28, 29,
30, 31]. The optimal cut-off value - the value above which model probabilities
are considered to be positive - was determined on the validation dataset by
repeatedly calculating the accuracy for different cut-offs. The confidence of
predictions was visualized. Receiver Operating Characteristic Curves (ROC)
were plotted, and the Area Under the Receiver Operating Characteristic Curve
(AUC) was calculated. Confusion matrices were generated to further illustrate
the model’s effectiveness. Given data imbalances across datasets, balanced
accuracy, sensitivity, and specificity were calculated. 95% confidence
intervals were estimated using bootstrapping with 1000 repetitions. DeLong’s
algorithm was used to compare the AUC between the two models for each dataset
[32]. McNemar’s test was used to compare model performance on the follow-up
data. A p-value $<$0.05 was considered statistically significant.

###  2.7 Ethics approval

PROOF, GESPIC, OptiRef, SIMACT, and VNCa/SWI studies were approved by the
local ethics committees of each study center in accordance with the local laws
and regulations and were conducted in accordance with the Declaration of
Helsinki and Good Clinical Practice. Written informed consent was obtained
from all patients.

##  3 Results

Expert readers identified radiographic sacroiliitis in 835 (66%) images from
the PROOF training dataset (n=1261) and in 145 (65%) from the PROOF validation
dataset (n=222). Regarding the independent test datasets, radiographic
sacroiliitis was present in 219 (50%) of 436 GESPIC images, 47 (29%) of 163
DAMACT images, and 61 (18%) of 340 OptiRef images. Classification performance
of the anatomy-aware and standard model was compared on the validation and the
test datasets.

###  3.1 Validation Dataset Model Comparison

Both the standard model and the anatomy-aware models exhibited strong
performance on the PROOF validation dataset. The AUC was 0.888 for the
standard model and 0.9 for the anatomy-aware model. For both models, the cut-
off value that results in the highest validation accuracy was calculated. This
cut-off value was 0.69 for the standard model and 0.59 for the anatomy-aware
model. These cut-offs were then used for all further predictions with the
respective model on the test datasets. For the standard model, this resulted
in an accuracy of 0.826, specificity of 0.844, and sensitivity of 0.807. For
the anatomy-aware model, the cut-off value resulted in an accuracy of 0.836,
specificity of 0.844, and sensitivity of 0.828. ROC curves and confusion
matrices for the validation dataset are provided in the supplementary
materials.

###  3.2 Test Dataset Model Comparison

Performance disparities between the models were dataset-dependent. Table 2
summarizes the classification metrics for both models and each of the
independent test datasets. In GESPIC, the standard model attained an AUC of
0.853, whereas the anatomy-aware model reached 0.899. For DAMACT, AUC scores
were 0.817 for the standard model and 0.846 for the anatomy-aware model. For
OptiRef, the AUC scores were 0.947 for the standard model and 0.957 for the
anatomy-aware model. The difference between the two models for GESPIC was
statistically significant (p=0.0047). For the DAMACT and OptiRef data, the
difference between the two models was not statistically significant (p = 0.18
and 0.22, respectively). Confusion matrices for each dataset and model can be
found in the Supplements.

Table 2: Classification performance for the standard and the anatomy-aware model on three different independent test datasets | Standard Model | Anatomy-Aware Model
---|---|---
GESPIC Test Dataset |  |
AUC | 0.853 (95% CI: 0.817, 0.887) | 0.899 (95% CI: 0.866, 0.928)
Balanced Accuracy | 0.770 (95% CI: 0.731, 0.808) | 0.821 (95% CI: 0.786, 0.853)
Sensitivity | 0.886 (95% CI: 0.847, 0.927) | 0.941 (95% CI: 0.908, 0.969)
Specificity | 0.654 (95% CI: 0.587, 0.716) | 0.700 (95% CI: 0.642, 0.760)
DAMACT Test Dataset |  |
AUC | 0.817 (95% CI: 0.740, 0.884) | 0.846 (95% CI: 0.779, 0.997)
Balanced Accuracy | 0.724 (95% CI: 0.647, 0.801) | 0.744 (95% CI: 0.670, 0.805)
Sensitivity | 0.681 (95% CI: 0.550, 0.818) | 0.851 (95% CI: 0.739, 0.945)
Specificity | 0.767 (95% CI: 0.687, 0.841) | 0.638 (95% CI: 0.548, 0.721)
OptiRef Test Dataset |  |
AUC | 0.947 (95% CI: 0.921, 0.968) | 0.957 (95% CI: 0.926, 0.979)
Balanced Accuracy | 0.850 (95% CI: 0.793, 0.903) | 0.906 (95% CI: 0.868, 0.942)
Sensitivity | 0.754 (95% CI: 0.638, 0.860) | 0.934 (95% CI: 0.870, 0.986)
Specificity | 0.946 (95% CI: 0.918, 0.971) | 0.878 (95% CI: 0.839, 0.910)
Figure 3: Receiver operating characteristics curves and area under the curves
for the standard model (A) and the anatomy-aware model (B).

###  3.3 Investigation of Model Activations

A comparison of the Grad-CAMs for both models revealed that the anatomy-aware
model exhibited more targeted activations, specifically focusing on the SIJ.
This attention to diagnostically relevant regions aligns with the New York
classification criteria, which categorize patients as positive for
radiographic sacroiliitis if they exhibit a score greater than or equal to
grade II bilaterally or greater than or equal to grade III unilaterally.

Figure 4: Grad-CAMs of the anatomy-aware model for predictions from the test
dataset. A: mild sacroiliitis is present on both sides, and the model’s
activations are equally focused on both sides; B: sacroiliitis of grade
$\geq$3 present, and the model has based its decision for radiographic axSpA
on only one joint and especially on the lesion; C: images without radiographic
axSpA, in those the model is focused equally on both sides and the full length
of the SIJ.

###  3.4 Evaluating Model Confidence Across Datasets

We performed an analysis of the models’ output probabilities across various
datasets to evaluate their confidence in predictions. The output probabilities
- values before applying the cut-off for binary classification - are closer to
ground truth (either 0 or 1) when the model is more confident. An ideal model
would provide probabilities that align closely with the ground truth.

In the GESPIC dataset, both models were more confident in predicting positive
cases for radiographic axSpA than in negative cases. For the DAMACT dataset,
both models show similar confidence values for the positive and negative
class. In the OptiRef dataset, both models showed a higher degree of
confidence predicting negative cases of non-radiographic axSpA.

Figure 5: Probabilities for the standard model (A) and anatomy-aware model
(B). 1 refers to r-axSpA and 0 to nr-axSpA. The bars are colored according to
the respective ground truth.

###  3.5 Identifying High-Risk Patients

The ability of deep learning models to detect subtle features in medical
images can be potentially leveraged for identifying patients at a higher risk
of disease progression (in our case - of development of definite radiographic
sacroiliitis). To test the hypothesis that the model is able to detect non-
radiographic patients with a high probability of radiographic sacroiliitis
development at baseline later on, we utilized the data of the 2-year follow-up
in the GESPIC dataset. This included a total of 251 patients, of which 135
were initially classified as non-radiographic axSpA. 25 of these patients
(18.5%) progressed to radiographic axSpA after two years. We analyzed the
predictions of both models for those patients at baseline. The anatomy-aware
model had 30 confident false positive predictions, of which 12 (40%) had a
progression within 2 years. This equals an odds ratio of 2.16 (95% CI: 1.19,
3.86) for progression after two years. The standard model had 48 confident
false positive predictions, of which 11 (22.9%) had a progression within two
years, which resulted in an odds ratio of 1.24 (95% CI: 0.60, 2.34). The
difference in model performance when considering the labels at the 2-year
follow-up was statistically significant (p=0.0002).

##  4 Discussion

This study shows incorporating anatomical awareness in artificial intelligence
algorithms augments their capacity to generalize, thereby enhancing their
utility and reliability in predicting definite radiographic sacroiliitis on
unseen data. Both the standard and anatomy-aware models exhibited comparable
learning abilities from the training data but substantial performance
differences on the different external test datasets used. Particularly, the
anatomy-aware model showed a higher consistency across datasets, with minimal
fluctuations in performance metrics, despite changes in disease prevalence and
severity. This improved performance also manifested in more targeted model
activations, as demonstrated by the Grad-CAMs. Here, the anatomy-aware model
displayed dual advantages. Firstly, it avoided distractions from other
anatomical structures, such as the pubic symphysis or hip joint, that often
sidetrack the standard model. Secondly, even when the sacrum appears in the
frame, the anatomy-aware model remains committed to focusing on the SIJ. This
suggests that the advantage of the anatomy-aware model lies not only in
eliminating distracting joints in preprocessing, but that the anatomy-aware
model has also learned to better locate and assess the SIJ.

Figure 6: Grad-CAMs for example images from GESPIC as output by the standard
model (A) and the anatomy-aware model (B).

Moreover, the anatomy-aware model shows a nuanced capability to identify
patients who may be on a trajectory toward disease progression in terms of
definite radiographic sacroiliitis development. Drawing from the GESPIC
dataset, the model expressed high confidence in diagnosing radiographic axSpA
in a specific subset of patients who were initially classified as not having
definite radiographic sacroiliitis by human experts. Specifically, these
patients were twice as likely to progress to radiographic axSpA within two
years. However, it is not clear if the model presciently forecasted the
development of radiographic sacroiliitis or if it rather more accurately
discerned early-stage markers that human readers might deem inconclusive for a
formal diagnosis. Independent of this, the model’s confident ’false positives’
could serve as an actionable flag for clinicians, guiding them to prioritize
these patients for more intensive surveillance. This could make the model a
valuable aid in identifying patients that could most benefit from a proactive
and focused clinical follow-up, thereby enhancing patient management
strategies in a nuanced manner. Compared to previous studies [6, 7], our work
has two key advantages: Firstly, this study utilizes automatic pre-processing,
with uniform settings for all images, instead of manually adjusting grayscale
levels for each image as done in [6], which is more representative of a real-
world clinical application. Secondly, while Üreten et al. also cropped the
pelvic radiographs to include only the SIJ before training a CNN [33]. They
evaluated the model performance on a small test dataset obtained from one
site. In comparison, this study used multiple independent test datasets,
providing a better picture of the model’s ability to generalize to images
taken from patients with different clinical characteristics. Furthermore, the
direct comparison between the standard and the anatomy-aware model allowed the
assessment of the impact that anatomical awareness has on the generalizability
of the model. To the best of our knowledge, there were no other studies
investigating the concept of anatomical awareness in pelvic radiographs at the
time of writing. There are also some limitations to this study. The first
Follow-Up examination that was available for this study was conducted after
two years. It is therefore unknown when exactly during this two-year period
the progression could have been diagnosed or if additional patients would
progress after the two years. It seems probable that the progression becomes
harder to predict the further it lies in the future. More data points with
smaller intervals between examinations could provide a clearer picture of the
model’s ability to identify high-risk patients.

### Conclusion

The incorporation of anatomical awareness can improve the diagnostic
performance and generalizability of deep neural networks in detecting
radiographic sacroiliitis. Additionally, neural networks may help identify
patients who are likely to progress to radiographic axSpA.

## Acknowledgements

The authors acknowledge the Scientific Computing of the IT Division at the
Charité - Universitätsmedizin Berlin for providing computational resources
that have contributed to the research results reported in this paper. URL:
https://www.charite.de/en/research/research_support_services/research_infrastructure/science_it/#c30646061
KKB is grateful for his participation in the BIH Charité Digital Clinician
Scientist Program funded by the Charité–Universitätsmedizin Berlin and the
Berlin Institute of Health.

## References

  * Linden et al. [1984] Sjef Van Der Linden, Hans A. Valkenburg, and Arnold Cats.  Evaluation of diagnostic criteria for ankylosing spondylitis.  _Arthritis & Rheumatism_, 27(4):361–368, 1984.  ISSN 1529-0131.  doi:10.1002/art.1780270401.
  * Sieper and Poddubnyy [2017] Joachim Sieper and Denis Poddubnyy.  Axial spondyloarthritis.  _The Lancet_ , 390(10089):73–84, 7 2017.  ISSN 0140-6736.  doi:10.1016/S0140-6736(16)31591-4.
  * Mandl et al. [2015] P. Mandl, V. Navarro-Compán, L. Terslev, P. Aegerter, D. van der Heijde, M. A. D’Agostino, X. Baraliakos, S. J. Pedersen, A. G. Jurik, E. Naredo, C. Schueller-Weidekamm, U. Weber, M. C. Wick, P. a. C. Bakker, E. Filippucci, P. G. Conaghan, M. Rudwaleit, G. Schett, J. Sieper, S. Tarp, H. Marzo-Ortega, M. Østergaard, and European League Against Rheumatism (EULAR).  Eular recommendations for the use of imaging in the diagnosis and management of spondyloarthritis in clinical practice.  _Annals of the Rheumatic Diseases_ , 74(7):1327–1339, 7 2015.  ISSN 1468-2060.  doi:10.1136/annrheumdis-2014-206971.  PMID: 25837448.
  * Yazici et al. [1987] H. Yazici, M. Turunç, H. Ozdoğan, S. Yurdakul, A. Akinci, and C. G. Barnes.  Observer variation in grading sacroiliac radiographs might be a cause of ’sacroiliitis’ reported in certain disease states.  _Annals of the Rheumatic Diseases_ , 46(2):139–145, 2 1987.  ISSN 0003-4967, 1468-2060.  doi:10.1136/ard.46.2.139.  publisher: BMJ Publishing Group Ltd section: Research Article PMID: 3827336.
  * van den Berg et al. [2014] R. van den Berg, G. Lenczner, A. Feydy, D. van der Heijde, M. Reijnierse, A. Saraux, A. Rahmouni, M. Dougados, and P. Claudepierre.  Agreement between clinical practice and trained central reading in reading of sacroiliac joints on plain pelvic radiographs: Results from the DESIR cohort.  _Arthritis & Rheumatology_, 66:2403–2411, 2014.  doi:10.1002/art.38738.  URL https://doi.org/10.1002/art.38738.
  * Bressem et al. [2021] Keno K. Bressem, Janis L. Vahldiek, Lisa Adams, Stefan Markus Niehues, Hildrun Haibel, Valeria Rios Rodriguez, Murat Torgutalp, Mikhail Protopopov, Fabian Proft, Judith Rademacher, Joachim Sieper, Martin Rudwaleit, Bernd Hamm, Marcus R. Makowski, Kay-Geert Hermann, and Denis Poddubnyy.  Deep learning for detection of radiographic sacroiliitis: achieving expert-level performance.  _Arthritis Research & Therapy_, 23(1):106, 4 2021.  ISSN 1478-6362.  doi:10.1186/s13075-021-02484-0.
  * Poddubnyy et al. [2021] Denis Poddubnyy, Fabian Proft, Kay-Geert A Hermann, Laura Spiller, Stefan M Niehues, Lisa C Adams, Mikhail Protopopov, Valeria Rios Rodriguez, Burkhard Muche, Judith Rademacher, Murat Torgutalp, Keno K Bressem, and Janis L Vahldiek.  Detection of radiographic sacroiliitis with an artificial neural network in patients with suspicion of axial spondyloarthritis.  _Rheumatology_ , 60(12):5868–5869, 12 2021.  ISSN 1462-0324.  doi:10.1093/rheumatology/keab636.
  * Poddubnyy et al. [2022] Denis Poddubnyy, Joachim Sieper, Servet Akar, Santiago Muñoz-Fernández, Hildrun Haibel, Maja Hojnik, Fabiana Ganz, and Robert D Inman.  Characteristics of patients with axial spondyloarthritis by geographic regions: Proof multicountry observational study baseline results.  _Rheumatology_ , 61(8):3299–3308, 8 2022.  ISSN 1462-0324.  doi:10.1093/rheumatology/keab901.
  * Rudwaleit et al. [2009] Martin Rudwaleit, Hildrun Haibel, Xenofon Baraliakos, Joachim Listing, Elisabeth Märker-Hermann, Henning Zeidler, Jürgen Braun, and Joachim Sieper.  The early disease stage in axial spondylarthritis: Results from the german spondyloarthritis inception cohort.  _Arthritis & Rheumatism_, 60(3):717–727, 2009.  ISSN 1529-0131.  doi:10.1002/art.24483.
  * Proft et al. [2020] Fabian Proft, Laura Spiller, Imke Redeker, Mikhail Protopopov, Valeria Rios Rodriguez, Burkhard Muche, Judith Rademacher, Anne-Katrin Weber, Susanne Lüders, Murat Torgutalp, Joachim Sieper, and Denis Poddubnyy.  Comparison of an online self-referral tool with a physician-based referral strategy for early recognition of patients with a high probability of axial spa.  _Seminars in Arthritis and Rheumatism_ , 50(5):1015–1021, 10 2020.  ISSN 1532-866X.  doi:10.1016/j.semarthrit.2020.07.018.  PMID: 32911279.
  * Diekhoff et al. [2017] Torsten Diekhoff, Kay-Geert A. Hermann, Juliane Greese, Carsten Schwenke, Denis Poddubnyy, Bernd Hamm, and Joachim Sieper.  Comparison of mri with radiography for detecting structural lesions of the sacroiliac joint using ct as standard of reference: results from the simact study.  _Annals of the Rheumatic Diseases_ , 76(9):1502–1508, 9 2017.  ISSN 0003-4967, 1468-2060.  doi:10.1136/annrheumdis-2016-210640.  publisher: BMJ Publishing Group Ltd section: Clinical and epidemiological research PMID: 28283515.
  * Deppe et al. [2021] Dominik Deppe, Kay-Geert Hermann, Fabian Proft, Denis Poddubnyy, Felix Radny, Mikhail Protopopov, Marcus R Makowski, and Torsten Diekhoff.  Ct-like images of the sacroiliac joint generated from mri using susceptibility-weighted imaging (swi) in patients with axial spondyloarthritis.  _RMD Open_ , 7(2):e001656, 5 2021.  ISSN 2056-5933.  doi:10.1136/rmdopen-2021-001656.  PMID: 34049998 PMCID: PMC8166621.
  * Diekhoff et al. [2022] Torsten Diekhoff, Iris Eshed, Felix Radny, Katharina Ziegeler, Fabian Proft, Juliane Greese, Dominik Deppe, Robert Biesen, Kay Geert Hermann, and Denis Poddubnyy.  Choose wisely: imaging for diagnosis of axial spondyloarthritis.  _Annals of the Rheumatic Diseases_ , 81(2):237–242, 2 2022.  ISSN 0003-4967, 1468-2060.  doi:10.1136/annrheumdis-2021-220136.  publisher: BMJ Publishing Group Ltd section: Imaging PMID: 34049855.
  * Fedorov et al. [2012] Andriy Fedorov, Reinhard Beichel, Jayashree Kalpathy-Cramer, Julien Finet, Jean-Christophe Fillion-Robin, Sonia Pujol, Christian Bauer, Dominique Jennings, Fiona Fennessy, Milan Sonka, John Buatti, Stephen Aylward, James V. Miller, Steve Pieper, and Ron Kikinis.  3d slicer as an image computing platform for the quantitative imaging network.  _Magnetic resonance imaging_ , 30(9):1323–1341, 11 2012.  ISSN 0730-725X.  doi:10.1016/j.mri.2012.05.001.  PMID: 22770690 PMCID: PMC3466397.
  * Diaz-Pinto et al. [2023] Andres Diaz-Pinto, Sachidanand Alle, Vishwesh Nath, Yucheng Tang, Alvin Ihsani, Muhammad Asad, Fernando Pérez-García, Pritesh Mehta, Wenqi Li, Mona Flores, Holger R. Roth, Tom Vercauteren, Daguang Xu, Prerna Dogra, Sebastien Ourselin, Andrew Feng, and M. Jorge Cardoso.  MONAI Label: A framework for AI-assisted Interactive Labeling of 3D Medical Images, April 2023.  URL http://arxiv.org/abs/2203.12362.  arXiv:2203.12362 [cs, eess].
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-Net: Convolutional Networks for Biomedical Image Segmentation, May 2015.  URL http://arxiv.org/abs/1505.04597.  arXiv:1505.04597 [cs].
  * Loshchilov and Hutter [2019] Ilya Loshchilov and Frank Hutter.  Decoupled Weight Decay Regularization, January 2019.  URL http://arxiv.org/abs/1711.05101.  arXiv:1711.05101 [cs, math].
  * Smith and Topin [2019] Leslie N. Smith and Nicholay Topin.  Super-convergence: very fast training of neural networks using large learning rates.  In Tien Pham, editor, _Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications_ , volume 11006, page 1100612. International Society for Optics and Photonics, SPIE, 2019.  doi:10.1117/12.2520589.  URL https://doi.org/10.1117/12.2520589.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 770–778, 2016.  doi:10.1109/CVPR.2016.90.
  * Qiu et al. [2017] Jianfeng Qiu, H. Harold Li, Tiezhi Zhang, Fangfang Ma, and Deshan Yang.  Automatic x-ray image contrast enhancement based on parameter auto-optimization.  _Journal of applied clinical medical physics_ , 18(6):218–223, 11 2017.  ISSN 1526-9914.  doi:10.1002/acm2.12172.  publisher-place: United States PMID: 28875594 PMCID: PMC5689921.
  * Zhang et al. [2018] Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz.  mixup: Beyond Empirical Risk Minimization, April 2018.  URL http://arxiv.org/abs/1710.09412.  arXiv:1710.09412 [cs, stat].
  * Szegedy et al. [2016] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the Inception Architecture for Computer Vision.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2818–2826, 2016.  doi:10.1109/CVPR.2016.308.
  * Selvaraju et al. [2020] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  _International Journal of Computer Vision_ , 128(2):336–359, 2 2020.  ISSN 0920-5691, 1573-1405.  doi:10.1007/s11263-019-01228-7.  arXiv:1610.02391 [cs].
  * Falcon and team [2019] William Falcon and The PyTorch Lightning team.  Pytorch lightning, 3 2019.  URL https://doi.org/10.5281/zenodo.7469930.  DOI: 10.5281/zenodo.7469930.
  * Cardoso et al. [2022] M. Jorge Cardoso, Wenqi Li, Richard Brown, Nic Ma, Eric Kerfoot, Yiheng Wang, Benjamin Murrey, Andriy Myronenko, Can Zhao, Dong Yang, Vishwesh Nath, Yufan He, Ziyue Xu, Ali Hatamizadeh, Andriy Myronenko, Wentao Zhu, Yun Liu, Mingxin Zheng, Yucheng Tang, Isaac Yang, Michael Zephyr, Behrooz Hashemian, Sachidanand Alle, Mohammad Zalbagi Darestani, Charlie Budd, Marc Modat, Tom Vercauteren, Guotai Wang, Yiwen Li, Yipeng Hu, Yunguan Fu, Benjamin Gorman, Hans Johnson, Brad Genereaux, Barbaros S. Erdal, Vikash Gupta, Andres Diaz-Pinto, Andre Dourson, Lena Maier-Hein, Paul F. Jaeger, Michael Baumgartner, Jayashree Kalpathy-Cramer, Mona Flores, Justin Kirby, Lee A. D. Cooper, Holger R. Roth, Daguang Xu, David Bericat, Ralf Floca, S. Kevin Zhou, Haris Shuaib, Keyvan Farahani, Klaus H. Maier-Hein, Stephen Aylward, Prerna Dogra, Sebastien Ourselin, and Andrew Feng.  MONAI: An open-source framework for deep learning in healthcare, November 2022.  URL http://arxiv.org/abs/2211.02701.  arXiv:2211.02701 [cs].
  * Detlefsen et al. [2022] Nicki Skafte Detlefsen, Jiri Borovec, Justus Schock, Ananya Harsh, Teddy Koker, Luca Di Liello, Daniel Stancl, Changsheng Quan, Maxim Grechkin, and William Falcon.  Torchmetrics - measuring reproducibility in pytorch, 2 2022.  URL https://github.com/Lightning-AI/metrics.  DOI: 10.21105/joss.04101.
  * Harris et al. [2020] Charles R. Harris, K. Jarrod Millman, van der Stéfan J. Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, van Marten H. Kerkwijk, Matthew Brett, Allan Haldane, del Jaime Fernández Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant.  Array programming with numpy.  _Nature_ , 585(7825):357–362, 9 2020.  doi:10.1038/s41586-020-2649-2.  publisher: Springer Science and Business Media LLC.
  * Pedregosa et al. [2011] Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay.  Scikit-learn: Machine learning in python.  _Journal of Machine Learning Research_ , 12(85):2825–2830, 2011.
  * McKinney [2010] Wes McKinney.  Data Structures for Statistical Computing in Python.  In Stéfan van der Walt and Jarrod Millman, editors, _Proceedings of the 9th Python in Science Conference_ , pages 56 – 61, 2010.  doi:10.25080/Majora-92bf1922-00a.
  * Hunter [2023] J. D. Hunter.  Matplotlib: A 2d graphics environment.  _Computing in Science & Engineering_, 9(3):90–95, 4 2023.  doi:10.1109/MCSE.2007.55.  original-date: 2011-02-19T03:17:12Z.
  * Waskom [2021] Michael L. Waskom.  seaborn: statistical data visualization.  _Journal of Open Source Software_ , 6(60):3021, 2021.  doi:10.21105/joss.03021.  publisher: The Open Journal.
  * Sun and Xu [2014] Xu Sun and Weichao Xu.  Fast implementation of delong’s algorithm for comparing the areas under correlated receiver operating characteristic curves.  _IEEE Signal Processing Letters_ , 21(11):1389–1393, 11 2014.  ISSN 1558-2361.  doi:10.1109/LSP.2014.2337313.  event-title: IEEE Signal Processing Letters.
  * Üreten et al. [2023] Kemal Üreten, Yüksel Maraş, Semra Duran, and Kevser Gök.  Deep learning methods in the diagnosis of sacroiliitis from plain pelvic radiographs.  _Modern Rheumatology_ , 33(1):202–206, 1 2023.  ISSN 1439-7595.  doi:10.1093/mr/roab124.

## Supplemental Material

### Label generation for the individual datasets

Label generation for the individual datasets For the PROOF dataset, images
were first assessed by the local readers, then by a central reader, who was
blinded to the results of the previous assessment. In the case of disagreement
between the local and central reader, the radiograph was further evaluated by
a second central reader. The final decision on the presence of definite
radiographic sacroiliitis was made based on the decision of two out of the
three readers. For the GESPIC dataset, all radiographs were scored
independently by two trained and calibrated central readers. For the OptiRef
dataset, the radiographs were scored as the consensus of a rheumatologist and
radiologist on the presence of definite radiographic sacroiliitis. In SIMACT
and VnCa, all radiographs were scored independently by three readers, who were
blinded to other results. The agreement of two out of the three readers was
used as the final score.

### Model Training Procedure

The segmentation model was trained without pre-trained weights, for 500
epochs, using a learning rate of $1\times 10^{-3}$ and a batch size of 8.
During training, the mean Dice Coefficient on the validation dataset was
monitored, and the model was saved on every improvement in the metric. For the
classification model, first, only the classification head of the model was
trained for 15 epochs on the smallest image size while the remaining layers
were frozen. During training, the Matthews Correlation Coefficient on the
validation dataset was monitored, and the model saved on every improvement in
the metric. The best-performing model was then loaded and trained on all
layers for another 15 epochs. This process was repeated for all image sizes
with varying hyperparameters. The batch size was 32 for the image size 416x628
and 64 for all other image sizes. The total training time was two hours. The
optimal values for the initial learning rate, the weight decay, the level of
mixup, the coefficient discriminative learning rate, the number of training
epochs, the batch size, and augmentation transforms were determined by testing
a wide range of possible values and selecting the hyperparameters that produce
the best classification accuracy. PyTorch Lightnings integration of mixed
precision training was used to speed up the training process and reduce memory
requirements.

Figure 7: Receiver operating characteristics curves and area under the curves
for the standard model and the anatomy-aware model, both on the validation
dataset. Figure 8: Confusion matrices for the standard model and the anatomy-
aware model, both on the validation dataset. Figure 9: Confusion matrices for
the standard model (A) and the anatomy-aware model (B). Figure 10: Sensitivity
and 1- Specificity (False positive rate) for the standard model and the
anatomy-aware model, both on the validation dataset. The Cut-off is 0.69 for
the standard model and 0.59 for the anatomy-aware model. Figure 11:
Sensitivity and 1- Specificity (False positive rate) for the standard model
(A) and the anatomy-aware model (B). The Cut-off is 0.69 for the standard
model and 0.59 for the anatomy-aware model.