11institutetext: Polytechnique Montréal, Montreal, Quebec, Canada
22institutetext: Centre Hospitalier de l’Université de Montréal, Montreal,
Quebec, Canada

# Comparing 3D deformations between longitudinal daily CBCT acquisitions using
CNN for head and neck radiotherapy toxicity prediction

William Trung Le 11 0000-0002-6106-7386 Chulmin Bang 22 Philippine Cordelle 11
Daniel Markel 22 Phuc Felix Nguyen‐Tan 22 Houda Bahig 22 Samuel Kadoury 11 2 2
0000-0002-3048-4291

###### Abstract

Adaptive radiotherapy is a growing field of study in cancer treatment due to
it’s objective in sparing healthy tissue. The standard of care in several
institutions includes longitudinal cone-beam computed tomography (CBCT)
acquisitions to monitor changes, but have yet to be used to improve tumor
control while managing side-effects. The aim of this study is to demonstrate
the clinical value of pre-treatment CBCT acquired daily during radiation
therapy treatment for head and neck cancers for the downstream task of
predicting severe toxicity occurrence: reactive feeding tube (NG),
hospitalization and radionecrosis. For this, we propose a deformable 3D
classification pipeline that includes a component analyzing the Jacobian
matrix of the deformation between planning CT and longitudinal CBCT, as well
as clinical data. The model is based on a multi-branch 3D residual
convolutional neural network, while the CT to CBCT registration is based on a
pair of VoxelMorph architectures. Accuracies of $85.8\%$ and $75.3\%$ was
found for radionecrosis and hospitalization, respectively, with similar
performance as early as after the first week of treatment. For NG tube risk,
performance improves with increasing the timing of the CBCT fraction, reaching
$83.1\%$ after the 5th week of treatment.

###### Keywords:

Toxicity prediction head and neck radiotherapy 3D CNN, deformable image
registration Jacobian matrix.

##  1 Introduction

Radiation therapy has seen an increase in usage as a form of primary cancer
treatment in recent years, due to significant improvements in imaging quality
[20]. However, radiotherapy is well known to be associated with negative side-
effects. Particularly in head and neck cancers (HNC), radiotherapy is
associated with significant toxicities affecting the upper aerodigestive track
causing impaired eating function or tissue necrosis [19], leading to needing
placement of a reactive feeding tube (NG) [3] or even hospitalization [12]. To
this day it remains an ongoing challenge to balance intensification vs de-
intensification of dose delivery in order to simultaneously maximize tumor
control and minimize affecting surrounding organs at risk (OAR).

In the past decade, it has become routine practice to include daily volumetric
scans during treatment to better localize the targeted tumor and monitor
anatomical changes. While not as high quality as the computed tomography scan
used for treatment planning (pCT), these serial or temporal cone-beam CTs
(CBCT) acquired before each radiotherapy fraction contain a wealth of imaging
biomarkers that are simply unused in the current standard of care.
Specifically, information on the deformation of the tissues and organs during
therapy due to mass reduction or tumor shrinkage can be potentially associated
with tumor control and toxicity outcomes. Dysphagia for example is often seen
correlated with anatomical changes in the surrounding areas of the digestive
track [3]. This information could then potentially be used to adapt the
radiotherapy plan (ART) in progress to minimize the risk of toxicities.
Recently, Muelas-Soria et al. [18] found it possible to determine the CBCT
fraction at which dosimetry re-planning would be required using a commercial
deformable image registration (DIR) platform (RayStation). Nonetheless, to our
knowledge no work yet has tried to use ART based on patient’s daily anatomical
changes observed during treatment to predictive toxicity risks.

Many of the leading state-of-the-art methods in medical image analysis involve
the use of deep neural networks (DNN) for efficient extraction of automatic
features and task-specific therapy biomarkers. Bibault et al. [2] used a DNN
method on radiomics features to evaluate treatment response for locally
advanced rectal cancer. However, methods leveraging convolutional neural
networks (CNN) have been shown to be particularly efficient at processing
large-scale 3D and 4D datasets, commonly used in radiology for diagnosis. Jin
et al. [9] used a pair of Siamese subnetworks to evaluate treatment response
which allowed analysis of dynamic information in longitudinal images, with
pre- and post-op scans specifically. Men et al. [16] used a residual CNN to
process pCT, organ segmentations and 3D dose distributions in order to predict
xerostomia risk in patients with HNC. FDG-PET/CT images were analyzed in
Diamant et al. [5] and Want et al. [24] using CNN architectures, the later
including dose distribution in the inputs. Recently, Le et al. [11] adapted
these techniques using a pseudo-3D version of the residual CNN [6], while also
integrating patient clinical data as additional input features.

In this work, we design a HNC toxicity outcome prediction pipeline that
analyses features obtained from the daily CBCTs, the deformations between
those CBCT and the baseline pCT, and the patient’s clinical records. The
proposed model is based on a multi-branch 3D version of the ResNet
architecture [6] processing 3D deformations along with an additional fully-
connected branch for processing clinical data, with latent space fusion using
concatenation before the final decision layers for binary output
classification. To encode the temporal aspect of a patient’s response to
treatment, a sub-step consisting of a pair of deep DIR model using the
VoxelMorph framework was used to capture both the transformation from
Housfield to CBCT space as well as the anatomical changes between the target
CBCTt fraction the the pCT, transformed into a Jacobian matrix to capture the
magnitude of the displacements. We aim to determine the feasibility of
predicting from radiotherapy-induced anatomical changes the risk of observing
the following toxicities during treatment: reactive NG tube, hospitalization
and radionecrosis.

The novel contributions proposed in this work are as follows:

  1. 1.

Analysis of CBCT images, which are yet clinically unproven for prognosis and
ART.

  2. 2.

Classification of severe HNC radio-toxicities using the Jacobian matrix
encoding of anatomical changes, learned using deep DIR models.

  3. 3.

Evaluation of the toxicity risk evolution based on CBCT fractions to maximize
mid-treatment intervention opportunities.

##  2 Materials and Methods

###  2.1 Dataset and preprocessing

A retrospective dataset of 292 patients of the Centre Hospitalier de
l’Université de Montréal that underwent radiation therapy between 2000 and
2018 was used, with histologically confirmed HNC with or without surgery and
concurrent chemotherapy. This dataset includes 120kVp 1.5mm resolution pCT
along with the associated OAR and tumor segmentation, between 30 to 35 daily
pre-treatment 120kVp CBCT with 2mm resolution, and clinical data: age at
diagnosis, sex, KPS, tumor location, smoker, alcohol consumption, cancer
severity in the form of TNM staging [10] (composed of primary tumor extent,
regional lymph involvement, distant metastases), p16 biomarker status, whether
the patient had a surgical tumor resection, and if the patient is undergoing
concurrent chemotherapy.

Patient clinical data was one-hot encoded. Tumor location includes:
oropharynx, larynx, nasopharynx, hypopharynx, oral cavity or unknown primary.
All patients requiring a feeding tube at onset of therapy were excluded from
the study.

All images were resampled isotropically to $2\times 2\times 2$mm and
normalized to have pixel intensities between 0 and 1. CT scans were cropped
around the tumor segmentation (GTV) to a fix size of $128\times 128\times 128$
to match the field of view of the CBCTs, which were also center-cropped to the
same shape.

###  2.2 Registration and classification models

Figure 1:  Multi-branch toxicity prediction model based on ResNet [6], adapted
to 3D multi-modal inputs for binary classification. Two branch types were
defined with their own sets of weights: a CBCT feature extractor, and an
anatomical deformation analysis branch using the Jacobian matrix of the DVF
between pCT and the target CBCTt. Masking was used in all images to include
only the HNC region. A third fully connected input branch processed clinical
data. Top left: deep DIR model trained to learn the CT $\rightarrow$ CBCT0
mapping. Bottom left: deep DIR model trained to learnt the CBCT0 $\rightarrow$
CBCTt mapping. Right: toxicity outcome classification model. Architecture
shown is for the training phase. During testing, CBCT0 is unused, and is
replaced by the aligned CT.

Our proposed prediction pipeline contains an initial deep DIR offline training
phase, followed by a second online classification phase.

In the initial step, the anatomical changes of the patient in response to
radiotherapy over time were computed, that is between pCT and CBCTt. While
based on related technologies and seemingly similar visually, CT and CBCT
present distinct characteristics in pixel intensities, edge resolution and
artifacting [4]. Older methods of DIR incorporated an initial intensity
correction preprocessing step so that the registration could be seen as a
mono-modal task [7, 13, 21]. More recent approaches leveraging deep learning
attempted to perform both tasks in a single fully automated way [4].
Nonetheless, we are proposing to follow the former two stage approach
precisely to distinguish between multi-modal mapping space from mono-modal
transformations resulting uniquely from morphological changes in response to
radiotherapy. Thus by replacing the preprocessing steps with a more modern
deep learning-based DIR approaches, we are able to train a pair of models
based on the VoxelMorph [1] framework that are capable of disentangling the
different stages of multi-modal registration. And this will become relevant as
our proposed method focuses specifically on extracting the transformation
related to the second stage: registration resulting from anatomical changes
before and during therapy.

To train under this proposed methodology, an initial model learned the mapping
from pCT and CBCT0 (see Figure 1). Independently, a second model was trained
to register CBCT that were deformed due to changes in anatomy resulting from
radiotherapy treatment CBCT0 $\rightarrow$ CBCTt). Fixing the input image to
CBCT0 has the advantage of resembling the pCT the most since no treatment
occurred between those two dates, which will become relevant at testing time.
At that stage then after both models are trained, the pCT is registered first
to CBCT0, then that image (Aligned CT in Figure 1) is then fed into the second
model and registered to the CBCT at the current fraction of treatment (not
shown).

Both models used the UNet architecture [22] with $(16,32,32,32)$ layers in the
encoder and $(32,32,32,32,32,16,16)$ layers in the decoder to generate the DVF
outputs, as well as a spatial transformer network (STN) [8] to learn the
deformation from the input to output space, as detailed in the original work
[1]. Due to the massive memory requirement for storing DVF, the final pCT
$\rightarrow$ CBCTt deformations were converted to its Jacobian matrix
$\mathbb{J}_{f}$ which consists of the partial derivative linear approximation
along each displacement vector, as in Eq. 1:

$\mathbb{J}_{f}=\left[\begin{array}[]{ccc}\dfrac{\partial\mathbf{f}(\mathbf{x})}{\partial
x_{1}}&\cdots&\dfrac{\partial\mathbf{f}(\mathbf{x})}{\partial
x_{n}}\end{array}\right]$ (1)

where the transformation function $f$ corresponds to the composition of the
two DIR models, as in Eq. 2.

$\mathbf{f}_{t}:pCT\rightarrow CBCT_{0}\rightarrow CBCT_{t}$ (2)

The multi-branch classification model itself is the 3D ResNet architecture —
using both bottleneck convolutional blocks and residual skip connections —
specifically the 34 and 50 layers variants as detailed in the original work by
He et. al [6] and provided by the MONAI library [17]. To integrate features
across the multiple inputs, separate branches were replicated without weights
sharing to process the CBCT and $\mathbb{J}_{f}$ images. These were then
masked to remove non anatomical artifacts using adaptive thresholding. An
additional fully-connected branch with 3 layers (64, 128, 256) and ReLU
activation integrated patient clinical data before fusion with the latent
embedding vector of the image features. Dropout of 40% and batch normalization
was used in every block of the clinical features path.

The final decision layer consisting of a single softmaxed fully-connected
layer taking the concatenation of the CBCT, $\mathbb{J}_{f}$ and clinical data
latent vectors produced a binary classification output. A different model was
trained for each event of reactive NG tube, hospitalization and radionecrosis
outcome appearing at any point during the $30$–$35$ days of treatment ($6$–$7$
weeks of 5 work days).

###  2.3 Training and evaluation

The sequential deep DIR models were trained offline to preprocess pCT/CBCTt
pairs into a $\mathbb{J}_{f}$ matrix using the VoxelMorph framework [1].
Inputs were first co-registered rigidly using the SimpleElastix software [15]
with normalized cross-correlation loss (NCC), AdaptiveSGD optimizer and 4
levels of resolutions. DIR models were both trained using the Adam optimizer
with the global NCC image reconstruction loss and regularized by the L2
gradient loss. In the first case, input pairs consisted of pCT/CBCT0 with an
L2 gradient regularization term of $\lambda=1.0$. In the second case, input
pairs consisted of CBCT0 and all available CBCTt with $t={1...35}$ with
$\lambda=0.5$. In both cases, the registration models were trained with a
$70\%$, $15\%$, $15\%$ split for training, validation and testing sets on
paired input images only. To quantitatively evaluate the performance of the
end-to-end registration model, target registration errors based on 3 manually
annotated landmarks was computed between the pCT and the final (30th) CBCT
fraction.

In the second online phase, a different model was trained for each of the 3
possible outcomes using a 5-fold cross-validation scheme. Each model was
trained to predict a binary target using the cross-entropy loss, weighted by
the inverse of the toxicity frequency to combat data imbalance: the toxicity
occurrence rate in the training dataset are $19.9\%$ for reactive NG tube,
$3.8\%$ for radionecrosis and $7.2\%$ for hospitalization. In the specific
sub-study on the impact of multi-modal image inputs, the feature extractor
branches used the hyperparameters defined by ResNet34, to account for the
large increase in GPU memory requirements. Otherwise, the ResNet50
architecture was used throughout this study [6]. In both the offline and
online models, the OneCycle learning scheduling policy [23] was used to
improve convergence time.

##  3 Results

(a) CBCT0

(b) CBCT5

(c) CBCT10

(d) CBCT15

(e) CBCT20

(f) CBCT25

(g) CBCT30

Figure 2:  Registration of pCT to CBCT at different fractions with
corresponding deformed OAR and tumor segmentation. Top: deformed pCT to the
corresponding CBCT. The following segmentations are visible: oral cavity in
pink, submandibular glands in yellow, GTV in blue and PTV in green. Bottom:
Reference CBCT with DVF as yellow arrows.

The multi-modal registration model mapping CT inputs to CBCT voxel space was
trained for for 100 epochs with early stopping with a batch size of 7. An
average validation loss of $-0.9438$ was obtained, with $99.9\%$ of the
$\mathbb{J}_{f}$ matrix showing a non-zero deformation amplitude. For the
temporal CBCT DIR model, a $-0.9862$ mean validation NCC was obtained with
deformations present in $99.4\%$ of the $\mathbb{J}_{f}$. Qualitative results
of the deformed pCT and segmentations are shown in Figure 2 along with the
reference CBCT fraction. Mean TRE for landmarks yielded $2.83mm\pm 0.89$ on
the 2D slice with visibly segmented tumor.

The classification model was trained for 100 epochs, with a batch size of 8
and the Adam optimizer with 0.0007 learning rate, except for the
$\mathbb{J}_{f}$/clinical ablation study, which was trained for 200 epochs.
Performance metrics reported in the experiments include balanced accuracy
(bAcc), specificity (Spec) and sensitivity (Sens). To evaluate the correlation
between the CBCT fraction and classification performance, a linear fit was
used with a statistical significance of $r^{2}\geq 0.9$. All models were
trained on Nvidia GeForce RTX 2080 Ti GPUs.

Toxicity | Clinical | $\mathbb{J}_{f}$ | bAcc (% $\pm$ std) | Spec (% $\pm$ std) | Sens (% $\pm$ std)
---|---|---|---|---|---
NG tube | ✓ | ✓ | 69.1 $\pm$ 4.5 | 75.9 $\pm$ 18.4 | 61.7 $\pm$ 16.1
| ✓ | 68.8 $\pm$ 6.3 | 75.8 $\pm$ 21.8 | 61.8 $\pm$ 17.8
✓ |  | 60.0 $\pm$ 5.7 | 55.4 $\pm$ 12.2 | 64.6 $\pm$ 17.5
Hospitalization | ✓ | ✓ | 75.3 $\pm$ 3.6 | 67.4 $\pm$ 16.6 | 83.1 $\pm$ 10.4
| ✓ | 72.9 $\pm$ 2.7 | 74.6 $\pm$ 12.0 | 71.1 $\pm$ 11.1
✓ |  | 56.4 $\pm$ 12.2 | 63.6 $\pm$ 21.2 | 49.3 $\pm$ 36.7
Radionecrosis | ✓ | ✓ | 85.8 $\pm$ 7.7 | 78.2 $\pm$ 15.6 | 93.3 $\pm$ 14.9
| ✓ | 74.3 $\pm$ 10.4 | 93.3 $\pm$ 14.9 | 55.3 $\pm$ 27.7
✓ |  | 81.0 $\pm$ 14.9 | 75.4 $\pm$ 4.1 | 86.7 $\pm$ 29.8

Table 1:  Ablation study of the classification performance for models trained
against $\mathbb{J}_{f}$ and clinical inputs. The baseline model contains only
the feed-forward path for classifying clinical data. Models with deformation
inputs in the form of the $\mathbb{J}_{f}$ matrix between the baseline CT and
the 10th day CBCT (chosen to balance between amount of deformation from
baseline and distance from the end of treatment) add an additional
ResNet50-like[6] 3D convolutional branch before the classification layers.
Figures are mean metrics over a 5-fold cross-validation with standard
deviation.

Table 1 shows the performance of the ablation study evaluating the effects of
using anatomical deformation and patient clinical data as inputs to the model.
Results show that a combination of inputs leads to an improvement in accuracy
vs baseline for each of reactive NG tube ($69.1\%>60.0\%$), hospitalization
($75.3\%>56.4\%$) and radionecrosis ($85.8\%>81.0\%$). Sensitivity was higher
than specificity for hospitalization ($83.1\%>67.4\%$) and radionecrosis
($93.3\%>78.2\%$), but lower for reactive NG tube ($61.7\%<75.9\%$). Notably,
reactive NG tube also has the lowest classification performance across all
models.

Toxicity | CBCT | $\mathbb{J}_{f}$ | bAcc (% $\pm$ std) | Spec (% $\pm$ std) | Sens (% $\pm$ std)
---|---|---|---|---|---
NG tube | ✓ | ✓ | 64.7 $\pm$ 5.3 | 52.1 $\pm$ 13.0 | 77.2 $\pm$ 9.8
| ✓ | 68.2 $\pm$ 4.8 | 66.6 $\pm$ 11.2 | 69.8 $\pm$ 13.7
✓ |  | 68.2 $\pm$ 6.7 | 78.1 $\pm$ 13.0 | 58.3 $\pm$ 20.5
Hospitalization | ✓ | ✓ | 79.6 $\pm$ 6.7 | 83.0 $\pm$ 5.4 | 76.3 $\pm$ 14.6
| ✓ | 64.4 $\pm$ 3.5 | 72.2 $\pm$ 15.5 | 56.6 $\pm$ 14.4
✓ |  | 73.6 $\pm$ 4.6 | 75.9 $\pm$ 6.2 | 71.3 $\pm$ 12.4
Radionecrosis | ✓ | ✓ | 79.0 $\pm$ 10.9 | 68.0 $\pm$ 26.4 | 90.0 $\pm$ 22.4
| ✓ | 74.7 $\pm$ 12.7 | 72.8 $\pm$ 18.4 | 76.7 $\pm$ 22.4
✓ |  | 79.7 $\pm$ 15.6 | 59.4 $\pm$ 31.2 | 100 $\pm$ 0.0

Table 2:  Ablation study of the classification performance for models trained
against $\mathbb{J}_{f}$ and CBCT inputs. All models include a clinical data
processing branch as in Table 1. Multi-branch classification model include a
separate ResNet34-like[6] 3D convolutional branch per input modality before
concatenation for the final classification binary output. Figures are mean
metrics over a 5-fold cross-validation with standard deviation.

Table 2 shows the effects on classification performance when adding either or
both of the CBCTs and the $\mathbb{J}_{f}$ to the inputs. Improvements were
found by substituting $\mathbb{J}_{f}$ for the CBCT volume for hospitalization
($73.6\%>64.4\%$) and radionecrosis ($79.7\%>74.7\%$) prediction, with no
false negative errors in the later case. Further improvements were found when
combining both $\mathbb{J}_{f}$ and CBCT inputs for hospitalization prediction
($79.6\%>73.6\%$).

Figure 3:  Risk evolution study. Balanced accuracy performance of predictive
model trained against $\mathbb{J}_{f}$ resulting from deformations after
patients received $t$ amount of radiotherapy fractions (day). Toxicity
outcomes: $\circ$ NG tube, $\blacktriangle$ hospitalization, $\times$
radionecrosis. Dashed lines: no linear correlation found between CBCT fraction
inputs and classification performance ($r^{2}<0.9$). Note that the baseline
models with deformations at fraction day 0 corresponds to the model with
clinical data only as inputs (not shown) as in Table 1.

Figure 3 shows the classification performance of the top-performing model with
$\mathbb{J}_{f}$ and clinical data inputs as found in Table 1 when trained
against deformations at increasing fractions. A positive correlation was found
between the predictive power and the amount of radiation received for reactive
NG tube classification ($r^{2}>0.9$), with a maximum accuracy of $83.1\%$ at
the 25th fraction. No correlation was found for radionecrosis or
hospitalization outcome prediction.

##  4 Discussion

In this study, we demonstrated the clinical value of longitudinal pre-
treatment CBCT for the downstream task of predicting radiotherapy toxicity
risk in HNC. The use of both the raw CBCT modality, despite not containing
Hounsfield data [14], as well as the deformation between the target CBCT at
the pCT, both allowed prediction of reactive NG tube, hospitalization and
radionecrosis, all at the 10th fraction. However, performance was not as could
be in the Jacobian + CBCT study, due to the lower model capacity used
(ResNet-34 instead of ResNet-50 in the Jacobian-only model), a trade-off that
was necessary arising from the memory limitation of the available GPU.

In addition, we showed that patient clinical data had a positive impact on
model performance when combined with the imaging inputs ($4.8\%$ improvements
for radionecrosis and $18.9\%$ for hospitalization), but lead to poor
predictive power when used alone, as is the case of reactive NG tube
prediction that barely performed better than random ($57.2\%$).

Furthermore, since predicting the risk of side effects loses clinical utility
as the treatment progresses given the decreasing opportunity for intervention,
we aimed to show whether there was a correlation between predictive power and
the CBCT fraction used. For reactive NG tube, performance improved over time,
which suggested that the model was able to detect the link between a change in
tissue mass and risk of dysphagia leading to requiring a feeding tube. For
radionecrosis and hospitalization however, no correlation was found, with all
models showing similar performances between the 5th and the 30th treatment
day. A possibility is that these toxicities may be related to the tumor
location and baseline radiological characteristics rather than the dynamic
changes, which compels further examination. Another would require correlating
the time at which these two toxicities occurred in the patient cohort and the
choice of fraction used as inputs to the models within the context of these
longitudinal correlation experiments. Moreover, given that the addition of
CBCT images to the inputs seemed to improve overall classification scores for
these two outcomes compared to baseline, additional investigation would be
required to properly estimate the model’s predictive ability that does not
take the lower model capacity trade-off. Finally, the main limitation the feel
that needs to be addressed is the absence of use of radiotherapy specific
imaging data such as tumor segmentation and dosimetry in combination with the
available pre-treatment CBCT, the subject of a ongoing study.

## References

  * [1] Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: Voxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging 38(8), 1788–1800 (2019)
  * [2] Bibault, J.E., Giraud, P., Housset, M., Durdux, C., Taieb, J., Berger, A., Coriat, R., Chaussade, S., Dousset, B., Nordlinger, B., et al.: Deep learning and radiomics predict complete response after neo-adjuvant chemoradiation for locally advanced rectal cancer. Scientific reports 8(1), 1–8 (2018)
  * [3] Brewczyński, A., Jabłońska, B., Mrowiec, S., Składowski, K., Rutkowski, T.: Nutritional support in head and neck radiotherapy patients considering hpv status. Nutrients 13(1), 57 (2020)
  * [4] Cao, Y., Fu, T., Duan, L., Dai, Y., Gong, L., Cao, W., Liu, D., Yang, X., Ni, X., Zheng, J.: Cdfregnet: A cross-domain fusion registration network for ct-to-cbct image registration. Computer Methods and Programs in Biomedicine 224, 107025 (2022). https://doi.org/https://doi.org/10.1016/j.cmpb.2022.107025, https://www.sciencedirect.com/science/article/pii/S0169260722004072
  * [5] Diamant, A., Chatterjee, A., Vallières, M., Shenouda, G., Seuntjens, J.: Deep learning in head & neck cancer outcome prediction. Scientific reports 9(1), 1–10 (2019)
  * [6] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [7] Hou, J., Guerrero, M., Chen, W., D’Souza, W.D.: Deformable planning ct to cone-beam ct image registration in head-and-neck cancer. Medical physics 38(4), 2088–2094 (2011)
  * [8] Jaderberg, M., Simonyan, K., Zisserman, A., et al.: Spatial transformer networks. Advances in neural information processing systems 28 (2015)
  * [9] Jin, C., Yu, H., Ke, J., Ding, P., Yi, Y., Jiang, X., Duan, X., Tang, J., Chang, D.T., Wu, X., et al.: Predicting treatment response from longitudinal images using multi-task deep learning. Nature communications 12(1), 1–11 (2021)
  * [10] Lababede, O., Meziane, M.A.: The eighth edition of tnm staging of lung cancer: Reference chart and diagrams. The oncologist 23(7), 844–848 (2018)
  * [11] Le, W.T., Vorontsov, E., Romero, F.P., Seddik, L., Elsharief, M.M., Nguyen-Tan, P.F., Roberge, D., Bahig, H., Kadoury, S.: Cross-institutional outcome prediction for head and neck cancer patients using self-attention neural networks. Scientific Reports 12(1), 1–17 (2022)
  * [12] Ling, D.C., Kabolizadeh, P., Heron, D.E., Ohr, J.P., Wang, H., Johnson, J., Kubicek, G.J.: Incidence of hospitalization in patients with head and neck cancer treated with intensity-modulated radiation therapy. Head & neck 37(12), 1750–1755 (2015)
  * [13] Lou, Y., Niu, T., Jia, X., Vela, P.A., Zhu, L., Tannenbaum, A.R.: Joint ct/cbct deformable registration and cbct enhancement for cancer radiotherapy. Medical image analysis 17(3), 387–400 (2013)
  * [14] Mah, P., Reeves, T., McDavid, W.: Deriving hounsfield units using grey levels in cone beam computed tomography. Dentomaxillofacial Radiology 39(6), 323–335 (2010)
  * [15] Marstal, K., Berendsen, F., Staring, M., Klein, S.: Simpleelastix: A user-friendly, multi-lingual library for medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 134–142 (2016)
  * [16] Men, K., Geng, H., Zhong, H., Fan, Y., Lin, A., Xiao, Y.: A deep learning model for predicting xerostomia due to radiation therapy for head and neck squamous cell carcinoma in the rtog 0522 clinical trial. International Journal of Radiation Oncology* Biology* Physics 105(2), 440–447 (2019)
  * [17] MONAI Consortium: MONAI: Medical Open Network for AI (9 2022), https://github.com/Project-MONAI/MONAI
  * [18] Muelas-Soria, R., García-Mollá, R., Morillo-Macías, V., Bonaque-Alandí, J., Sorribes-Carreras, P., García-Piñón, F., Ferrer-Albiach, C.: The usefulness of adaptative radiotherapy in prostate cancer: How, when, and who? Biomedicines 10(6), 1401 (2022)
  * [19] Nabil, S., Samman, N.: Risk factors for osteoradionecrosis after head and neck radiation: a systematic review. Oral surgery, oral medicine, oral pathology and oral radiology 113(1), 54–69 (2012)
  * [20] Pan, H.Y., Haffty, B.G., Falit, B.P., Buchholz, T.A., Wilson, L.D., Hahn, S.M., Smith, B.D.: Supply and demand for radiation oncology in the united states: Updated projections for 2015 to 2025. International Journal of Radiation Oncology* Biology* Physics 96(3), 493–500 (2016)
  * [21] Park, S., Plishker, W., Quon, H., Wong, J., Shekhar, R., Lee, J.: Deformable registration of ct and cone-beam ct with local intensity matching. Physics in Medicine & Biology 62(3), 927 (2017)
  * [22] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
  * [23] Smith, L.N., Topin, N.: Super-convergence: Very fast training of neural networks using large learning rates. In: Artificial intelligence and machine learning for multi-domain operations applications. vol. 11006, pp. 369–386. SPIE (2019)
  * [24] Wang, C., Liu, C., Chang, Y., Lafata, K., Cui, Y., Zhang, J., Sheng, Y., Mowery, Y., Brizel, D., Yin, F.F.: Dose-distribution-driven pet image-based outcome prediction (ddd-piop): a deep learning study for oropharyngeal cancer imrt application. Frontiers in Oncology 10, 1592 (2020)