11institutetext: University College London, UK 22institutetext: Stanford
University, USA 33institutetext: University of Aberdeen, UK

# Task-guided Domain Gap Reduction for Monocular Depth Prediction in Endoscopy

Anita Rau Project conducted while at University College London.1122 Binod
Bhattarai⋆ 1133 Lourdes Agapito 11 Danail Stoyanov 11

###### Abstract

Colorectal cancer remains one of the deadliest cancers in the world. In recent
years computer-aided methods have aimed to enhance cancer screening and
improve the quality and availability of colonoscopies by automatizing sub-
tasks. One such task is predicting depth from monocular video frames, which
can assist endoscopic navigation. As ground truth depth from standard in-vivo
colonoscopy remains unobtainable due to hardware constraints, two approaches
have aimed to circumvent the need for real training data: supervised methods
trained on labeled synthetic data and self-supervised models trained on
unlabeled real data. However, self-supervised methods depend on unreliable
loss functions that struggle with edges, self-occlusion, and lighting
inconsistency. Methods trained on synthetic data can provide accurate depth
for synthetic geometries but do not use any geometric supervisory signal from
real data and overfit to synthetic anatomies and properties. This work
proposes a novel approach to leverage labeled synthetic and unlabeled real
data. While previous domain adaptation methods indiscriminately enforce the
distributions of both input data modalities to coincide, we focus on the end
task, depth prediction, and translate only essential information between the
input domains. Our approach results in more resilient and accurate depth maps
of real colonoscopy sequences. The project is available here:
https://github.com/anitarau/Domain-Gap-Reduction-Endoscopy

###### Keywords:

Depth prediction Domain adaptation Self-supervision Endoscopy.

##  1 Introduction

Colorectal Cancer is treatable if detected early, but patient outcome relies
on the skill of the performing colonoscopist and complete diagnostic
examination of the colon. To improve navigation during colonoscopy and assist
endoscopists in ensuring complete examination, computer-assisted mapping and
3D reconstruction could help detect missed surfaces manifesting as holes in
the colon map reconstruction [3, 6]. Such surgical 3D environment maps could
also be used for robotic systems and automation, but despite rapid advances in
endoscopic artificial intelligence systems for polyp detection [1], mapping
technologies remain challenging to implement robustly. Traditional methods
require reliable features to be matched between frames, but colonoscopic
images suffer from illumination inconsistency and a lack of texture.

Figure 1: The proposed network reduces the domain gap between synthetic and
real images without fully closing it. We translate only domain- and task-
specific information like water which is present in real images but not in
synthetic ones.

A featureless way to obtain a 3D model of the colon is to directly learn
frame-wise depth and the relative camera pose between frames. But obtaining
ground truth training data for real colonoscopy frames is currently
unfeasible, as this would require a depth sensor to be integrated into a
standard colonoscope. Instead, self-supervised methods [12, 16] do not require
any ground truth data and use warping-errors to optimize depth and pose
predictions mutually. While such methods work well on homogeneous surfaces
[17], they are challenged by the self-occluding tubular shape of the colon and
the view-dependent illumination during colonoscopy.

An alternative to unlabeled real data is synthetically generated data with
ground truth depth. Chen et al. propose to first train a network on synthetic
data only and in a second, independent step, train the initialized network on
real images with self-supervision [5]. However, this approach does not account
for the domain shift between real and synthetic images. Other methods have
used Generative Adversarial Networks (GANs) to reduce the appearance domain
gap, some of which Figure 2 depicts. Mahmood et al. [10] propose a multi-stage
pipeline first mapping real examples to the synthetic domain, followed by an
independent depth network trained on synthetic data only. However,
independently training each sub-net might lead to sub-optimal results.
Integrating domain adaptation and depth prediction into a mutual framework,
Rau et al. [15] propose to train a single network on real and synthetic
images. Mathew et al. propose a variation of a cycle GAN that maps virtual
images to real images and vice versa [11]. One common drawback of these GAN-
based methods is the holistic translation from one domain to another without
considering domain- and task-specific components. Itoh et al. [8] are more
deliberate about their choice of translation and decompose information based
on a Lambertian-reflection mode; however, this hand-crafted decomposition is
not guaranteed to extract and translate the most helpful information. All the
translations mentioned above are difficult and distract from the main
objective: predicting depth.

Rather than aligning one domain to another, images should reduce the domain
gap between real and synthetic images only to the extent that it benefits the
end task [13]. Our approach is end-to-end trainable and learns from real and
synthetic data accounting for their different geometries by using separate
depth losses. As Figure 1 shows, the resulting network translates unknown
geometric structures like water which is not present in the synthetic dataset.
To the best of our knowledge, our method is the first to integrate synthetic
data through a domain-adaptation framework into self-supervised depth
estimation in colonoscopy.

$X_{S/R}$ = Synthetic/Real image, $X^{\prime}$ = Generated image, $Y_{S/R}$ =
Synthetic/Real depth,
$Y^{\prime}$ = Generated depth. $G$ = Generator, $T$ = Task (depth) net, $D$ =
Discriminator.

Figure 2: Comparisons of different domain adaptation methods ([15, 10, 11, 8,
5]) for depth prediction in colonoscopy. Depiction inspired by [19].

##  2 Methods

A standard GAN-based approach to depth prediction can map real and synthetic
images to the real domain [19], the synthetic/depth domain [15], or both [8,
11]. To map an image $X_{1}\in\mathcal{X}_{1}$ to a different domain
$\mathcal{X}_{2}$, $X_{1}$ is passed through a generator $\mathcal{G}$. The
output $\mathcal{G}(X_{1})$ is then passed through the discriminator
$\mathcal{D}$ which compares it to known images from $\mathcal{X}_{2}$.
Minimizing

$\displaystyle\mathbb{E}_{\mathcal{X}_{2}}[\text{log}(\mathcal{D}(X_{2}))]+\mathbb{E}_{\mathcal{X}_{1}}[\text{log}(1-\mathcal{D}(\mathcal{G}(X_{1})))],$
(1)

forces $\mathcal{G}$ to learn the distribution of $\mathcal{X}_{2}$ ([8, 11,
19, 15]).

There are two issues with this approach: (i) these GANs assume that real and
synthetic depths come from the same distribution, which is not necessarily
true; (ii) the domain adaptation is not guided by the end task. Depth losses
for synthetic data are employed but there is no geometric supervision for
predicted real depths. The domain adaptation network thus has no incentive to
translate images such that the most accurate real depths are predicted. Our
approach solves both issues.

###  2.1 Domain Gap Reduction

Our method maps as little information as possible to a mutual domain, allowing
the network to focus on the end task. This concept was proposed by SharinGAN
[13] for depth prediction from calibrated stereo cameras in urban settings.

Figure 3: Overview of our training and testing pipelines. Our network maps
real and synthetic images into a new domain $\mathcal{X^{\prime}}$. These
translated images are passed through the DepthNet. Synthetic images are
supervised with an $L_{1}$ error; real images are self-supervised using a warp
loss. During testing, inputs are passed through the Generator and DepthNet
only.

Figure 3 shows an overview of our approach. First, a GAN maps synthetic and
real images to a mutual and end-task-specific domain. Being in the same
domain, the synthetic and real images can now learn depth-specific features
from one another. Let $\mathcal{X_{R}}$ and $\mathcal{X_{S}}$ denote the real
and synthetic domain and let each image
$\in\\{\mathcal{X_{R}},\mathcal{X_{S}}\\}$ consist of domain-agnostic and
domain-specific information.

Domain-agnostic information $I$ is shared between the domains. Such
information should encompass the underlying geometry of the colon. We actively
avoid adaptation of $I$ as it is unnecessary. Domain-specific information
could be blood vessels that are visible in real images but not in synthetic
ones. Such domain-specific information can be end-task-specific, $\delta_{R}$
and $\delta_{S}$, or unspecific, $\delta^{\prime}_{R}$ and
$\delta^{\prime}_{S}$. The end task in our case is depth prediction but other
tasks can be adapted to the same concept. Blood vessels do not encode relevant
information about the geometry and are $\delta^{\prime}_{R}$; water and
shadows, on the other hand, contain information about shape and are
$\delta_{S}$. The domain gap between $\delta^{\prime}_{R}$ and
$\delta^{\prime}_{S}$ is negligible, as the depth net will learn to ignore
such information. But the domain gap between $\delta_{R}$ and $\delta_{S}$
will affect the training of the depth network. If $\delta_{R}$ and
$\delta_{S}$ are first mapped into a shared domain, real and synthetic data
can complement each another.

Let $x$ be a feature of image $X$. We want to learn a mapping
$f:\mathcal{X_{R}}\cup\mathcal{X_{S}}\rightarrow\mathcal{X^{\prime}};x\mapsto
f(x)$, such that $f(x)=x$ if
$x\in\\{\delta^{\prime}_{S},\delta^{\prime}_{R},\mathcal{I}\\}$, and $f(x)\neq
x$ if $x\in\\{\delta_{S},\delta_{R}\\}$. But how do we learn such a mapping?

Instead of mapping one domain to the other, both domains can be mapped into a
mutual one moving the means of the distributions together [2]:

$\displaystyle
L_{GAN}=\mathbb{E}_{\mathcal{X_{S}}}[\mathcal{D}(\mathcal{G}(X_{S}))]-\mathbb{E}_{\mathcal{X_{R}}}[\mathcal{D}(\mathcal{G}(X_{R}))].$
(2)

To translate only crucial information in an image while retaining most of it,
we use a reconstruction loss that penalizes translation by comparing the
generator’s input with its output:

$\displaystyle L_{R}$
$\displaystyle=\|\mathcal{G}(X_{S})-X_{S}\|^{2}_{2}+\|M(\mathcal{G}(X_{R})-X_{R})\|_{1}.$
(3)

We experimentally found the $L1$-loss to lead to more similar reconstructions
of small details in the real images and applied a specularity mask $M$ based
on the real images’ RGB values.

Now, instead of having to learn how to translate a synthetic image to the real
domain, or vice-versa, the network only needs to solve how to translate some
information. To encourage that only task-relevant information is translated,
we pass the generator’s output through a depth net. The depth losses from both
domains must then be back-propagated as described in the next section.

###  2.2 Depth supervision

As labels for synthetic data exist, synthetic depths are supervised with an
$L_{1}$-loss between the prediction and ground truth:

$\displaystyle L_{S}$ $\displaystyle=\|Y_{S}^{\prime}-Y_{S}\|_{1}.$ (4)

But as we miss ground truth for real data, the supervision for the real domain
is less straight-forward. SharinGAN proposes to use stereo images for
supervision. But in endoscopy we have to fall back to monocular video. We
therefore propose to incorporate self-supervised geometric supervision for
real images. Self-supervised losses help generalize to real anatomies but tend
to converge to local minima. Additional synthetic supervision can help guide
the optimization of self-supervised models.

For warping-based self-supervision we pass a second image, $X^{t+1}$, through
the same generator and subsequently input both images into a WarpNet, which
outputs a 6D pose vector $\mathbf{p}$ allowing us to warp image $X^{t+1}$ to
look like image $X^{t}$. We refer to this warped image as
$X^{t+1\rightarrow{t}}$. The warp loss $L_{W}$ is computed as proposed in [12]
allowing a direct comparison of both models:

$\displaystyle L_{W}$
$\displaystyle=1*L_{\text{photo}}+0.5*L_{\text{geo}}+0.1*L_{\text{smooth}}.$
(5)

It consists of a photometric loss comparing an image to its warped
counterpart:

$\displaystyle L_{\text{photo}}=\sum$
$\displaystyle\|\mathbf{T}(X^{t})-X^{t+1\rightarrow{t}}\|_{2},$ (6)

where $\mathbf{T}$ is the brightness-aware transformation of $X$ according to
[12]. Unlike [12] we do not incorporate the structural similarity index
measure (SSIM) in the warp loss [18], making SSIM a fair evaluation measure on
the test set. The geometric consistency loss is based on $Y^{\prime t}$ warped
to $t+1$, and $Y^{\prime t+1}$ backwards interpolated to $\tilde{Y}^{\prime
t+1}$ and the smooth loss supports convergence:

$\displaystyle L_{\text{geo}}=\frac{\|Y^{\prime
t\rightarrow{t+1}}-\tilde{Y}^{\prime t+1}\|_{1}}{Y^{\prime
t\rightarrow{t+1}}+\tilde{Y}^{\prime t+1}},\quad\text{and }\quad
L_{\text{smooth}}=\sum(\exp^{-\nabla X^{t}}\cdot\nabla Y^{\prime t})^{2}.$ (7)

The final loss is a sum of the GAN-loss, reconstruction loss, and depth
losses:

$\displaystyle L$
$\displaystyle=\omega_{G}L_{GAN}+\omega_{R}L_{R}+0.5\cdot(\omega_{S}L_{S}+\omega_{W}L_{W}).$
(8)

Now that the task losses from both domains are back-propagated through the
generator, the domain adaptation is guided by the end task and issue (ii) is
addressed. Lastly, we observe that our network does not assume that real and
synthetic depths are identically distributed (issue i); Figure 2 illustrates
that we only input RGB images ($X_{S},X_{R}$) to a mutual discriminator, not
depths.

###  2.3 Implementation details

Our DepthNet is the architecture used both in EndoSLAM [12] and SharinGAN
[13], allowing a direct comparison of the methods. For further comparability,
we use the WarpNet proposed in [12]. We use SharinGAN’s generator but replace
the transposed convolutional layers with interpolation-based upsampling. We
replace SharinGAN’s discriminator with the lightweight discriminator proposed
in Pix2Pix [7] reducing training time by almost half to 8 hours on one NVIDIA
A100-80GB GPU. The loss weights are chosen based on grid search and are:
$\omega_{G}=1,\omega_{R}=10,\omega_{S}=100,\omega_{W}=1$. We train our network
on 3,162 image pairs generated from 1,300 real colonoscopy frames of the
EndoMapper111https://www.synapse.org/Synapse:syn26707219/wiki/615178 dataset
[4]. All training images were extracted from a single video, as only two
videos in the dataset provide camera intrinsics, and one was held out for
testing. The synthetic dataset consists of 11,000 frames from the Unity-based
SimCol222https://www.ucl.ac.uk/interventional-surgical-sciences/simcol3d-data
dataset [14].

##  3 Experiments

Evaluating a method that bypasses the need for training data is not
straightforward, because the absence of test data is inherent to the task. Our
evaluation thus first focuses on a qualitative comparison. We then
quantitatively compare various reprojection losses for baselines trained in a
self-supervised fashion. Lastly, we show that our method generalizes across
patients and datasets.

*SharinGAN without virtual supervision of real images as no stereo data is available.

Figure 4: Comparison of different methods on test images. EndoSLAM and the
variation of SharinGAN fail to generalize to test data. AF-SfMLearner
generalizes more robustly but suffers from large artefacts. We highlight some
inconsistencies in pix2pix-ucl through magenta boxes. Our method is more
resilient to specular highlights, water, and bubbles than the baselines and
leads to smoother depth maps where the geometry is even (box in f) while
preserving crisp edges (arrow in h) and details (arrows in c).

Figure 5: Examples of input test images, their translated counterparts, and a
difference map, where yellow denotes larger difference. The GAN mainly
translates specularities and water in the real image, and shadows in synthetic
images (see bottom right corner).

Qualitative comparison: We compare our method to two self-supervised
approaches and two domain-adaptation-based algorithms. Our baselines are the
self-supervised approaches EndoSLAM [12] and AF-SfMLearner [16] with all
parameters set to their default values. The domain-adaptation-based baselines
are: (1) a modification of SharinGAN [13], SharinGAN*, in which we omit the
virtual supervision of the real images; (2) the extension of Pix2Pix [7],
referred to as pix2pix-ucl [15]. Figure 4 depicts results on real test images.
These test images are from the same patient but different sections of the
colon than the train images. EndoSLAM fails to generalize to unseen scenes.
Although the network converged on the training data, it fails to predict
useful depth maps on test images. AF-SfMLeraner predicts largely sensible
depth maps but struggles with artifacts like water, stool, and specular
highlights. SharinGAN* predicts the overall shape well but fails to preserve
details. Pix2pix-ucl, preserves details but the resulting depth maps are
patchy and sometimes inaccurate. See, for instance, the highlighted
inconsistency in map (e). Further, the gradient of these depth maps is uneven,
with only very few pixels assuming high depths (mostly in wrong locations).
Our method is the most robust one. It learns from synthetic images, such as
SharinGAN or pix2pix-ucl, but incorporating the warping loss helps understand
structures that would otherwise be misinterpreted.

In Figure 5 we investigate how our GAN works. We plot an image, its translated
version, and a difference map for a real and a synthetic example. We can
observe that only domain-specific and task-relevant information is translated.
In the real image, specularities and water are translated the most (yellow).
Specularities encapsulate information about surface normals, while water
puddles have specific geometric properties that are not present in the
synthetic data. The synthetic example shows that the network hallucinates a
strong shadow in the lower right corner, because Unity’s renderer does not
produce entirely realistic shadows.

Quantitative comparison:

Table 1: Comparison of the different warping-based methods on 1,006 real test
images.

| Photo. loss (Eq. 6) $\downarrow$ | Geo. loss (Eq. 7) $\downarrow$ | SSIM $\uparrow$
---|---|---|---
AF-SfMLearner | .096 $\pm$ .081 $\ddagger$ | .069 $\pm$ .040 | .686 $\pm$ .134 ${\dagger}$
EndoSLAM | .076 $\pm$ .035 ${\dagger}$ | .061 $\pm$ .033 ${\dagger}$ | .641 $\pm$ .104 ${\dagger}$
Proposed | .076 $\pm$ .036 ${\dagger}$ | .036 $\pm$ .031 ${\dagger}$ | .659 $\pm$ .110

${\dagger}$) Loss used for training as well. $\ddagger$) Related loss used for
training as well.

For methods using warping during training, we can evaluate the warping losses
on real test images. These indirectly give us information about the accuracy
of the depth [5]. As our network is supervised by the warping loss and the
synthetic $L_{1}$-loss, one might assume that our network results in a larger
warp error than EndoSLAM, which is trained with warp loss only. However, in
Table 1 we observe that our network results in a comparable photometric loss
but a significantly smaller geometric loss. Comparing the SSIM between the
methods, we observe that our model produces higher structural similarity than
EndoSLAM, although EndoSLAM uses an SSIM-loss during training and our approach
does not. A direct comparison of the self-supervised model EndoSLAM and our
GAN suggests that synthetic data benefits our shared training approach. AF-
SfMLearner produces the highest SSIM, though it was trained with the SSIM-loss
while the proposed method was not. This evaluation has limitations. Warping
errors evaluate the quality of depth and pose prediction mutually, and the two
tasks can compensate each other. We also investigated EndoMapper’s provided
point clouds as potential pseudo ground truth but found them too noisy and
sparse to be useful.

Generalization to new patients:

Figure 6: Generalization to different patients. Our method predicts accurate
depths even when trained on only one other patient. Failure cases due to
extreme shade, tools, and very large specular highlights, that are not in the
training set, are indicated by orange frames.

The EndoMapper dataset provides COLMAP results for two of the patients. These
pseudo-labels are helpful as they provide camera intrinsics and because COLMAP
rejects images that are too blurred or too occluded and thus neither useful to
extract features nor for our purposes. We found that training on fewer but
qualitatively better sequences improves results. We trained our model on one
of the two patients with COLMAP labels and evaluated it on the other. Results
are shown in Figure 6. We can observe that the model generalizes well to a
different patient, even when the colon is filled with water as in image (g,
top row) or when geometries are peculiar as in image (b, bottom row). We also
show failure cases in the bottom row. In image (e), the model does not
generalize to extensive shadow, probably cause by an occluded light source. In
image (f), the model falsely locates the retroflexed scope viewing itself in
the background. In image (g), the model does not generalize to the
extraordinary large specular highlight. However, none of these extreme cases
were present in the training data. Nonetheless, the model predicts sharp,
accurate, and robust depth maps on most frames of an unseen patient, even when
trained on a single anatomy and procedure only.

Figure 7: Depth predictions on the LDPolypVideo dataset. Depicted are results
on test images from a different procedure than the training data. Depths are
accurate and robust to interlacing artefacts. Failure cases are indicated by
orange frames.

Generalization to different datasets: We repeat our experiments on a second
publicly available dataset: the
LDPolypVideo333https://github.com/dashishi/LDPolypVideo-Benchmark dataset [9],
a dataset for polyp detection that conveniently offers polyp-free colonoscopy
videos. These videos can be used for our purposes as most frames focus on the
lumen rather than the mucosa. As the dataset does not provide camera
intrinsics we cannot rule out that the network learns a consistent but skewed
geometry. We trained our model on frames from one colonoscopy sequence and
applied it to images of a different procedure. We use the same synthetic
dataset and hyper-parameters as in the previous experiments. But unlike the
EndoMapper dataset, the sequences used for this experiment are only a few
minutes long and show only a small section of a colon. Accordingly, the model
is only trained on a fraction of the geometries observed in our first
experiment. Nonetheless, the model can generalize to a different patient
predicting accurate and sharp depth maps and is highly robust to interlacing
artefacts as illustrated in Figure 7.

##  4 Conclusions

Learning-based depth prediction has seen significant advances in recent years
but requires labels for training, which are not available for colonoscopy.
This work addresses the question how unlabeled real data and cheap, labeled
synthetic data can be used in a mutual framework without overfitting to the
geometry of the synthetic data. At the core of this work is the idea that
domain adaptation is a challenging task that should only be addressed to the
extent that it benefits the end task. Rather than indiscriminately translating
entire images from one domain to another, and accounting only for appearance
domain gaps, we propose task-guided domain gap reduction.

Our experiments show that our model learns to translate only task- and domain-
specific information in real and synthetic input images. The network learns
that water and air bubbles are specific to real data and that rendered shadows
in synthetic data differ from real data. Accounting for these task-specific
differences leads to geometrically consistent depth maps, outperforming
previous domain translation and self-supervised models. We demonstrate that
our results are more consistent with the smooth surfaces of the colon, more
robust to unseen geometries, and still preserve details and edges. In the
future, other tasks could benefit from task-guided domain gap reduction.

####  4.0.1 Acknowledgements

This work was supported by the Wellcome/EPSRC Centre for Interventional and
Surgical Sciences (WEISS) [203145Z/16/Z]; Engineering and Physical Sciences
Research Council (EPSRC) [EP/P027938/1, EP/ R004080/1, EP/P012841/1]; The
Royal Academy of Engineering Chair in Emerging Technologies scheme; and the
EndoMapper project by Horizon 2020 FET (GA 863146). All datasets used in this
work are publicly available and linked in this manuscript. The code for this
project is publicly available on Github. For the purpose of open access, the
author has applied a CC BY public copyright license to any author-accepted
manuscript version arising from this submission.

## References

  * [1] Ahmad, O.F., et al.: Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified delphi method. Endoscopy 53(09), 893–901 (2021)
  * [2] Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: ICML. pp. 214–223. PMLR (2017)
  * [3] Armin, M.A., et al.: Automated visibility map of the internal colon surface from colonoscopy video. IJCARS 11(9), 1599–1610 (2016)
  * [4] Azagra, P., et al.: Endomapper dataset of complete calibrated endoscopy procedures. arXiv preprint arXiv:2204.14240 (2022)
  * [5] Cheng, K., et al.: Depth estimation for colonoscopy images with self-supervised learning from videos. In: MICCAI. pp. 119–128 (2021)
  * [6] Freedman, D., et al.: Detecting deficient coverage in colonoscopies. IEEE TMI 39(11), 3451–3462 (2020)
  * [7] Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: CVPR. pp. 1125–1134 (2017)
  * [8] Itoh, H., et al.: Unsupervised colonoscopic depth estimation by domain translations with a lambertian-reflection keeping auxiliary task. IJCARS 16(6), 989–1001 (2021)
  * [9] Ma, Y., et al.: Ldpolypvideo benchmark: A large-scale colonoscopy video dataset of diverse polyps. In: MICCAI. pp. 387–396 (2021)
  * [10] Mahmood, F., Durr, N.J.: Deep learning and conditional random fields-based depth estimation and topographical reconstruction from conventional endoscopy. Medical image analysis 48, 230–243 (2018)
  * [11] Mathew, S., Nadeem, S., Kumari, S., Kaufman, A.: Augmenting colonoscopy using extended and directional cyclegan for lossy image translation. In: Proceedings of CVPR. pp. 4696–4705 (2020)
  * [12] Ozyoruk, K.B., et al.: Endoslam dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Medical image analysis 71, 102058 (2021)
  * [13] PNVR, K., Zhou, H., Jacobs, D.: Sharingan: Combining synthetic and real data for unsupervised geometry estimation. In: Proceedings of CVPR. pp. 13974–13983 (2020)
  * [14] Rau, A., Bhattarai, B., Agapito, L., Stoyanov, D.: Bimodal camera pose prediction for endoscopy. IEEE Transactions on Medical Robotics and Bionics pp. 1–1 (2023). https://doi.org/10.1109/TMRB.2023.3320267
  * [15] Rau, A., et al.: Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. IJCARS 14(7), 1167–1176 (2019)
  * [16] Shao, S., et al.: Self-supervised monocular depth and ego-motion estimation in endoscopy: Appearance flow to the rescue. Medical image analysis 77, 102338 (2022)
  * [17] Turan, M., et al.: Unsupervised odometry and depth learning for endoscopic capsule robots. In: IROS. pp. 1801–1807 (2018)
  * [18] Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13(4), 600–612 (2004)
  * [19] Zheng, C., Cham, T.J., Cai, J.: T2net: Synthetic-to-realistic translation for solving single-image depth estimation tasks. In: Proceedings of the European Conference on Computer Vision. pp. 767–783 (2018)