[3][htbp]

[3][htbp]

# The Second Monocular Depth Estimation Challenge

Jaime Spencer University of Surrey C. Stella Qian Aston University Michaela
Trescakova University of Southampton Chris Russell Amazon Simon
Hadfield11footnotemark: 1 Erich W. Graf33footnotemark: 3 Wendy J.
Adams33footnotemark: 3 Andrew J. Schofield22footnotemark: 2 James Elder York
University Richard Bowden11footnotemark: 1 Ali Anwar imec-University of
Antwerp Hao Chen Zhejiang University Xiaozhi Chen DJI Technology Kai Cheng
University of Science and Technology of China Yuchao Dai Northwestern
Polytechnical University Huynh Thai Hoa DeltaX Sadat Hossain1111footnotemark:
11 Jianmian Huang Independent Mohan Jing99footnotemark: 9 Bo
Li1010footnotemark: 10 Chao Li VIVO Baojun Li1212footnotemark: 12 Zhiwen
Liu1313footnotemark: 13 Stefano Mattoccia University of Bologna Siegfried
Mercelis66footnotemark: 6 Myungwoo Nam1111footnotemark: 11 Matteo
Poggi1414footnotemark: 14 Xiaohua Qi99footnotemark: 9 Jiahui
Ren1010footnotemark: 10 Yang Tang East China University of Science and
Technology Fabio Tosi1414footnotemark: 14 Linh Trinh66footnotemark: 6 S. M.
Nadim Uddin1111footnotemark: 11 Khan Muhammad Umair1111footnotemark: 11
Kaixuan Wang88footnotemark: 8 Yufei Wang1010footnotemark: 10 Yixing
Wang1313footnotemark: 13 Mochu Xiang1010footnotemark: 10 Guangkai
Xu99footnotemark: 9 Wei Yin88footnotemark: 8 Jun Yu99footnotemark: 9 Qi
Zhang1313footnotemark: 13 Chaoqiang Zhao1515footnotemark: 15

###### Abstract

This paper discusses the results for the second edition of the Monocular Depth
Estimation Challenge (MDEC). This edition was open to methods using any form
of supervision, including fully-supervised, self-supervised, multi-task or
proxy depth. The challenge was based around the SYNS-Patches dataset, which
features a wide diversity of environments with high-quality dense ground-
truth. This includes complex natural environments, e.g. forests or fields,
which are greatly underrepresented in current benchmarks.

The challenge received eight unique submissions that outperformed the provided
SotA baseline on any of the pointcloud- or image-based metrics. The top
supervised submission improved relative F-Score by 27.62%, while the top self-
supervised improved it by 16.61%. Supervised submissions generally leveraged
large collections of datasets to improve data diversity. Self-supervised
submissions instead updated the network architecture and pretrained backbones.
These results represent a significant progress in the field, while
highlighting avenues for future research, such as reducing interpolation
artifacts at depth boundaries, improving self-supervised indoor performance
and overall natural image accuracy.

##  1 Introduction

mde refers to the task of predicting the distance from the camera to each
image pixel. Unlike traditional geometric correspondence and triangulation
techniques, this requires only a single image. Despite the ill-posed nature of
the problem, deep learning has shown rapid improvements in this field.

Unfortunately, many existing approaches have focused solely on training and
evaluating in an automotive urban setting. This puts into question their
ability to adapt to previously unseen environments. The proposed Monocular
Depth Estimation Challenge (MDEC) aims to mitigate this by evaluating models
on a complex dataset consisting of natural, agricultural, urban and indoor
scenes. Furthermore, this is done in a zero-shot fashion, meaning that the
models must be capable of generalizing.

The first edition of MDEC [77] focused on benchmarking self-supervised
approaches. The submissions outperformed the baseline [78, 25] in all image-
based metrics (AbsRel, MAE, RMSE), but provided slightly inferior pointcloud
reconstructions [62] (F-Score). The second edition of MDEC, detailed in this
paper, ran in conjunction with CVPR2023. This edition was open to any form of
supervision, _e.g_. supervised, self-supervised or multi-task. The aim was to
evaluate the state of the field as a whole and determine the gap between
different supervision strategies.

The challenge was once again centered around SYNS-Patches [1, 78]. This
dataset was chosen due its diversity, which includes urban, residential,
industrial, agricultural, natural and indoor scenes. Furthermore, SYNS-Patches
contains dense high-quality LiDAR ground-truth, which is exceedingly rare in
outdoor environments. This ensures that the evaluations accurately reflect the
capabilities of each model.

Eight teams out of the 28 final submissions outperformed the State-of-the-Art
(SotA) baseline in either pointcloud- or image-based metrics. Half of these
submission were supervised using ground-truth depths, while the remaining half
were self-supervised with the photometric reconstruction loss [25, 28]. As
expected, supervised submissions typically outperformed self-supervised ones.
However, the novel self-supervised techniques generally outperformed the
provided baseline, even in pointcloud reconstructions. The remainder of the
paper will provide the technical details of each submission, analyze their
results on SYNS-Patches and discuss potential directions for future research.

##  2 Related Work

Supervised. Eigen  _et al_. [22] introduced the first end-to-end CNN for
monocular depth estimation (MDE), which made use of a scale-invariant loss and
a coarse-to-fine network. Further improvements to the network architecture
included the use of CRFs [53, 100], regression forests [72], deeper
architectures [88, 67], multi-scale prediction fusion [60] and transformer-
based encoders [66, 15, 9]. Alternatively, depth estimation was formulated as
a discrete classification problem [24, 49, 7, 8]. In parallel, novel losses
were proposed in the form of gradient-based regression [51, 84], the berHu
loss [47], an ordinal relationship loss [14] and scale/shift invariance [67].

Recent approaches focused on the generalization capabilities of MDE by
training with collections of datasets [82, 23, 69, 67, 66, 7]. This relied on
the availability of ground-truth annotations, including automotive data LiDAR
[27, 32, 38], RGB-D/Kinect [61, 16, 79], SfM reconstructions[51, 50], optical
flow/disparity estimation [88, 67] or crowd-sourced annotations [14]. These
annotations varied in accuracy, which may have impacted the final model’s
performance. Furthermore, this increased the requirements for acquiring data
from new sources, making it challenging to scale to larger amounts of data.

Self-Supervised. Instead of relying on costly annotations, Garg  _et al_. [25]
proposed an algorithm based on view synthesis and the photometric consistency
across stereo pairs. Monodepth [28] incorporated differentiable bilinear
interpolation [42], virtual stereo prediction and a SSIM+L1 reconstruction
loss. SfM-Learner [108] required only monocular video supervision by replacing
the known stereo transform with a pose estimation network.

Artifacts due to dynamic objects were reduced by incorporating uncertainty
[45, 93, 65], motion masks [31, 12, 20], optical flow [98, 68, 57] or the
minimum reconstruction loss [29]. Meanwhile, robustness to unreliable
photometric appearance was improved via feature-based reconstructions [105,
76, 99] and proxy-depth supervision [45, 73, 86]. Developments in network
architecture design included 3D (un-)packing blocks [32], positional encoding
[30], transformer-based encoders [106, 2], sub-pixel convolutions [64],
progressive skip connections [58] and self-attention decoders [43, 91, 107].

Challenges & Benchmarks. The majority of MDE approaches have been centered
around automotive data. This includes popular benchmarks such as Kitti [27,
81] or the Dense Depth for Autonomous Driving Challenge [32]. The Robust
Vision Challenge series [104], while generalization across multiple datasets,
has so far consisted only of automotive [27] and synthetic datasets [10, 70].

More recently, Ignatov  _et al_. introduced the Mobile AI Challenge [40],
investigating efficient MDE on mobile devices in urban settings. Finally, the
NTIRE2023[102] challenge, concurrent to ours, targeted high-resolution images
of specular and non-lambertian surfaces.

The Monocular Depth Estimation Challenge series [77]—the focus of this
paper—is based on the MonoDepth Benchmark [78], which provided fair
evaluations and implementations of recent SotA self-supervised MDE algorithms.
Our focus lies on zero-shot generalization to a wide diversity of scenes. This
includes common automotive and indoor scenes, but complements it with complex
natural, industrial and agricultural environments.

!t]syns_catdata:syns_cat

!t]distribdata:distrib

!t]synsdata:syns

##  3 The Monocular Depth Estimation Challenge

The second edition of the Monocular Depth Estimation
Challenge111https://codalab.lisn.upsaclay.fr/competitions/10031 was organized
on CodaLab [63] as part of a CVPR2023 workshop. The initial development phase
lasted four weeks, using the SYNS-Patches validation split. The leaderboard
for this phase was anonymous, where all method scores were publicly available,
but usernames remained hidden. Each participant could see the metrics for
their own submission.

The final challenge stage was open for two weeks. In this case, the
leaderboard was completely private and participants were unable to see their
own scores. This encouraged evaluation on the validation split rather than the
test split. Combined with the fact that all ground-truth depths were withheld,
the possibility of overfitting due to repeated evaluations was severely
limited.

This edition of the challenge was extended to any form of supervision, with
the objective of providing a more comprehensive overview of the field as a
whole. This allowed us to determine the gap between different techniques and
identify avenues for future research. We report results only for submissions
that outperformed the baseline in any pointcloud-/image-based metric on the
Overall dataset.

Dataset. The challenge is based on the SYNS-Patches dataset [1, 78], chosen
due to the diversity of scenes and environments. A breakdown of images per
category and some representative examples are shown in Table
LABEL:tbl:data:syns_cat and Figure LABEL:fig:data:syns. SYNS-Patches also
provides extremely high-quality dense ground-truth LiDAR, with an average
coverage of 78.20% (including sky regions). Given the dense ground-truth,
depth boundaries were obtained using Canny edge-detection on the log-depth
maps. This allows us to compute additional fine-grained metrics for these
challenging regions. As outlined in [78], the images are manually checked to
remove dynamic object artifacts.

Evaluation. Participants provided the unscaled disparity prediction for each
dataset image. The evaluation server bilinearly upsampled the predictions to
the target resolution and inverted them into depth maps. Self-supervised
methods trained with stereo pairs and supervised methods using LiDAR or RGB-D
data should be capable of predicting metric depth. Despite this, in order to
ensure comparisons are as fair as possible, the evaluation aligned predictions
with the ground-truth using the median depth. We set a maximum depth threshold
of 100 meters.

Metrics. We follow the metrics used in the first edition of the challenge
[77], categorized as image-/pointcloud-/edge-based. Image-based metrics
represent the most common metrics (MAE, RMSE, AbsRel) computed using pixel-
wise comparisons between the predicted and ground-truth depth map. Pointcloud-
based metrics [62] (F-Score, IoU, Chamfer distance) instead evaluate the
reconstructed pointclouds as a whole. In this challenge, we report
reconstruction F-Score as the leaderboard ranking metric. Finally, edge-based
metrics are computed only at depth boundary pixels. This includes
image-/pointcloud-based metrics and edge accuracy/completion metrics from
IBims-1 [46].

##  4 Challenge Submissions

We outline the technical details for each submission, as provided by the
authors. Each submission is labeled based on the supervision used, including
ground-truth (D), proxy ground-truth (D*) and monocular (M) or stereo (S)
photometric support frames. The first half represent supervised methods, while
the remaining half are self-supervised.

### Baseline – S

_ J. Spencer11footnotemark: 1<EMAIL_ADDRESS>C.
Russell44footnotemark: 4<EMAIL_ADDRESS>S. Hadfield11footnotemark: 1
<EMAIL_ADDRESS>R. Bowden11footnotemark: 1<EMAIL_ADDRESS>_

Challenge organizers submission from the first edition.
Network. ConvNeXt-B encoder [56] with a base Monodepth decoder [59, 28] from
[78].
Supervision. Self-supervised with a stereo photometric loss [25] and edge-
aware disparity smoothness [28].
Training. Trained for 30 epochs on Kitti Eigen-Zhou with an image resolution
of $192\times 640$.

### Team 1: DJI&ZJU – D

_ W. Yin 8 8footnotemark:  8<EMAIL_ADDRESS>K. Cheng 9 9footnotemark:  9
<EMAIL_ADDRESS>G. Xu 9 9footnotemark:  9<EMAIL_ADDRESS>H.
Chen 7 7footnotemark:  7<EMAIL_ADDRESS>B. Li1010footnotemark: 10
<EMAIL_ADDRESS>K. Wang 8 8footnotemark:  8<EMAIL_ADDRESS>X. Chen 8
8footnotemark:  8<EMAIL_ADDRESS>_

Network. ConvNeXt-Large [56] encoder, pretrained on ImageNet-22k [21], and a
LeReS decoder [97] with skip connections and a depth range of
$\left[0.3,150\right]$ meters.
Supervision. Supervised using ground-truth depths from a collection of
datasets [32, 36, 92, 16, 13, 17, 26, 3, 90, 6, 103]. The final loss is
composed of the SILog loss [22], pairwise normal regression loss [97], virtual
normal loss [95] and a random proposal normalization loss (RPNL). RPNL
enhances the local contrast by randomly cropping patches from the
predicted/ground-truth depth and applying median absolute deviation
normalization [75].
Training. The network was trained using a resolution of $512\times 1088$. In
order to train on mixed datasets directly with metric depth, all ground-truth
depths were rescaled as $\acs{depth-pred}^{\prime}=\nicefrac{{\acs{depth-
pred}f_{c}}}{{f}},$ where $f$ is the original focal length and $f_{c}$ is an
arbitrary focal length. This way, the network assumed all images were taken by
the same pinhole camera, which improved convergence.

### Team 2: Pokemon – D

_ M. Xiang1010footnotemark: 10<EMAIL_ADDRESS>J.
Ren1010footnotemark: 10<EMAIL_ADDRESS>Y. Wang1010footnotemark: 10
<EMAIL_ADDRESS>Y. Dai1010footnotemark: 10<EMAIL_ADDRESS>_

Network. Two-stage architecture. The first part was composed of a SwinV2
backbone [54] and a modified NeWCRFs decoder [100] with a larger attention
window. The second stage used an EfficientNet [80] with 5 inputs (RGB, low-res
depth and high-res depth) to refine the high-resolution depth.
Supervision. Supervised training using LiDAR/synthetic depth and stereo
disparities from a collection of datasets [38, 5, 94, 17, 16, 83, 96, 92, 37,
89, 71, 22, 61, 39, 88, 18, 34, 85, 6, 11, 84]. Losses included the SILog loss
[22] ($\lambda=0.85$) for metric datasets, SILog ($\lambda=1$) for scale-
invariant training, the Huber disparity loss for Kitti disparities and an
affine disparity loss [67] for datasets with affine ambiguities.
Training. The final combination of losses depended on the ground-truth
available from each dataset, automatically mixed by learning an uncertainty
weight for each dataset [44]. Since each dataset contained differently-sized
images, they were resized to have a shorter side of 352 and cropped into
square patches. Some datasets used smaller crops of size $96\times 352$, such
that the deepest feature map fell entirely into the self-attention window
($11\times 11$). A fusion process based on [60] merged low-/high-resolution
predictions into a consistent high-resolution prediction.

### Team 3: cv-challenge – D

_ C. Li1313footnotemark: 13<EMAIL_ADDRESS>Q. Zhang1313footnotemark: 13
<EMAIL_ADDRESS>Z. Liu1313footnotemark: 13<EMAIL_ADDRESS>Y.
Wang1313footnotemark: 13<EMAIL_ADDRESS>_

Network. Based on ZoeDepth [9] with a BEiT384-L backbone [4].
Supervision. Supervised with ground-truth depth from Kitti and NYUD-v2 [61]
using the SILog loss.
Training. The original ZoeDepth [9] and DPT [66] were pretrained on a
collection of 12 datasets. The models were then finetuned on Kitti ($384\times
768$) or NYUD-v2 ($384\times 512$) for outdoor/indoor scenes, respectively.
Different models were deployed on an automatic scene classifier. The fine-
tuned models were combined with a content-adaptive multi-resolution merging
method [60], where patches were combined based on the local depth cue density.
Since the transformer-based backbone explicitly captured long-term structural
information, the original double-estimation step was omitted.

### Team 4: DepthSquad – D

_ M. Nam1111footnotemark: 11<EMAIL_ADDRESS>H. T. Hoa1111footnotemark: 11
<EMAIL_ADDRESS>K. M. Umair1111footnotemark: 11<EMAIL_ADDRESS>S.
Hossain1111footnotemark: 11<EMAIL_ADDRESS>S. M. N. Uddin1111footnotemark: 11
<EMAIL_ADDRESS>_

Network. Based on the PixelFormer architecture [2] which used a Swin [55]
encoder and self-attention decoder blocks with cross-attention skip
connections. Disparity was predicted as a discrete volume [7], with the final
depth map given as the weighted average using the bin probabilities.
Supervision. Supervised using the SILog loss w.r.t. the LiDAR ground-truth.
Training. The model was trained on the Kitti Eigen-Zhou (KEZ) split using
images of size $370\times 1224$ for 20 epochs. Additional augmentation was
incorporated in the form of random cropping and rotation, left-right flipping
and CutDepth [41]. When predicting on SYNS-Patches, images were zero-padded to
$384\times 1248$ to ensure the compatibility of the training resolution. These
borders were remove prior to submission.

### Team 5: imec-IDLab-UAntwerp – MS

_ L. Trinh 6 6footnotemark:  6<EMAIL_ADDRESS>A. Anwar 6
6footnotemark:  6<EMAIL_ADDRESS>S. Mercelis 6 6footnotemark:  6
<EMAIL_ADDRESS>_

Network. Pretrained ConvNeXt-v2-Huge [87] encoder with an HR-Depth decoder
[58], modified with deformable convolutions [19]. The pose network instead
used ResNet-18 [35].
Supervision. Self-supervised using the photometric loss [29] and edge-aware
smoothness.
Training. Trained on the Kitti Eigen-Benchmark (KEB) split with images of size
$192\times 640$. The network was trained for a maximum of 30 epochs, with the
encoder remaining frozen after 6 epochs.

### Team 6: GMD – MS

_ B. Li1212footnotemark: 12<EMAIL_ADDRESS>J. Huang1212footnotemark: 12
<EMAIL_ADDRESS>_

Network. ConvNeXt-XLarge [56] backbone and an HR-Depth [58] decoder.
Supervision. Self-supervised based on the photometric loss [29].
Training. Trained on KEZ using a resolution of $192\times 640$.

### Team 7: MonoViTeam – MSD*

_ C. Zhao1515footnotemark: 15<EMAIL_ADDRESS>M.
Poggi1414footnotemark: 14<EMAIL_ADDRESS>F. Tosi1414footnotemark: 14
<EMAIL_ADDRESS>Y. Tang1515footnotemark: 15<EMAIL_ADDRESS>S.
Mattoccia1414footnotemark: 14<EMAIL_ADDRESS>_

Network. MonoViT [106] architecture, composed of MPViT [48] encoder blocks and
a self-attention decoder.
Supervision. Self-supervised on Kitti Eigen (KE) using the photometric loss
[29] (stereo and monocular support frames) and proxy depth regression.
Regularized using edge-aware disparity smoothness [28] and depth gradient
consistency w.r.t. the proxy labels.
Training. Proxy depths were obtained by training a self-supervised RAFT-Stereo
network [52] on the trinocular Multiscopic [101] dataset. The stereo network
was trained for 1000 epochs using $256\times 480$ crops. The monocular network
was trained on KE for 20 epochs using images of size $320\times 1024$.

### Team 8: USTC-IAT-United – MS

_ J. Yu 9 9footnotemark:  9<EMAIL_ADDRESS>M. Jing 9 9footnotemark:  9
<EMAIL_ADDRESS>X. Qi 9 9footnotemark:  9<EMAIL_ADDRESS>_

Network. Predictions were obtained as a mixture of multiple networks: DiffNet
[107], FeatDepth [74] and MonoDEVSNet [33]. DiffNet and FeatDepth used a
ResNet backbone, while MonoDEVSNet used DenseNet [38].
Supervision. Self-supervised using the photometric loss [29].
Training. The three models were trained with different resolutions: $320\times
1024$, $376\times 1242$, $384\times 1248$, respectively. All predictions were
interpolated to $376\times 1242$ prior to ensembling using a weighted average
with coefficients $\left\\{0.35,0.3,0.35\right\\}$.

##  5 Results

Participant submissions were evaluated on SYNS-Patches [1, 78]. As previously
mentioned, this paper only discusses submissions that outperformed the
baseline in any pointcloud-/image-based metric across the Overall dataset.
Since both challenge phases ran independently and participants were
responsible for generating the predictions, we cannot guarantee that the
testing/validation metrics used the same model. We therefore report results
only for the test split. All methods were median aligned w.r.t. the ground-
truth, regardless of the supervision used. This ensures that the evaluations
are identical and comparisons are fair.

!t]resultsres:results

!t]depthres:depth_viz

###  5.1 Quantitative Results

Table LABEL:tbl:res:results shows the overall performance for each submission
across the whole dataset, as well as each category. Each subset is ordered
using F-Score performance. We additionally show the ranking order based on
Overall F-Score for ease of comparison across categories.

The Overall top F-Score and AbsRel were obtained by Team DJI&ZJU , supervised
using ground-truth depths from a collection of 10 datasets. This represents a
relative improvement of 27.62% in F-Score (13.72% – Baseline) and 18% in
AbsRel (29.66% – OPDAI) w.r.t. the first edition of the challenge [77]. The
top-performing self-supervised method was Team imec-IDLab-UAntwerp , which
leveraged improved pretrained encoders and deformable decoder convolutions.
This submission provided relative improvements of 16.61% F-Score and 4.04%
AbsRel over the first edition.

As expected, supervised approaches using ground-truth depth generally
outperformed self-supervised approaches based on the photometric error.
However, it is interesting to note that supervising a model with only
automotive data (_e.g_. Team DepthSquad , trained on KEZ) was not sufficient
to guarantee generalization to other scene types. Meanwhile, as discussed in
[78], improving the pretrained backbone (Teams imec-IDLab-UAntwerp & GMD ) is
one of the most reliable ways of increasing performance. Alternative
contributions, such as training with proxy depths (MonoViTeam ) or ensembling
different architectures (USTC-IAT-United ), can improve traditional image-
based results but typically result in slightly inferior reconstructions.

The top submission (DJI&ZJU ) consistently outperformed the other submissions
across each scene category, demonstrating good generalization capabilities.
However, Teams Pokemon & cv-challenge provided slightly better pointcloud
reconstructions in Natural scenes. We theorize this might be due to the use of
additional outdoor datasets, while DJI&ZJU primarily relies on automotive
data. It is further interesting to note that self-supervised approaches such
as Teams imec-IDLab-UAntwerp & GMD outperformed even some supervised methods
in Urban reconstructions, despite training only on Kitti.

Finally, supervised methods provided the largest improvement in Indoor scenes,
since self-supervised approaches were limited to urban driving datasets.
DJI&ZJU relied on Taskonomy and DIML, Pokemon on ScanNet, SceneNet, NYUD-v2
and more and cv-challenge made use of ZoeDepth [9] pretrained on the DPT
dataset collection [66]. This demonstrates the need for more varied training
data in order to generalize across multiple scene types.

###  5.2 Qualitative Results

Figure LABEL:fig:res:depth_viz shows visualizations for each submission’s
predictions across varied scene categories. Generally, all approaches struggle
with thin structures, such as the railings in images two and five or the
branches in image four. Models vary between ignoring these thin objects
(Baseline), treating them as solid objects (USTC-IAT-United ) and producing
inconsistent estimates (cv-challenge ). Self-supervised methods are more
sensitive to image artifacts (_e.g_. saturation or lens flare in images one
and three) due to their reliance on the photometric loss. Meanwhile,
supervised methods can be trained to be robust to the artifacts as long as the
ground-truth is correct.

Object boundaries still present challenging regions, as demonstrated by the
halos produced by most approaches. Even Team DJI&ZJU , while reducing the
intensity of these halos, can sometimes produce over-pixelated boundaries.
However, it is worth pointing out that many submissions significantly improve
over the Baseline predictions [78]. In particular, Teams cv-challenge , imec-
IDLab-UAntwerp & GMD show much greater levels of detail in Urban and
Agricultural scenes, reflected by the improved Edge-Completion metric in Table
LABEL:tbl:res:results. This is particularly impressive given the self-
supervised nature of some of these submissions.

Unfortunately, self-supervised approaches show significantly inferior
performance in Indoor settings, as they lack the data diversity to generalize.
This can be seen by the fact that many self-supervised approaches produce
incorrect scene geometry and instead predict ground-planes akin to outdoors
scenes.

Images six, thirteen and sixteen highlight some interesting complications for
monocular depth estimation. Transparent surfaces, such as the glass, are not
captured when using LiDAR or photometric constraints. As such, most approaches
ignore them and instead predict the depth for the objects behind them.
However, as humans, we know that these represent solid surfaces and obstacles
that cannot be traversed. It is unclear how an accurate supervision signal
could be generated for these cases. This calls for more flexible depth
estimation algorithms, perhaps relying on multi-modal distributions and
discrete volumes.

##  6 Conclusions & Future Work

This paper has summarized the results for the second edition of MDEC. Most
submissions provided significant improvements over the challenge baseline.
Supervised submissions typically focused on increasing the data diversity
during training, while self-supervised submissions improved the network
architecture.

As expected, there is still a performance gap between these two styles of
supervision. This is particularly the case in Indoor environments. This
motivates the need for additional data sources to train self-supervised
models, which are currently only trained on automotive data. Furthermore,
accurate depth boundary prediction is still a highly challenging problem. Most
methods frequently predicted “halos”, representative of interpolation
artifacts between the foreground and background.

Future challenge editions may introduce additional tracks for metric _vs_.
relative depth prediction, as predicting metric depth is even more
challenging. We hope this competition will continue to bring researchers into
this field and strongly encourage any interested parties to participate in
future editions of the challenge.

### Acknowledgments

This work was partially funded by the EPSRC under grant agreements
EP/S016317/1, EP/S016368/1, EP/S016260/1, EP/S035761/1.

## References

  * [1] Wendy J Adams, James H Elder, Erich W Graf, Julian Leyland, Arthur J Lugtigheid, and Alexander Muryy.  The Southampton-York Natural Scenes (SYNS) dataset: Statistics of surface attitude.  Scientific Reports, 6(1):35805, 2016.
  * [2] Ashutosh Agarwal and Chetan Arora.  Attention attention everywhere: Monocular depth prediction with skip attention.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5861–5870, 2023.
  * [3] Manuel López Antequera, Pau Gargallo, Markus Hofinger, Samuel Rota Bulò, Yubin Kuang, and Peter Kontschieder.  Mapillary planet-scale depth dataset.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 589–604. Springer, 2020.
  * [4] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei.  BEit: BERT pre-training of image transformers.  In International Conference on Learning Representations, 2022.
  * [5] Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, et al.  Arkitscenes–a diverse real-world dataset for 3d indoor scene understanding using mobile rgb-d data.  arXiv preprint arXiv:2111.08897, 2021.
  * [6] Zuria Bauer, Francisco Gomez-Donoso, Edmanuel Cruz, Sergio Orts-Escolano, and Miguel Cazorla.  Uasol, a large-scale high-resolution outdoor stereo dataset.  Scientific data, 6(1):162, 2019.
  * [7] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka.  Adabins: Depth estimation using adaptive bins.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021.
  * [8] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka.  Localbins: Improving depth estimation by learning local distributions.  In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part I, pages 480–496. Springer, 2022.
  * [9] Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller.  Zoedepth: Zero-shot transfer by combining relative and metric depth.  arXiv preprint arXiv:2302.12288, 2023.
  * [10] Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black.  A naturalistic open source movie for optical flow evaluation.  In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12, pages 611–625. Springer, 2012.
  * [11] Yohann Cabon, Naila Murray, and Martin Humenberger.  Virtual kitti 2.  arXiv preprint arXiv:2001.10773, 2020.
  * [12] Vincent Casser, Soeren Pirk, Reza Mahjourian, and Anelia Angelova.  Depth prediction without the sensors: Leveraging structure for unsupervised learning from monocular videos.  In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8001–8008, 2019.
  * [13] Ming-Fang Chang, John Lambert, Patsorn Sangkloy, Jagjeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, and James Hays.  Argoverse: 3d tracking and forecasting with rich maps.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [14] Weifeng Chen, Zhao Fu, Dawei Yang, and Jia Deng.  Single-image depth perception in the wild.  Advances in neural information processing systems, 29, 2016.
  * [15] Zeyu Cheng, Yi Zhang, and Chengkai Tang.  Swin-depth: Using transformers and multi-scale fusion for monocular-based depth estimation.  IEEE Sensors Journal, 21(23):26912–26920, 2021.
  * [16] Jaehoon Cho, Dongbo Min, Youngjung Kim, and Kwanghoon Sohn.  Diml/cvl rgb-d dataset: 2m rgb-d images of natural indoor and outdoor scenes.  arXiv preprint arXiv:2110.11590, 2021.
  * [17] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.  The cityscapes dataset for semantic urban scene understanding.  In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  * [18] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner.  Scannet: Richly-annotated 3d reconstructions of indoor scenes.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017.
  * [19] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei.  Deformable convolutional networks.  In Proceedings of the IEEE international conference on computer vision, pages 764–773, 2017.
  * [20] Qi Dai, Vaishakh Patil, Simon Hecker, Dengxin Dai, Luc Van Gool, and Konrad Schindler.  Self-supervised object motion and depth estimation from video.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
  * [21] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [22] David Eigen and Rob Fergus.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture.  In International Conference on Computer Vision, pages 2650–2658, 2015.
  * [23] Jose M Facil, Benjamin Ummenhofer, Huizhong Zhou, Luis Montesano, Thomas Brox, and Javier Civera.  Cam-convs: Camera-aware multi-scale convolutions for single-view depth.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11826–11835, 2019.
  * [24] Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao.  Deep ordinal regression network for monocular depth estimation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2002–2011, 2018.
  * [25] Ravi Garg, Vijay Kumar, Gustavo Carneiro, and Ian Reid.  Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue.  In European Conference on Computer Vision, pages 740–756, 2016\.
  * [26] Mathias Gehrig, Willem Aarents, Daniel Gehrig, and Davide Scaramuzza.  Dsec: A stereo event camera dataset for driving scenarios.  IEEE Robotics and Automation Letters, 6(3):4947–4954, 2021.
  * [27] A Geiger, P Lenz, C Stiller, and R Urtasun.  Vision meets robotics: The KITTI dataset.  International Journal of Robotics Research, 32(11):1231–1237, 2013\.
  * [28] Clement Godard, Oisin Mac Aodha, and Gabriel J. Brostow.  Unsupervised Monocular Depth Estimation with Left-Right Consistency.  Conference on Computer Vision and Pattern Recognition, pages 6602–6611, 2017.
  * [29] Clement Godard, Oisin Mac Aodha, Michael Firman, and Gabriel Brostow.  Digging Into Self-Supervised Monocular Depth Estimation.  International Conference on Computer Vision, 2019-Octob:3827–3837, 2019.
  * [30] Juan Luis Gonzalez Bello and Munchurl Kim.  PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation with Neural Positional Encoding and Distilled Matting Loss.  In Conference on Computer Vision and Pattern Recognition, pages 6847–6856, 2021.
  * [31] Ariel Gordon, Hanhan Li, Rico Jonschkowski, and Anelia Angelova.  Depth from videos in the wild: Unsupervised monocular depth learning from unknown cameras.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8977–8986, 2019.
  * [32] Vitor Guizilini, Ambrus Ambrus, Sudeep Pillai, Allan Raventos, and Adrien Gaidon.  3D packing for self-supervised monocular depth estimation.  Conference on Computer Vision and Pattern Recognition, pages 2482–2491, 2020.
  * [33] Akhil Gurram, Ahmet Faruk Tuna, Fengyi Shen, Onay Urfalioglu, and Antonio M López.  Monocular depth estimation through virtual-world supervision and real-world sfm self-supervision.  IEEE Transactions on Intelligent Transportation Systems, 23(8):12738–12751, 2021.
  * [34] Ankur Handa, Viorica Pătrăucean, Simon Stent, and Roberto Cipolla.  Scenenet: An annotated model generator for indoor scene understanding.  In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages 5737–5743. IEEE, 2016.
  * [35] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
  * [36] John Houston, Guido Zuidhof, Luca Bergamini, Yawei Ye, Long Chen, Ashesh Jain, Sammy Omari, Vladimir Iglovikov, and Peter Ondruska.  One thousand and one hours: Self-driving motion prediction dataset.  In Conference on Robot Learning, pages 409–418. PMLR, 2021.
  * [37] Yiwen Hua, Puneet Kohli, Pritish Uplavikar, Anand Ravi, Saravana Gunaseelan, Jason Orozco, and Edward Li.  Holopix50k: A large-scale in-the-wild stereo image dataset.  arXiv preprint arXiv:2003.11172, 2020.
  * [38] Xinyu Huang, Peng Wang, Xinjing Cheng, Dingfu Zhou, Qichuan Geng, and Ruigang Yang.  The apolloscape open dataset for autonomous driving and its application.  IEEE transactions on pattern analysis and machine intelligence, 42(10):2702–2719, 2019.
  * [39] Braden Hurl, Krzysztof Czarnecki, and Steven Waslander.  Precise synthetic image and lidar (presil) dataset for autonomous vehicle perception.  In 2019 IEEE Intelligent Vehicles Symposium (IV), pages 2522–2529. IEEE, 2019.
  * [40] Andrey Ignatov, Grigory Malivenko, David Plowman, Samarth Shukla, and Radu Timofte.  Fast and accurate single-image depth estimation on mobile devices, mobile ai 2021 challenge: Report.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2545–2557, 2021.
  * [41] Yasunori Ishii and Takayoshi Yamashita.  Cutdepth: Edge-aware data augmentation in depth estimation.  arXiv preprint arXiv:2107.07684, 2021.
  * [42] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu.  Spatial Transformer Networks.  In Advances in Neural Information Processing Systems, volume 28, 2015.
  * [43] Adrian Johnston and Gustavo Carneiro.  Self-Supervised Monocular Trained Depth Estimation Using Self-Attention and Discrete Disparity Volume.  In Conference on Computer Vision and Pattern Recognition, pages 4755–4764, 2020.
  * [44] Alex Kendall and Yarin Gal.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?  In Advances in Neural Information Processing Systems, volume 30, 2017.
  * [45] Maria Klodt and Andrea Vedaldi.  Supervising the New with the Old: Learning SFM from SFM.  In European Conference on Computer Vision, pages 713–728, 2018\.
  * [46] Tobias Koch, Lukas Liebel, Friedrich Fraundorfer, and Marco Körner.  Evaluation of CNN-Based Single-Image Depth Estimation Methods.  In European Conference on Computer Vision Workshops, pages 331–348, 2018.
  * [47] Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab.  Deeper depth prediction with fully convolutional residual networks.  International Conference on 3D Vision, pages 239–248, 2016.
  * [48] Youngwan Lee, Jonghee Kim, Jeffrey Willette, and Sung Ju Hwang.  Mpvit: Multi-path vision transformer for dense prediction.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7277–7286, 2022.
  * [49] Ruibo Li, Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, and Lingxiao Hang.  Deep attention-based classification network for robust depth prediction.  In Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part IV 14, pages 663–678. Springer, 2019.
  * [50] Zhengqi Li, Tali Dekel, Forrester Cole, Richard Tucker, Noah Snavely, Ce Liu, and William T Freeman.  Mannequinchallenge: Learning the depths of moving people by watching frozen people.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(12):4229–4241, 2020.
  * [51] Zhengqi Li and Noah Snavely.  Megadepth: Learning single-view depth prediction from internet photos.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2041–2050, 2018.
  * [52] Lahav Lipson, Zachary Teed, and Jia Deng.  Raft-stereo: Multilevel recurrent field transforms for stereo matching.  In 2021 International Conference on 3D Vision (3DV), pages 218–227. IEEE, 2021.
  * [53] Fayao Liu, Chunhua Shen, and Guosheng Lin.  Deep convolutional neural fields for depth estimation from a single image.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5162–5170, 2015.
  * [54] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al.  Swin transformer v2: Scaling up capacity and resolution.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12009–12019, 2022.
  * [55] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  * [56] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11976–11986, 2022.
  * [57] Chenxu Luo, Zhenheng Yang, Peng Wang, Yang Wang, Wei Xu, Ram Nevatia, and Alan Yuille.  Every pixel counts ++: Joint learning of geometry and motion with 3d holistic understanding.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10):2624–2641, 2020.
  * [58] Xiaoyang Lyu, Liang Liu, Mengmeng Wang, Xin Kong, Lina Liu, Yong Liu, Xinxin Chen, and Yi Yuan.  HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation.  AAAI Conference on Artificial Intelligence, 35(3):2294–2301, 2021\.
  * [59] Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox.  A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation.  Conference on Computer Vision and Pattern Recognition, pages 4040–4048, 2016.
  * [60] S Mahdi H Miangoleh, Sebastian Dille, Long Mai, Sylvain Paris, and Yagiz Aksoy.  Boosting monocular depth estimation models to high-resolution via content-adaptive multi-resolution merging.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9685–9694, 2021.
  * [61] Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus.  Indoor segmentation and support inference from rgbd images.  In ECCV, 2012.
  * [62] Evin Pinar Örnek, Shristi Mudgal, Johanna Wald, Yida Wang, Nassir Navab, and Federico Tombari.  From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction.  arXiv preprint, 2022.
  * [63] Adrien Pavao, Isabelle Guyon, Anne-Catherine Letournel, Xavier Baró, Hugo Escalante, Sergio Escalera, Tyler Thomas, and Zhen Xu.  Codalab competitions: An open source platform to organize scientific challenges.  Technical report, 2022.
  * [64] Sudeep Pillai, Rareş Ambruş, and Adrien Gaidon.  SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation.  In International Conference on Robotics and Automation, pages 9250–9256, 2019.
  * [65] Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mattoccia.  On the Uncertainty of Self-Supervised Monocular Depth Estimation.  In Conference on Computer Vision and Pattern Recognition, pages 3224–3234, 2020.
  * [66] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.  Vision transformers for dense prediction.  In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12179–12188, October 2021.
  * [67] René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun.  Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.  IEEE transactions on pattern analysis and machine intelligence, 2020\.
  * [68] Anurag Ranjan, Varun Jampani, Lukas Balles, Kihwan Kim, Deqing Sun, Jonas Wulff, and Michael J Black.  Competitive collaboration: Joint unsupervised learning of depth, camera motion, optical flow and motion segmentation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12240–12249, 2019.
  * [69] Haoyu Ren, Mostafa El-Khamy, and Jungwon Lee.  Deep robust single image depth estimation neural network using scene understanding.  In CVPR Workshops, volume 2, 2019.
  * [70] Stephan R Richter, Zeeshan Hayder, and Vladlen Koltun.  Playing for benchmarks.  In Proceedings of the IEEE International Conference on Computer Vision, pages 2213–2222, 2017.
  * [71] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind.  Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 10912–10922, 2021.
  * [72] Anirban Roy and Sinisa Todorovic.  Monocular depth estimation using neural regression forest.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5506–5514, 2016.
  * [73] Rui, Stückler Jörg, Cremers Daniel Yang Nan, and Wang.  Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry.  In European Conference on Computer Vision, pages 835–852, 2018\.
  * [74] Chang Shu, Kun Yu, Zhixiang Duan, and Kuiyuan Yang.  Feature-Metric Loss for Self-supervised Learning of Depth and Egomotion.  In European Conference on Computer Vision, pages 572–588, 2020\.
  * [75] Dalwinder Singh and Birmohan Singh.  Investigating the impact of data normalization on classification performance.  Applied Soft Computing, 97:105524, 2020.
  * [76] Jaime Spencer, Richard Bowden, and Simon Hadfield.  DeFeat-Net: General monocular depth via simultaneous unsupervised representation learning.  In Conference on Computer Vision and Pattern Recognition, pages 14390–14401, 2020.
  * [77] Jaime Spencer, C Stella Qian, Chris Russell, Simon Hadfield, Erich Graf, Wendy Adams, Andrew J Schofield, James H Elder, Richard Bowden, Heng Cong, et al.  The monocular depth estimation challenge.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 623–632, 2023.
  * [78] Jaime Spencer, Chris Russell, Simon Hadfield, and Richard Bowden.  Deconstructing self-supervised monocular reconstruction: The design decisions that matter.  Transactions on Machine Learning Research, 2022.  Reproducibility Certification.
  * [79] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers.  A benchmark for the evaluation of rgb-d slam systems.  In Proc. of the International Conference on Intelligent Robot Systems (IROS), Oct. 2012.
  * [80] Mingxing Tan and Quoc Le.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks.  In International Conference on Machine Learning, volume 97, pages 6105–6114, 2019.
  * [81] Jonas Uhrig, Nick Schneider, Lukas Schneider, Uwe Franke, Thomas Brox, and Andreas Geiger.  Sparsity Invariant CNNs.  International Conference on 3D Vision, pages 11–20, 2018.
  * [82] Benjamin Ummenhofer, Huizhong Zhou, Jonas Uhrig, Nikolaus Mayer, Eddy Ilg, Alexey Dosovitskiy, and Thomas Brox.  Demon: Depth and motion network for learning monocular stereo.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5038–5047, 2017.
  * [83] Igor Vasiljevic, Nick Kolkin, Shanyi Zhang, Ruotian Luo, Haochen Wang, Falcon Z Dai, Andrea F Daniele, Mohammadreza Mostajabi, Steven Basart, Matthew R Walter, et al.  Diode: A dense indoor and outdoor depth dataset.  arXiv preprint arXiv:1908.00463, 2019.
  * [84] Chaoyang Wang, Simon Lucey, Federico Perazzi, and Oliver Wang.  Web stereo video supervision for depth prediction from dynamic scenes.  In 2019 International Conference on 3D Vision (3DV), pages 348–357. IEEE, 2019.
  * [85] Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer.  Tartanair: A dataset to push the limits of visual slam.  In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4909–4916. IEEE, 2020.
  * [86] Jamie Watson, Michael Firman, Gabriel Brostow, and Daniyar Turmukhambetov.  Self-supervised monocular depth hints.  International Conference on Computer Vision, 2019-Octob:2162–2171, 2019.
  * [87] Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie.  Convnext v2: Co-designing and scaling convnets with masked autoencoders.  arXiv preprint arXiv:2301.00808, 2023.
  * [88] Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo.  Monocular relative depth perception with web stereo data supervision.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 311–320, 2018.
  * [89] Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, and Zhiguo Cao.  Structure-guided ranking loss for single image depth prediction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 611–620, 2020.
  * [90] Pengchuan Xiao, Zhenlei Shao, Steven Hao, Zishuo Zhang, Xiaolin Chai, Judy Jiao, Zesong Li, Jian Wu, Kai Sun, Kun Jiang, et al.  Pandaset: Advanced sensor suite dataset for autonomous driving.  In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 3095–3101. IEEE, 2021.
  * [91] Jiaxing Yan, Hong Zhao, Penghui Bu, and YuSheng Jin.  Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation.  In International Conference on 3D Vision, pages 464–473, 2021.
  * [92] Guorun Yang, Xiao Song, Chaoqin Huang, Zhidong Deng, Jianping Shi, and Bolei Zhou.  Drivingstereo: A large-scale dataset for stereo matching in autonomous driving scenarios.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 899–908, 2019.
  * [93] Nan Yang, Lukas von Stumberg, Rui Wang, and Daniel Cremers.  D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry.  In Conference on Computer Vision and Pattern Recognition, pages 1278–1289, 2020.
  * [94] Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang, Yufan Ren, Lei Zhou, Tian Fang, and Long Quan.  Blendedmvs: A large-scale dataset for generalized multi-view stereo networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1790–1799, 2020.
  * [95] Wei Yin, Yifan Liu, and Chunhua Shen.  Virtual normal: Enforcing geometric constraints for accurate and robust depth prediction.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10):7282–7295, 2021.
  * [96] Wei Yin, Xinlong Wang, Chunhua Shen, Yifan Liu, Zhi Tian, Songcen Xu, Changming Sun, and Dou Renyin.  Diversedepth: Affine-invariant depth prediction using diverse data.  arXiv preprint arXiv:2002.00569, 2020.
  * [97] Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen.  Learning to recover 3d scene shape from a single image.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 204–213, 2021.
  * [98] Zhichao Yin and Jianping Shi.  Geonet: Unsupervised learning of dense depth, optical flow and camera pose.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1983–1992, 2018.
  * [99] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell.  Bdd100k: A diverse driving dataset for heterogeneous multitask learning.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2636–2645, 2020.
  * [100] Weihao Yuan, Xiaodong Gu, Zuozhuo Dai, Siyu Zhu, and Ping Tan.  Neural window fully-connected crfs for monocular depth estimation.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3906–3915, 2022.
  * [101] Weihao Yuan, Yazhan Zhang, Bingkun Wu, Siyu Zhu, Ping Tan, Michael Yu Wang, and Qifeng Chen.  Stereo matching by self-supervision of multiscopic vision.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5702–5709. IEEE, 2021.
  * [102] Pierluigi Zama Ramirez, Tosi Fabio, Luigi Di Stefano, Radu Timofte, Alex Costanzino, Matteo Poggi, Samuele Salti, Stefano Mattoccia, Jun Shi, Dafeng Zhang, Yong A, Yixiang Jin, Dingzhe Li, Chao Li, Zhiwen Liu, Qi Zhang, Yixing Wang, and Shi Yin.  NTIRE 2023 challenge on hr depth from images of specular and transparent surfaces.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.
  * [103] Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese.  Taskonomy: Disentangling task transfer learning.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3712–3722, 2018.
  * [104] Oliver Zendel, Angela Dai, Xavier Puig Fernandez, Andreas Geiger, Vladlen Koltun, Peter Kontschieder, Adam Kortylewski, Alina Kuznetsova, Tsung-Yi Lin, Torsten Sattler, Daniel Scharstein, Hendrik Schilling, Jonas Uhrig, and Jonas Wulff.  Robust Vision Challenge 2022 — robustvision.net.  http://robustvision.net/index.php.
  * [105] Huangying Zhan, Ravi Garg, Chamara Saroj Weerasekera, Kejie Li, Harsh Agarwal, and Ian M. Reid.  Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction.  Conference on Computer Vision and Pattern Recognition, pages 340–349, 2018.
  * [106] Chaoqiang Zhao, Youmin Zhang, Matteo Poggi, Fabio Tosi, Xianda Guo, Zheng Zhu, Guan Huang, Yang Tang, and Stefano Mattoccia.  Monovit: Self-supervised monocular depth estimation with a vision transformer.  International Conference on 3D Vision, 2022.
  * [107] Hang Zhou, David Greenwood, and Sarah Taylor.  Self-Supervised Monocular Depth Estimation with Internal Feature Fusion.  In British Machine Vision Conference, 2021.
  * [108] Tinghui Zhou, Matthew Brown, Noah Snavely, and David G. Lowe.  Unsupervised Learning of Depth and Ego-Motion from Video.  Conference on Computer Vision and Pattern Recognition, pages 6612–6619, 2017.

  *[MDEC]: Monocular Depth Estimation Challenge
  *[LiDAR]: Light Detection and Ranging
  *[SotA]: State-of-the-Art
  *[MDE]: monocular depth estimation
  *[KEZ]: Kitti Eigen-Zhou
  *[KEB]: Kitti Eigen-Benchmark
  *[KE]: Kitti Eigen