# Augment and Criticize: Exploring Informative Samples for Semi-Supervised
Monocular 3D Object Detection

Zhenyu Li1, Zhipeng Zhang2, Heng Fan3, Yuan He2, Ke Wang2, Xianming Liu1,
Junjun Jiang1
1Harbin Institute of Technology, 2DiDi Chuxing, 3University of North Texas
This work was done when Zhenyu was an intern at DiDi.Corresponding author
(jiangjunjun@hit.edu.cn).

###### Abstract

In this paper, we improve the challenging monocular 3D object detection
problem with a general semi-supervised framework. Specifically, having
observed that the bottleneck of this task lies in lacking reliable and
informative samples to train the detector, we introduce a novel, simple, yet
effective ‘Augment and Criticize’ framework that explores abundant informative
samples from unlabeled data for learning more robust detection models. In the
‘Augment’ stage, we present the Augmentation-based Prediction aGgregation
(APG), which aggregates detections from various automatically learned
augmented views to improve the robustness of pseudo label generation. Since
not all pseudo labels from APG are beneficially informative, the subsequent
‘Criticize’ phase is presented. In particular, we introduce the Critical
Retraining Strategy (CRS) that, unlike simply filtering pseudo labels using a
fixed threshold (e.g., classification score) as in 2D semi-supervised tasks,
leverages a learnable network to evaluate the contribution of unlabeled images
at different training timestamps. This way, the noisy samples prohibitive to
model evolution could be effectively suppressed. To validate our framework, we
apply it to MonoDLE [36] and MonoFlex [62]. The two new detectors, dubbed
3DSeMo${}_{\text{DLE}}$ and 3DSeMo${}_{\text{FLEX}}$, achieve state-of-the-art
results with remarkable improvements for over 3.5% $AP_{3D/BEV}(Easy)$ on
KITTI, showing its effectiveness and generality. Code and models will be
released.

##  1 Introduction

Figure 1: Motivation and Proposal. The differences between our method (red)
and previous semi-supervised learning framework (green) in pseudo label (PL)
generation and student model retraining. The introduced framework can improve
detection recall by observing different views of an image (red dots in (a)),
and dynamically determine when to discard an unlabeled sample during training
(line-chart in (b)) by the learnable critical module.

Monocular 3D (Mono3D) object detection is an essential and economical way for
intelligent agents to perceive 3D information in the real world. However,
since the depth information is inevitably lost during the 3D-to-2D projection
process [16], identifying 3D objects under pure monocular premise is ill-posed
in the first place. Fortunately, the recent triumphs in deep learning shed
light on circumventing the complex mathematical optimization [38] and tackling
this challenging problem in a data-driven manner [32, 36, 61, 62, 53]. Despite
of the improvements, state-of-the-art Mono3D models still lag far behind human
perception capabilities. On the contrary, deep learning has surpassed human-
level performance on image classification tasks [20]. One possible reason
behind such a performance gap is the notable dataset volume differences: for
example, Mono3D models are trained with 3.7K images from KITTI [18], while
classification models are trained with 1.2 million images from ImageNet [14].

Why not just simply scale up the dataset volume then? It turns out that 3D
annotations are more expensive than 2D ones. High-quality Mono3D datasets need
LiDARs for 3D ground truth. Data collection needs carefully calibrated and
synchronized different sensors. Taking this into consideration, we believe a
data-driven but annotation-efficient approach is necessary to bring the
performances of modern Mono3D models to the next level.

Naturally, semi-supervised learning, which can absorb knowledge from limited
annotated samples meanwhile exploit beneficial information from the enormous
unlabeled data, becomes a reasonable choice for this problem. Surprisingly, it
has been rarely explored in monocular 3D detection, despite its success in
numerous 2D vision tasks [48, 4, 46, 55, 56, 64, 60, 58, 31, 59]. We suspect
that the noise in the low-quality pseudo labels during training eventually
overwhelms the benefits brought by abundant extra data. Thus, how to
effectively find reliable and informative samples from unlabeled data, is
crucial to apply semi-supervised learning in Mono3D tasks. In particular, two
challenges are faced: _How to robustly generate high-quality pseudo labels
from unlabeled data_ and _How to properly leverage these pseudo labels for
effective learning_.

For the first challenge, prior works in 2D tasks [56, 8] alleviate the pseudo
label quality issue by handpicking a threshold to filter putative detection
boxes from unlabeled data (see Fig. 1). Such strategy might not work well in
the Mono3D setting. Firstly, generating precise pseudo labels from images
alone is already very difficult (notice the gap between vision-only and multi-
modal 3D detection results on public benchmarks); secondly, using a
handcrafted selection strategy may overfit to a specific model, reducing the
resilience of semi-supervised models. Thus, _a robust pseudo label generation
strategy is needed to tackle the first challenge._ To this end, we introduce a
simple yet effective Augmentation-based Prediction aGgregation strategy,
dubbed APG, that aims at robustly generating pseudo labels for unlabeled data.
The core idea is to aggregate predictions from different observations of an
image, which we find effectively reduces the detection biases and improves the
robustness of pseudo label generation (see Fig. 1).

For the second challenge, since not all the pseudo labels are beneficially
informative (even for the proposed APG), classification scores are usually
used in 2D tasks [8] to adapt the pseudo detection boxes for training.
However, it ignores that the contribution of each sample during model training
should vary across training iterations and the model’s parameters. Therefore,
_a more adaptive mechanism is needed to guide the training on unlabeled data._
For that, we propose a ‘Criticize’ module for the second challenge. More
specifically, in this stage, a Critical Retraining Strategy (CRS) is imposed
to adaptively update the model with noisy pseudo labels. In particular, CRS
contains a memory bank to preserve evaluation images and a criticize module to
determine which pseudo label benefits more when updating the model. Given a
pseudo label, the optimization loss of the model roughly indicates the
benefits this particular pseudo label would bring, should this pseudo label be
used for back-propagation. The ‘Criticize’ module discards the pseudo label if
the model would be updated towards unsatisfactory directions. As training
proceeds, the Criticize module can better adapt to the model optimization
progress through cyclical updating of the memory bank.

To summarize, we propose a novel ‘Augment and Criticize’ framework to approach
the two challenges in semi-supervised Mono3D object detection. Results on
different methods and benchmarks show remarkable improvement. Our
contributions are summarized as follow,

(1) _We propose a novel ‘Augment and Criticize’ framework for semi-supervised
Mono3D object detection._

(2) _We propose an augmented-based prediction aggregation to improve the
quality of pseudo labels for unlabeled data._

(3) _We propose a critical retraining strategy that adaptively evaluates each
pseudo label for effective model training._

(4) _We integrate our semi-supervised framework into different methods, and
results evidence its effectiveness._

##  2 Related Work

Figure 2: ‘Augment and Criticize’ Framework. We present the three major steps
in our semi-supervised scheme, including standard supervised training, pseudo
label generation with APG, and retraining with CRS.

###  2.1 Monocular 3D Object Detection

3D object detection is a fundamental task for agents to perceive the
surrounding 3D world [2]. Benefiting from the ubiquitous availabilities of
cameras, Mono3D methods have great potential for wide real-world deployment,
thus have received extensive attention recently [6, 15, 62, 44, 36, 34, 22].
Earlier attempts devoted massive efforts to the ill-posed depth estimation
problem. By adopting an isolated depth model [54] to generate pseudo point
cloud or lifting 2D features to 3D space [42], 3D detectors can be applied on
such pseudo point cloud or 3D features for object identification. Despite
promising results, the hefty computation overhead entailed by dense depth
estimation prohibits such methods from practical applications. By moving depth
estimation into an auxiliary head, [32, 36, 62, 53] enabled end-to-end model
training with a neater framework. These methods simultaneously predict object
centers in the 2D images and their corresponding depth in the 3D space. Object
3D locations and dimensions can then be easily recovered with camera
calibration. Representative methods like SMOKE [32] and MonoDLE [36] adopt
CenterNet-like architectures [65], whereas FCOS3D [53] and PGD [52] extend the
2D FCOS detector [49] into a 3D detection model. In this paper, we aim to
design a general semi-supervised framework, which is agnostic to specific
model designs, to push the envelope of modern Mono3D object detectors.

###  2.2 Semi-Supervised Learning

Semi-Supervised Learning (SSL) is attractive because of its capability to
further unveil the power of machine learning with abundant cheap unlabeled
data [50, 50, 39, 25, 4, 46, 40]. Due to the space limitation, this section
only reviews self-training-based methods, which is one of the most engaging
directions in SSL [37, 43]. In general, self-training-based semi-supervised
learning methods first train a teacher model with a small set of human-
annotated data. The teacher model then generates pseudo labels on a much
larger set of unlabeled data. Finally, a student model is trained with both
human-labeled and self-annotated data. Such a paradigm has demonstrated great
success in image classification [48, 4, 46, 55], semantic segmentation [58,
31, 59], and 2D object detection [56, 64, 60]. While different applications
usually require additional bells and whistles, the core components of semi-
supervised learning remain unchanged: how to generate high-quality pseudo-
label, and how to retrain student models effectively. Mean-Teacher [48]
proposes temporal ensembling to facilitate retraining. Soft-Teacher [56]
utilizes the classification score to reweight loss and imposed 2D box jitter
to filter unreliable pseudo labels. ST++ [58] adopts strong augmentations on
the unlabeled samples and leverages evolving stability during training to
prioritize high-quality labels. Compared with well-studied 2D tasks, it is
much more challenging for Mono3D detection to collect reliable pseudo labels.
Although such issue can be alleviated by introducing multi-view consistency
[28], compared with abundant single-view datasets, high-quality stereo or
multi-view datasets are much harder to collect (device-wise multi-view).
Besides, learning consistency among video frames is vulnerable to moving
objects (temporal-wise multi-view). Thus we believe, it is the single-view
scenario that semi-supervised learning can make the most impact. In this
paper, we focus on the design of effective semi-supervised learning frameworks
for Mono3D object detection.

##  3 Method

Our proposed semi-supervised framework (Fig. 2) is detailed in this section.
We first recap the definition of semi-supervised Mono3D object detection task
and introduce the vanilla self-training scheme in Sec. 3.1. The augmentation-
based prediction aggregation (APG) for robustly generating high-quality pseudo
labels is described in Sec. 3.2, and the critical retraining strategy (CRS)
for adaptively learning from unlabeled data is described in Sec. 3.3.

###  3.1 Preliminary

Task Definition. Given an image sample $x$ in the labeled dataset, its ground
truth label $y$ contains information about the category, location, dimension,
and orientation of objects visible in $x$. Semi-supervised Mono3D object
detection aims to acquire knowledge from both precisely annotated dataset
$\mathcal{D}_{l}=\\{x_{i}^{l},y_{i}^{l}\\}_{i=1}^{N_{l}}$ and unlabeled
dataset $\mathcal{D}_{u}=\\{x_{j}^{u}\\}_{j=1}^{N_{u}}$, where $N_{u}\gg
N_{l}$.

Vanilla Self-Training Scheme. As a prominent branch in semi-supervised
learning [1, 58], self-training works by iteratively optimizing a model with
the help of pseudo-labels on the unlabeled samples. A vanilla self-training
[58] pipeline contains three major steps: 1) Standard Supervised Training
which trains a teacher model $M_{t}$ on the labeled dataset $\mathcal{D}_{l}$,
2) Pseudo Label Generation which predicts pseudo labels
$\\{\hat{y}=M_{t}(x_{j})|x_{j}\in\mathcal{D}_{u}\\}$ on the unlabeled dataset
$\mathcal{D}_{u}$, and 3) Retraining with Noisy Labels which learns a student
model $M_{s}$ for final evaluation. Using $M_{s}$ as the new teacher, the step
2 and 3 can be repeated until satisfactory performance is obtained.

In this paper, we elaborately investigate the pseudo label generation (step 2)
and retraining strategy (step 3), which are the most crucial parts of the
self-training scheme. For simplicity, we don’t iteratively perform step 2 and
3.

###  3.2 Augmentation-Based Prediction Aggregation

To obtain high-quality pseudo labels, previous 2D semi-supervised learning
methods [53, 56, 27, 64, 8] resort to a suitable threshold $\tau{}$ to filter
predicted boxes. However, it is non-trivial to determine an optimal threshold
for each different method, especially in the ill-posed Mono3D object
detection. A higher threshold may bring tons of false negatives (FN),
decreasing the quantity of useful pseudo labels. In contrast, a lower
threshold may introduce more false positives (FP), resulting in adverse
noises. In order to alleviate the dependency on such a handcrafted threshold,
we propose the APG strategy to effectively aggregate predictions from
different observations of the same image sample to improve the robustness of
pseudo-label generation.

Figure 3: Illustration of APG. We aggregate the predictions from $K$ different
transformations of an unlabeled image. The by-product reliability scores
estimated by MLE is adopted in retraining to measure the uncertainty of pseudo
labels.

The proposed algorithm, that is illustrated in Alg. 1 and Fig. 3, consists of
three steps:

1) Firstly, given an input image from the unlabeled dataset
$x_{j}^{u}\in\mathcal{D}_{u}$, the teacher model $M_{t}$ predicts the
detection results for $x_{j}^{u}$ and its $K$ augmented images. Let
$\mathcal{P}_{r}$ denote the raw prediction of $x_{j}^{u}$, and
$\mathcal{P}_{f}^{0}$ and $\\{\mathcal{P}_{f}^{k}\\}_{k=1}^{K}$ represent the
post-processed (by the pre-defined threshold $\tau$) predictions of
$x_{j}^{u}$ and the augmented images, respectively.

2) Secondly, for each prediction $p_{i}$ in $\mathcal{P}_{f}^{0}$, we apply
the kNN clustering algorithm to find its nearest neighbors in
$\\{\mathcal{P}_{f}^{k}\\}_{k=1}^{K}$, that forms a cluster. $p_{i}$ is
considered as a pseudo label for $x_{j}^{u}$. Intuitively, the number of
assigned predictions $n$ in the cluster indicates the difficulty degree in
detecting an object, and the variance $\sigma$ by Maximum Likelihood
Estimation (MLE) measures the uncertainty of $p_{i}$. With the classification
score $s$, these by-products are combined by Eq. 1 to demonstrate $p_{i}$’s
reliability, which is then used to weight the loss of each unlabeled data in
retraining.

$w=\gamma_{1}\times
s+(1-\gamma_{1})\times\exp{(-\frac{\sigma}{n}*\gamma_{2})},$ (1)

We set $\gamma_{1}=0.6$ and $\gamma_{2}=6$ in our model, respectively.

3) Finally, for the unused predictions in
$\\{\mathcal{P}_{f}^{k}\\}_{k=1}^{K}$, they would be self-clustered. The
cluster centers are treated as reference points, whose closest prediction in
$\mathcal{P}_{r}$ are selected as pseudo labels. Their uncertainties are
measured by Eq. 1.

Moreover, inspired by successful attempts at auto data augmentations [13, 29],
we resort to the Tree-Structured Parzen Estimators (TPE) [3] to automatically
pick the $K$ transformations and their hyper-parameters (_e.g._ , resize
ratio). More details are presented in supplementary materials.

Input: Predictions of different observations,

Raw prediction $\mathcal{P}_{r}$ of an unlabeled image,

Filtered prediction $\mathcal{P}_{f}^{0}$,
$N=\text{len}(\mathcal{P}_{f}^{0})$,

Filtered predictions $\\{\mathcal{P}_{f}^{k}\\}_{k=1}^{K}$ of augmented
images,

Threshold $\tau$ for kNN

Output: Aggregated prediction $\mathcal{P}$

Initialize set
$\mathcal{S}\leftarrow\\{\\{p_{1}\\},\cdots,\\{p_{N}\\}\\},p_{n}\in\mathcal{P}_{f}^{0}$

for  _image observation $k\in\\{1,\dots,K\\}$_ do

for  _prediction $p_{i}\in\mathcal{P}_{f}^{k}$_ do

$(\text{index }j,\text{distance }l)$ $\leftarrow$ kNN$(p_{i},\mathcal{S})$

if  _$l <\tau$_ then

$\mathcal{S}_{j}\leftarrow\mathcal{S}_{j}\cup\\{p_{i}\\}$ # append $p_{i}$ to
clusters

else

$\mathcal{S}_{j}\leftarrow\mathcal{S}_{j}\cup\\{\\{p_{i}\\}\\}$ # create new
clusters

for  _loop index n, cluster set $\\{p^{m}\\}_{m=1}^{M}\in\mathcal{S}$ _ do

location $\mu$, variance $\sigma=$ MLE $(\\{p^{m}\\}_{j=m}^{M})$

if  _$n <N$_ then

$\mathcal{P}\leftarrow\mathcal{P}\cup\\{(\mathcal{S}_{n}[0],\sigma)\\}$

else

$\mathcal{P}\leftarrow\mathcal{P}\cup\\{(\text{NearestSearch}(\mu,\mathcal{P}_{r}),\sigma)\\}$

return _$\mathcal{P}$_

Algorithm 1 APG Aggregation Pseudocode

###  3.3 Critical Retraining Strategy

Generated pseudo labels inevitably contain noises, thus it is crucial to find
informative ones that benefit model evolution. Previous methods (_e.g_., [56])
use the box jitter scores as proxies for the pseudo label quality. However, it
might be hard to replicate the success of such strategy in the Mono 3D
detection due to the inferior performance of the teacher model $M_{t}$. The
uncertainty measurements of pseudo labels provided by the APG module can
enhance the stability of student model retraining, but it still suffers from
the fixed weight of each sample. We argue that the contribution of each sample
during model training should adapt to the model’s state as training proceeds
[23, 17].

To this end, we propose a learning-based critical module to adaptively find
the informative unlabeled data, which may provide a new perspective for semi-
supervised Mono3D object detection to retrain a better student model.

Figure 4: Illustration of CRS. We adopt a critical module to discriminate
whether a sample from the unlabeled data benefits model convergence. The
memory bank is cyclically updated.

Specifically, the critical module first evaluates the effect of a training
sample from the unlabeled dataset, and then assigns it with a 0-1 binary flag
indicating whether to back-propagate its gradients. From a reinforcement
learning perspective, we regard the Mono3D detector (student model) as an
agent, the model’s weight parameters as the state, the input image and the
output of the model as an observation. At state $\mathcal{S}$, a detection
loss $\mathcal{L}_{unsup}$ for the agent can be calculated based on the given
observation $\mathcal{O}$. If the gradients of $\mathcal{L}_{unsup}$ are back-
propagated, the state will be updated to $\mathcal{S}^{{}^{\prime}}$ and the
model output will be updated to $\mathcal{O}^{{}^{\prime}}$. The critical
module then evaluates whether $\mathcal{S}^{{}^{\prime}}$ is the optimal
choice of updated $\mathcal{S}$ based on the observations $\mathcal{O}$ and
$\mathcal{O}^{{}^{\prime}}$.

At each training step, an input image $x_{i}^{u}$ from the unlabeled data is
fed into the detector $M$ (agent), obtaining the detection predictions
$\mathbf{D}$ (classification and regression response maps),

$\mathbf{D}=M(x_{i}^{u}|\mathcal{S}),$ (2)

With the pseudo label $\hat{y}_{i}^{u}$, we can get the training loss,

$\mathcal{L}_{unsup}=\mathcal{L}(\mathbf{D},\hat{y}_{i}^{u}),$ (3)

We take one ‘trial’ gradient descent step to obtain the updated model
$M^{\prime}$ with parameters $\mathcal{S}^{{}^{\prime}}$. Then, the critical
module evaluates the effectiveness of this update
($\mathcal{S}\rightarrow\mathcal{S}^{{}^{\prime}}$),

$v=\mathcal{C}(x_{i}^{u},\mathbf{D},\mathbf{D}^{{}^{\prime}}|\Psi),$ (4)

where $\mathbf{D}^{{}^{\prime}}$ is the detection predictions of the updated
model $M^{\prime}$ on $x_{i}^{u}$, and $\Psi$ is the parameter of the critical
module. During training, we chop off a certain number of samples with the
lowest evaluation value $v$ (see Fig. 4).

To guarantee the critical module can provide reliable feedback, we propose a
reward function to supervise the training of the critical network,

$r=\mathcal{L}(M(x_{i}|\mathcal{S}),y_{i})-\mathcal{L}(M^{\prime}(x_{i}|\mathcal{S}^{{}^{\prime}}),y_{i}),$
(5)

where $(x_{i},y_{i})$ denotes samples from the training set of the labeled
dataset $\mathcal{D}_{l}$. The L2 loss [5] is applied to $v$ and $r$ for
supervising the learning of critical module. During training, we alternately
update the detector and critical module.

Notably, it’s impractical to evaluate all samples to get a reliable reward $r$
due to the unaffordable computation cost. Motivated by the self-supervised
method MoCo [10], we employ a memory bank (queue) to buffer the training
samples in $\mathcal{D}_{l}$ and cyclically update it. After tons of steps
updating, the knowledge of all samples for evaluation are encoded to the
weight parameters of the critical network, making it capable of predicting
accurate indicator.

##  4 Experiments

In this section, we first recap the experimental setup in Sec. 4.1. Then, we
respectively present the evaluation results (Sec. 4.2), ablation studies and
analysis (Sec. 4.3) to demonstrate the effectiveness of the proposed methods.

Method | Reference | Extra data | Test $AP_{3D}$ | Test $AP_{BEV}$ | Val $AP_{3D}$
---|---|---|---|---|---
Easy | Mod. | Hard | Easy | Mod. | Hard | Easy | Mod. | Hard
PatchNet [35] | ECCV2020 | Depth | 15.68 | 11.12 | 10.17 | 22.97 | 16.86 | 14.97 | - | - | -
D4LCN [15] | CVPRW2020 | 16.65 | 11.72 | 9.51 | 22.51 | 16.02 | 12.55 | - | - | -
DDMP-3D [51] | CVPR2021 | 19.71 | 12.78 | 9.80 | 28.08 | 17.89 | 13.44 | - | - | -
Kinematic3D [6] | ECCV2020 | Multi-frames | 19.07 | 12.72 | 9.17 | 26.69 | 17.52 | 13.10 | 19.76 | 14.10 | 10.47
MonoRUn [9] | CVPR2021 | LiDAR | 19.65 | 12.30 | 10.58 | 27.94 | 17.34 | 15.24 | 20.02 | 14.65 | 12.61
CaDDN [41] | CVPR2021 | 19.17 | 13.41 | 11.46 | 27.94 | 18.91 | 17.19 | 23.57 | 16.31 | 13.84
MonoDTR [22] | CVPR2022 | 21.99 | 15.39 | 12.73 | 28.59 | 20.38 | 17.14 | 24.52 | 18.57 | 15.51
AutoShape [33] | ICCV2021 | CAD | 22.47 | 14.17 | 11.36 | 30.66 | 20.08 | 15.59 | 20.09 | 14.65 | 12.07
SMOKE [32] | CVPRW2020 | None | 14.03 | 9.76 | 7.84 | 20.83 | 14.49 | 12.75 | 14.76 | 12.85 | 11.50
MonoPair [12] | CVPR2020 | 13.04 | 9.99 | 8.65 | 19.28 | 14.83 | 12.89 | 16.28 | 12.30 | 10.42
RTM3D [26] | ECCV2020 | 13.61 | 10.09 | 8.18 | - | - | - | 19.47 | 16.29 | 15.57
PGD [52] | CoRL2022 | 19.05 | 11.76 | 9.39 | 26.89 | 16.51 | 13.49 | 19.27 | 13.23 | 10.65
MonoRCNN [44] | ICCV2021 | 18.36 | 12.65 | 10.03 | 25.48 | 18.11 | 14.10 | 16.61 | 13.19 | 10.65
Zhang et al.${}_{\text{DLE}}$ [63] | CVPR2022 | 20.25 | 14.14 | 12.42 | 28.85 | 17.72 | 17.81 | 20.82 | 15.64 | 13.82
GUPNet [34] | ICCV2021 | 20.11 | 14.20 | 11.77 | - | - | - | 22.76 | 16.46 | 13.72
HomoLoss${}_{\text{FLEX}}$ [19] | CVPR2022 | 21.75 | 14.94 | 13.07 | 29.60 | 20.68 | 17.81 | 23.04 | 16.89 | 14.90
Mix-Teaching${}_{\text{FLEX}}$ [57]† | Arxiv2022 | Unlabeled | 21.88 | 14.34 | 11.86 | 30.52 | 19.51 | 16.45 | 25.54 | 19.35 | 16.51
MonoDLE [36] | CVPR2021 | None | 17.23 | 12.26 | 10.29 | 24.79 | 18.89 | 16.00 | 17.45 | 13.66 | 11.68
3DSeMo${}_{\text{DLE}}$ | Ours | Unlabeled | 23.11 | 15.58 | 13.58 | 30.99 | 21.78 | 18.64 | 22.87 | 17.65 | 14.83
Improvement | Ours | v.s. baseline | +5.88 | +3.32 | +3.29 | +6.20 | +2.89 | +2.64 | +5.42 | +3.99 | +3.15
MonoFlex [62]† | CVPR2021 | None | 19.94 | 13.89 | 12.07 | 28.23 | 19.75 | 16.89 | 23.44 | 17.43 | 14.67
3DSeMo${}_{\text{FLEX}}$ | Ours | Unlabeled | 23.55 | 15.25 | 13.24 | 32.57 | 21.21 | 18.07 | 27.35 | 20.87 | 17.66
Improvement | Ours | v.s. baseline | +3.61 | +1.36 | +1.17 | +4.34 | +1.46 | +1.18 | +3.91 | +3.44 | +2.99

Table 1: Comparision with state-of-the-art (SOTA) Methods. We present the
evaluation results of ‘CAR’ in the KITTI test and validation sets. †denotes
the reproduction results. For fair comparisons, we train Mix-Teaching with the
same data volume as our method.

.

###  4.1 Experimental Setup

Dataset. We mainly conduct our experiments on KITTI [18], which contains 7,481
images for training and 7,518 images for testing. Following [11], we split the
original training set into 3,712 training and 3,769 validation samples. We
picked 151 unlabeled video sequences from KITTI. After removing the duplicated
samples in the training set, we obtained 33,507 unlabeled samples for semi-
supervised training purposes, roughly 10 times larger than the annotated
training set. A smaller subset of the training data is held out for the TPE
hyper-parameter search. It is observed that previous works on semi-supervised
Mono 3D detection only evaluate KITTI [18]. To further show the generality of
our method, we design a toy experiment on nuScenes (see Sec. 4.3.5).

Metrics. In our experiments, we utilize the average precision (AP) as the
metrics (both in 3D and bird’s eye view) to evaluate and compare different
methods. To prove the effectiveness of the proposed APG, we calculate the
detection recall to measure the qualities of the generated pseudo labels (see
Sec. 4.3.3). Following [45], all evaluation results on validation and test
sets are based on AP@40.

Implementation Details. We integrate the proposed semi-supervised framework to
classical Mono3D detectors MonoDLE [36] and MonoFlex [62]. Unless otherwise
specified, the proposed APG augments an input image from the unlabeled dataset
to $K=9$ different views. While the initial threshold for filtering detection
boxes is set as 0.65, other predictions with confidence scores lower than 0.65
will be used in the center aggregation algorithm (see Sec. 3.2). For the
proposed CRS, we construct the critical module with ResNet-18 [21] network and
modify the output dimension of the last fully connected layer to 1. Other
layers are initialized with the standard pre-trained weights on ImageNet [14].
Notably, the critical module is not used during inference. For a batch size of
8, we chop off the 2 samples with lowest evaluation value $v$ in CRS training.
For retraining, we initialize the student model with the weights from the
teacher model trained on the labeled dataset.

For fair comparisons, we reproduce the baseline methods MonoDLE [36] and
MonoFlex [62] based on the official codes provided by the authors. While most
Mono3D methods are trained on a single GPU, we adopt 8 A6000 GPUs in all
experiments to facilitate training with a larger data volume. Ablations are
conducted based on MonoDLE unless otherwise specified. Configs will be
released.

###  4.2 Main Results

Method | Ped. $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.5$ | Cyc. $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.5$
---|---|---
Easy | Mod. | Hard | Easy | Mod. | Hard
Baseline | 9.64 | 6.55 | 5.44 | 4.59 | 2.66 | 2.45
Ours | 10.78 | 7.26 | 6.05 | 7.04 | 4.24 | 3.56
Improvement | +1.14 | +0.71 | +0.61 | +2.45 | +1.58 | +1.11

Table 2: ‘Pedestrian’ and ‘Cyclist’ Performances in KITTI test set.

We apply the proposed framework to MonoDLE [36] and MonoFlex [62], and
evaluate our methods on the official test and validation sets of KITTI. Tab. 1
and Tab. 2 present quantitative comparisons of our method with other state-of-
the-art counterparts on the KITTI leaderboard. It shows that by effectively
leveraging larger volumes of unlabeled data, our proposed semi-supervised
strategy significantly boosts the performance of the baseline methods. In
particular, our approach respectively improves the baseline MonoDLE
$\mathtt{AP_{3D}(Mod.)}$ and $\mathtt{AP_{BEV}(Mod.)}$ by +3.32%/+2.89% on the
test set without any tricks (_e.g.,_ test-time augmentation). The gains on
$\mathtt{AP(Easy)}$ of our 3DSeMo${}_{\text{DLE}}$ surprisingly exceeds +5% on
all metrics and data splits. When integrating our method to MonoFlex [30], it
achieves gains of +3.61/1.36 on $\mathtt{AP_{3D}(Easy/Mod.)}$, respectively,
evidencing the generality of our framework.

###  4.3 Ablation Studies and Analysis

This section presents more in-depth analyses to demonstrate the effectiveness
of each proposed component.

####  4.3.1 Influence of the Unlabeled Data Volume

We compare the fully supervised baseline with our semi-supervised model
trained with different amounts of unlabeled data in Tab. 3. It shows that our
semi-supervised method outperforms the baseline under different unlabeled data
volume, which proves that our method can effectively exploit the useful
information in unlabeled data and the generated pseudo labels. Notably, even
only using 20% unlabeled data (about 3350 samples), our approach can obtain
+1.55% $\mathtt{AP_{3D}(Mod.)}$ and +1.25% $\mathtt{AP_{BEV}(Mod.)}$
improvements on car and pedestrian, respectively. When using all the unlabeled
data, our method outperforms the baseline with a favorable margin,
demonstrating the profound potentials of semi-supervised learning in boosting
Mono3D detection.

Method | Car | Ped. | Cyc.
---|---|---|---
Baseline | 13.66 | 4.45 | 2.52
\+ 20% Unlabeled Data | 15.21 | 5.21 | 2.51
\+ 50% Unlabeled Data | 16.50 | 6.37 | 2.76
\+ 100% Unlabeled Data | 17.65 | 8.25 | 3.47

Table 3: Influence of the Unlabeled Data Volume (AP3D Mod.).

####  4.3.2 Component-wise Analysis

To understand the effect of each component, we incrementally apply the
proposed APG and CRS to the baseline detector MonoDLE [36] with
$\mathtt{Car~{}AP_{3D}(Mod.)}$ of 13.66. As shown in Tab. 4, the vanilla self-
training strategy (see Sec. 3.1) improves the baseline model for 2.11% on
$\mathtt{Car~{}AP_{3D}(Mod.)}$ without bells and whistles. Subsequently, we
apply the proposed APG and CRS to the model, respectively. It shows that both
of them can significantly improve $\mathtt{Car~{}AP_{3D}(Mod.)}$ by about
2.5%, which validates our argument that robust pseudo label generation and
finding informative samples are both crucial for semi-supervised Mono3D object
detection. Last but not least, while maintaining the superiority on
$\mathtt{Car}$, combining the proposed APG and CRS can pre-eminently improve
$\mathtt{Ped.(Mod.)~{}AP_{3D}}$ for about 1.4%. In Mono3D object detection,
the pedestrian is more challenging than car because of the much smaller object
size, for which a slight prediction shift leads to drastic degradation of IoU.
The pseudo labels of pedestrian thus contain much more noise. Therefore, the
gains on pedestrian prove CRS’s ability in adaptively selecting informative
samples from severely noisy pseudo labels.

Method | Car $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.7$ | Ped. $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.5$
---|---|---
Easy | Mod. | Hard | Easy | Mod. | Hard
Baseline | 16.77 | 13.66 | 11.54 | 5.53 | 4.45 | 3.33
Plainest Self-Training | 20.14 | 15.77 | 13.27 | 7.27 | 5.99 | 4.74
\+ CRS | 22.64 | 17.53 | 14.59 | 9.43 | 6.81 | 5.71
\+ APG | 22.71 | 17.56 | 14.68 | 9.35 | 6.77 | 5.58
\+ APG + CRS | 22.87 | 17.65 | 14.83 | 10.99 | 8.25 | 6.72

Table 4: Component-wise Analysis. We compare the effects of each proposed
component to demonstrate the effectiveness and rationality of our framework.
Figure 5: Effectiveness of APG. We demonstrate the robustness of APG on both
pseudo label quality (recall) and 3D object detection performance (AP@40).
Results show that our method is less sensitive to threshold change.

####  4.3.3 Analysis of APG

Robustness. Previous semi-supervised methods usually filter detection boxes to
generate pseudo labels by applying a threshold $\tau$ on the classification
score. However, as presented in Fig. 5 (the blue solid line), it suffers from
a drastic degradation on $\mathtt{Recall}$ when enlarging the threshold.
Besides, its detection performance is sensitive to threshold change (the  cyan
dotted line). In contrast, the performances of our APG (the red dotted lines)
are more stable, which proves its robustness in generating pseudo labels. We
select $\tau=0.65$ in our model based on the observation of this experiment,
with which the APG can boost the $\mathtt{Car~{}(Mod.)~{}AP_{3D}}$ for 1.06%,
as shown in Tab. 5. Though we need to set an initial threshold in APG, our
experiment (the red solid line) show that it is less sensitive to threshold
change. Fig. 5 also proves that geometry-based augmentation (the green solid
line, i.e., geometry transform) is superior to the content-based counterpart
(the red solid line, i.e., color enhencement) in improving the quality of the
generated pseudo labels. This may attribute to their different mechanism that
content-based transformations only marginally modify the context, while
geometry-based transformations can significantly migrate the position and
scale distribution of objects, which are the common reasons for false
negatives in Mono3D object detection (see Fig. 1 again). Notably, the
transformations are automatically learned by TPE, which can be effortlessly
integrated into other detectors. All details, codes, and results about TPE
will be released for reproduction.

| Method | Car $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.7$ | Ped. $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.5$
---|---|---|---
| +cls. | +loc. | Easy | Mod. | Hard | Easy | Mod. | Hard
① | w/o APG | 21.75 | 16.32 | 14.15 | 8.34 | 6.04 | 4.80
② | w/ APG | 22.66 | 17.38 | 14.67 | 7.71 | 5.88 | 4.74
③ | ✓ |  | 22.51 | 17.44 | 14.63 | 9.02 | 6.52 | 5.33
④ | ✓ | ✓ | 22.71 | 17.56 | 14.68 | 9.35 | 6.77 | 5.58
⑤ | DenseTeacher [64] | 22.83 | 17.12 | 14.36 | 7.78 | 5.94 | 4.75

Table 5: Effectiveness of Reweighting Strategy. With the by-product of APG to
reweight samples during retraining, the student model obtains impressive
gains, especially on pedestrian.

Influence of sample weight. While APG improves the overall recall of pseudo
labels, it inevitably introduces more noise to challenging categories (e.g.,
pedestrian) as shown in Tab. 5. As a result, the
$\mathtt{Ped.~{}(Mod.)~{}AP_{3D}}$ drops 0.16% (② $v.s.$ ①). To alleviate
this, we weight the loss of each unlabeled sample in the retraining phase with
the by-product clues from the proposed APG (see Sec. 3.2 and Eq. 1). As shown
in Tab. 5, when adopting classification score to weight samples as in previous
works [56], it can improve the performance of hard category ( ③ $v.s.$ ②).
Introducing the clues from APG can further enhance the detection of
pedestrian, obtaining 0.89% improvement on the
$\mathtt{Ped.~{}(Mod.)~{}AP_{3D}}$ (④ $v.s.$ ②). Threshold-free approaches,
such as DenseTeacher [64], however, does not bring benefits to the Mono3D SSL
task. The discrepancies between 2D and 3D detections is to blame for such
failure.

####  4.3.4 Analysis of CRS

Different strategies for selecting informative samples. The proposed CRS aims
to adaptively separate informative samples from noisy ones. To demonstrate the
superiority of CRS, we compare against some alternative strategies which have
demonstrated success in other tasks. The compared counterparts include 1)
filtering samples with the quality score of pseudo labels introduced in Eq. 1,
2) the bbox jitter proposed for 2D detection in [56]. We tailor the 2D box
jitter strategy for 3D detection, and details are presented in the
supplementary material. As shown in Tab. 6, bbox jitter causes performance
degradation because of its unreliable quality measurement for pseudo labels (②
v.s. ①). ③ throws away unreality samples based on classification and location
scores (see Eq. 1). It shows that ③ only slightly improves pedestrian
detection performance, however still lagging behind our proposed CRS (⑤).
Besides the unreliability of detection scores and box jitter in Mono3D,
another underlying reason for the advance of CRS is that ① and ② are static
strategies where the filtering indicator of a sample holds along the
retraining phase. Conversely, the indicator learned by the critical module
changes in different retraining timestamps, as shown in Fig. 6. It both
intuitively and theoretically makes sense that the importance of a sample
should be mutative in training.

Method Car $\text{AP}_{3D}$ $\text{IoU}\geqslant 0.7$ Ped. $\text{AP}_{3D}$
$\text{IoU}\geqslant 0.5$ Easy Mod. Hard Easy Mod. Hard ① Baseline 22.71 17.56
14.68 9.35 6.77 5.58 ② \+ 3D bbox jitter filter [56] 22.50 16.71 14.40 8.36
6.30 5.07 ③ \+ cls.&loc. weight filter 21.86 17.20 14.31 9.04 7.25 5.75 ④ \+
CRS filter w/o critical module 21.41 16.58 14.05 8.05 6.23 5.01 ⑤ \+ CRS
filter w/ critical module 22.87 17.65 14.83 10.99 8.25 6.72

Table 6: Effectiveness of CRS. We compare our critical module with other
counterparts in filtering pseudo labels. The prominent improvement on
pedestrian indicates that the CRS can effectively find informative samples
during training. Figure 6: Adaptive Reward of CRS. Vanilla filter-based
strategies invariably drop out or preserve a samples whereas our proposed
critical module predicts adaptive rewards during retraining.

Learnable or not. The proposed CRS learns the filtering indicator with a
learnable critical module (Eq. 4). Yet intuitively, we can simply determine
the contribution of a sample by the training loss before and after the model
updating with Eq. 5. To validate the necessity of the proposed scheme, we
prohibit the critical module and directly leverage the reward calculated in
Eq. 5 as the indicator to select samples during retraining. As shown in Tab. 6
④, unsurprisingly, this naive strategy degrades the overall performance
because of its biased optimization objective. In particular, the strategy of ④
can only access several samples during calculating the reward, lacking the
global vision of the evaluation set. In contrast, the learning-based critical
module encodes the knowledge of the whole dataset to its weights parameters
through cyclically updating the memory bank, which can provide stable and
effective filtering indicators for model retraining (see ⑤).

Method | KITTI Validation
---|---
0-20 | 20-40 | 40-$\infty$ | Avg. | AP3D
Baseline | 0.511 | 1.243 | 2.639 | 1.172 | 13.66
Ours | 0.446 | 1.161 | 2.293 | 1.024 | 17.65
Method | nuScenes Validation
0-20 | 20-40 | 40-$\infty$ | Avg. | AP
Baseline | 0.652 | 1.607 | 4.429 | 1.990 | 29.25
Ours | 0.547 | 1.500 | 3.661 | 1.669 | 32.21

Table 7: Evaluation on KITTI validation and nuScenes frontal validation cars
with depth MAE ($\downarrow$) and AP ($\uparrow$).

####  4.3.5 Evaluation on nuScenes

It is noticed that recent semi-supervised works only evaluate KITTI [18]. To
further show the generalization and potential of our method, we conduct a toy
experiment on nuScenes [7]. Knowing there is no established semi-supervised
Mono3D prototype on nuScenes, we roughly divide the official training set of
28,130 images into two subsets: 3,375 labeled ones and 24,755 unlabeled ones.
Evaluations are performed on the official validation set consisting of 6,019
images. Following the approach in [24], we only consider the frontal view cars
and use mean absolute error (MAE) of the depth and average precision (AP) to
measure the prediction accuracy. We refer the readers to [24] for more details
about the criteria. As shown in Tab. 7, the proposed 3DSeMo shows consistent
improvement across different distance ranges to ego car, again demonstrating
our method’s generation. We hope our attempt can drive more interest in semi-
supervised 3D object detection.

##  5 Conclusion and Limitation

In this paper, we present the ‘Augment and Criticize’ policies to construct a
general framework for self-training-based semi-supervised monocular 3D object
detection. The proposed APG aggregates predictions from different views of
unlabeled images for robust label generation. The CRS adopts a learnable
critical module to measure the reward of each pseudo sample and filter noisy
ones to enhance model training. Extensive experiments and analyses demonstrate
the effectiveness of our approach.

We hope our work could enlighten more research on semi-supervised monocular 3D
object detection. There exists one limitation that we leave to future work: As
presented in Tab. 3, detection performance grows with the unlabeled data
volume yet it has not impressively plateaued. Naturally, restocking more
unlabeled samples from other sources (e.g. Waymo [47] and nuScenes [7]) could
further enhance the detection methods. However, the domain gap in different
sources may compromise the effectiveness of semi-supervised learning. In
future work, we will devote more effort to mitigate this gap for exploiting
more unlabeled data, which we believe can further facilitate semi-supervised
Mono3D tasks.

## References

  * [1] Massih-Reza Amini, Vasilii Feofanov, Loic Pauletto, Emilie Devijver, and Yury Maximov.  Self-training: A survey.  arXiv preprint arXiv:2202.12040, 2022.
  * [2] Eduardo Arnold, Omar Y Al-Jarrah, Mehrdad Dianati, Saber Fallah, David Oxtoby, and Alex Mouzakitis.  A survey on 3d object detection methods for autonomous driving applications.  IEEE Transactions on Intelligent Transportation Systems, 20(10):3782–3795, 2019.
  * [3] James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl.  Algorithms for hyper-parameter optimization.  NeurIPS, 24, 2011.
  * [4] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel.  Mixmatch: A holistic approach to semi-supervised learning.  NeurIPS, 32, 2019.
  * [5] Peter J Bickel and Kjell A Doksum.  Mathematical statistics: basic ideas and selected topics, volumes I-II package.  Chapman and Hall/CRC, 2015.
  * [6] Garrick Brazil, Gerard Pons-Moll, Xiaoming Liu, and Bernt Schiele.  Kinematic 3d object detection in monocular video.  In ECCV, pages 135–152. Springer, 2020.
  * [7] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.  nuscenes: A multimodal dataset for autonomous driving.  In CVPR, pages 11621–11631, 2020.
  * [8] Binbin Chen, Weijie Chen, Shicai Yang, Yunyi Xuan, Jie Song, Di Xie, Shiliang Pu, Mingli Song, and Yueting Zhuang.  Label matching semi-supervised object detection.  In CVPR, pages 14381–14390, 2022.
  * [9] Hansheng Chen, Yuyao Huang, Wei Tian, Zhong Gao, and Lu Xiong.  Monorun: Monocular 3d object detection by reconstruction and uncertainty propagation.  In CVPR, pages 10379–10388, 2021.
  * [10] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He.  Improved baselines with momentum contrastive learning.  arXiv preprint arXiv:2003.04297, 2020.
  * [11] Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Andrew G Berneshawi, Huimin Ma, Sanja Fidler, and Raquel Urtasun.  3d object proposals for accurate object class detection.  NeurIPS, 28, 2015.
  * [12] Yongjian Chen, Lei Tai, Kai Sun, and Mingyang Li.  Monopair: Monocular 3d object detection using pairwise spatial relationships.  In CVPR, pages 12093–12102, 2020.
  * [13] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le.  Autoaugment: Learning augmentation policies from data.  arXiv preprint arXiv:1805.09501, 2018.
  * [14] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In CVPR, pages 248–255. Ieee, 2009.
  * [15] Mingyu Ding, Yuqi Huo, Hongwei Yi, Zhe Wang, Jianping Shi, Zhiwu Lu, and Ping Luo.  Learning depth-guided convolutions for monocular 3d object detection.  In CVPRW, pages 1000–1001, 2020.
  * [16] David Eigen, Christian Puhrsch, and Rob Fergus.  Depth map prediction from a single image using a multi-scale deep network.  NeurIPS, 27, 2014.
  * [17] Tianxiang Gao and Vladimir Jojic.  Sample importance in training deep neural networks.  2016\.
  * [18] Andreas Geiger, Philip Lenz, and Raquel Urtasun.  Are we ready for autonomous driving? the kitti vision benchmark suite.  In CVPR, pages 3354–3361, 2012.
  * [19] Jiaqi Gu, Bojian Wu, Lubin Fan, Jianqiang Huang, Shen Cao, Zhiyu Xiang, and Xian-Sheng Hua.  Homography loss for monocular 3d object detection.  In CVPR, pages 1080–1089, 2022.
  * [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In ICCV, pages 1026–1034, 2015.
  * [21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, pages 770–778, 2016.
  * [22] Kuan-Chih Huang, Tsung-Han Wu, Hung-Ting Su, and Winston H Hsu.  Monodtr: Monocular 3d object detection with depth-aware transformer.  In CVPR, pages 4012–4021, 2022.
  * [23] Angelos Katharopoulos and François Fleuret.  Not all samples are created equal: Deep learning with importance sampling.  In ICML, pages 2525–2534, 2018.
  * [24] Abhinav Kumar, Garrick Brazil, Enrique Corona, Armin Parchami, and Xiaoming Liu.  Deviant: Depth equivariant network for monocular 3d object detection.  In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part IX, pages 664–683. Springer, 2022.
  * [25] Dong-Hyun Lee et al.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In ICMLW, page 896, 2013.
  * [26] Peixuan Li, Huaici Zhao, Pengfei Liu, and Feidao Cao.  Rtm3d: Real-time monocular 3d detection from object keypoints for autonomous driving.  In ECCV, pages 644–660. Springer, 2020.
  * [27] Zhenyu Li, Zehui Chen, Ang Li, Liangji Fang, Qinhong Jiang, Xianming Liu, and Junjun Jiang.  Unsupervised domain adaptation for monocular 3d object detection via self-training.  arXiv preprint arXiv:2204.11590, 2022.
  * [28] Qing Lian, Yanbo Xu, Weilong Yao, Yingcong Chen, and Tong Zhang.  Semi-supervised monocular 3d object detection by multi-view consistency.  In ECCV, 2022.
  * [29] Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim.  Fast autoaugment.  NeurIPS, 32, 2019.
  * [30] Xianpeng Liu, Nan Xue, and Tianfu Wu.  Learning auxiliary monocular contexts helps monocular 3d object detection.  In AAAI, pages 1810–1818, 2022.
  * [31] Yuyuan Liu, Yu Tian, Yuanhong Chen, Fengbei Liu, Vasileios Belagiannis, and Gustavo Carneiro.  Perturbed and strict mean teachers for semi-supervised semantic segmentation.  In CVPR, pages 4258–4267, 2022.
  * [32] Zechen Liu, Zizhang Wu, and Roland Tóth.  Smoke: Single-stage monocular 3d object detection via keypoint estimation.  In CVPRW, pages 996–997, 2020.
  * [33] Zongdai Liu, Dingfu Zhou, Feixiang Lu, Jin Fang, and Liangjun Zhang.  Autoshape: Real-time shape-aware monocular 3d object detection.  In ICCV, pages 15641–15650, 2021.
  * [34] Yan Lu, Xinzhu Ma, Lei Yang, Tianzhu Zhang, Yating Liu, Qi Chu, Junjie Yan, and Wanli Ouyang.  Geometry uncertainty projection network for monocular 3d object detection.  In ICCV, pages 3111–3121, 2021.
  * [35] Xinzhu Ma, Shinan Liu, Zhiyi Xia, Hongwen Zhang, Xingyu Zeng, and Wanli Ouyang.  Rethinking pseudo-lidar representation.  In ECCV, 2020.
  * [36] Xinzhu Ma, Yinmin Zhang, Dan Xu, Dongzhan Zhou, Shuai Yi, Haojie Li, and Wanli Ouyang.  Delving into localization errors for monocular 3d object detection.  In CVPR, pages 4721–4730, 2021.
  * [37] Geoffrey J McLachlan.  Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis.  Journal of the American Statistical Association, 70(350):365–369, 1975.
  * [38] Arsalan Mousavian, Dragomir Anguelov, John Flynn, and Jana Kosecka.  3d bounding box estimation using deep learning and geometry.  In CVPR, pages 7074–7082, 2017.
  * [39] Alec Radford, Luke Metz, and Soumith Chintala.  Unsupervised representation learning with deep convolutional generative adversarial networks.  arXiv preprint arXiv:1511.06434, 2015.
  * [40] Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko.  Semi-supervised learning with ladder networks.  Advances in neural information processing systems, 28, 2015.
  * [41] Cody Reading, Ali Harakeh, Julia Chae, and Steven L Waslander.  Categorical depth distribution network for monocular 3d object detection.  In CVPR, pages 8555–8564, 2021.
  * [42] Thomas Roddick, Alex Kendall, and Roberto Cipolla.  Orthographic feature transform for monocular 3d object detection.  arXiv preprint arXiv:1811.08188, 2018.
  * [43] Henry Scudder.  Probability of error of some adaptive pattern-recognition machines.  IEEE Transactions on Information Theory, 11(3):363–371, 1965.
  * [44] Xuepeng Shi, Qi Ye, Xiaozhi Chen, Chuangrong Chen, Zhixiang Chen, and Tae-Kyun Kim.  Geometry-based distance decomposition for monocular 3d object detection.  In ICCV, pages 15172–15181, 2021.
  * [45] Andrea Simonelli, Samuel Rota Bulo, Lorenzo Porzi, Manuel López-Antequera, and Peter Kontschieder.  Disentangling monocular 3d object detection.  In ICCV, pages 1991–1999, 2019.
  * [46] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  In NeurIPS, pages 596–608, 2020.
  * [47] Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, et al.  Scalability in perception for autonomous driving: Waymo open dataset.  In CVPR, pages 2446–2454, 2020.
  * [48] Antti Tarvainen and Harri Valpola.  Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results.  NeurIPS, 30, 2017.
  * [49] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  Fcos: Fully convolutional one-stage object detection.  In ICCV, pages 9627–9636, 2019.
  * [50] Jesper E Van Engelen and Holger H Hoos.  A survey on semi-supervised learning.  Machine Learning, 109(2):373–440, 2020.
  * [51] Li Wang, Liang Du, Xiaoqing Ye, Yanwei Fu, Guodong Guo, Xiangyang Xue, Jianfeng Feng, and Li Zhang.  Depth-conditioned dynamic message propagation for monocular 3d object detection.  In CVPR, pages 454–463, 2021.
  * [52] Tai Wang, ZHU Xinge, Jiangmiao Pang, and Dahua Lin.  Probabilistic and geometric depth: Detecting objects in perspective.  In CoRL, pages 1475–1485. PMLR, 2022.
  * [53] Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin.  Fcos3d: Fully convolutional one-stage monocular 3d object detection.  In ICCV, pages 913–922, 2021.
  * [54] Xinshuo Weng and Kris Kitani.  Monocular 3d object detection with pseudo-lidar point cloud.  In ICCVW, pages 0–0, 2019.
  * [55] Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le.  Self-training with noisy student improves imagenet classification.  In CVPR, pages 10687–10698, 2020.
  * [56] Mengde Xu, Zheng Zhang, Han Hu, Jianfeng Wang, Lijuan Wang, Fangyun Wei, Xiang Bai, and Zicheng Liu.  End-to-end semi-supervised object detection with soft teacher.  In ICCV, pages 3060–3069, 2021.
  * [57] Lei Yang, Xinyu Zhang, Li Wang, Minghan Zhu, Chuang Zhang, and Jun Li.  Mix-teaching: A simple, unified and effective semi-supervised learning framework for monocular 3d object detection.  arXiv preprint arXiv:2207.04448, 2022.
  * [58] Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, and Yang Gao.  St++: Make self-training work better for semi-supervised semantic segmentation.  In CVPR, pages 4268–4277, 2022.
  * [59] Jianlong Yuan, Jinchao Ge, Qi Qian, Zhibin Wang, Fan Wang, and Yifan Liu.  Semi-supervised semantic segmentation with mutual knowledge distillation.  arXiv preprint arXiv:2208.11499, 2022.
  * [60] Fangyuan Zhang, Tianxiang Pan, and Bin Wang.  Semi-supervised object detection with adaptive class-rebalancing self-training.  In AAAI, pages 3252–3261, 2022.
  * [61] Renrui Zhang, Han Qiu, Tai Wang, Xuanzhuo Xu, Ziyu Guo, Yu Qiao, Peng Gao, and Hongsheng Li.  Monodetr: Depth-aware transformer for monocular 3d object detection.  arXiv preprint arXiv:2203.13310, 2022.
  * [62] Yunpeng Zhang, Jiwen Lu, and Jie Zhou.  Objects are different: Flexible monocular 3d object detection.  In CVPR, pages 3289–3298, 2021.
  * [63] Yunpeng Zhang, Wenzhao Zheng, Zheng Zhu, Guan Huang, Dalong Du, Jie Zhou, and Jiwen Lu.  Dimension embeddings for monocular 3d object detection.  In CVPR, pages 1589–1598, 2022.
  * [64] Hongyu Zhou, Zheng Ge, Songtao Liu, Weixin Mao, Zeming Li, Haiyan Yu, and Jian Sun.  Dense teacher: Dense pseudo-labels for semi-supervised object detection.  arXiv preprint arXiv:2207.02541, 2022.
  * [65] Xingyi Zhou, Dequan Wang, and Philipp Krähenbühl.  Objects as points.  arXiv preprint arXiv:1904.07850, 2019.