# Teachers in concordance for pseudo-labeling of 3D sequential data

Awet Haileslassie Gebrehiwot1∗, Patrik Vacek1, David Hurych2, Karel
Zimmermann1, Patrick Perez2,
Tomáš Svoboda1
_1 Czech Technical University in Prague, FEE, Dept of Cybernetics_
_2 Valeo.ai_ * Corresponding author: E-mail<EMAIL_ADDRESS>__

###### Abstract

Automatic pseudo-labeling is a powerful tool to tap into large amounts of
sequential unlabeled data. It is especially appealing in safety-critical
applications of autonomous driving where performance requirements are extreme,
datasets large, and manual labeling is very challenging. We propose to
leverage the sequentiality of the captures to boost the pseudo-labeling
technique in a teacher-student setup via training multiple teachers, each with
access to different temporal information. This set of teachers, dubbed
_Concordance_ , provides higher quality pseudo-labels for the student training
than standard methods. The output of multiple teachers is combined via a novel
pseudo-label confidence-guided criterion. Our experimental evaluation focuses
on the 3D point cloud domain in urban driving scenarios. We show the
performance of our method applied to multiple model architectures with tasks
of 3D semantic segmentation and 3D object detection on two benchmark datasets.
Our method, using only 20% of manual labels, outperforms some of the fully
supervised methods. Special performance boost is achieved for classes rarely
appearing in the training data, _e.g._ , bicycles and pedestrians. _The
implementation of our approach is publicly available at
https://github.com/ctu-vras/T-Concord3D._

##  I Introduction

In many machine learning problems, the supervision via complete annotation of
training data is required to achieve state-of-the-art performance. Therefore,
an effective way to increase the performance of a model is to add more
annotated training data [12]. However, this approach is neither scalable nor
sustainable, as it requires painstaking manual labeling by human annotators.
To make matters worse, annotation tasks such as semantic segmentation of
videos and sequences of point clouds are very complex to accomplish. Reducing
annotation needs is therefore a crucial and active research field. Pseudo-
labeling [14] has emerged as a versatile and powerful tool: A teacher model
trained on a small amount of labeled data is used to automatically annotate
lots of unlabeled data; The student model is trained on the combination of the
small labeled set and the large pseudo-labeled set. In this work, we introduce
different ways to boost pseudo-labeling when dealing with temporally ordered
data streams. The proposed framework is instantiated and evaluated in the
context of 3D point-cloud analysis for driving applications.

Figure 1: Proposed Concordance of teachers for pseudo-labeling of sequences. A
set $\mathcal{D}^{\ell}$ of sequences with central frame labeled, and a larger
set $\mathcal{D}^{\mathrm{u}}$ of unannotated ones, are available for
training; \raisebox{-.9pt}{1}⃝ Multiple offline teachers are trained with full
supervision on $\mathcal{D}^{\ell}$, each with a different temporal range
towards future and past frames; \raisebox{-.9pt}{2}⃝ The teachers are run on
$\mathcal{D}^{\mathrm{u}}$ to produce pseudo-labels (PLs) for central frames;
\raisebox{-.9pt}{3}⃝ Sequences with the most confident PLs according to
Concordance of teachers are selected, forming the pseudo-labeled set
$\mathcal{D}^{\mathrm{p}}$. The white box depicts the discarded PLs;
\raisebox{-.9pt}{4}⃝ The student is trained on
$\mathcal{D}^{\ell}\cup\mathcal{D}^{\mathrm{p}}$, to work online with past and
current frames only

In contrast to unordered data, the sequentiality of temporally ordered data
grants the student and teacher access to a meaningful temporal context. In
particular, the teacher can also access future data, a form of privileged
information [31] that is available at the time of pseudolabeling, but not at
inference time. Consequently, the teacher can benefit from both, the past
frames, like the student, and the future ones, thus making the most of the
temporal consistency over an extended time window. We noticed that the range
of this temporal window has a crucial influence on teacher’s performance, as
it captures different temporal context. In fact, the complexity of spatio-
temporal events at work in 3D driving scenes would require the teacher to
operate simultaneously at different temporal ranges. Learning a large enough
teacher capable of modeling the aforementioned complexity would require a huge
amount of labels, contradicting the motivation of easing the annotation. We
take a more practical approach where multiple complementary teachers are
trained, each operating in its own temporal range. The _Concordance_ of these
teachers is used to assess the confidence of the extracted pseudo-labels (PLs)
and to eventually select the most confident ones for student training.

We experimentally demonstrate that Concordance-based pseudo-labeling (i)
achieves competitive performance with state-of-the-art fully supervised
methods [34, 20, 36, 35, 7, 8, 38] with only a fraction of labeled data, (ii)
outperforms pseudo-labeling methods that do not leverage multiple teachers
[12], and (iii) surpasses usual ensemble methods, which do not explicitly
consider the different temporal nature of each teacher.

Our approach (Fig. 1) only assumes that two sets of sequences are available:
The first one with the central frames annotated and the second one larger and
devoid of annotation. Several teachers are trained on the first set to predict
the label of the middle frame of an input sequence. Once trained, all the
offline models are run on the second dataset to produce pseudo-labeling of the
central frames. Based on time-aware Concordance sample selection, the most
promising automatically annotated samples are weighted and added to the first
set. The resulting large labeled set, with the future frames not available at
input, is used to train the final online model. We put this framework to work
for different spatio-temporal perception tasks on outdoor point clouds (PCs)
acquired with a laser scanner (lidar). Lidar PCs are naturally captured as
sequences of successive scans. It is common practice to accumulate them over
short time intervals to densify the PC to be analysed. In contrast, our
approach takes advantage of the sequence and its temporal ordering. We
demonstrate this on the tasks of 3D detection and 3D semantic segmentation in
driving scenes on two architectures [26, 38] and two datasets (see Figure 2).
Our contributions to pseudo-labeling of temporal data are:

  1. 1.

An effective way to aggregate time-ordered both unannotated/annotated 3D
scans; Leveraging such privileged information improves teacher’s performance
for 3D semantic segmentation and object detection tasks.

  2. 2.

A novel confidence-guided criterion for better pseudo-labels selection and
loss function guidance.

  3. 3.

A novel weighting of pseudo-labels via the Concordance of teachers with
different temporal ranges.

##  II Related Work

Spatial and temporal consistency in point clouds. Lidar PCs are unstructured
data. PointNet architecture [23] directly consumes raw PCs to extract features
with permutation invariance and global features for classification. PointNet++
[24] further improves the extraction of local features. The architecture of
MeteorNet [18] works with multiple input PCs to extract features from
additional available points with consistent temporal properties. Another
approach using sparse 4D CNN [6] for spatiotemporal perception helps in
detection tasks and improves robustness. Spatial frame synchronization to the
reference scan and the addition of one additional channel of encoded time [10]
lead to further performance gain. This approach was also exploited in [22] as
an extension of multiview fusion [40]. Temporal information for PC
densification based on tracking of previously detected objects was used in
[22] for the automatic annotation of data. Conversely, they use a top-
performance detection model pre-trained in a fully supervised way. We focus on
building a set of teacher models (Concordance) from a minimal amount of
annotated data and apply distillation through pseudo-labeling.

Knowledge Distillation. Training the student model is usually done by
distilling knowledge in the feature or output space [9]. Work [9] distills an
ensemble of teachers into a single student model and [17] extends the idea
using different architectures suitable for different tasks as teachers. [5]
shows that larger models are not necessarily better teachers due to parameter
complexity mismatch. [21] introduces a multistep knowledge distillation with
an intermediate-sized network (teacher assistant) to bridge the gap between
student and teacher complexity. The benefit of using the teacher network can
also come from exploiting privileged information [32, 13]. We incorporate
future data in sequential frames when training the teacher models.

Semi-supervised learning. pseudolabeling technique for semi-supervised
learning was introduced in [14], to help training with a few labeled data in
combination with large amounts of unlabeled data [15]. A framework based on
the mean teacher as a semi-supervised learning technique for 3D improves the
object detection task [33] with unlabeled data. One can train directly on
pseudolabels after filtering them using a confidence threshold based on the
predicted “objectness” and class probability to filter low-quality
pseudolabels [33]. Proposed pseudo-labeling in [3] for 3D object detection in
lidar scans to leverage more widely available unlabeled data led to
performance gains with additional data augmentation strategies and massive
unlabeled dataset.

Comparing the consistency of augmented versions of data is a commonly used
method for semi-supervised learning [28]. Data perturbations for 3D object
detection were examined in [37] with additional proposed consistency loss. A
semi-supervised learning to 3D semantic segmentation with guided point
contrastive loss has been proposed in [12]. They have shown that the use of
pseudolabel and confidence thresholding can help to improve feature learning
in both indoor and outdoor scenes. We experimentally compare our approach with
this relevant method in Section IV.

##  III Method

One key component is the use of temporal privileged information in the form of
several unlabeled future scans to boost the performance of pseudo-labeling
teachers. Combining these scans with the current and past scans, all aligned
by odometry, gives access to a richer spatial and temporal description of the
scene (see Fig. 1). As a result, better teachers can be trained, and their
combination will in turn improve the quality of the pseudo-labels and the
performance of the student trained on them. A selection of pseudo-labels based
on the Concordance of teachers further improves the performance.

Point-cloud notations. A point cloud $X=\\{\bm{x}^{k}\\}_{k=1}^{K}$ is a
finite orderless collection of 3D points, where the number of points $K$ is
assumed constant over time for notational convenience. We consider symmetric
time-ordered PC sequences of the form
$X_{\mathop{\mbox{-}}n:n}=(X_{\mathop{\mbox{-}}n},\ldots,X_{\mathop{\mbox{-}}1},X_{0},X_{1},\ldots,X_{n})$
composed of a reference scan $X_{0}$, preceded by $n$ past scans and followed
by $n$ future ones. All scans in the sequence are transformed into the
coordinate system of the reference one.

Figure 2: Proposed architectures to aggregate sequence of 3D point clouds. We
build a modified version of Cylinder3D for semantic segmentation and PointRCNN
for object detection that can aggregate multiple frames inside its spatio-
temporal encoder. The output of the spatio-temporal encoder-decoder are
extracted point features $\bm{h}(\bm{x})$. For object detection, Only features
from the reference frame, i.e., $\bm{h}(\bm{x_{0}})$ for $\bm{x}_{0}\in
X_{0}$, are used to train PointRCNN for object detection. Here, we use full
set of features $\bm{h}(\bm{x})$ for semantic segmentation due to the setup of
the state of the art Cylinder3D architecture

Time-aware feature extraction. We build on a modified Cylinder3D [38] for
semantic segmentation and PointRCNN [26] for object detection. Both
architectures consist of MLP modules responsible for attaching rich spatial
features to individual scan points, followed by task-dependent modules. In
contrast to single-frame perception, we must handle successive scans;
therefore, we propose a time-aware extension of their original MLPs.

Given a sequence $X_{\mathop{\mbox{-}}n:n}$ of point clouds, the backbone must
attach a feature vector $\bm{h}(\bm{x}_{0})$ to each point $\bm{x}_{0}$ in the
reference scan $X_{0}$. The backbone module is a fully connected encoder-
decoder that relies on local grouping of points. We perform a spatio-temporal
grouping step by which each point $\bm{x}_{0}\in X_{0}$ is associated with a
spatio-temporal neighborhood $\mathcal{N}(\bm{x}_{0})$. The spatio-temporal
neighborhood of each point $\bm{x}_{0}\in X_{0}$ (Fig. 3) is defined as:

$\mathcal{N}(\bm{x}_{0})\\!=\\!\big{\\{}\bm{x}_{t}\\!\in\\!X_{t}:\|\bm{x}_{t}\\!-\\!\bm{x}_{0}\|\leq
r(|t|),\;t\in\llbracket-n,n\rrbracket\big{\\}},$ (1)

where $r$ is an increasing function as in [18]. The feature of each
$\bm{x}_{0}\in X_{0}$ is finally defined as:

$\bm{h}(\bm{x}_{0})=\max_{\bm{x}_{t}\in\mathcal{N}(\bm{x}_{0})}\big{\\{}\phi(\bm{x}_{t}-\bm{x}_{0},t)\big{\\}},$
(2)

that is, by max-pooling of time-aware pairwise features over the spatio-
temporal neighborhood, where $\phi$ is an MLP to be trained, and
$t\in\llbracket-n,n\rrbracket$.

The construction of the spatio-temporal neighborhoods obeys the intuition that
the maximum distance an object can travel between two scans increases with the
object’s speed and the temporal separation between the scans. Thus, the
maximum spatial distance for grouping points should increase with their
temporal distance.

Figure 3: Time-aware neighborhood of a point in the reference scan. Points
from different times are presented in different colors, where circled green
point ${\bm{x}}_{0}$ is from the (green) reference frame $X_{0}$. Its spatio-
temporal neighborhood $\mathcal{N}({\bm{x}}_{0})$ is composed of all points in
scan $X_{t}$ in a spatial radius $r(|t|)$, for $t=-n,\cdots,n$. Crossed points
are excluded from this spatio-temporal neighborhood

Task-dependent modules. The previous feature extraction method provides point-
wise feature vectors along with their corresponding 3D positions for the
points in the reference coordinate frame of $X_{0}$. The feature vectors are
then fed into subsequent task-dependent modules.

3D Object Detection: We consider the 3D detection of vehicles with our multi-
frame adaptation of PointRCNN [26]. Here, the bounding box labels are not
associated with specific input points, but rather with a specific 3D position.
If we consider box labels from all frames, we would not know which points on
input are associated with each one of them. Therefore, we choose to use the
bounding box labels only for the reference scan $X_{0}$ and mask the box
labels from other frames. This differs from semantic segmentation, where each
point has its own exclusive class probability in each frame.

3D Semantic Segmentation: We adapt Cylinder3D [39] and extend it into a semi-
supervised approach. A trained Cylinder3D classifier semantically labels each
point into corresponding categories. Here, we use labels for all temporal
input instants.

Training data. Let us denote $N$ the maximum past or future time horizon
around reference frames. We are interested in learning under challenging
conditions, where we have access to only a small set $\mathcal{D}^{\ell}$ of
labeled synchronized point cloud sequences $X_{\mathop{\mbox{-}}N:N}$. Scan
labeling is a collection of per-point semantic labels for segmentation and
vehicle bounding boxes for detection. We also assume the access to a large set
$\mathcal{D}^{\mathrm{u}}$ of unlabelled point cloud sequences for pseudo
labeling. We mainly consider two types of teacher models for training: (i)
$T_{\mathop{\mbox{-}}n:n}$ with access to future frames, and (ii)
$T_{\mathop{\mbox{-}}n:0}$, which has access only to past frames.

To understand the effect of distilling a teacher model with access to
privileged information $T_{\mathop{\mbox{-}}n:n}$, we have performed an
experiment where we train a student model by the pseudo-labels provided by one
teacher with and without privileged information. As shown in Fig. 4 distilling
a single teacher with privileged information in a student model
$S_{\mathop{\mbox{-}}m:0}\shortleftarrow T_{\mathop{\mbox{-}}n:n}$ provides
the best performance (red on top) over the baseline supervised student (black
pillars) and over distilling a single teacher without privileged information
into a student model $S_{\mathop{\mbox{-}}m:0}\shortleftarrow
T_{\mathop{\mbox{-}}n:0}$ (blue pillars).

Figure 4: Impact of knowledge distillation. Performance of distilling
privileged information into student in 3D object detection task.

Concordance and selection of pseudo-labels. Inspired by our finding from Fig.
4, we want to fully exploit the information contained in the available scan
sequences of length $(2N+1)$. We propose Concordance of teachers, where we
train a set of $N$ teachers with a varying span of the temporal context. It
means that teacher $T_{\mathop{\mbox{-}}1:1}$ is trained on the subsequences
$X_{\mathop{\mbox{-}}1:1}$, teacher $T_{\mathop{\mbox{-}}2:2}$ on
$X_{\mathop{\mbox{-}}2:2}$ and so on, for the sake of simplicity, we further
drop the distinction between these sequences and assume that given a sequence
$X\in\mathcal{D}^{\textrm{u}}$ any network will crop the temporal range
appropriately. Given the Concordance of teachers

$\mathcal{T}^{\\{1,\dots,N\\}}=\\{T_{\mathop{\mbox{-}}1,1},T_{\mathop{\mbox{-}}2,2},\dots,T_{\mathop{\mbox{-}}N,N}\\},$
(3)

we process every unlabeled sequence $X\in\mathcal{D}^{\textrm{u}}$ by all
trained teachers independently. Since the nature of outputs is slightly
different for the semantic segmentation (pixel-wise class probabilities) and
for the object detection (3D bounding boxes with class probabilities), we
split the Concordance description at this point to avoid any
misinterpretation.

Concordance for semantic segmentation. We estimate the pseudo-label $k^{*}$
and its confidence $c$ for each output pixel. Given the pixel, each teacher
$T\in\mathcal{T}^{\\{1,\dots,N\\}}$ provides a vector of class probabilities
${\hat{\mathbf{y}}^{T}}$. We then estimate the teacher-wise pseudo-labels
$k^{*}(T)$

$k^{*}(T)=\operatorname*{argmax}_{k\in K}{\hat{\mathbf{y}}^{T}}_{k},$ (4)

We pay special attention to the teacher with the strongest opinion, that is,
the one with the highest output value $\hat{\mathbf{y}}_{k}^{T}$. In
particular, we denote this teacher as $T^{*}$, its pseudo-label $k^{*}$ and
the score of this pseudo-label as $y^{*}=\hat{\mathbf{y}}_{k^{*}}^{T^{*}}$.

We determine the pseudo-label of the given output pixel as $k^{*}$. The
confidence of $k^{*}$ is defined as the weighted sum of two criteria: (i) the
score $y^{*}$ of the pseudo-label and (ii) the number of other teachers $T$
that predict the same class, that is, for which $k^{*}=k^{*}(T)$. The trade-
off between these two criteria is determined by a nonnegative weight $\lambda$
as follows:

$\hat{c}=y^{*}+\lambda\sum_{T\in\mathcal{T}\setminus
T^{*}}\mathds{1}\llbracket k^{*}=k^{*}(T)\rrbracket\,.$ (5)

To preserve compatibility with training loss Eq. (7), this confidence is
clipped:

$c=\min(1,\hat{c})\,.$ (6)

Concordance for object detection. Each teacher provides a different set of 3D
bounding boxes (bbs) and class probabilities. To measure the Concordance, we
need to extract bbs that correspond to a single physical object. To this end,
we greedily search for clusters of bbs with mutual intersection-over-union
above a user-defined threshold. The algorithm starts by building the first
cluster from the strongest bb. Once there are no more bbs with a sufficient
IoU with the strongest bb, we stop building the cluster, suppress all
associated bbs and continue by building a following cluster from the remaining
bbs. The class probabilities $\mathbf{y}^{T}$ corresponding to each single
cluster are then used directly to compute the pseudo-label and its confidence
by the same procedure as for the semantic segmentation. Following the standard
practice in pseudo-labeling (e.g., [19] in object detection or [16] in
semantic segmentation), the final selection of pseudo-labels is obtained by
thresholding the confidence. Individual pseudo-labels with confidence below
the chosen threshold are masked out of the loss function and treated as a
Don’t Care class, since they are potentially incorrect predictions. The final
selection of pseudo-labels and associated data form the training set
$\mathcal{D}^{\mathrm{p}}$.

Training the student. The student model only performs inference online and,
therefore, is learned only with past input sequences. Following pseudo-
labeling through the teacher-student framework [14, 27], we train the student
model on both the human-labeled set and pseudo-labeled one. The loss function
for training the student is summed as follows:

$\mathcal{L}=\frac{1}{M}\sum_{(\mathbf{y},c)\in\mathcal{D}^{\mathrm{p}}}c~{}\mathcal{L}_{\text{task}}(\hat{\mathbf{y}},\mathbf{y}),$
(7)

where $\mathbf{y}$ is one hot label encoding vector, $\mathbf{\hat{y}}$ are
model predictions, $c$ is the corresponding confidence from our Concordance
selection. Samples collected from the original human-labeled dataset have
$c=1$. The original loss function of the task-dependent module is denoted
$\mathcal{L}_{\text{task}}$. All data are sampled from the combined dataset
$\mathcal{D}^{\mathrm{p}}$, and M is the number of samples in
$\mathcal{D}^{\mathrm{p}}$.

##  IV Experiments

TABLE I: LiDAR semantic segmentation performance on SemanticKITTI single-scan test set. Our method, $S_{0:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ (‘Ours (20%)’) utilizes only 20% of the GT annotation, the remaining 80% of training data being automatically annotated, to achieve a performance comparable with state-of-the-art methods. ‘Cylinder3D (20%)’ denotes Cylinder3D [38] trained, like our method, with only 20% of ground-truth annotation; all other results are obtained from the literature, where full GT annotation is used. Performance in IoU percentages, per class and averaged, the higher the better.  |  mIoU |  car |  bicycle |  motorcycle |  truck |  o-vehicle |  person |  bicyclist |  motorcyclist |  road |  parking |  sidewalk |  o-ground |  building |  fence |  vegetation |  trunk |  terrain |  pole |  traffic-sign
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Darknet53 [1] | 49.9 | 86.4 | 24.5 | 32.7 | 25.5 | 22.6 | 36.2 | 33.6 | 4.7 | 91.8 | 64.8 | 74.6 | 27.9 | 84.1 | 55.0 | 78.3 | 50.1 | 64.0 | 38.9 | 52.2
RandLNet [11] | 50.3 | 94.0 | 19.8 | 21.4 | 42.7 | 38.7 | 47.5 | 48.8 | 4.6 | 90.4 | 56.9 | 67.9 | 15.5 | 81.1 | 49.7 | 78.3 | 60.3 | 59.0 | 44.2 | 38.1
RangeNet+ [20] | 52.2 | 91.4 | 25.7 | 34.4 | 25.7 | 23.0 | 38.3 | 38.8 | 4.8 | 91.8 | 65.0 | 75.2 | 27.8 | 87.4 | 58.6 | 80.5 | 55.1 | 64.6 | 47.9 | 55.9
PolarNet [36] | 54.3 | 93.8 | 40.3 | 30.1 | 22.9 | 28.5 | 43.2 | 40.2 | 5.6 | 90.8 | 61.7 | 74.4 | 21.7 | 90.0 | 61.3 | 84.0 | 65.5 | 67.8 | 51.8 | 57.5
SquezSgv3 [34] | 55.9 | 92.5 | 38.7 | 36.5 | 29.6 | 33.0 | 45.6 | 46.2 | 20.1 | 91.7 | 63.4 | 74.8 | 26.4 | 89.0 | 59.4 | 82.0 | 58.7 | 65.4 | 49.6 | 58.9
KPConv [30] | 58.8 | 96.0 | 32.0 | 42.5 | 33.4 | 44.3 | 61.5 | 61.6 | 11.8 | 88.8 | 61.3 | 72.7 | 31.6 | 95.0 | 64.2 | 84.8 | 69.2 | 69.1 | 56.4 | 47.4
Salsanext [7] | 59.5 | 91.9 | 48.3 | 38.6 | 38.9 | 31.9 | 60.2 | 59.0 | 19.4 | 91.7 | 63.7 | 75.8 | 29.1 | 90.2 | 64.2 | 81.8 | 63.6 | 66.5 | 54.3 | 62.1
FusionNet [35] | 61.3 | 95.3 | 47.5 | 37.7 | 41.8 | 34.5 | 59.5 | 56.8 | 11.9 | 91.8 | 68.8 | 77.1 | 30.8 | 92.5 | 69.4 | 84.5 | 69.8 | 68.5 | 60.4 | 66.5
Cylinder3D [38] | 61.8 | 96.1 | 54.2 | 47.6 | 38.6 | 45.0 | 65.1 | 63.5 | 13.6 | 91.2 | 62.2 | 75.2 | 18.7 | 89.6 | 61.6 | 85.4 | 69.7 | 69.3 | 62.6 | 64.7
Torando [8] | 63.1 | 94.2 | 55.7 | 48.1 | 40.0 | 38.2 | 63.6 | 60.1 | 34.9 | 89.7 | 66.3 | 74.5 | 28.7 | 91.3 | 65.6 | 85.6 | 67.0 | 71.5 | 58.0 | 65.9
Cylinder3d (20%) | 51.9 | 92.1 | 31.7 | 28.5 | 25.1 | 22.1 | 49.6 | 32.4 | 26.4 | 86.9 | 47.2 | 67.5 | 12.6 | 88.7 | 55.7 | 83.2 | 64.4 | 64.8 | 53.4 | 53.9
Ours (20%) | 58.9 | 92.9 | 46.4 | 36.6 | 35.1 | 27.3 | 62.4 | 54.0 | 24.0 | 90.0 | 60.8 | 72.1 | 22.2 | 92.0 | 65.6 | 84.6 | 70.3 | 63.7 | 59.3 | 60.2
TABLE II: LiDAR semantic segmentation performance on SemanticKITTI multi-scan test set. Moving object classes are prefixed with ‘mv’; ‘CylAsy3D(20%)’ denotes Cylinder3D [41] trained, like our method, with only 20% of ground-truth annotation. We have omitted cyclists and bicyclists categories from this table as all methods reported $\sim$ 0.0 IoU.  |  mIoU |  car |  bicycle |  motorcycle |  truck |  o-vehicle |  person |  road |  parking |  sidewalk |  o-ground |  building |  fence |  vegetation |  trunk |  terrain |  pole |  traffic-sign |  mv-car |  mv-truck |  mv-other |  mv-person |  mv-biclist |  mv-motor
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
TangCov [29] | 34.1 | 84.9 | 2.0 | 18.2 | 21.1 | 18.5 | 1.6 | 83.9 | 38.3 | 64.0 | 15.3 | 85.8 | 49.1 | 79.5 | 43.2 | 56.7 | 36.4 | 31.2 | 40.3 | 1.1 | 6.4 | 1.9 | 30.1 | 42.2
DarkNet53 [1] | 41.6 | 84.1 | 30.4 | 32.9 | 20.0 | 20.7 | 7.5 | 91.6 | 64.9 | 75.3 | 27.5 | 85.2 | 56.5 | 78.4 | 50.7 | 64.8 | 38.1 | 53.3 | 61.5 | 14.1 | 15.2 | 0.2 | 28.9 | 37.8
SpSeqnet [25] | 43.1 | 88.5 | 24.0 | 26.2 | 29.2 | 22.7 | 6.3 | 90.1 | 57.6 | 73.9 | 27.1 | 91.2 | 66.8 | 84.0 | 66.0 | 65.7 | 50.8 | 48.7 | 53.2 | 41.2 | 26.2 | 36.2 | 2.3 | 0.1
KPConv [30] | 51.2 | 93.7 | 44.9 | 47.2 | 42.5 | 38.6 | 21.6 | 86.5 | 58.4 | 70.5 | 26.7 | 90.8 | 64.5 | 84.6 | 70.3 | 66.0 | 57.0 | 53.9 | 69.4 | 0.5 | 0.5 | 67.5 | 67.4 | 47.2
CylAsy3D [41] | 51.5 | 93.8 | 67.6 | 63.3 | 41.2 | 37.6 | 12.9 | 90.4 | 66.3 | 74.9 | 32.1 | 92.4 | 65.8 | 85.4 | 72.8 | 68.1 | 62.6 | 61.3 | 68.1 | 0.0 | 0.1 | 63.1 | 60.0 | 0.4
CylAsy3D (20%) | 42.1 | 89.4 | 35.2 | 22.9 | 16.3 | 15.9 | 11.6 | 88.1 | 53.9 | 69.2 | 12.6 | 88.6 | 56.8 | 83.2 | 65.7 | 61.3 | 53.2 | 59.2 | 65.8 | 0.0 | 0.0 | 43.3 | 47.9 | 12.8
Ours (20%) | 47.2 | 93.0 | 45.3 | 35.7 | 27.4 | 19.4 | 14.4 | 90.5 | 61.4 | 75.0 | 15.6 | 91.3 | 62.1 | 83.3 | 69.3 | 64.0 | 59.7 | 63.6 | 77.4 | 0.0 | 0.0 | 64.0 | 57.5 | 9.5

The experiments show that our method, with only 20% of the training data being
manually annotated, is competitive with state-of-the-art fully-supervised
semantic segmentation methods that use full annotation of the training data.
Furthermore, our method achieves state-of-the-art performance in semi-
supervised semantic segmentation of LiDAR scans. Finally, we show that it can
also generalize to different tasks such as 3D object detection.

###  IV-A Datasets

We evaluate our approach on the Argoverse dataset [4] for 3D object detection,
particularly for the vehicle category, and the large-scale driving scene
SemanticKITTI dataset [1] for 3D semantic segmentation. In every experiment,
we first warp all $2n$ past and future LiDAR scans into the reference scan
coordinate system. Once combined, these scans appear as captured by the same
sensor at different times, with static objects being densified and moving ones
being smudged (see LiDAR data in Fig. 5(b)).

Argoverse [4] provides a large collection of LiDAR sequences, with 3D
bounding-box labels available for each frame. For these experiments, we
utilize the first $\sim\,10\%$ of human-labeled sequences to form
$\mathcal{D}^{\ell}$, the remaining $\sim\,90\%$ being gathered without
annotation in $\mathcal{D}^{\mathrm{u}}$ and thus made available for pseudo-
labeling.

SemanticKITTI [1] provides a large-scale driving-scene point cloud for 3D
semantic segmentation. The dataset consists of 22 sequences with a split of
sequences from 00 to 10 for training (08 is reserved for validation) and from
11 to 21 for testing. The dataset has two challenges, namely _single-scan_
with only 19 class categories, and _multi-scan_ with 25 class categories,
including 19 from single-scan and 6 moving-object categories. For this
experiment, we cut each training sequence into two parts, the first 20% for
human-labeled dataset $\mathcal{D}^{\ell}$ and the latter 80% for unlabeled
dataset $\mathcal{D}^{\mathrm{u}}$.

###  IV-B 3D Semantic Segmentation

In the 3D semantic segmentation experiment, we train a student model on
pseudo-labels generated by the concordance of teachers. Here, we utilize
Cylinder3D and the output class probabilities for all individual scan points
are treated with our confidence-guided criterion (Eqs. 4, 5, 6). Training is
done by optimizing the cross-entropy loss and the Lovasz-softmax loss [2]
weighted by our confidence-guided criterion, as in Eq. 7, for multi-class
semantic segmentation. The standard mean Intersection over Union (mIoU) metric
is used for evaluation.

Single-scan semantic segmentation. In this experiment, we compare the results
of our method with the LiDAR segmentation state of the art on SemanticKITTI
single-scan test set. As shown in Table I, our method
$S_{0:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ trained with only
$20\%$ of ground truth (GT) and $80\%$ of pseudo-labels outperforms all
methods based on 3D-to-2D projection and fully-annotated training data,
including Darknet53 [1], SqueezeSegv3 [34], RangeNet++ [20] and PolarNet [36].
Moreover, our method shows comparable results to voxel partition and 3D
convolution-based methods, including FusionNet [35], Salsanext [7], Torando
[8] and Cylinder3D111The authors of Cylinder3D reported a performance of 67.8
mIoU in [41]. However, they have added the validation dataset to the training
and used test-time augmentation (TTA), which is not the standard evaluation
setup. Hence, we are not comparing with this performance. [38]. Further, we
present some qualitative results in Fig. 5(a), which show that our model helps
to improve the segmentation quality as compared to its supervised-only
counterparts.

Multi-scan semantic segmentation. Compared to the single-scan set-up, the
multi-scan segmentation in SemanticKITTI has six more categories accounting
for moving objects (car, truck, other-vehicle, person, bicyclist and
motorcyclist). In this experiment, all methods utilize multiple input point
clouds. In Table II, we show that our method, with only $20\%$ of human-
labeled training data, outperforms methods that use full manual annotations,
namely TangConv[29], DarkNet53[1] and SpSeqnet[25], and is on par with
KPConv[30] and CylAsy3D[41]. Our method outperforms all others on the moving-
car and traffic-sign categories.

TABLE III: Semi-supervised learning on SemanticKITTI validation set. Performance in mIoU (%) with ratio of labeled data varying in {20%, 30%, 40%}. ‘Guided-Point-SSL’ denotes [12] semi-supervised models; ‘$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!T_{\mathop{\mbox{-}}2:2}$’ and ‘$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$’ denote our approach with distillation from a single teacher and a concordance of teachers respectively. |  | Labeled data |
---|---|---|---
method | 20% | 30% | 40%
Guided Point SSL [12] | 58.8 | 59.4 | 59.9
$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!T_{\mathop{\mbox{-}}2:2}$ (Ours) | 59.0 | 59.3 | 59.9
$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ (Ours) | 60.6 | 61.6 | 62.8

Comparison to state-of-the-art semi-supervised method. To further assess the
merit of our approach, we compare it to the most recent semi-supervised
segmentation work, Guided-Point-SSL [12]. As shown in Table. III, while our
method learned from a single teacher
($S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!T_{\mathop{\mbox{-}}2:2}$) is
on par with Guided-Point-SSL, once learned from Concordance of teachers
($S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$), it
outperforms Guided-Point-SSL with $20\%$, $30\%$ and $40\%$ labeled data by
1.8, 2.2 and 2.9 mIoU points respectively.

###  IV-C 3D object detection

The models are learned as in the original PointRCNN [26] design, with the
exception of the confidence-guided criterion and the usage of multiple frames
at input. We perform this by weighting the classification and regression
branches of the original loss function of the PointRCNN by Eq. 7.

TABLE IV: Results of 3D object detection. Detection performance (AP percentage) of proposed students trained with 10% human-labeled training data and of oracle model trained with full GT annotation. $S_{\mathop{\mbox{-}}5:0}$ | $S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!T_{\mathop{\mbox{-}}5:5}$ | $S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{3,4,5\\}}$ | Oracle
---|---|---|---
(Baseline) | (Ours) | (Ours) |
51.1 | 56.7 | 58.3 | 62.6

In Table. IV, we show that the student
$S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{3,4,5\\}}$
learned from the concordance of teachers achieved 58.3 AP when trained with
10% human-labeled training data, outperforming the student learned from a
single teacher
($S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!T_{\mathop{\mbox{-}}5:5}$) by
1.6 AP as well as the baseline student ($S_{\mathop{\mbox{-}}5:0}$) by 7.2 AP.
Moreover, it closes in large part the gap between the baseline and the
‘Oracle’ (the model $S_{\mathop{\mbox{-}}5:0}$ trained with 100$\%$ GT labeled
training data)222Since the argoverse leaderboard does not contain information
about the methods used for the reported performance, we are unable compare to
the state of the art on this dataset..

(a) A scene from SemanticKITTI with its GT semantic segmentation (1st column)
showing that our method
$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ (3rd
column) can segment cars (blue), side-walks (brown) and drivable area (pink)
better than the supervised baseline based on Cylinder3D (2nd column). Correct
and incorrect segmentation are indicated by green ✓and red ✗ respectively.
Both methods utilize 20% of human-labeled data.

(b) A scene from Argoverse showing that student
$S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{3,4,5\\}}$ can
be robust to severe occlusions. Some of the rightly detected boxes do not
contain GT vehicle points since they correspond to vehicles occluded in the
current frame and partially occluded in the past frames, as it can be observed
in the ring of camera images. While they are hard to detect due to the small
number of points, our student succeeds in detecting them thanks to the
knowledge distilled by the Concordance of teachers.

Figure 5: Qualitative results. Examples of (a) 3D semantic segmentation and
(b) object detection.

##  V Ablation Studies

Concordance of teachers vs. Ensemble. To improve the quality of pseudo-
labeling, we apply our concordance-driven criterion (Eq. 5), which is a
confidence-guided criterion using the temporal consistency of multiple
teachers with access to different time windows. In particular, we initialize
several teachers operating on different temporal ranges.

TABLE V: Concordance of teachers vs. ensemble of teachers. The student trained
with concordance of teachers outperforms the one trained with ensemble of
teachers, in both (a) 3D object detection and (b) semantic segmentation.

(a) Object detection (AP) $S_{\mathop{\mbox{-}}3:0}\\!\shortleftarrow\\!\mathcal{E}^{\\{3,3\\}}$ | 54.3
---|---
$S_{\mathop{\mbox{-}}3:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{2,3\\}}$ | 55.6
$S_{\mathop{\mbox{-}}4:0}\\!\shortleftarrow\\!\mathcal{E}^{\\{4,4\\}}$ | 57.2
$S_{\mathop{\mbox{-}}4:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{3,4\\}}$ | 57.7
$S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!\mathcal{E}^{\\{5,5\\}}$ | 58.0
$S_{\mathop{\mbox{-}}5:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{4,5\\}}$ | 58.2

(b) Semantic seg. (mIoU) $S_{\mathop{\mbox{-}}1:0}\\!\shortleftarrow\\!\mathcal{E}^{\\{1,1,1\\}}$ | 59.5
---|---
$S_{\mathop{\mbox{-}}1:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ | 60.6
$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!\mathcal{E}^{\\{2,2,2\\}}$ | 59.9
$S_{\mathop{\mbox{-}}2:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ | 60.6
$S_{\mathop{\mbox{-}}3:0}\\!\shortleftarrow\\!\mathcal{E}^{\\{3,3,3\\}}$ | 60.0
$S_{\mathop{\mbox{-}}3:0}\\!\shortleftarrow\\!\mathcal{T}^{\\{1,2,3\\}}$ | 60.9

To show that the temporal diversity among teachers beats the various training
initializations of the standard ensemble, we performed a comparison with an
ensemble of teachers; Following the Concordance notation, the Ensemble of
teachers is denoted
$\mathcal{E}^{\\{N,\dots,N\\}}={\\{T^{1}_{\mathop{\mbox{-}}n,n},\dots,T^{K}_{\mathop{\mbox{-}}N,N}\\}}$,
each teacher $T^{k}_{\mathop{\mbox{-}}n,n}$ operating in _the same_ temporal
range but with a different random initialization. As shown in Table. V(b), the
student taught by a Concordance of teachers outperforms the one taught by the
Ensemble of teachers in both 3D object detection and 3D semantic segmentation.

Selection of pseudo-labels. We perform an ablation study that demonstrates the
benefits of the proposed confidence-guided criterion. The baseline here is the
standard method, as used in [19, 39, 33], which tunes a single confidence-
based threshold (CT) for all pseudo-samples. The comparison is depicted in
Fig. 6. Models with the proposed selection criterion outperform models with
only CT across different confidence thresholds.

Figure 6: Ablation on PL’s selection strategies. Detection performance as a
function of threshold $\theta$ and influence of our proposed confidence-guided
criterion over the standard confidence-based threshold (CT) [19, 39, 33]

##  VI Conclusion

We propose a novel pseudo-labeling framework that leverages spatio-temporal
information from unlabeled sequences of point clouds. We demonstrated its
merit in two 3D perception tasks on publicly available datasets. The reported
performance gains stem from: (i) A better selection of the final pseudo-labels
via the concordance of multiple teachers operating at different temporal
ranges; (ii) A novel pseudo-label confidence-guided criterion. Thanks to the
privileged information available in the different temporal ranges, the
Concordance of teachers delivers strong pseudo-labeled samples. Using manual
labeling of only 20% of training data, our method achieves state-of-the-art
performance in semi-supervised 3D semantic segmentation and competes even with
methods that use the full set of labels on this task. By the nature of our
pseudo-labeling framework, the proposed approach is complementary to other
techniques that use sequential data, and can thus be combined with them to
further boost the performance.

##  VII Acknowledgment

This work was supported in part by OP VVV MEYS funded project
CZ.02.1.01/0.0/0.0/16_019/0000765 “Research Center for Informatics”, and by
Grant Agency of the CTU Prague under Project SGS22/111/OHK3/2T/13. Authors
want to thank Valeo company for a support.

## References

  * [1] J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, C. Stachniss, and J. Gall.  SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences.  In Proc. of the IEEE/CVF International Conf. on Computer Vision (ICCV), 2019.
  * [2] Maxim Berman, Amal Rannen Triki, and Matthew B Blaschko.  The lovász-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks.  In CVPR, 2018.
  * [3] Benjamin Caine et al.  Pseudo-labeling for scalable 3d object detection.  arXiv, abs/2103.02093, 2021.
  * [4] Ming-Fang Chang, John W Lambert, Patsorn Sangkloy, Jagjeet Singh, Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, and James Hays.  Argoverse: 3D tracking and forecasting with rich maps.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [5] Jang Hyun Cho and Bharath Hariharan.  On the efficacy of knowledge distillation.  In IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
  * [6] Christopher B. Choy, JunYoung Gwak, and Silvio Savarese.  4d spatio-temporal convnets: Minkowski convolutional neural networks.  In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 3075–3084. Computer Vision Foundation / IEEE, 2019.
  * [7] Tiago Cortinhal, George Tzelepi, and Eren Aksoy.  Salsanext: Fast, uncertainty-aware semantic segmentation of lidar point clouds for autonomous driving.  In 15th International Symposium ISVC 2020.
  * [8] Martin Gerdzhev, Ryan Razani, Ehsan Taghavi, and Liu Bingbing.  Tornado-net: multiview total variation semantic segmentation with diamond inception module.  In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 9543–9549. IEEE, 2021.
  * [9] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al.  Distilling the knowledge in a neural network.  In NIPS Deep Learning and Representation Learning Workshop, 2015\.
  * [10] Peiyun Hu, Jason Ziglar, David Held, and Deva Ramanan.  What you see is what you get: Exploiting visibility for 3d object detection.  In CVPR 2020, pages 10998–11006, 06 2020.
  * [11] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham.  Randla-net: Efficient semantic segmentation of large-scale point clouds.  In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11108–11117, 2020.
  * [12] Li Jiang, Shaoshuai Shi, Zhuotao Tian, Xin Lai, Shu Liu, Chi-Wing Fu, and Jiaya Jia.  Guided point contrastive learning for semi-supervised point cloud semantic segmentation.  In Proc. of the IEEE/CVF International Conference on Computer Vision, pages 6423–6432, 2021.
  * [13] John Lambert, Ozan Sener, and Silvio Savarese.  Deep learning under privileged information using heteroscedastic dropout.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8886–8895, 2018.
  * [14] D. Lee.  Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks.  In International Conference on Machine Learning (ICML). Workshop Challenges in Representation Learning, 2013.
  * [15] Li-Jia Li, Gang Wang, and Fei-Fei Li.  OPTIMOL: automatic online picture collection via incremental model learning.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2007.
  * [16] Yuexiang Li, J. Chen, Xinpeng Xie, Kai Ma, and Y. Zheng.  Self-loop uncertainty: A novel pseudo-label for semi-supervised medical image segmentation.  The Medical Image Computing and Computer Assisted Intervetion, (MICCAI), pages 614–623, 09 2020.
  * [17] I Liu et al.  Knowledge flow: Improve upon your teachers.  In 7th International Conference on Learning Representations, (ICLR), 2019.
  * [18] Xingyu Liu et al.  Meteornet: Deep learning on dynamic 3d point cloud sequences.  In ICCV, 2019.
  * [19] Yen-Cheng Liu, Chih-Yao Ma, Zijian He, Chia-Wen Kuo, Kan Chen, Peizhao Zhang, Bichen Wu, Zsolt Kira, and Peter Vajda.  Unbiased teacher for semi-supervised object detection.  In International Conference on Learning Representations, (ICLR), 2021.
  * [20] Andres Milioto, Ignacio Vizzo, Jens Behley, and C. Stachniss.  Rangenet ++: Fast and accurate lidar semantic segmentation.  2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4213–4220, 2019.
  * [21] Seyed-Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, and Hassan Ghasemzadeh.  Improved knowledge distillation via teacher assistant: Bridging the gap between student and teacher.  In 34th AAAI Conference on Artificial Intelligence, 2019.
  * [22] Charles R Qi, Yin Zhou, Mahyar Najibi, Pei Sun, Khoa Vo, Boyang Deng, and Dragomir Anguelov.  Offboard 3d object detection from point cloud sequences.  In CVPR, 2021.
  * [23] Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas.  PointNet: Deep learning on point sets for 3D classification and segmentation.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  * [24] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas.  Pointnet++ and three layers of features fusion for occlusion three-dimensional ear recognition based on one sample per person.  Symmetry, 12(1):78, 2020.
  * [25] Hanyu Shi, Guosheng Lin, Hao Wang, Tzu-Yi Hung, and Zhenhua Wang.  Spsequencenet: Semantic segmentation network on 4d point clouds.  In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [26] Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li.  PointRCNN: 3D object proposal generation and detection from point cloud.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–779, 2019.
  * [27] Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng MaXiaoyu Tao, and Nanning Zheng.  Transductive semi-supervised deep learning using min-max features.  In Proc. of the European Conference on Computer Vision (ECCV), September 2018.
  * [28] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  In Advances in Neural Information Processing Systems, volume 33, pages 596–608. Curran Associates, Inc., 2020.
  * [29] Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian-Yi Zhou.  Tangent convolutions for dense prediction in 3d.  In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3887–3896, 2018.
  * [30] Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas.  Kpconv: Flexible and deformable convolution for point clouds.  In Proc. of the IEEE/CVF international conference on computer vision, pages 6411–6420, 2019.
  * [31] Vladimir Vapnik et al.  A new learning paradigm: Learning using privileged information.  Neural Networks, 22(5):544–557, 2009.  Advances in Neural Networks Research: IJCNN2009.
  * [32] Vladimir Vapnik and Rauf Izmailov.  Learning using privileged information: Similarity control and knowledge transfer.  Journal of Machine Learning Research, 16(61):2023–2049, 2015.
  * [33] He Wang, Yezhen Cong, Or Litany, Yue Gao, and Leonidas J Guibas.  3DIoUMatch: Leveraging IoU prediction for semi-supervised 3d object detection.  arXiv preprint arXiv:2012.04355, 2020.
  * [34] Chenfeng Xu, Bichen Wu, Zining Wang, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka.  Squeezesegv3: Spatially-adaptive convolution for efficient point-cloud segmentation.  In European Conference on Computer Vision, pages 1–19. Springer, 2020.
  * [35] Feihu Zhang, Jin Fang, Benjamin W. Wah, and Philip H. S. Torr.  Deep fusionnet for point cloud semantic segmentation.  In ECCV, 2020.
  * [36] Yang Zhang, Zixiang Zhou, Philip David, Xiangyu Yue, Zerong Xi, Boqing Gong, and Hassan Foroosh.  Polarnet: An improved grid representation for online lidar point clouds semantic segmentation.  In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9601–9610, 2020.
  * [37] Na Zhao, Tat-Seng Chua, and Gim Hee Lee.  Sess: Self-ensembling semi-supervised 3d object detection.  In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [38] Hui Zhou, Xinge Zhu, Xiao Song, Yuexin Ma, Zhe Wang, Hongsheng Li, and Dahua Lin.  Cylinder3d: An effective 3d framework for driving-scene lidar semantic segmentation.  arXiv preprint arXiv:2008.01550, 2020.
  * [39] Qiang Zhou, Chaohui Yu, Zhibin Wang, Qi Qian, and Hao Li.  Instant-teaching: An end-to-end semi-supervised object detection framework.  In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4081–4090, June 2021.
  * [40] Yin Zhou, Pei Sun, Yu Zhang, Dragomir Anguelov, Jiyang Gao, Tom Ouyang, James Guo, Jiquan Ngiam, and Vijay Vasudevan.  End-to-end multi-view fusion for 3d object detection in lidar point clouds.  In CoRL 2019, 2019.
  * [41] Xinge Zhu, Hui Zhou, Tai Wang, Fangzhou Hong, Yuexin Ma, Wei Li, Hongsheng Li, and Dahua Lin.  Cylindrical and asymmetrical 3d convolution networks for lidar segmentation.  In Proc. of the IEEE/CVF conference on computer vision and pattern recognition, pages 9939–9948, 2021.


#  Vibrational effects in the quantum dynamics of the H + D${}_{2}^{+}$ charge
transfer reaction

O. Ronceroa , V. Andrianarijaonab,c , A. Aguadod and C. Sanz-Sanzd CONTACT O.
Roncero. Email<EMAIL_ADDRESS>aInstituto de Física Fundamental, IFF-
CSIC, c/ Serrano 123, 28006 Madrid, Spain, b Department of Physics , Pacific
Union College, Angwin, California, United States, c Current address:
Department of Physics and Engineering, Southern Adventist University,
Collegedale, Tennessee, United States dUnidad Asociada UAM-CSIC, Departamento
de Química Física Aplicada, Facultad de Ciencias M-14, Universidad Autónoma de
Madrid, 28049, Madrid, Spain

###### Abstract

The H + D${}_{2}^{+}$(v=0,1 and 2) charge transfer reaction is studied using
an accurate wave packet method, using recently proposed coupled diabatic
potential energy surfaces. The state-to-state cross section is obtained for
three different channels: non-reactive charge transfer, reactive charge
transfer, and exchange reaction. The three processes proceed via the
electronic transition from the first excited to the ground electronic state.
The cross section for the three processes increases with the initial
vibrational excitation. The non-reactive charge transfer process is the
dominant channel, whose branching ratio increases with collision energy, and
it compares well with experimental measurements at collision energies around
0.5 eV. For lower energies the experimental cross section is considerably
higher, suggesting that it corresponds to higher vibrational excitation of
D${}_{2}^{+}$(v) reactants. Further experimental studies of this reaction and
isotopic variants are needed, where conditions are controlled to obtain a
better analysis of the vibrational effects of the D${}_{2}^{+}$ reagents.

###### keywords:

charge transfer, quantum dynamics, non-adiabatic dynamics

This is an accepted manuscript of an article that will be published by Taylor
& Francis in “Molecular Physics” on 2021, will be avilable on line:
http://wwww.tandfonline.com/10.1080/00268976.2021.1948125

##  1 Introduction

H2 is the most abundant molecule in the interstellar medium. The gas phase
routes to form molecular hydrogen present very slow rate constants, and its
formation in local galaxies is attributed to reactions on cosmic grains and
ices[1, 2]. However, in environments where grains and ice do not exist, like
in the Early Universe, one key process is the formation of H2 in gas phase.
One of such processes is the charge transfer reaction

$\displaystyle{\rm H}+{\rm H}_{2}^{+}\rightarrow{\rm H}_{2}+{\rm H}^{+}.$ (1)

This reaction has been studied experimentally at rather high energies, $>$ 1
keV, by Karpas et al[3] and by McCartney et al.[4]. Andrianarijaona et al[5,
6] measured the H + D${}_{2}^{+}$ $\rightarrow$ H+ +D2 charge transfer cross
section in a broad energy range, from 0.1eV/u-10 keV/u, thus including
energies more relevant for the astrophysical environment. At this point it is
worth mentioning that the D${}_{2}^{+}$ reactant is excited vibrationally
since it is generated by photoionization or electronic impact. The main goal
of this work is to compare the simulated cross sections for each
D${}_{2}^{+}$(v) vibrational state with the experimental measurements. In this
line, the measurements made in the H + D${}_{2}^{+}$ reaction [5, 6] can then
be considered as benchmark for the comparison with the theoretical
simulations, which conclusions can be generalized to other isotopes and to
similar reactions for other cations.

From the theoretical point of view, reaction (1) has been studied by several
authors [7, 8, 9, 10, 11, 12, 13], using different and increasingly more
accurate coupled diabatic potential energy surfaces (PESs)[14, 15, 16, 17,
18]. Last et al. [7] used a quantum method based on negative imaginary
potential combined with a variational quantum method in a L2 basis set, with
the helicity decoupling approximation, and using approximated PESs. Krstic[9]
used a close coupling method based on the infinite order sudden approximation
(IOSA) with Delves hyperspherical coordinates. Gosh et al. [12] and Sanz-Sanz
et al. [13] used accurate quantum wave packet methods, in hyperspherical and
Jacobi coordinates respectively, and used different coupled diabatic PESs
based on accurate ab initio calculations. All these calculations showed an
important enhancement on a particular vibrational state of H2 due to a quasi-
degeneracy of H${}_{2}^{+}$(v=0) and H2(v’=4). This effect depends strongly on
the long-range behavior of the PESs, very accurately described in the work of
Aguado et al[19], and already used by Sanz-Sanz et al. [13]. In this latter
work, the charge transfer reaction was studied for several isotopic variants.
Among them the H+ D${}_{2}^{+}$(v=0) $\rightarrow$ H+ \+ D2 charge transfer
reaction was studied and compared with the experimental values [5, 6], showing
good agreement for collision energies around 0.5 eV, but some differences
appear at lower energies. These differences were attributed to vibrational
excitation of the D${}_{2}^{+}$, not taken into account by Sanz-Sanz et al.
[13]. The goal of this work is to analyze the effect of vibrational excitation
of D${}_{2}^{+}$, which is expected to present important discrepancies due to
the variation of the energy difference between the initial D${}_{2}^{+}$(v)
and final D2(v’) vibrational levels.

The manuscript is distributed as follows. A brief description of the
theoretical method is presented in section 2. In section 3, the quantum
results are described and compared with the experimental results. Finally, in
section 4 are outlined some conclusions.

##  2 Theoretical method

In this work we use the coupled diabatic PESs developed by Aguado et al. [19]
based on a Diatomics-in-Molecules (DIM) 3$\times$3 matrix description[20, 21],
corrected with diagonal and non-diagonal three body-terms [22] to fit the
energies obtained in Multi-Reference Configuration interaction (MRCI)
calculations with complete basis set extrapolation (CBS). Dealing with a
system of two electrons, MRCI calculations give full configuration
interactions (FCI) results within the space spanned by the orbital set. To
obtain near FCI results cfor complete basis set, a complete basis set
extrapolation method was performed. These PESs include the long-range
interaction very accurately. The dominant terms for reactants H +
H${}_{2}^{+}$ channel are the charge-induced dipole and charge-induced
quadrupole dispersion interactions, varying as $R^{-4}$ and $R^{-6}$
respectively. For the products H+ \+ H2 channel, the main long-range terms are
the charge-quadrupole and the charge-induced dipole dispersion energies, which
vary as $R^{-3}$ and $R^{-4}$ respectively. The DIM diabatic representation is
non diagonal for the description of any H${}_{2}^{+}$ \+ H fragment.
Therefore, a transformation to a new diabatic representation should be done,
in which the PESs are diagonal in the reactants channel while they are non-
diagonal in the two product rearrangement channels, as described by Sanz-Sanz
et al. [13]. The features of the potential in the reactant channel, as a
function of Jacobi r variable, are shown in Fig. 1. At long distances, the
potentials of D2 and D${}_{2}^{+}$ cross at $r=r_{c}$=1.323Å, as shown in Fig.
1.a. The amplitude of D${}_{2}^{+}(v)$ vibrational state is non-zero at this
distance, and, in this outer classical turning point region, the
D${}_{2}^{+}(v)$ levels have a matching overlap with different D2(v’)
vibrational states. As the two reactants approach each other, the degeneracy
between the two first adiabatic states dissappears: the ground state gets
stabilized forming the very stable HD${}_{2}^{+}$ system, which correlates to
H++D2 and D++HD adiabatic asymptotes, while the excited adiabatic state
becomes repulsive (see Fig. 1.b).

(a) Potential of D2 and D${}_{2}^{+}$ at very long distance of H atom, showing
the vibrational levels on each. The energies of D${}^{+}_{2}$(v) are 2.056,
2.251 and 2.439 eV, for v=0, 1 and 2, respectively. The energies of D2(v’) are
1.902, 2.202, 2.487 and 2.758 for v’= 5,6, 7 and 8, respectively.

(b) Adiabatic potential energies of the first 2 singlet electronic states as a
function of R, for $r=r_{c}$ and at a T-shaped configuration. The diabatic
coupling between the two diabatic energies are also shown.

Figure 1: Features of the 3x3 PES’s describing the H+D${}_{2}^{+}$
$\rightarrow$ H+ \+ D2 charge transfer process, where $r$ is the D2
internuclear distance and $R$ is the distance between H and the center-of-mass
of D2.

The reactive and charge transfer collisions have been studied with a quantum
wave packet method using the MADWAVE3 code [23, 24], using the parameters
listed in Table 1 of Ref. [13]. The wave packet is propagated for each total
angular momentum $J$, using a modified Chebyshev propagator[25, 26, 27, 28,
29]. The wave packet is represented in reactant Jacobi coordinates and is
transformed to product Jacobi coordinates [30] at each iteration to extract
the individual state-to-state reaction probabilities, $P^{J}_{\alpha
vj\Omega\rightarrow\alpha^{\prime}v^{\prime}j^{\prime}\Omega^{\prime}}(E)$,
where $\alpha$ denotes the rearrangement channel and the electronic state,
$v,j$ correspond to the vibrational and rotational state of the diatomic
fragment, and $\Omega$ is the projection of the total angular momentum in the
z-axis of the body-fixed frame. The integral state-to-state reactive and
inelastic cross sections are calculated using the partial wave expansion as

$\displaystyle\sigma_{\alpha
vj\rightarrow\alpha^{\prime}v^{\prime}j^{\prime}}(E)$ $\displaystyle=$
$\displaystyle q_{e}{\pi k_{\alpha
vj}^{-2}\over{(2j+1)}}\sum_{J\Omega\Omega^{\prime}}(2J+1)P^{J}_{\alpha
vj\Omega\rightarrow\alpha^{\prime}v^{\prime}j^{\prime}\Omega^{\prime}}(E),$

where $q_{e}=1/4$ is the electronic partition function - note that this factor
was not included in the previous study [13] -. There are four electronic
states correlating to H(2S)+D${}_{2}^{+}(^{2}\Sigma_{g}^{+})$ asymptote, one
singlet and three triplet states. The three triplet states are not connected
to the H+ \+ H2(X${}^{1}\Sigma_{g}^{+}$) channel, and therefore cannot undergo
charge transfer reactions. In this study, $P^{J}(E)$ are calculated with the
MADWAVE3 code for all values with $J<15$ and for $J$ = 20, 30, 40, 50, 60, 70
and 80. The rest of reaction probabilities for all the intermediate
$J^{\prime}s$ are obtained by a linear interpolation based on the J-shifting
approximation, as described before [31, 32], to save computational effort. In
these calculations, the maximum projection, $\Omega$, considered in the
dynamic calculations is 23. In a previous study[13], the role of the trunction
of $\Omega_{max}$ was checked, giving excellent results for the NCRT channel,
and an accuracy better than 2-3 $\%$ for the other two channels.

In what follows, three processes are distinguished

$\displaystyle\begin{array}[]{lccr}{\rm H}+{\rm
D}_{2}^{+}(v,j=0,J)&\longrightarrow&{\rm H}^{+}+{\rm
D}_{2}(v^{\prime},j^{\prime})&{\tiny{\rm non-
reactive\,charge\,transfer(NRCT)}}\\\ &&&\\\ &\longrightarrow&{\rm
HD}^{+}(v^{\prime},j^{\prime})+{\rm D}&{\tiny{\rm exchange\,reaction(ER)}}\\\
&&&\\\ &\longrightarrow&{\rm HD}(v^{\prime},j^{\prime})+{\rm D}^{+}&{\tiny{\rm
reactive\,charge\,transfer(RCT)}}.\\\ \end{array}$ (8)

Charged diatomic products correspond to the first excited electronic singlet
adiabatic state, 2${}^{1}A^{\prime}$, while the product with a charged atomic
fragment corresponds to the ground adiabatic electronic singlet state,
1${}^{1}A^{\prime}$.

The H(${}^{2}S$)+ H${}_{2}^{+}(^{2}\Sigma_{g})$ entrance channel also
correlates with the lowest triplet state of H${}_{3}^{+}$(${}^{3}A^{\prime}$).
This triplet state presents a shallow well at collinear geometry, and has
access to the exchange products, as described before[19, 33]. Therefore, the
triplet state contributes to the exchange reaction, ER, but does not
contribute to the other two charge transfer processes, NRCT and RCT. In the
present calculations dynamical calculations are all performed for the singlet
states. Therefore, the cross section for the RCT channel presented below is
not complete, since they should include the contributions arising from the
triplet states.

##  3 Results

The vibrationally resolved reaction probabilities for $J$=0 are shown in Fig.
2 for H+D${}_{2}^{+}$(v) collisions, for v=0,1 and 2, in the left, middle and
right panels, respectively. The probabilities for the individual final
vibrational states, v’, are included for the three different processes, NRCT
(top panels), ER (middle panels) and RCT (bottom panels). The reaction in the
excited adiabatic electronic state has an extremely high barrier. However, as
the two fragments get closer, the electronic couplings bring a large portion
of the wave packet to the ground adiabatic state correlating to the two charge
transfer channels, NRCT and RCT. The NRCT reaction probabilities are very
close to those of RCT at low energies, except when the D2(v’) level is close
to the D${}_{2}^{+}$(v) state considered, D2(v’=6) for v=0 and 1 and D2(v’=7)
for v=2. According to Fig. 1.a, the final population of D2(v’) with an energy
slightly above than that of the initial D${}_{2}^{+}$(v) is always enhanced.
This is due to the interaction with the H atom, which stabilizes the ground
electronic state, connected to H+\+ D2 asymptote, while the excited state
becomes repulsive (see Fig. 1.b).

Figure 2: Vibrationally resolved reaction probability for $J$=0
H+D${}_{2}^{+}$(v=0,j=0) (left panels), H+D${}_{2}^{+}$(v=1,j=0) (middle
panels), and H +D${}_{2}^{+}$(v=2) (right panels), towards the inelastic
charge transfer, NRCT (H++D2, in the top panels), the exchange reactive
channel, ER (H+HD+, in the middle panels) and reactive charge transfer, RCT,
channels (D+ +HD, in the bottom panels).

For lower energies, the amplitude transfered to the ground electronic state
enters the deep well of HD${}_{2}^{+}$, forming long lived resonances. In this
well, the energy is transfered among all possible modes and the reaction
becomes nearly statistical, as reported for the reaction dynamics in the
ground electronic state [29, 34, 35]. This explains why NRCT and RCT show
similar populations, except for near-resonant v’ levels. This also allows the
transition back to the excited electronic state and explains the appearance of
the ER probabilities in the middle panels of Fig. 2, that cannot reach
directly the excited adiabatic electronic state.

The amplitude of the D${}_{2}^{+}$(v) vibrational state extends over larger
radial distances as v increases, becoming larger at the region of the crossing
between the two electronic states. This can explain why all the probabilities,
NRCT, ER and RCT, increase with v.

As the total angular momentum $J$ increases, the situation gradually changes.
The reaction probabilites for the three vibrational states studied are shown
in Fig. 3 for the 3 different rearrangent channels. As the rotational barrier
increases, the reaction probabilities shift towards higher energy, as
expected. This shift, however, depends on the mechanism. Thus, the two
reactive processes, ER and RCT, have always larger shifts or effective
barriers. As the barrier increases with $J$, the reaction probabilities
decreases.

Figure 3: Reaction probability for different $J$ values in
H+D${}_{2}^{+}$(v=0) (left panels), H+D${}_{2}^{+}$(v=1) (middle panels), and
H +D${}_{2}^{+}$(v=2) (right panels), towards the inelastic charge transfer,
NRCT (H++D2, in the top panels), the exchange reactive channel, ER (H+HD+, in
the middle panels) and the reactive charge transfer, RCT, channels (D+ +HD, in
the bottom panels).

This situation is somehow different for the NRCT channel, which is essentially
inelastic and has lower effective barriers. As shown in the top panel of Fig.
3, the effective barrier decreases progressively when going from v=0 to v=2.
The reason is that the energy difference between the initial (v) and final
(v’) vibrational levels also decreases. Thus, the electronic coupling becomes
more effective at longer distances for higher v’, because lower couplings can
induce the transition.

The resonant structure appearing at low energy progressively dissapears as $J$
increases because the rotational barrier increases and the wave packet can not
reach the deep well in the ground electronic state. Finally, as the reactive
processes decrease, the NRCT proccess increases in intensity with $J$,
becoming dominant. The lower effective barriers observed for the NCRT channel
makes necessary to include more partial waves $J$ to converge the
corresponding cross section in Eq. (2), up to $J$=140, and they were
extrapolated using the J-shifting approximation from the calculated $J$=80.

The cross section for the NRCT, ER, and RCT channels are shown in Fig. 4.
Clearly the ER channel is one order of magnitude lower than the other two.
NRCT and RCT are of the same order at low energies, but as collision energy
increases, RCT decreases while NRCT is either constant or increases for v=1
and 2. At higher energy, it is expected that this difference increases,
becoming dominant the NRCT channel. It should be noted that in the case of
H+H${}_{2}^{+}$ the two charge transfer processes, RCT and NRCT, are
indistinguishible [13].

Figure 4: Reaction cross section for H+D${}_{2}^{+}$(v=0, 1 and 2) towards the
inelastic charge transfer, NRCT (H++D2, in the top panel), the exchange
reactive channel, ER (H+HD+, in the middle panel) and the reactive charge
transfer, RCT, channels (D+ +HD, in the bottom panel). The experimental values
are taken from Ref.[5, 6].

The present theoretical results for each initial v are compared with the
experimental cross sections in the top panel of Fig. 4. In the experimental
work by Andrianarijaona et al[5, 6] on the H + D${}_{2}^{+}$ collisions, the
H+ products were detected, i.e. they provide a direct evidence on the NRCT
cross section. The theoretical NRCT cross section increases as the initial
D${}^{+}_{2}$(v) vibrational state increases, and, for v=2 at collision
energies around 0.5 eV, it matches the corresponding experimental
measurements. This seems to indicate that measurements made at lower
temperature could be due to more vibrationally excited initial D${}_{2}^{+}$
reactants. In the experiments, the D${}_{2}^{+}$ were produced by a
CEA/Grenoble all-permanent magnet Electron Cyclotron Resonance ion source
(ECR) [5, 6], and their vibrational distributions depend on the ECR ion source
parameters, which are not coupled with the ion beam energy. Unfortunately,
these experiments did not include any direct nor indirect measurement of the
vibrational distributions of these D${}_{2}^{+}$ ions. It is expected, though,
that slight changes in the ion source conditions affect the vibrational
distributions of the ions as observed in the case of molecular ions, including
D${}_{2}^{+}$, produced by Duoplasmatron sources [36]. A more detailed
experimental work needs to be done with controlled conditions on the
vibrational excitation of the D${}_{2}^{+}$ reactant ions, specially at low
collision energies, before a factual conclusion could be drawn. The 3-D
imaging technique similar to the one used by Urbain et al. [37] was proven to
be efficient to measure vibrational distributions of H${}_{2}^{+}$ issued from
the primary reaction H+ +H2. In this technique, two position sensitive
detectors detect the positions and time of flights of fragments, giving access
to the kinetic energy release, which definitely contains information on the
vibrational state of the molecular ions. Given the similarity between Urbain
et al..’s molecular ions and the ones in Andrianarijaona et al,’s experiment
[5, 6], the 3-D imagine technique would be an appropriate fit to provide data
that will accomplish our goal. The theoretical simulations presented in this
work are expected to provide a good guidance to interpret such measurements.
Moreover, this should also be extended to the H${}_{2}^{+}$ \+ H reaction due
to its astrophysical relevance, especially in Early Universe models.

In all the mechanisms, the cross section increases with the vibrational
excitation. This is explained, as in the reaction probabilities, by the
increase of the overlap between the initial D${}_{2}^{+}$(v) and dominant
D2(v’) (see Fig. 1.a), which acts as a doorway for the three mechanisms. The
amplitude of the initial vibrational D${}_{2}^{+}$(v) wave function around the
electronic curve crossing region increases, thus favoring the electronic
transition.

Figure 5: Vibrationally resolved state-to-state cross section for
H+D${}_{2}^{+}$(v=0, 1 and 2) towards the inelastic charge transfer, NRCT
(H++D2(v’), in the left panels), and the reactive charge transfer, RCT,
channels (D+ +HD(v’), in the right panels).

The vibrationally resolved state-to-state cross sections for the NRCT and RCT
channels are shown in Fig. 5. There is a clear difference between the two
mechanisms. For RCT, all cross sections decrease with energy, which could be
explained by the near statistical mechanisms, due to the presence of many
resonances associated to the deep HD${}_{2}^{+}$ well that mediate the
reactivity. For NRCT, however, there are several cases: there is always a
dominant v’: v’=6 and 7 for the cases of v=0,1 and v=2, respectively. This
inelastic channel is formed by an electronic transition taking place at rather
long distances. The rest of v’ in the NRCT channel show a progresively
decreasing cross section with decreasing v’, probably due to transitions from
vd’.

Figure 6: NRCT probabilities obtained in H + D${}_{2}^{+}$(v=2) collisions at
several $J$ values using the “exact” wave packet results (WP) and the
approximated time independent close coupling (TICC) results, as described in
the text.

The reason why the NRCT channels v’=6 and 7 for D${}_{2}^{+}$(v=0,1) and v’=7
for D${}_{2}^{+}$(v=2) are dominant at high energy can be easily interpreted
in terms of the reaction probabilities at high energies and high angular
momentum. As $J$ increases, the rotational barrier also increases. This
barrier avoids the wave packet to reach short distances between the two
reactants, and therefore to enter in the deep insertion well of the ground
electronic state, where the reaction takes place. However, the charge transfer
between the initial D${}_{2}^{+}$(v) and nearly resonant D2(v’) can takes
places at rather long distances, at which small electronic couplings can
induce transitions between close lying electronic states. Under these
circumstances, as reaction probabilities decreases, the inelastic NRCT becomes
dominant towards a particular final D2(v’) state.

In order to check this model, we have performed approximate time-independent
close coupling (TICC) calculations, using reactant Jacobi coordinates,
including only one vibrational channel in the initial excited electronic
state, D${}_{2}^{+}$(v=2), and one vibrational state in the final ground
electronic state, D2(v’=7), with 20 rotational channels in each, even $j$
values from 0 to 40. The differential equations are integrated using the
renormalized Numerov method[38, 39] for distances between 2 and 20 bohr, with
300 grid points. Also, the centrifugal sudden approximation is assumed,
keeping only one helicity, $\Omega$=0, since the Coriolis coupling are
expected to play a negligible role at the long distances for which the
electronic transition takes place at high total angular momentum. These TICC
calculations include the NRCT only, and moreover, only consider a possible CT
vibrational product, D2(v’=7). However, the WP presented above include all
rearrangment channels, ER and RCT, and all accessible vibrational states of
the products, becoming much more demanding computationally.

The NRCT probabilities obtained in these model calculations are compared in
Fig. 6 with the accurate wave packet calculations described before for $J$=50,
60 and 70 (denotd by WP). Clearly, as $J$ increases, the agreement between the
two methods improves. This demonstrate the validity of the model. The
oscillations obtained as a function of collision energy are due to the
interference between the entrance and final channels energy difference[40,
41].

This behavior obtained at high $J$ values shows that NRCT is the dominant
reaction channel at high energy. This is valid not only for the reaction
studied in this work but also for all the other isotopic variants, since it is
attributed to the crossing between the neutral and cationic diatomic
fragments, which takes place at very long distances between the reactants.

The TICC model has been extended to calculate the NRCT cross section in a
wider energy range, by solving the close-coupling equations for $J$= 0 to 300.
The simulated NRCT cross sections are compared with the experimental results
of Andrianarijaona et al.[5, 6]. The TICC cross section is very close to the
wave packet results for D2(v’=7) below 1 eV, but is lower than the total WP
NRCT cross section, since it does not take into account the other v’ levels.
As discussed above, the quasi degeneracy between initial D${}_{2}^{+}$(v=2)
and final D2(v’=7) produces a very efficient electronic transition giving rise
to the charge transfer. At low energy, for which relatively low total angular
momentum $J$ are dominant, the rotational barriers are low and the dynamics in
the ground electronic state can access to the deep well of H${}_{3}^{+}$.
Within this well there is an effective energy transfer towards the vibrational
levels of D2(v’) and also access to HD products in other rearrangment
channels. This explains why TICC results are so different to the accurate WP
results below 1eV, since TICC neglects these processes. However, at high
angular momenta, which dominate at high collision energy, the rotational
barrier does not allow the H and D${}^{+}_{2}$ or H++D2 reactants to approach
each other at distances where the vibrational energy transfer or the exchange
reaction can take place. This explains why the TICC approximation yields
results in rather good agreement with the experimental results for collision
energies between 3 and 9 eV.

Figure 7: NRCT cross section obtained in H + D${}_{2}^{+}$(v=2) collisions
comparing the total NRCT “exact” wave packet results (WP), the state-to-state
wave packet cross section for D2(v’=7), and the approximated time independent
close coupling (TICC) results. The experimental values are taken from Ref.[5,
6].

##  4 Conclusions

The reactive cross sections for the H + D${}_{2}^{+}$ charge transfer reaction
increase with the initial vibrational excitation of the D${}_{2}^{+}$(v)
reactant, due to the increase of the amplitude of the vibrational
wavefunctions of D${}_{2}^{+}$ and D2 in the curve crossing and their mutual
overlaps. The dynamics is dominated by the electronic transition from the
excited to the ground electronic state, where the reaction takes place in the
deep insertion well of H${}_{3}^{+}$. The charge transfer electronic
transition is dominated by the crossing of the two D2/D${}_{2}^{+}$ occurring
at rather large distance between reactants, where it can take place between
close lying vibrational states. For this reason, the reaction shows a rather
marked dependency on the initial vibrational excitation. Moreover, this
feature determines that the non-reactive charge transfer becomes the dominant
channel as the collision energy increases, for which the high total angular
momentum introduces a barrier avoiding the access to the insertion minimum.

The non-reactive charge transfer cross section is compared to the experimental
measurements in the 0.01-1 eV collision energy range studied theoretically. A
good agreement is found around 0.5 eV. At lower collision energies, the
experimental cross-section are larger than the experimental ones. This is
attributted to a possible higher vibrational excitation in the generation of
D${}_{2}^{+}$ reagents. Also, a TICC model, including only the initial
D${}_{2}^{+}(v=2)$ and the near resonant D2(v’=7) vibrational states, shows
good agreement with the experimental cross-section in the 3-9 eV collision
energy range. It is concluded that a more detailed experimental study,
focusing on the control of experimental conditions used to generate
D${}_{2}^{+}$, are needed to further analyze the vibrational effects of
reagents on the charge transfer reaction, specially at low energies. For these
studies, the theoretical simulations can be of important help.

## Acknowledgement(s)

We acknowledge computing time at Cibeles (UAM) under RES computational grant
AECT-2021-1-0011.

## Funding

The research leading to these results has received fundings from Ministerio de
Ciencia, Investigación y Universidades (MICIU) (Spain) under grant
FIS2017-83473-C2. V. Andrianarijaona is supported by the National Science
Foundation through Grant No. PHY - 1530944

## References

  * [1] S.C.O. Glover, Astrophys. J. 584, 331 (2003).
  * [2] V. Wakelam, E. Bron, S. Cazaux, F. Dulieu, C. Gry, P. Guillard, E. Habart, L. Hornekær, S. Morisset, G. Nyman, V. Pirronello, S.D. Price, V. Valdivia, G. Vidali and N. Watanabe, Molecular Astrophysics 9, 1 – 36 (2017).
  * [3] Z. Karpas, V. Anicich and W.T. Huntress, J. Chem. Phys. 70, 2877 (1979).
  * [4] P.C.E. McCartney, C. McGrath, J.W. McConkey, M.B. Shah and J. Geddes, J. Phys. B: At. Mol. Opt. Phys. 32, 5103 (1999).
  * [5] V.M. Andrianarijaona, J.J. Rada, R. Rejoub and C.C. Havener, J. Phys.: Conf. Ser. 194, 012043 (2009).
  * [6] V.M. Andrianarijaona, L.M. Wegley, A.Z. Watson, M. Andrianarijaona, C.P. DeGuzman, K. Kim, E.J. Nuss, J.J. Taylor, R.L. Wilson, R.T. Zhang, D.G. Seely and C.C. Havener, AIP Conference Proceedings 2160, 070005 (2019).
  * [7] I. Last, M. Gilibert and M. Baer, J. Chem. Phys. 107, 1451 (1997).
  * [8] H. Kamisaka, W. Bian, K. Nobusada and H. Nakamura, J. Chem. Phys. 116, 654 (2002).
  * [9] P.S. Krstíc, Phys. Rev. A 66, 042717 (2002).
  * [10] P.S. Krstic and R.K. Janev, Phys. Rev. A 67, 022708 (2003).
  * [11] L.F. Errea, A. Macias, L. Méndez, I. Rabadán and A. Riera, Nuclear Instruments and Methods Phys. Research B 235, 362 (2005).
  * [12] S. Ghosh, T. Sahoo, M. Baer and S. Adhikari, J. Phys. Chem. A 125, 731 (2021).
  * [13] C. Sanz-Sanz, A. Aguado and O. Roncero, J. Chem. Phys. 154, 104104 (2021).
  * [14] R. Preston and J. Tully, J. Chem. Phys. 54, 4297 (1971).
  * [15] A. Ichihara, J. Chem. Phys. 103, 2109 (1995).
  * [16] V.G. Ushakov, K. Nobusada and V.I. Osherov, Phys. Chem. Chem. Phys. 3, 63 (2001).
  * [17] S. Mukherjee, D. Mukhopadhyay and S. Adhikari, The Journal of Chemical Physics 141 (20), 204306 (2014).
  * [18] B. Mukherjee, K. Naskar, S. Mukherjee, S. Ghosh, T. Sahoo and S. Adhikari, International Reviews in Physical Chemistry 38, 287 (2019).
  * [19] A. Aguado, O. Roncero and C. Sanz-Sanz, PCCP 23, 7735 (2021).
  * [20] F.O. Ellison, N.T. Huff and J.C. Patel, J. Am. Chem. Soc. 85, 3544 (1963).
  * [21] J.C. Tully, Adv. Chem. Phys. 42, 63 (1980).
  * [22] L.P. Viegas, A. Alijah and A.J.C. Varandas, J Chem Phys 126 (7), 074309 (2007).
  * [23] A. Zanchet, O. Roncero, T. González-Lezana, A. Rodríguez-López, A. Aguado, C. Sanz-Sanz and S. Gómez-Carrasco, J. Phys. Chem. A 113, 14488 (2009).
  * [24] O. Roncero, https://github.com/octavioroncero/madwave3 (2021).
  * [25] V.A. Mandelshtam and H.S. Taylor, J. Chem. Phys. 102, 7390 (1995).
  * [26] G.J. Kroes and D. Neuhauser, J. Chem. Phys. 105, 8690 (1996).
  * [27] R. Chen and H. Guo, J. Chem. Phys. 105, 3569 (1996).
  * [28] S.K. Gray and G.G. Balint-Kurti, J. Chem. Phys. 108, 950 (1998).
  * [29] T. González-Lezana, A. Aguado, M. Paniagua and O. Roncero, J. Chem. Phys. 123, 194309 (2005).
  * [30] S. Gómez-Carrasco and O. Roncero, J. Chem. Phys. 125, 054102 (2006).
  * [31] E. Aslan, N. Bulut, J.F. Castillo, L. Bañares, F.J. Aoiz and O. Roncero, Astrophys. J. 739, 31 (2012).
  * [32] A. Zanchet, B. Godard, N. Bulut, O. Roncero, P. Halvick and J. Cernicharo, ApJ 766, 80 (2013).
  * [33] C. Sanz, O. Roncero, C. Tablero, A. Aguado and M. Paniagua, J. Chem. Phys. 114, 2182 (2001).
  * [34] T. González-Lezana, O. Roncero, P. Honvault, J.M. Launay, N. Bulut, F.J. Aoiz and L. Bañares, J. Chem. Phys. 125, 094314 (2006).
  * [35] T. González-Lezana and P. Honvault, Int. Rev. Phys. Chem 33, 371 (2014).
  * [36] V.M. Andrianarijaona, PhD Thesis, Université Catholique de Louvain (2002).
  * [37] X. Urbain, N. de Ruette, V.M. Andrianarijaona, M.F. Martin, L.F. Menchero, L. Errea, L. Méndez, I. Rabadan and B. Pons, Phys. Rev. Lett. 111, 203201 (2013).
  * [38] F.X. Gadéa, H. Berriche, O. Roncero, P. Villarreal and G. Delgado-Barrio, J. Chem. Phys. 107, 10515 (1997).
  * [39] O. Roncero, A. Aguado and S. Gómez-Carrasco, in _Gas-phase chemistry in space: from elementary particles to complex organic molecules_ , edited by Lique, F and Faure, A, AAS-IOP Astronomy (IOP publishing ltd, Dirac house, Temple Back, Bristol BS1 6BE, England, 2019).
  * [40] G.A. Parker, A. Laganà, S. Crocchianti and R.T. Pack, J. Chem. Phys. 102, 1238 (1995).
  * [41] T. P. Tsien and R. T Pack, Chem. Phys. Lett. 6 (1970).