# PointNetPGAP-SLC: A 3D LiDAR-based Place Recognition Approach with Segment-
level Consistency Training for Mobile Robots in Horticulture

T. Barros1, L. Garrote1, P. Conde1, M.J. Coombes2, C. Liu2, C. Premebida1,
U.J. Nunes1 1T.Barros, L.Garrote, P.Conde, C.Premebida and U.J.Nunes are with
the University of Coimbra, Institute of Systems and Robotics, Department of
Electrical and Computer Engineering, Portugal. E-mails:{tiagobarros, garrote,
pedro.conde, cpremebida<EMAIL_ADDRESS>and C.Liu are with the
Dept. of Aeronautical and Automotive Engineering, LUCAS Lab, Loughborough
University, UK. E-mails:{M.J.Coombes<EMAIL_ADDRESS>
###### Abstract

This paper addresses robotic place recognition in horticultural environments
using 3D-LiDAR technology and deep learning. Three main contributions are
proposed: (i) a novel model called PointNetPGAP, which combines a global
average pooling aggregator and a pairwise feature interaction aggregator; (ii)
a Segment-Level Consistency (SLC) model, used only during training, with the
goal of augmenting the contrastive loss with a context-specific training
signal to enhance descriptors; and (iii) a novel dataset named HORTO-3DLM
featuring sequences from orchards and strawberry plantations. The experimental
evaluation, conducted on the new HORTO-3DLM dataset, compares PointNetPGAP at
the sequence- and segment-level with state-of-the-art (SOTA) models, including
OverlapTransformer, PointNetVLAD, and LOGG3D. Additionally, all models were
trained and evaluated using the SLC. Empirical results obtained through a
cross-validation evaluation protocol demonstrate the superiority of
PointNetPGAP compared to existing SOTA models. PointNetPGAP emerges as the
best model in retrieving the top-1 candidate, outperforming PointNetVLAD (the
second-best model). Moreover, when comparing the impact of training with the
SLC model, performance increased on four out of the five evaluated models,
indicating that adding a context-specific signal to the contrastive loss leads
to improved descriptors. The dataset and the code will be made publicly
available at https://github.com/Cybonic/PointNetPGAP-SLC.git

Figure 1: Geolocation, 3D maps and recording setup of the HORTO-3DLM dataset.
Four data sequences were recorded over two years at different locations in the
UK, using a Husky mobile platform equipped with a 32-beam Velodyne sensor and
ZED-F9P RTK GPS/GNSS system.

00footnotetext: 1T.Barros, L.Garrote, P.Conde, C.Premebida and U.J.Nunes are
with the University of Coimbra, Institute of Systems and Robotics, Department
of Electrical and Computer Engineering, Portugal. E-mails: _{_ tiagobarros,
garrote, pedro.conde, cpremebida<EMAIL_ADDRESS>2
M.J.Coombes and C.Liu are with the Dept. of Aeronautical and Automotive
Engineering, LUCAS Lab, Loughborough University, UK. E-mails:{M.J.Coombes,
<EMAIL_ADDRESS>
##  I INTRODUCTION

Autonomous robots have become more common in modern agriculture, where they
have been used for various tasks such as the application of pesticides and
fertilizers, infrastructure inspection (e.g., fences and irrigation systems),
and fruit and vegetable harvesting [1]. Agricultural robots require precise
and accurate localization systems to operate reliably. In horticultural
environments, plantations may be located within polytunnels, greenhouses or in
places with lack of GNSS coverage. Therefore, in such environments, a GNSS-
based localization systems may not be a reliable localization source.
Alternatively, most localization systems rely on map-based approaches, where
Simultaneous Localization and Mapping (SLAM) can be used to generate or update
maps [2].

As part of SLAM systems, or as a standalone global localization approach, 3D
LiDAR-based place recognition has seen significant advancements, driven
primarily by the autonomous driving community, with the goal of achieving
long-term localization [3, 4]. Despite recent efforts to adapt these methods
to natural [5] and agricultural environments [6], major challenges still
remain largely unresolved.

This work addresses the 3D LiDAR place recognition problem for robots
operating in horticultural environments. Horticultural crops present unique
challenges due to varying planting arrangements, both between different plant
species and within the same species. Factors such as inter-row spacing, plant
arrangement and density affect the rows’ permeability to LiDAR scans, allowing
beams to penetrate multiple rows. This phenomenon generates sparse and highly
overlapping scans with poor geometries, which leads to inter-row descriptor
ambiguity.

To cope with these challenges, we propose three key contributions:

(i) PointNetPGAP: A lightweight 3D LiDAR-based place recognition model that
utilizes PointNet [7] for local feature extraction and aggregates the local
features by combining a global average pooling with another aggregator that
explores pairwise feature interactions into a single descriptor.

(ii) Segment-Level Consistency (SLC) model: A model introduced during training
to address the inter-row descriptor ambiguity problem. Specifically, we split
the crops in various segments, and use the SLC model to provide a training
signal, in addition to the contrastive loss, that differentiates scans from
distinct rows/segments, even if they are highly overlapped. The SLC model
consists of a Multi-layer Perceptron (MLP) which predicts the segment class
for each descriptor and computes the loss based on the corresponding ground-
truth. By relying on the descriptors as inputs, the SLC model is easily
integrated with any place recognition model that outputs descriptors.

(iii) Horticulture 3D Localization and Mapping (HORTO-3DLM) Dataset: A new
dataset from horticultural environments comprising data from three orchards
and one strawberry plantation using a 3D LiDAR sensor and RTK-GPS mounted on a
mobile platform, as shown in Fig. 1. Although, in this work, this dataset is
primarily employed for place recognition, it can also be utilized for other
localization and mapping tasks.

We benchmark PointNetPGAP against state-of-the-art (SOTA) models, namely
OverlapTransformer [8], PointNetVLAD [9], and LOGG3D-Net [10] on the
HORTO-3DLM dataset using a cross-validation evaluation protocol. The results
demonstrate that PointNetPGAP offers a more effective and efficient solution
compared to the aforementioned SOTA models. Furthermore, we train all models
with the SLC approach and report the results both at the sequence- and
segment-level. Notably, all models except OverlapTransformer exhibit
performance improvements, which is primarily driven by an increase in
performance in segments with higher ambiguity.

The remainder of this paper is organized as follows. In Section II, we discuss
related work. Section III outlines the proposed method. The experimental
evaluation and the results are presented and discussed in Section IV, while
the concluding remarks are provided in Section V.

Figure 2:  Top: Retrieval framework using PointNetPGAP to generate global
descriptors. The framework comprises the following key stages: T1) An input
scan $P$ is processed by PointNet to extract local features $F$; T2) These
local features are aggregated into a global descriptor $D$ by concatenating
the global average representation of local features with pairwise feature
interactions into a single representation; T3) The global descriptor queries
the database to retrieve the top-$k$ most similar places. Bottom: Training
scheme with the following stages: B1) A training tuple consisting of scans,
including an anchor-positive pair and $m$ negatives, is input into the model,
which generates the corresponding descriptors; B2) In the SLC model, the
descriptors are input into an MLP that predicts the corresponding segment
classes. The loss is computed using the Negative Log-likelihood loss based on
the predictions and ground-truth segment labels; B3) The LazyTriplet loss is
calculated based on the distances between the anchor-positive pair and the
anchor-negatives; B4) Finally, both losses are combined using a weight
$\alpha$.

##  II RELATED WORK

Place recognition has been the subject of much research over the past decade,
particularly in recent years, with a focus on 3D-LiDAR approaches [3]. LiDAR
sensory data has been recognized as a relevant modality to achieve robustness
in place recognition, primarily due to its relative invariance to challenging
variations such as different seasonal or lighting conditions, which are
important in horticultural robotics.

###  II-A 3D LiDAR-based Place Recognition

In the context of 3D LiDAR-based methods, although hand-crafted descriptors
are still being used [11], Deep Learning (DL) techniques have emerged as the
most prevalent choice for place modeling [3]. DL-based place recognition
models can be broadly split into two main components: a local feature
extraction module that maps the input scan to a local feature space, and a
global feature aggregation module that aggregates the local features into a
global descriptor.

As for the feature extraction module, approaches can be categorized based on
how the point clouds are processed: those that extract features directly from
point clouds, such as PointNetVLAD [9], and those that resort to proxy
representations, such as voxels [12, 13], polar coordinates [14], or depth
range images [15, 8]. Recent works such as MinkLoc3D [13], LCDNet [2], and
LOGG3D-Net [10] tend to primarily use voxel-based approaches, employing sparse
convolutions such as PV-RCNN [16] or 4D Spatio-Temporal ConvNets [17], which
have shown higher feature extraction capabilities in urban-like environments.
In this work, we resort to PointNet [18] due to its efficiency and lower
computational demands (as demonstrated in Section IV-F).

Regarding aggregation strategies, most 3D LiDAR place recognition frameworks
rely on first-order aggregation approaches such as NetVLAD [9, 19, 2], which
computes the residuals w.r.t. to feature clusters, generalized-mean pooling
(GeM) [13] or global max pooling (MAC) [20] which operate directly on the
feature space. Inspired by these works, we propose an aggregation approach
that combines, in a single descriptor, a first-order and a second-order
aggregation approach. Specifically, we combine the global average pooling
aggregator with a pairwise feature interaction aggregator into a single
descriptor. We compare our hypothesis with the SOTA models LOGG3D-Net [10],
PointNetVLAD [9], and OverlapTransformer [8], and show through results on
real-world datasets that the combination of these two statistical-based
aggregators yields higher performance than each aggregator individually and
also the SOTA models.

###  II-B Datasets

3D LiDAR-based place recognition methods have gained traction as robust and
scalable global localization approaches, primarily driven by the autonomous
vehicle community, which works mainly on road/city environments [3]. Works
such as MinkLoc3D [13], PointNetVLAD [9], LCDNet [2], or LOGG3D-Net [10] are
evaluated on urban environments, using available 3D LiDAR urban datasets such
as KITTI odometry [21], MulRan [22], Oxford RobotCar [23, 9], or KITTI-360
[24].

Less studied is 3D LiDAR place recognition in field or agricultural
environments. One possible reason for this is the lack of available datasets
which, recently, has been partially compensated by datasets such as Wild-
places [25], RELLIS-3D [26], ORFD [27], TreeScope [6], or VineLiDAR [28],
showing the increasing interest in such environments. Thus, given the lack of
real-world data, especially in horticulture, we propose a novel dataset
HORTO-3DLM that includes 3D LiDAR data of semi-structured horticulture
environments, particularly from fruit trees namely apples and strawberries.
The goal of HORTO-3DLM is to contribute with real-world quality data suitable
to place recognition/localization in the domains related to field and
agricultural robotics.

##  III Proposed Approach

This section introduces the PointNetPGAP and the proposed training regime,
where a traditional contrastive learning approach is combined with the
proposed SLC model. Figure 2 outlines both PointNetPGAP in a retrieval
framework (Top) and PointNetPGAP in the proposed training scheme (Bottom),
with the combination of both the contrastive and the Negative Log-Likelihood
(NLL) losses.

###  III-A Problem Formulation

A retrieval-based place recognition framework comprises two stages (as
illustrated in Fig. 2): firstly (denoted by T1 and T2), a LiDAR scan
$P\in\mathbb{R}^{n\times 3}$ with $n$ points is mapped to a descriptor
$D\in\mathbb{R}^{d}$, such that $\Theta:\mathbb{R}^{n\times
3}\rightarrow\mathbb{R}^{d}$; then (the 2nd stage: T3), the descriptor $D$
queries a database for the $k$ nearest neighbors (KNN) based on a similarity
metric. The $k$ retrieved candidates represent the top-$k$ candidates for
potential revisited places. In this work, we focus on the first part (i.e., T1
and T2) and follow a retrieval framework (i.e., T3) as described in [15].

###  III-B PointNetPGAP

PointNetPGAP, here defined as a mapping function $\Theta$ (outlined in Fig. 2
by T1, T2), can be decomposed into two functions:
$\Theta\equiv\varphi\circ\phi$, where $\phi:\mathbb{R}^{n\times
3}\rightarrow\mathbb{R}^{n\times c}$ maps the input scan
$P\in\mathbb{R}^{n\times 3}$ to hyper-dimensional features
$F\in\mathbb{R}^{n\times c}$; and $\varphi:\mathbb{R}^{n\times
c}\rightarrow\mathbb{R}^{d}$ aggregates the local features $F$ into a global
descriptor $D\in\mathbb{R}^{d}$ with $d$ dimensions.

####  III-B1 Local Feature Extraction (T1)

In this work, we follow the same approach as in [9] and implement $\phi$ based
on PointNet [7], a DL-based network that operates directly on LiDAR scans.
Hence, it receives a LiDAR scan $P\in\mathbb{R}^{n\times 3}$ with $n$ points
and returns a local feature representation $F\in\mathbb{R}^{n\times c}$ with
$c$ dimensions.

####  III-B2 Feature Aggregation (T2)

As for the aggregation (i.e., $\varphi$), this work proposes the fusion of two
statistically-inspired aggregators: a global average representation of the
local features, also known as global average pooling, and an aggregator based
on pairwise feature interactions.

The global average representation, hereafter referred to as GAP, is computed
as follows:

$\bar{F}=\frac{1}{n}\sum_{f_{j}\in F}f_{j},$ (1)

where $f_{j}\in\mathbb{R}^{c}$ represents the $j$-th feature point, and
$\bar{F}\in\mathbb{R}^{c}$ represents the global average representation. As
for the aggregator based on the pairwise feature interactions, it is computed
using the outer product of $F$ as follows:

$G=\frac{1}{n}F\,F^{T},$ (2)

where $G\in\mathbb{R}^{c\times c}$ represents pairwise feature interactions of
the local features, which is flattened to form a vector representation
$G^{\prime}\in\mathbb{R}^{c^{2}}$. Finally, to create the final descriptor
$D$, both representations are concatenated into a single vector
$D^{*}=[G^{\prime},\bar{F}]\in\mathbb{R}^{c+c^{2}}$, which is fed to a full-
connected layer $\text{FC}:\mathbb{R}^{c+c^{2}}\rightarrow\mathbb{R}^{d}$, and
normalized using the $L_{2}$-norm:

$D=L_{2}\text{-norm}(\text{FC}(D^{*})).$ (3)

###  III-C Network Training

Training PointNetPGAP involves a traditional contrastive approach using the
LazyTriplet loss [9], along with the proposed SLC model. The training process
is illustrated at the bottom of Fig. 2 and is divided into four main stages,
labeled B1, B2, B3, and B4.

####  III-C1 Training Data (B1)

Place recognition methods traditionally resort to contrastive learning
techniques, which learn to distinguish between similar (positive pairs) and
dissimilar (negative pairs) data samples. In place recognition for urban
environments, positive pairs are defined only by a fixed radius, which is a
sufficient condition given the scale of the trajectories and revisits in these
environments. However, to perform place recognition in crops with narrow rows,
the radius is a necessary but not sufficient condition. The search conditions
must guarantee that the search occurs along the row and does not transgress
into neighboring rows. Splitting the crops into unique segments and confining
the search within the segment ensures the necessary conditions.

Hence, in this work, a positive pair refers to a pair of scans that must meet
the following constraints: (1) both scans must be within a $r_{th}$ range (in
meters); (2) both scans must be from different revisits, meaning that the
positive scan cannot be the immediate previous scan of the anchor; and (3)
both scans must be from the same segment (see Fig. 3). Scan pairs that do not
meet these three constraints belong to the set of negatives.

As such, the model is trained with tuples of scans (that satisfy the
aforementioned conditions) $\tau^{P}=(P^{A},P^{P},\\{P^{N}_{j}\\})$ where
$P^{A}\in\mathbb{R}^{n\times 3}$ is the anchor, $P^{P}\in\mathbb{R}^{n\times
3}$ is the closest positive scan w.r.t. $P^{A}$ in the Euclidean space, while
$\\{P^{N}_{j}|P^{N}_{j}\in\mathbb{R}^{n\times 3}\\}_{j=1}^{m}$ is a set of $m$
randomly selected negatives. The scan tuple $\tau^{P}$ is mapped to the
descriptor space (as described in Section III-B), generating the corresponding
descriptor tuple $\tau^{D}=(D^{A},D^{P},\\{D^{N}_{j}\\})$.

####  III-C2 Segment-level consistency model (B2)

The SLC model introduces an additional signal during training, which is more
context-related than the one generated by the contrastive loss. The goal is to
resolve ambiguities created by scans that are highly overlapped but belong to
neighboring segments. The SLC model is framed as a classification problem,
aiming to predict the segment class using the descriptors as inputs. The loss
between the predictions and the corresponding ground-truth segment class
$s\in[1,2,...,L]$, where $L$ is the number of segments in a sequence (as shown
in Fig. 3), is computed based on the NLL Loss. Hence, given a training tuple
of descriptors $\tau^{D}$, and the corresponding tuple of ground-truth segment
classes $\tau^{S}=(s^{A},s^{P},\\{s^{N}_{j}\\})$, all descriptors
$D_{i}\in\tau^{D}$ are fed to an MLP to predict the corresponding segment
class, such that:

$\mathbf{p}_{i}=\text{MLP}(D_{i}),$ (4)

where $\mathbf{p}_{i}=[p_{1},p_{2},...,p_{L}]$ with $p_{k}\in[0,1]$
representing the segment prediction scores which are used, together with the
respective ground-truth segment label $s_{i}\in\tau^{S}$, to compute the loss
as follows:

$\mathcal{L}_{S}=\sum_{i}NLLLoss(\mathbf{p}_{i},s_{i}).$ (5)

####  III-C3 Contrastive Loss (B3)

As for the contrastive cost function, the LazyTriplet loss [9] is used. Thus,
given a descriptor tuple $\tau^{D}=(D^{A},D^{P},\\{D^{N}_{j}\\})$ the loss is
computed as follows:

$\mathcal{L}_{T}=\text{max}(d_{AP}-d_{AN}+m,0),$ (6)

where $d_{AP}=\|D^{A}-D^{P}\|_{2}$ is the Euclidean distance between the
anchor and the positive,
$d_{AN}=\min\limits_{j}(\|D^{A}-\\{D^{N}_{j}\\}\|_{2})$ is the Euclidean
distance between the anchor and the hardest negative i.e., the negative that
is the closest w.r.t. $D^{A}$ in the descriptor space, while $m$ designates a
margin value.

####  III-C4 Final Loss (B4)

Finally, both $\mathcal{L}_{T}$ and $\mathcal{L}_{S}$ are combined as follows:

$\mathcal{L}=\alpha\mathcal{L}_{T}+(1-\alpha)\mathcal{L}_{S},$ (7)

where $\alpha$ is adjustable weight.

##  IV EXPERIMENTAL EVALUATION

To support the experiments under realistic and challenging cross-horticultural
domains, we introduce a new dataset comprising four sequences from distinct
horticultural crops, including detailed annotations for ground truth
evaluation. This section outlines the evaluation protocol, implementation
details, and training procedures, including preprocessing steps and the
architecture of the proposed approach. Additionally, quantitative metrics and
qualitative retrieval results are reported, showcasing the effectiveness of
the approach. Furthermore, we conduct a runtime analysis to assess the
computational efficiency of the PointNetPGAP.

Figure 3: 2D and 3D visualizations of the sequences’ paths. The 2D
representation shows the individual segments, while the 3D representation
outlines the overlapping paths.

###  IV-A The HORTO-3DLM Dataset

For real-world deployment, it is important to capture specific conditions that
may induce unique challenges in agricultural environments. For instance,
complex conditions may arise due to varying field layouts (row length, row
width, open field, within tunnels, etc.), which can differ from plant type to
plant type and even within the same plant species. Another aspect that may
present challenges is the changing appearance of the same plantation due to
plant growth and flowering stages.

Hence, we created a dataset called Horticulture 3D Localization and Mapping
(HORTO-3DLM)***https://github.com/Cybonic/HORTO-3DLM.git, which was recorded
in the United Kingdom over two years at different locations. The dataset
includes data collected from four crops: two orchards (apples and cherries),
and one strawberry plantation within polytunnels with a table-top growing
system. All sequences incorporate various revisits.

The data collection process involved teleoperating a Clearpath Husky mobile
robot equipped with a Velodyne VLP32 3D LiDAR (10Hz) and a ZED-F9P RTK-GPS
(5Hz). Figure 1 showcases the geolocation, the 3D maps of the four sequences,
and the recording setup mounted on the mobile robot, while Table I provides
more detailed information regarding each sequence. The three sequences from
orchards are denominated OJ22, ON22, and OJ23, while the sequence from the
strawberry plantation is designated SJ23.

TABLE I: Summary of the four sequences. The various sequence names represent
the first letter of plantation type, month and year of the recording. For
example, an orchard sequence recorded in November 2022 is named ON22. The
columns have the following meaning: name of the sequence (Sequence), month of
recording (M), year of recording (Y), number of rows each sequence comprises
(N.Rs), number of segments each sequence comprises (N.Segs), traveled distance
(D) and plantation type.
---
Sequence |  M |  Y |  Total Scans |  N.Rs |  N.Segs |  D [m] |  Plantation Type
Orchards |  |  |  |  |  |  |
ON22 |  Nov. |  2022 |  7974 |  4 |  6 |  514 |  Apple (open)
OJ22 |  July |  2022 |  4361 |  3 |  5 |  206 |  Apple (open)
OJ23 |  June |  2023 |  7229 |  3 |  5 |  459 |  Cherry (open)
Strawberries |  |  |  |  |  |  |
SJ23 |  June |  2023 |  6389 |  3 |  5 |  742 |  tunnels
TABLE II: Training and testing statistics used in the experiments.
---
| Cross-validation |
Sequence |  Test Anchors |  Training Anchors |  Seq. Training Anchors
OJ22 |  1797 |  1605 |  100
OJ23 |  4801 |  1193 |  512
ON22 |  4993 |  1210 |  495
SJ23 |  3509 |  1107 |  598

###  IV-B Methodology for Place Recognition

From the proposed HORTO-3DLM dataset described in Section IV-A, each sequence
was split into various segments (S1, S2, …, S6) as illustrated in Fig. 3.
Besides the rows, we also included the extremities of the fields (i.e., S3 and
S4) as segments. Thus, for each data point (LiDAR scan), there exists a
corresponding segment class.

Following the protocol defined in Section III-C1, $r_{th}$ was set to 2 m for
the training data, which means that a positive pair exists whenever an anchor-
positive pair is within a range of 2 m. Additionally, from all anchor-positive
pairs, only the anchors that were at least 0.5 m apart were selected for
training. For evaluation purposes, $r_{th}$ was set to 10 m, and all anchors
were used.

###  IV-C Evaluation Protocol

The evaluation of the proposed approach is based on a k-fold cross-validation
protocol, with $k=4$, which consists of training on three sequences and
testing on the sequence that was left out. This is repeated four times to
obtain results on all sequences.

The dataset split for the cross-validation protocol is outlined in Table II.
The entries have the following meaning: the first column (Sequence) identifies
the sequence on which a given model was tested, meaning the model was trained
on the remaining sequences; the second column (Test Anchors) refers to the
number of anchors/loops with which the model was tested; the third column
(Training Anchors) refers to the number of anchors/loops the model was trained
with, corresponding to the sum of the training anchors of the remaining
sequences; the fourth column (Seq. Training Anchors) refers to the number of
training anchors each sequence contains individually.

As for the performance metrics, the results obtained from the proposed
experiments are reported using Recall for the top-k retrieved candidates
(i.e., Recall@k).

###  IV-D Training and Implementation Details

All models were trained and evaluated under the same conditions. The input
scans were downsampled to 10k points, and randomly rotated around the z-axis.
The parameters of PointNetPGAP were the following: we used the original
architecture of PoinNet, with T-Net disabled, descriptor size was set to 256
dimensions ($d=256$), the local features were set to 16 dimensions ($c=16$),
the MLP was configured with three layers with 256, 64, and 6 neurons,
respectively, while the weight of the final loss was set to 0.5
($\alpha=0.5$). The SOTA models were implemented based on the original code.

All models were trained with 200 epochs on an NVIDIA GeForce RTX 3090 GPU,
using the closest positive and 20 negatives ($m=20$) for each anchor. The
margin value of the contrastive loss was set to 0.5 ($m=0.5$), and the model
parameters were optimized using the AdamW optimizer with a learning rate
($L_{r}$) of 0.0001 and a weight decay ($W_{d}$) of 0.0005. Moreover, all
proposed experiments were conducted on Python 3.8, and PyTorch with CUDA 11.6.

###  IV-E Retrieval Evaluation and Discussion

This section presents and discusses the empirical results. Firstly, we conduct
an ablation study on the proposed PointNetPGAP model, delineating the
contribution of each aggregator to the performance. Secondly, we benchmark
PointNetPGAP, both trained with and without the SLC model, against SOTA
models: PointNetVLAD [9], LOGG3D-Net [10], and OverlapTransformer [8]. Each of
these models adopts distinct strategies for handling scans: PointNetVLAD
processes scans directly, OverlapTransformer processes scans as range images,
and LOGG3D-Net relies on a voxel-based representation and network. The main
results are presented in Table III and Fig. 4, while results at the segment-
level are presented in Fig. 3. Furthermore, a qualitative assessment is
outlined in Fig. 6. Finally, the efficiency of the models is evaluated through
runtime analysis. Throughout this section, PointNetPGAP will be abbreviated as
PNPGAP, and PNPGAP-SLC when trained without and with the SLC model,
respectively.

####  IV-E1 Ablation Study

The results of the ablation study are reported in Table IV. For this study, we
report the Recall@1 and Recall@1% of each aggregation approach separately and
combined. The results indicate that combining both aggregators yields higher
performance compared to each aggregator individually, improving performance by
0.6 percentage points (pp) for retrieving the Top-1% candidates and 4.7 pp for
retrieving the Top-1 candidate. These results suggest that the real impact of
combining these aggregation approaches is on modeling the closest neighbor.

TABLE III: Recall@1 performance of the models trained: (without SLC)/(with
SLC). The bolded scores represent the highest value between trained without
and with SLC, while the underling indicates the highest score in each column.

Model OJ22 OJ23 ON22 SJ23 MEAN OverlapTransformer [8] 0.084/0.075-0.009
0.153/0.164+0.011 0.221/0.194-0.027 0.084/0.073-0.011 0.136/0.127-0.009 LOGG3D
[10] 0.357/0.394+0.037 0.309/0.329+0.020 0.412/0.470+0.058 0.254/0.285+0.031
0.333/0.370+0.037 PointNetVLAD [9] 0.560/0.659+0.099 0.485/0.502+0.017
0.611/0.625+0.014 0.676/0.717+0.041 0.583/0.630+0.043 PNPGAP(ours)
0.873/0.880+0.007 0.643/0.688+0.045 0.685/0.734+0.049 0.678/0.712+0.034
0.719/0.754+0.034

OJ22

OJ23

ON22

SJ23

Figure 4: Retrieval performance for the top-25 candidates on the four
sequences. Figure 5: Recall@1 performance at the segment-level for the four
sequences. For more details on the segments, please see Fig. 3. TABLE IV:
Ablation study of PNPGAP, where GAP represents the global average pooling
aggregator and PFI represents the aggregator based on pairwise feature
interactions. The bolded scores represent the highest values in each column.

GAP PFI Recall@1 Recall@1% OJ22 OJ23 ON22 SJ24 MEAN OJ22 OJ23 ON22 SJ24 MEAN ✓
0.819 0.589 0.687 0.714 0.701 0.928 0.909 0.962 0.975 0.944 ✓ 0.827 0.613
0.685 0.713 0.710 0.962 0.925 0.950 0.988 0.956 ✓ ✓ 0.880 0.688 0.734 0.723
0.756 0.943 0.938 0.985 0.983 0.962

####  IV-E2 Comparison to SOTA

PNPGAP emerges as the best model when compared to the SOTA models,
outperforming PointNetVLAD (the second-best model) by 13.6 pp without the SLC
model and by 12.4 pp with the SLC model when retrieving the top-1 candidate.
This is also evident in Fig. 6, where a qualitative assessment is shown on the
paths, displaying the true positive predictions of each model when retrieving
the top-1 candidate.

When analyzing the performance of retrieving the top-25 candidates (Fig. 4),
LOGG3D-Net, which relies on a voxel-based network for feature extraction,
exhibits the highest increase in performance. Meanwhile, OverlapTransformer,
which relies on range images and on a CNN-based network for feature
extraction, shows poor performance across all sequences. The inadequate
performance of OverlapTransformer is also apparent in Fig. 6, where in some
segments the model fails to predict any loops correctly. This suggests the
unsuitability of this model for agricultural environments.

At the segment-level, most models perform best in segments 3 and 4,
representing the extremities of the crops. As illustrated in Fig. 1, these
segments are distinctive, while segments representing the rows exhibit higher
similarities and thus create ambiguities among neighboring rows, resulting in
lower retrieval performance.

Figure 6: True positive predictions for retrieving the top-1 candidate.

####  IV-E3 Training with the SLC Model

The results when adding the SLC model at training time (presented in Table
III) indicate that this additional training signal improves the performance
throughout all sequences and models (except OverlapTransformer), with an
average performance increase of 3.7 pp for LOGG3D, 4.3 pp for PointNetVLAD,
and 3.34 pp for the proposed PNPGAP model.

These findings can be further analyzed in more detail in Fig. 3, where a
comparison between PNPGAP and PNPGAP-SLC is made at the segment-level. The
results show that, while the performance in segments with high performance
remains mostly unchanged, segments such as those corresponding to rows (i.e.,
1, 2, and 5) increase performance, indicating that the SLC model produces
better descriptors that are less susceptible to ambiguities.

###  IV-F Runtime Analysis

TABLE V: The average runtime per batch, where the batch size is 20 scans.
---
|  Param. [M] |  OJ22 [ms] |  OJ23 [ms] |  ON22 [ms] |  SJ23 [ms] |  MEAN [ms]
OverlapTransformer |  48.2 |  12.5 |  11.5 |  11.4 |  11.5 |  11.7
LOGG3D |  8.8 |  108.3 |  115.9 |  108.0 |  82.0 |  103.5
PointNetVLAD |  19.8 |  23.3 |  22.1 |  22.1 |  22.4 |  22.5
PNPGAP (ours) |  0.4 |  3.3 |  3.5 |  3.3 |  3.4 |  3.38

The processing capacity and model size are crucial factors for real-world
implementation, particularly in resource-constrained robotics applications.
Throughout the runtime experiment, the results presented in Table V were
obtained using identical hardware specifications as detailed in Section IV-D:
an AMD Ryzen 9 5900X 12-Core CPU with 64 GB RAM and a GeForce RTX 3090 24GB
GPU. The reported outcomes specifically denote the duration, expressed in
milliseconds (ms), that each model required for processing a batch of 20
scans. The retrieval duration was omitted from this experiment because it
remains uniform across all models. The size of the models is reported through
the count of parameters, denominated in millions (M). PNPGAP emerges as one of
the smaller models, comprising 0.4 million parameters and demanding on average
4.4 ms per batch.

A comparative analysis against other SOTA models reveals that PNPGAP has
approximately 5% of the size of LOGG3D-Net and is 31 times faster.
Furthermore, in comparison to PointNetVLAD, PNPGAP is only 2% of its size and
is 6 times faster. These outcomes highlight the computational efficiency of
the proposed PNPGAP model.

##  V CONCLUSIONS

In this letter, we addressed the 3D LiDAR place recognition problem in
horticultural settings by introducing a novel aggregation approach that fuses
two statistic-inspired aggregators into a model called PointNetPGAP; an SLC
model to handle inter-row descriptor ambiguities; and a novel dataset,
HORTO-3DLM, which includes four sequences: three from orchards and one from a
strawberry plantation.

The experimental evaluation was conducted on the HORTO-3DLM dataset, and the
proposed PointNetPGAP model was benchmarked against SOTA models. An ablation
study demonstrated the superiority of the combined aggregators, while the
results at both the sequence and segment levels show the proposed model
outperforming SOTA models while being more efficient.

Regarding the proposed training approach, all models were trained with and
without the SLC model. The results showed that, except for one model,
incorporating the SLC model during training led to an increase in performance.
This improvement was primarily observed in segments with high inter-row
similarity. This suggests that providing a context-specific training signal,
as offered by the SLC model, may be crucial for effectively training place
recognition models in environments where LiDAR scans overlap significantly,
even when originating from different rows.

## ACKNOWLEDGMENTS

This work has been supported by the project GreenBotics (ref. PTDC/EEI-
ROB/2459/2021), funded by Fundação para a Ciência e a Tecnologia (FCT),
Portugal, and by Innovate UK under the Grant No. 10072930 and 10073332. It was
also partially supported by FCT through grant UIDB/00048/2020 and under the
PhD grant with reference 2021.06492.BD. The authors would also like to thank
Dr Charles Whitfield at NIAB East Malling for facilitating orchard data
collection campaigns.

## References

  * [1] R. Sparrow and M. Howard, “Robots in agriculture: prospects, impacts, ethics, and policy,” _precision agriculture_ , vol. 22, pp. 818–833, 2021.
  * [2] D. Cattaneo, M. Vaghi, and A. Valada, “LCDNet: Deep Loop Closure Detection and Point Cloud Registration for LiDAR SLAM,” _IEEE Transactions on Robotics_ , vol. 38, no. 4, pp. 2074–2093, 2022.
  * [3] T. Barros, R. Pereira, L. Garrote, C. Premebida, and U. J. Nunes, “Place recognition survey: An update on deep learning approaches,” _arXiv preprint arXiv:2106.10458_ , 2021.
  * [4] T. Barros, L. Garrote, M. Aleksandrov, C. Premebida, and U. J. Nunes, “TReR: A Lightweight Transformer Re-Ranking Approach for 3D LiDAR Place Recognition,” in _2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC)_. IEEE, 2023.
  * [5] J. Guo, P. V. Borges, C. Park, and A. Gawel, “Local Descriptor for Robust Place Recognition using LiDAR Intensity,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 1470–1477, 2019.
  * [6] D. Cheng, F. C. Ojeda, A. Prabhu, X. Liu, A. Zhu, P. C. Green, R. Ehsani, P. Chaudhari, and V. Kumar, “TreeScope: An Agricultural Robotics Dataset for LiDAR-Based Mapping of Trees in Forests and Orchards,” _arXiv preprint arXiv:2310.02162_ , 2023.
  * [7] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , July 2017, pp. 652–660.
  * [8] J. Ma, J. Zhang, J. Xu, R. Ai, W. Gu, and X. Chen, “OverlapTransformer: An Efficient and Yaw-Angle-Invariant Transformer Network for LiDAR-Based Place Recognition,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 3, pp. 6958–6965, 2022.
  * [9] M. Angelina Uy and G. Hee Lee, “PointNetVLAD Deep Point Cloud Based Retrieval for Large-Scale Place Recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 4470–4479.
  * [10] K. Vidanapathirana, M. Ramezani, P. Moghadam, S. Sridharan, and C. Fookes, “LoGG3D-Net: Locally Guided Global Descriptor Learning for 3D Place Recognition,” in _2022 International Conference on Robotics and Automation (ICRA)_ , 2022.
  * [11] F. Ou, Y. Li, and Z. Miao, “Place Recognition of Large-Scale Unstructured Orchards with Attention Score Maps,” _IEEE Robotics and Automation Letters_ , vol. 8, no. 2, pp. 958–965, 2023.
  * [12] S. Siva, Z. Nahman, and H. Zhang, “Voxel-Based Representation Learning for Place Recognition Based on 3D Point Clouds,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020.
  * [13] J. Komorowski, “MinkLoc3D: Point Cloud Based Large-Scale Place Recognition,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , 2021, pp. 1790–1799.
  * [14] L. Li, X. Kong, X. Zhao, T. Huang, W. Li, F. Wen, H. Zhang, and Y. Liu, “RINet: Efficient 3D lidar-based place recognition using rotation invariant neural network,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 2, pp. 4321–4328, 2022.
  * [15] T. Barros, L. Garrote, R. Pereira, C. Premebida, and U. J. Nunes, “AttDLNet: Attention-Based Deep Network for 3D LiDAR Place Recognition,” in _ROBOT2022: Fifth Iberian Robotics Conference: Advances in Robotics, Volume 1_. Springer, 2022, pp. 309–320.
  * [16] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 10 529–10 538.
  * [17] C. Choy, J. Gwak, and S. Savarese, “4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 3075–3084.
  * [18] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 652–660.
  * [19] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “NetVLAD: CNN Architecture for Weakly Supervised Place Recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 5297–5307.
  * [20] G. Tolias, R. Sicre, and H. Jégou, “Particular object retrieval with integral max-pooling of CNN activations,” _arXiv preprint arXiv:1511.05879_ , 2015.
  * [21] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? The KITTI vision benchmark suite,” in _2012 IEEE Conference on Computer Vision and Pattern Recognition_ , 2012, pp. 3354–3361.
  * [22] G. Kim, Y. S. Park, Y. Cho, J. Jeong, and A. Kim, “Mulran: Multimodal range dataset for urban place recognition,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 6246–6253.
  * [23] W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The Oxford RobotCar dataset,” _The International Journal of Robotics Research_ , vol. 36, no. 1, pp. 3–15, 2017.
  * [24] J. Xie, M. Kiefel, M.-T. Sun, and A. Geiger, “Semantic Instance Annotation of Street Scenes by 3D to 2D Label Transfer,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 3688–3697.
  * [25] J. Knights, K. Vidanapathirana, M. Ramezani, S. Sridharan, C. Fookes, and P. Moghadam, “Wild-Places: A Large-Scale Dataset for Lidar Place Recognition in Unstructured Natural Environments,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023.
  * [26] P. Jiang, P. Osteen, M. Wigness, and S. Saripalli, “RELLIS-3D Dataset: Data, Benchmarks and Analysis,” in _2021 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2021, pp. 1110–1116.
  * [27] C. Min, W. Jiang, D. Zhao, J. Xu, L. Xiao, Y. Nie, and B. Dai, “ORFD: A Dataset and Benchmark for Off-Road Freespace Detection,” in _2022 International Conference on Robotics and Automation (ICRA)_. IEEE, 2022.
  * [28] A. Prabhu, X. Liu, I. Spasojevic, Y. Wu, Y. Shao, D. Ong, J. Lei, P. C. Green, P. Chaudhari, and V. Kumar, “UAVs for forestry: Metric-semantic mapping and diameter estimation with autonomous aerial robots,” _Mechanical Systems and Signal Processing_ , vol. 208, p. 111050, 2024.