# RankED: Addressing Imbalance and Uncertainty in Edge Detection
Using Ranking-based Losses

Bedrettin Cetinkaya, Sinan Kalkan†, Emre Akbas†
Department of Computer Engineering and METU ROMER Robotics Center
Middle East Technical University, Ankara, Turkey
{bckaya, skalkan<EMAIL_ADDRESS>
###### Abstract

Detecting edges in images suffers from the problems of (P1) heavy imbalance
between positive and negative classes as well as (P2) label uncertainty owing
to disagreement between different annotators. Existing solutions address P1
using class-balanced cross-entropy loss and dice loss and P2 by only
predicting edges agreed upon by most annotators. In this paper, we propose
RankED, a unified ranking-based approach that addresses both the imbalance
problem (P1) and the uncertainty problem (P2). RankED tackles these two
problems with two components: One component which ranks positive pixels over
negative pixels, and the second which promotes high confidence edge pixels to
have more label certainty. We show that RankED outperforms previous studies
and sets a new state-of-the-art on NYUD-v2, BSDS500 and Multi-cue datasets.
Code is available at https://ranked-cvpr24.github.io.

††footnotetext: Equal contribution.

##  1 Introduction

Figure 1: (a) Current approaches threshold label certainties and class-
balanced cross-entropy loss for training edge detectors. (b) With RankED, we
propose a unified approach which ranks positives over negatives to handle the
imbalance problem and sorts positives with respect to their certainties.

Detecting contours of objects in a given image is a fundamental problem in
Computer Vision. It has been approached as a machine learning problem since
the introduction of the influential BSDS dataset [1]. As with any learning-
based approach, characteristics of the training data affects performance. One
striking issue regarding ground-truth contour data is that contours are rare
events. For example, in the BSDS dataset, only 7% of all pixels within an
image are marked as edge pixels111Although some studies call such high-level,
semantic edges as “contour” and low-level edges as “edge”, we follow the
recent literature [44, 39, 12, 15, 55] and use the term “edge” for contours in
the rest of the paper.. This creates a significant imbalance between the
positive (edge) and negative (non-edge) classes, which hinders the training of
machine learning models. Another important issue observed in edge data is the
uncertainty regarding the ground-truth annotations. There exist a non-trivial
amount of variation between the annotations produced by different human
annotators, which essentially creates noise in the supervisory signal.

These two issues of edge ground-truth data, namely the imbalance and
uncertainty, have long been known, however, efforts to address them have
remained limited. For the imbalance problem, although there is a vast
literature on long-tailed and imbalance learning (see, e.g., [53, 46]),
researchers have only explored using Dice Loss [10, 11, 12] and weighted
cross-entropy loss [49, 29, 20, 44, 39], which mitigate the problem to a
certain extent. However, as we show in this paper, they are far from finding
the optimal solution. The label uncertainty has been tackled using edge pixels
agreed by most annotators [49, 29, 20, 39, 12] (Figure 1(a)). A recent
exception is the study by Zhou et al. [55] who proposed jointly learning the
mean and variances of labels using multi-variate Gaussian distributions. To
the best of our knowledge, this is the only work in literature to attempt the
uncertainty problem.

In this paper, we address both the imbalance and the uncertainty problems
encountered in edge detection using ranking-based losses. We draw inspiration
from the recent success of ranking-based losses in object detection [6, 41,
40, 36] and instance segmentation [36]. Similar to edge detection, the data in
these problems also manifest high imbalance between classes [35]. In
particular, AP Loss [6] and RS Loss [36] have shown to outperform focal loss
and weighted cross-entropy in these problems. Their success stems from the
ability to naturally balance the gradients for different classes [34].
Inspired by these works, we propose RankED, which basically uses a ranking-
based loss function to learn from data (Figure 1(b)). When a training image
has only one annotation, then RankED tries to rank edge pixels above non-edge
pixels. When there are multiple annotations per training image, RankED not
only tries to rank edge pixels above non-edge pixels but also to sort edge
pixels with respect to their annotation certainty.

Contributions. Our main contributions are as follows:

  * •

We propose RankED, a new ranking-based loss function for edge detection.

  * •

RankED simultaneously addresses the imbalance and uncertainty issues commonly
encountered in edge detection datasets.

  * •

Our experiments on three edge detection datasets (BSDS, NYUDv2, MultiCue) show
that when integrated with Swin-Transformer, RankED consistently outperforms
all SOTA models in average precision (AP).

##  2 Related Work

Edge Detection. Before the rise of deep learning, the conventional edge
detection methods were based on hand-crafted filters such as Sobel [25], Canny
[4], and Laplacian of Gaussian [32]. Deep learning has enabled detecting edges
directly from data and has provided high quality results [49, 3, 10, 23, 20,
26, 31, 38, 42, 44, 50, 39, 55].

Earlier deep learning based methods [2, 42, 3] used features extracted from
fixed patches. In this approach, CNNs detect edges using extracted patches
around candidate contour points. Following studies extended this patch-based
approach with end-to-end learning. These studies focused on detecting edges at
multiple scales [49] by combining features at different layers [29] and in
cascades [20, 15] and making these networks faster [44]. Recently, attention-
based modules and transformers have been shown to provide significant
improvements [39, 12].

Loss Functions for Edge Detection. Edge detection is generally tackled using
score-based classification losses, e.g., Cross Entropy (CE) Loss [49, 20, 44,
39]. However, owing to the severe imbalance problem between positives
($\mathcal{P}$) and negatives ($\mathcal{N}$), CE Loss is often weighted with
a class balancing (CB) factor, e.g. as follows [49]:

$\mathcal{L}_{CB}=\sum_{i}-\beta{{y}}_{i}\log({{p}}_{i})-(1-\beta)(1-{{y}}_{i})\log(1-{{p}}_{i}),$
(1)

where ${{p}}_{i}$ is the edge prediction probability for pixel $i$ and
${{y}}_{i}$ is its target. The terms are multiplied with weights to mitigate
the imbalance issue: $\beta$ = $|\mathcal{N}|/|\mathcal{N}\cup\mathcal{P}|$,
$(1-\beta)$ = $|\mathcal{P}|/|\mathcal{N}\cup\mathcal{P}|$. With these
weights, class-balanced loss function applies a higher penalty for edge pixels
as they are rare.

A more common approach is to use an adaptation of the Dice Loss [45] for edge
detection [10, 11, 12]:

$\mathcal{L}_{DICE}=\frac{\sum_{i}{{p}}_{i}^{2}+\sum_{i}{{y}}_{i}^{2}}{2\sum_{i}{{p}}_{i}{{y}}_{i}},$
(2)

where $i$ runs over all pixels. Dice Loss is often combined with CE Loss [10,
11, 12]:

$\mathcal{L}_{final}=\alpha\mathcal{L}_{DICE}+\beta\mathcal{L}_{CE},$ (3)

where $\alpha$ and $\beta$ balance the two loss functions.

Although there are studies that propose regularization terms to improve edge
detection quality (e.g., sharpness [12]); Cross Entropy, Class Balanced Cross
Entropy and Dice are the de facto losses used for training deep edge
detectors. To the best of our knowledge, there are no ranking-based losses
proposed as an alternative.

Using Uncertainty in Edge Detection. Although uncertainty has been shown to be
a useful measure for various Computer Vision problems such as classification
[24], object detection [27, 19], depth estimation [37, 21], and semantic
segmentation [22, 54], there is only one study [55] employing uncertainty in
edge detection. Zhou et al. [55] propose an uncertainty-aware edge detector
(UAED) that exploits the multi-label nature of the edge detection problem.
Their method jointly learns the mean and variance of given inputs and
constructs multi-variate Gaussian distribution using predicted mean and
variance. Moreover, their method gives more importance to the loss of pixels
with higher uncertainty.

Figure 2: An overview of RankED. RankED introduces two novel loss functions
for edge detection: $\mathcal{L}_{\textrm{Rank}}$ ($\S$ 3.1) for ranking
positive pixels over background pixels, and $\mathcal{L}_{\textrm{Sort}}$
($\S$ 3.2) for sorting positive pixels with respect to their (un)certainties
($\S$ 3.3).

Comparative Summary We note from the literature reviewed above and their
summary in Table 1 that there are no studies that adopt a ranking approach to
edge detection. There is only one study [55] utilizing label uncertainty in
edge detection, which, however, does not use a ranking-based approach.

Method | Ranking-based | Uncertainty?
---|---|---
Loss Function?
BDCN [20] (CVPR ’19) | ✗ | ✗
PiDiNet [44] (ICCV ’21) | ✗ | ✗
EDTER [39] (CVPR ’22) | ✗ | ✗
FCL-Net [51] (NN ’22) | ✗ | ✗
ACTD [12] (NeuroCom. ’23) | ✗ | ✗
CHRNet [15] (Pat.Rec. ’23) | ✗ | ✗
UAED [55] (CVPR ’23) | ✗ | ✓
RankED (This paper) | ✓ | ✓

Table 1: A comparative summary with recent studies.

##  3 Methodology: RankED

RankED is a ranking-based solution with two novel components for edge
detection (Figure 2):

  * •

Ranking Positives over Negatives ($\mathcal{L}_{\textrm{Rank}}$): RankED is
supervised with a loss function that ranks the classification scores of
positives (edge pixels) over negatives (non-edge pixels). For this, we use a
differentiable approximation of Average Precision as a loss function,
following its successful applications in object detection [5].

  * •

Sorting Positives with Respect to their Uncertainties
($\mathcal{L}_{\textrm{Sort}}$): Positive pixels tend to have different
uncertainties, which can be used to adjust their supervision. RankED gives
more priority for positive pixels with higher certainty. For this, we adapt
the loss function proposed by Oksuz et al. [36] for sorting positives with
respect to their localization qualities in object detection.

###  3.1 Ranking Positive Edge Pixels over Negatives

To rank the scores of positive edge pixels higher than those of the negative
pixels, we adapt the ranking-based loss function proposed by Chen et al. [5]
for object detection. Chen et al. approximated Average Precision as a
differentiable ranking based loss function, which was later shown by Oksuz et
al. [34] to be very robust to extreme positive-negative imbalance. As edge
detection exhibits a severe imbalance between positive and negative classes
(positive pixels are 1%, 3%, 7% of all pixels for Multicue-boundary, Multi-cue
edge, and BSDS datasets, respectively), a ranking-based loss maximizing
Average Precision (AP) can be beneficial for edge detection.

We define $\mathcal{L}_{\textrm{Rank}}$, the loss for ranking positive edge
pixels above negatives in terms of Average Precision by counting positives and
negatives as follows (following the notation introduced in Section 2 and Chen
et al. [6]):

$\mathcal{L}_{\textrm{Rank}}=1-\mathrm{AP}=1-\frac{1}{|\mathcal{P}|}\sum_{i\in\mathcal{P}}\frac{\operatorname{rank}^{+}(i)}{\operatorname{rank}(i)},\\\
$ (4)

where $\operatorname{rank}^{+}(i)$ is the rank of pixel $i$ among positives
($\mathcal{P}$) defined as
$\operatorname{rank}^{+}(i)=\sum_{j\in\mathcal{P}}H(x_{ij})$ with
$x_{ij}={{p}}_{j}-{{p}}_{i}$. Similarly, $\operatorname{rank}(i)$ is the rank
of pixel $i$ among all predictions:
$\operatorname{rank}(i)=\sum_{j\in\mathcal{P}\cup\mathcal{N}}H(x_{ij})$. These
rank definitions rely on a step function, $H(\cdot)$, with a
$\delta$-approximation around the step, as suggested by Chen et al.:

$\small H(x)=\left\\{\begin{aligned} &0\,,&x<-\delta\\\
&\frac{x}{2\delta}+0.5\,,&-\delta\leq x\leq\delta\\\
&1\,,&\delta<x.\end{aligned}\right.$ (5)

Plugging these definitions into Eq. 4 yields:

$\displaystyle\mathcal{L}_{\textrm{Rank}}$
$\displaystyle=1-\frac{1}{|\mathcal{P}|}\sum_{i\in\mathcal{P}}\frac{\sum_{j\in\mathcal{P}}H\left(x_{ij}\right)}{\sum_{j\in\mathcal{P}\cup\mathcal{N}}H\left(x_{ij}\right)},$
(6)
$\displaystyle=\frac{1}{|\mathcal{P}|}\sum_{i\in\mathcal{P}}\sum_{j\in\mathcal{N}}L^{\textrm{Rank}}_{ij},$

where $L^{\textrm{Rank}}_{ij}$, coined as the primary term, is defined as

$L^{\textrm{Rank}}_{ij}=\frac{H\left(x_{ij}\right)}{\sum_{k\in\mathcal{P}\cup\mathcal{N}}H\left(x_{ik}\right)}.$
(7)

###  3.2 Sorting Positives with their Uncertainties

Edge detection datasets contain annotations from multiple annotators, leading
to (aleatoric) uncertainty in ground truth labels. In RankED, we favor
accurate prediction of positive edge pixels with high certainty over those
with low certainty. To this end, we use the sorting objective introduced by RS
Loss [36], where object detection hypotheses were sorted by their localization
qualities.

Following RS Loss [36], we define our sorting loss as:

$\mathcal{L}_{\mathrm{Sort}}=\frac{1}{|\mathcal{P}|}\sum_{i\in\mathcal{P}}\left(\ell_{\mathrm{Sort}}(i)-\ell_{\mathrm{Sort}}^{*}(i)\right),$
(8)

where $\ell_{\mathrm{Sort}}(i)$ and $\ell_{\mathrm{Sort}}^{*}(i)$ are a
summation of current sorting error and target sorting error, respectively. We
define the sorting error $\ell_{\mathrm{Sort}}(i)$ as the average uncertainty
of pixels with higher confidence than pixel $i$ (i.e., $H(x_{ij})=1$):

$\displaystyle\ell_{\mathrm{Sort}}(i)$
$\displaystyle=\frac{1}{\operatorname{rank}^{+}(i)}\sum_{j\in\mathcal{P}}H\left(x_{ij}\right)(1-c_{j}),$
(9)

where $c_{j}\in[0,1]$ is the label certainty of pixel $j$ (explained in
Section 3.3), and $\operatorname{rank}^{+}(i)$ and
$\mathrm{H}\left(x_{ij}\right)$ are as defined in Section 3.1.

The target error $\ell_{\mathrm{Sort}}^{*}(i)$ is defined as the average
uncertainty ($1-c_{j}$) over each positive pixel $j\in\mathcal{P}$ with higher
confidence ($H(x_{ij})=1$) and higher certainty ($c_{j}\geq c_{i}$),
calculated as:

$\displaystyle\ell_{\mathrm{Sort}}^{*}(i)$
$\displaystyle=\frac{\sum_{j\in\mathcal{P}}\mathrm{H}\left(x_{ij}\right)\left[c_{j}\geq
c_{i}\right](1-c_{j})}{\sum_{j\in\mathcal{P}}\mathrm{H}\left(x_{ij}\right)\left[c_{j}\geq
c_{i}\right]},$ (10)

where $[P]$, the Iverson Bracket, is 1 if $P$ is True and 0 otherwise.

We plug the definitions of $\ell_{\mathrm{Sort}}(i)$ and
$\ell_{\mathrm{Sort}}^{*}(i)$ into $\mathcal{L}_{\mathrm{Sort}}$ to obtain the
primary terms as in AP Loss as follows:

$L^{\textrm{Sort}}_{ij}=\left(\ell_{\mathrm{Sort}}(i)-\ell_{\mathrm{Sort}}^{*}(i)\right)\frac{H(x_{ij})[c_{j}>c_{i}]}{\sum_{k}H(x_{ik})[c_{k}>c_{i}]},$
(11)

which is zero if $i,j\notin\mathcal{P}$.

###  3.3 Computing Pixel Uncertainties

Previous studies create labels for the multi-label datasets as follows: (i)
Merge $n$ annotations $\\{\mathbf{y}^{a}\\}_{a=1}^{n}$ for one RGB image
provided by $n$ annotators:
${\mathbf{y}^{*}\leftarrow\oplus_{a=1}^{n}\mathbf{y}^{a}}$. (ii) Binarize
${\mathbf{y}^{*}}$ using a chosen threshold $\tau$ to obtain the training
target ${\mathbf{y}}$:

$y_{i}=\begin{cases}0,\text{ for }{y}^{*}_{i}<\tau,\\\ 1,\text{ otherwise ,
}\end{cases}$ (12)

where $i\in\mathcal{P}\cup\mathcal{N}$.

This approach neglects both pixel-wise and label-wise uncertainties. Precise
labeling of edges by hand is difficult. Most edge pixels annotated by humans
do not overlap with low-level edges in RGB images. Hence, the evaluation
procedure of edge detection tolerates localization error to match edges in
predicted and ground-truth results [14]. Therefore, a simple merging operation
ignores this pixel-wise uncertainty in training. Moreover, binarizing
$\mathbf{y}^{*}$ not only leads to a loss of information about how many times
edge pixels are labeled by multiple annotators but also causes a loss of edges
that are rarely labeled among these annotations.

To this end, we propose calculating a certainty map $\mathbf{c}$ which
preserves the level of agreement among annotators (Algorithm 1). For this,
first we take the pixel-wise logical-OR of the annotations:
$\tilde{\mathbf{y}}=\operatorname{OR}_{a=1}^{n}{\mathbf{y}^{a}}$
($\tilde{y}_{i}$ for pixel $i$). Then, we define the certainty $c_{i}$ for
pixel $i$ as an average of how many annotations match an edge pixel in its
$d$-vicinity in $\\{\mathbf{y}^{a}\\}_{a=1}^{n}$:

$c_{i}=\frac{1}{n}\sum_{a=1}^{n}\textrm{CP}(\tilde{y}_{i},\mathbf{y}^{a},d),$
(13)

where $\textrm{CP}(\tilde{y}_{i},\mathbf{y}^{a},d)$ is a commonly used
function in the edge detection benchmarks [1, 14] to find match predictions
($\tilde{y}_{i}\in\tilde{\mathbf{y}}$) with the ground truth
($\mathbf{y}^{a}$) within a $d$-Manhattan distance (CP: Correspond Pixels).

Input:  \- Set of $n$ annotations $\\{\mathbf{y}^{a}\\}_{a=1}^{n}$.
\- Maximum distance tolerance for overlap: ${d}$.
Output: Certainty map, $\mathbf{c}$ ($c_{i}$: for pixel $i$).

1:$\tilde{\mathbf{y}}\leftarrow\mathbf{y}^{1}\operatorname{OR}\mathbf{y}^{2}\operatorname{OR}...\operatorname{OR}\mathbf{y}^{n}$.
$\triangleright$ Combine annotations.

2:for each edge pixel $i$ in $\tilde{\mathbf{y}}$ do

3:
$c_{i}\leftarrow\frac{1}{n}\sum_{a=1}^{n}\textrm{CP}(\tilde{y}_{i},\mathbf{y}^{a},{d})$
$\triangleright$ Eq. 13.

4:end for

Algorithm 1 Computing the certainty map $\mathbf{c}$ of annotations.

###  3.4 Overall Loss Function

We combine the ranking and sorting components as follows:

$\mathcal{L}_{\textrm{Overall}}=\mathcal{L}_{\textrm{Rank}}+\alpha\mathcal{L}_{\textrm{Sort}},$
(14)

where $\alpha$ is a coefficient to control influence of sorting loss
$\mathcal{L}_{\textrm{Sort}}$. In the case of a single-annotation dataset
(i.e., $n=1$), we are unable to calculate an uncertainty map / the sorting
loss $\mathcal{L}_{\textrm{Sort}}$; therefore, we set $\alpha$ to 0 for such
datasets.

###  3.5 Optimization

The counting-based definitions of $\mathcal{L}_{\textrm{Rank}}$ and
$\mathcal{L}_{\textrm{Sort}}$ prohibit an autograd-based calculation of
$\partial\mathcal{L}_{o}/\partial p_{i}$ where $o\in\\{\textrm{Rank,
Sort}\\}$. To address this issue, we follow the error-driven update mechanism
of Chen et al. [6] that approximates such a gradient as follows:

$\begin{gathered}\begin{aligned} \frac{\partial\mathcal{L}_{o}}{\partial
p_{i}}=\sum_{j}L^{o}_{ji}t^{o}_{ji}-\sum_{j}L^{o}_{ij}t^{o}_{ij},\end{aligned}\end{gathered}$
(15)

where $t^{o}_{kl}=1$ if $y_{k}=1$ and $y_{l}=0$ for $o=\textrm{Rank}$;
$t^{o}_{kl}=1$ if $y_{k}=1$ and $y_{l}=1$ for $o=\textrm{Sort}$. See Chen et
al. [6] for the derivation details.

##  4 Experiments

###  4.1 Experimental Setup and Details

Datasets. To evaluate our method, we carried out comprehensive experiments on
three commonly used datasets:

BSDS500 [1] is one of the most popular datasets for evaluating edge detection.
It officially contains 200 images for training, 100 images for validation, and
200 images for testing. Each image has 321 × 481 resolution and is manually
labeled by at least five different annotators, which creates aleatoric
uncertainty in the ground-truth labels.

NYUDv2 [43] contains 1449 paired RGB and Depth images with 576 × 448
resolution. Edge detection labels are extracted using the boundaries of
segmented objects. It has a single ground-truth edge map for each RGB image.
It officially contains 381 images for training, 414 for validation, and 654
images for testing.

Multi-cue [33] contains 100 natural images with 1280 × 720 resolution. Each
image has six edge and five boundary labels. The dataset does not provide
training and testing splits. We randomly allocated 80 images for training, and
20 images for testing following previous work [39, 55].

Method | SS | MS
---|---|---
ODS | OIS | AP | ODS | OIS | AP
Canny [4] (PAMI’86) | .611 | .676 | .520 | - | - | -
gPb-UCM [1] (PAMI’10) | .729 | .755 | .745 | - | - | -
SCG [48] (NeurIPS’12) | .739 | .758 | .773 | - | - | -
SE [14] (PAMI’14) | .743 | .764 | .800 | - | - | -
OEF [18] (CVPR’15) | .746 | .770 | .815 | - | - | -
DeepEdge [2] (CVPR’15) | .753 | .772 | .807 | - | - | -
DeepContour [42] (CVPR’15) | .757 | .776 | .790 | - | - | -
HED [49] (ICCV’15) | .788 | .808 | .840 | - | - | -
DeepBoundary [26] (ICLR’15) | .789 | .811 | .789 | .803 | .820 | .848
CEDN [52] (CVPR’16) | .788 | .804 | - | - | - | -
RDS [28] (CVPR’16) | .792 | .810 | .818 | - | - | -
COB [31] (ECCV’16) | .793 | .820 | .859 | - | - | -
AMH-Net [50] (NeurIPS’17) | .798 | .829 | .869 | - | - | -
RCF [29] (CVPR’17) | .798 | .815 | - | - | - | -
CED [47] (CVPR’17) | .803 | .820 | .871 | - | - | -
LPCB [10] (ECCV’18) | .800 | .816 | - | - | - | -
BDCN [20] (CVPR’19) | .806 | .826 | .847 | - | - | -
DSCD [9] (ACMMM’20) | .802 | .817 | - | - | - | -
FCL-Net [51] (NN’22) | .807 | .822 | - | .816 | .833 | -
EDTER [39] (CVPR’22) | .824 | .841 | .880 | .840 | .858 | .896
UAED [55] (CVPR’23) | .829 | .847 | .892 | .837 | .855 | .897
ACTD [12] (Neurocomp.’23) | .817 | .836 | .839 | - | - | -
RankED (Ranking Only) | .822 | .838 | .886 | .829 | .850 | .900
RankED (Ranking & Sorting) | .824 | .840 | .895 | .837 | .855 | .911

Table 2: Quantitative results on BSDS dataset [1]. SS and MS represent single-
scale and multi-scale, respectively. Bold: Best result. Underline: Second-best
results.

Method | SS+VOC | MS+VOC
---|---|---
ODS | OIS | AP | ODS | OIS | AP
LDC [11] ( ACMMM’21) | .812 | .826 | .857 | .819 | .834 | .860
PiDiNet [44] (ICCV’21) | .807 | .823 | - | - | - | -
FCL-Net [51] (NN’22) | .815 | .834 | - | .826 | .845 | -
EDTER [39] (CVPR’22) | .832 | .847 | .886 | .848 | .865 | .903
UAED [55] (CVPR’23) | .838 | .855 | .902 | .844 | .864 | .905
CHRNet [15] (Pat. Rec.’23) | .787 | .788 | .801 | .830 | .853 | .870
ACTD [12] (Neurocomp.’23) | .821 | .837 | .850 | .826 | .842 | .854
RankED (Ranking Only) | .833 | .848 | .901 | .844 | .860 | .916

Table 3: Quantitative results on BSDS dataset [1] using extra Pascal Context
Data [16] in training. SS and MS represent single-scale and multi-scale,
respectively. The best and second-best results are shown with bold and
underlined texts, respectively.

Implementation and Training Details. We use the MMSegmentation toolbox [7] to
implement our method RankED. The official implementations of AP Loss [5] and
RS Loss [36] are based on for-iterations, which lead to long training times.
Therefore, we developed fully-vectorized implementations, boosting training
$\sim$4.5x compared to iteration-based ones. While the fully-vectorized
implementation requires huge GPU memory – at least 45 GB for 320×320 input
resolution with the base model of Swin-Transformer [30], we also provide a
semi-vectorized implementation which speeds up about 2.5× and requires 20 GB
of memory for the same input and model settings.

For the NYUD-v2 dataset, we use the official training and validation sets for
training. We apply the following data augmentations like in previous studies
[49, 39]: (i) Scaling with 0.5 and 1.5, (ii) horizontal flip, and (iii)
rotation with degrees 90, 180, and 270.

For the BSDS500 dataset, we use the official training and validation sets for
training. By following the literature [20, 39], we apply horizontal flip,
rotation with 16 different angles between [0,360], and scaling with 0.5 and
1.5. For experiments on using additional training data, we use the Pascal
Context Dataset [16].

For the Multi-cue dataset, we randomly choose 80 images for training, and the
remaining images for testing. We repeat this procedure 3 times and report
their average with standard deviation. We apply horizontal flip and rotation
with 90, 180, and 360 degrees as the data augmentation.

For all datasets, we randomly crop RGB images to 320 $\times$ 320 resolution.
We do not use any threshold to select positive pixels (i.e., $\tau=0$). We
take the $\delta$ parameter in $H(\cdot)$ (Eq. 5) as 0.4 for NYUD-v2 and 0.1
for BSDS and Multi-cue datasets in $\mathcal{L}_{\textrm{Rank}}$. In
$\mathcal{L}_{\textrm{Sort}}$, $\delta$ is used as 0.1 for BSDS and Multi-cue
datasets. The learning rate is 1e-6 for BSDS and 1e-5 for NYUD-v2 & Multicue
datasets.

The number of iterations is 200k for NYUD-v2 and BSDS datasets and 140k for
Multicue dataset because of the risk of overfitting. Batch size is 1. The
remaining settings of the optimizer are the same as the base model of Swin-
Transformer [30] for the segmentation task. Also, we use pre-trained weights
of ImageNet-22k [8] on 384x384 images. The sorting loss coefficient $\alpha$
(Eq. 14) is used as 1 and 2 for Multicue and BSDS datasets, respectively.

For testing, we followed the standard edge detection evaluation protocol [13]
which includes non-maximum suppression (NMS) and skeleton-based edge thinning,
respectively. Also, we set the maximum allowed distance ($d$ in Eq. 13) as
0.011 for NYUD-v2, 0.0075 for BSDS500 & Multicue datasets similar to previous
studies [49, 20, 39].

Performance Measures. We use the following three commonly used measures:
Optimal Dataset Scale (ODS), Optimal Image Scale (OIS), and Average Prevision
(AP). To calculate ODS, one threshold is used for binarizing the predicted
edge maps in the dataset, whereas different thresholds are used for each image
in OIS. Also, ODS and OIS are essentially $F_{1}$ scores, showing the best
result at only one point on the precision-recall curve. On the other hand, AP
gives an insight into the general performance of the model.

Unlike previous studies, we present uncertainty-aware results (UaR) using
these measures for multi-label datasets. In this type of result, we separately
report these three measures for different values of certainty (e.g., AP @
$c_{i}\geq 0.2$, which means AP if $c_{i}$ is thresholded at 0.2).

###  4.2 Experiment 1: Comparison with SOTA

This section compares RankED with the state-of-the-art (SOTA) methods
including hand-crafted and deep-learning-based methods. We report the results
of SOTA methods from their publications.

On BSDS500. We compare the performance of our model with SOTA methods, as
shown in Tables 2 and 3, for different input and training data settings:
Single Scale (SS), Multi-scale (MS), with additional Pascal-VOC dataset in
Single Scale (SS+VOC), and with additional Pascal-VOC dataset in Multi-Scale
(MS+VOC). The results suggest that our method provides the best AP for all
input and training data settings – with only one exception at SS+VOC, where
RankED is 0.1 AP points behind UAED [55]. RankED’s ODS and OIS performances
are on par with others. Since the Pascal VOC dataset is not a multi-label
dataset (i.e. no uncertainty), we cannot apply the sorting loss on it.

On NYUD-v2. We provide results for RGB images in Table 4. Our RankED achieves
the best results in all three measures: 78.0% ODS, 79.3% OIS, 82.6% AP. HHA
and RGB-HHA results are reported in the supplementary material.

Method | ODS | OIS | AP
---|---|---|---
gPb-ucm [1] (PAMI’11) | .632 | .661 | .562
Silberman et al. [43] (ECCV’12) | .658 | .661 | -
gPb+NG [17] (CVPR’13) | .687 | .716 | .629
OEF [18] (CVPR’15) | .651 | .667 | -
HED [49] (ICCV’15) | .720 | .734 | .734
RCF [29] (CVPR’17) | .729 | .742 | -
AMH-Net [50] (NeurIPS’17) | .744 | .758 | .765
LPCB [10] (ECCV’18) | .739 | .754 | -
BDCN [20] (CVPR’19) | .748 | .763 | .770
PiDiNet [44] (ICCV’21) | .733 | .747 | .715
EDTER [39] (CVPR’22) | .774 | .789 | .797
ACTD [12] (Neurocomp.’23) | .762 | .774 | -
RankED (R) | .780 | .793 | .826

Table 4: Quantitative comparisons on NYUD-v2 [43] for RGB images. The best and
second-best results are shown with bold and underlined texts, respectively. R:
Ranking only.

On Multicue. We compare our method with SOTA deep learning-based methods using
both edge and boundary labels of Multi-cue dataset, as shown in Table 5.
RankED clearly outperforms all SOTA methods for edge and boundary detection in
all metrics. On edge detection, it yields +5.5, +4.3, and +2.3 percentage
point improvements over the second-best results for ODS, OIS, and AP metrics,
respectively. Similarly, on boundary, it gives +9.9, +9.5, and +6.8 percentage
point improvements over the second-best results in ODS, OIS, and AP metrics,
respectively.

Method | ODS | OIS | AP
---|---|---|---
Human [33] (VR’16) | .750±.024 | - | -
Multicue [33] (VR’16) | .830±.002 | - | -
HED [49] (ICCV’15) | .851±.014 | .864±.011 | -
RCF [29] (CVPR’17) | .857±.004 | .862±.004 | -
BDCN [20] (CVPR’19) | .891±.001 | .898±.002 | .935±.002
DSCD [9] (ACMMM’20) | .871±.007 | .876±.002 | -
LDC [11] (ACMMM’21) | .881±.012 | .893±.011 | -
PiDiNet [44] (ICCV’21) | .855±.007 | .860±.005 | -
FCL-Net [51] (NN’22) | .875±.005 | .880±.005 | -
EDTER [39] (CVPR’22) | .894±.005 | .900±.003 | .944±.002
UAED [55] (CVPR’23) | .895±.002 | .902±.001 | .949±.002
CHRNet [15] (Pat. Rec.’23) | .907±.001 | .922±.001 | -
ACTD [12] (Neurocomp.’23) | .890±.011 | .905±.009 | -
RankED (${R}$) | .951±.002 | .953±.001 | .962±.004
RankED (${R+S}$) | .962±.003 | .965±.003 | .973±.006
| (a) Edge |  |
Method | ODS | OIS | AP
Human [33] (VR’16) | .760±.017 | - | -
Multicue [33] (VR’16) | .720±.014 | - | -
HED [49] (ICCV’15) | .814±.011 | .822±.008 | .869±.015
RCF [29] (CVPR’17) | .817±.004 | .825±.005 | -
BDCN [20] (CVPR’19) | .836±.001 | .846±.003 | .893±.001
DSCD [9] (ACMMM’20) | .828±.003 | .835±.004 | -
LDC [11] (ACMMM’21) | .839±.012 | .853±.006 | -
PiDiNet [44] (ICCV’21) | .818±.003 | .830±.005 | -
FCL-Net [51] (NN’22) | .834±.016 | .840±.016 | -
EDTER [39] (CVPR’22) | .861±.003 | .870±.004 | .919±.003
UAED [55] (CVPR’23) | .864±.004 | .872±.006 | .927±.006
CHRNet [15] (Pat. Rec.’23) | .859±.001 | .863±.001 | -
ACTD [12] (Neurocomp.’23) | .852 ±.004 | .863±.008 | -
RankED (${R}$) | .954 ±.004 | .958± .005 | .992 ±.002
RankED (${R+S}$) | .963 ±.002 | .967±.002 | .995±.001
| (b) Boundary |  |

Table 5: Quantitative results on Multicue Dataset [33]. The best and second-best results are shown with bold and underlined texts, respectively. $R$: Ranking only. $R+S$: Ranking & Sorting. |  Low Certainty (High Uncertainty) $\xleftrightarrow{\makebox[142.26378pt]{}}$ High Certainty (Low Uncertainty)
---|---
| $\mathbf{c}>0.0$ | $\mathbf{c}\geq 0.2$ | $\mathbf{c}\geq 0.4$ | $\mathbf{c}\geq 0.6$ | $\mathbf{c}\geq 0.8$ | $\mathbf{c}=1.0$
Method | ODS | OIS | AP | ODS | OIS | AP | ODS | OIS | AP | ODS | OIS | AP | ODS | OIS | AP | ODS | OIS | AP
HED [49] (ICCV’15) | .788 | .804 | .840 | .794 | .810 | .846 | .812 | .829 | .864 | .833 | .850 | .887 | .852 | .868 | .902 | .880 | .893 | .926
BDCN [20] (CVPR’19) | .815 | .830 | .872 | .821 | .837 | .880 | .842 | .859 | .899 | .864 | .881 | .919 | .884 | .900 | .933 | .914 | .926 | .956
PiDiNet [44] (ICCV’21) | .807 | .823 | .856 | .810 | .826 | .864 | .829 | .846 | .883 | .851 | .867 | .904 | .871 | .887 | .923 | .899 | .911 | .947
EDTER [39] (CVPR’22) | .824 | .841 | .880 | .826 | .842 | .881 | .848 | .864 | .899 | .870 | .885 | .918 | .890 | .905 | .936 | .920 | .932 | .962
RankED (${R}$) | .822 | .838 | .886 | .828 | .844 | .890 | .849 | .866 | .909 | .870 | .886 | .924 | .889 | .904 | .939 | .916 | .923 | .962
2-19[.4pt/2pt] RankED (${R+S}$) | .824 | .840 | .895 | .831 | .847 | .900 | .854 | .870 | .918 | .877 | .892 | .936 | .897 | .911 | .951 | .926 | .929 | .970
Table 6: Uncertainty-aware results on BSDS dataset. All SOTA results are
computed using official weights. Also, $c$ represents the certainty map
mentioned in Algorithm 1. While case $c>0.0$ contains all ground-truth edges
in all labels, case $c=1.0$ contains only ground-truth edges that are labeled
in all labels. $R$: Only ranking, $R+S$: Ranking & Sorting.

###  4.3 Experiment 2: Uncertainty-aware Evaluation

This section provides uncertainty-aware evaluation (UaR) of RankED and SOTA
methods which published their weights on the BSDS dataset as shown in Table 6.
As most studies published their models for only the BSDS dataset, we could not
do a similar analysis on the Multi-cue dataset.

For the BSDS dataset, we define six uncertainty levels. For example, level
$\mathbf{c}$ $>$ 0.0 contains all ground-truth edges (high + low
uncertainties) in all labels. Therefore, this case is equivalent to the
standard evaluation of edge detectors. On the other hand, level
$\mathbf{c}=1.0$ (lowest uncertainty) contains only ground-truth edges which
are labeled by all annotators.

The results in Table 6 suggest a similar behavior for all models: They are
more successful on low-uncertainty edges than high-uncertainty ones. While the
uncertainty level is increased (i.e., $\mathbf{c}$ goes from 1.0 to 0.0), the
scores of ODS, OIS, and AP measures decrease.

Our method RankED gives better results when the uncertainty level is
decreased. For example, our method gives nearly the same performance with
EDTER [39] for the ODS and OIS measures in level $\mathbf{c}$ $>$ 0.0, and the
performance of our model increases, as the uncertainty level decreases. This
can be explained by our sorting task because it assigns higher importance to
lower uncertainty pixels.

This set of experiments and our uncertainty-aware results (UaR) reveal that
edge detection models give a better performance when the uncertainty levels
are decreased and most of the performance loss is caused by higher uncertainty
pixels, especially labeled by only one annotator. Therefore, focusing on those
pixels can provide more performance gain for the future study. UaR results of
Multi-cue dataset are reported in the supplementary material.

###  4.4 Experiment 3: Ablation Analysis

Different loss functions. To show the effectiveness of our method, we perform
step-by-step comparisons by including class-balanced cross-entropy and the
combination of cross-entropy and dice losses [10] with the same model and
input settings on NYUD-v2, BSDS, and Multi-cue datasets. The results in Table
7 suggest that ranking systematically improves the results over the commonly
used loss functions in the literature. Moreover, we see that, whenever
feasible, using the sorting component provides gains compared to only using
ranking.

Dataset | Loss | ODS | OIS | AP
---|---|---|---|---
NYUD-v2 | CECB | .775 | .789 | .802
2-5[.4pt/2pt] | CE + DICE | .779 | .791 | .807
2-5[.4pt/2pt] |  RankED (${R}$) | .780 | .793 | .826
BSDS | CECB | .820 | .831 | .871
2-5[.4pt/2pt] | CE + DICE | .821 | .836 | .872
2-5[.4pt/2pt] |  RankED (${R}$) | .822 | .838 | .886
2-5[.4pt/2pt] |  RankED (${R+S}$) | .824 | .840 | .895
Multicue Edge | CECB | .926±.006 | .926±.007 | .880±.001
2-5[.4pt/2pt] | CE + DICE | .930±.005 | .931±.004 | .883±.006
2-5[.4pt/2pt] |  RankED (${R}$) | .951±.002 | .953±.001 | .962±.004
2-5[.4pt/2pt] |  RankED (${R+S}$) | .962±.003 | .965±.003 | .973±.006
Multicue Boundary | CECB | .934±.004 | .943±.003 | .981±.003
2-5[.4pt/2pt] | CE + DICE | .947±.004 | .954±.004 | .984±.004
2-5[.4pt/2pt] |  RankED (${R}$) | .954 ±.004 | .958± .005 | .992 ±.002
2-5[.4pt/2pt] |  RankED (${R+S}$) | .963 ±.002 | .967±.002 | .995±.001

Table 7: Quantitative comparison between our method and commonly used loss
functions in edge detection. Bold numbers show the best results for
corresponding columns and datasets. While OursR represents using only ranking
loss, $R$: Ranking. $R+S$: Ranking & Sorting. CECB: Class-balanced cross-
entropy.

Our certainty map computation vs. label averaging. We also conduct an
experiment to show the efficiency of our certainty map $\mathbf{c}$ on BSDS
dataset. In this experiment, as shown in Table 8, we use pixel-wise averaging
and use it as the certainty map
($\mathbf{c}\leftarrow\sum_{a}\mathbf{y}^{a}$). In this experiment, all
settings are the same except for the labels used. Our proposed certainty map
$\mathbf{c}$ gives better performance than the standard labels obtained by
pixel-wise averaging in all metrics because we consider pixel-wise uncertainty
using a dataset-specific distance toleration like an evaluation protocol of
edge detection. In this way, we overlap edges in multi-label that do not
overlap in case of pixel-wise averaging and these overlaps provide labels that
are more informative to the sorting task in terms of uncertainty.

Prioritize Uncertain Pixels. Our sorting loss prioritizes low-uncertainty
pixels while Zhou et al. [55] give higher priority to high-uncertainty edges.
To validate our approach, we integrate their uncertainty weighting scheme into
our loss function as follows: $(1-c_{i})\cdot\mathcal{L}(i)$ for pixel $i$.
The results in Table 8 suggest that this significantly drops the performance
of our model. Therefore, giving more importance to low-uncertainty edges
provides a better performance in RankED.

Label Type | ODS | OIS | AP
---|---|---|---
Label Averaging as Certainty ($\mathbf{c}\leftarrow\sum_{a}\mathbf{y}^{a}$) | 0.818 | 0.831 | 0.880
Loss Weighting with Uncertainty ($1-c_{i}$) | 0.789 | 0.813 | 0.849
RankED | 0.824 | 0.840 | 0.895

Table 8: Comparison with pixel-wise averaging of labels as certainty, and
uncertainty-weighted loss.

###  4.5 Experiment 4: Qualitative Comparison

We also provide a qualitative comparison between SOTA models and ours on BSDS
dataset, as shown in Figure 3. These results are presented after the post-
processing step using thresholds of OIS measure. While our model detects
prominent edges better, it may miss low-level details. See the Supp. Mat. for
more results.

###  4.6 Experiment 5: Running Time Comparisions

Although ranking-based loss functions give superior performance [6, 36], they
suffer from long training time due to for-loops in their official
implementations. Our vectorized implementations solve this problem as shown in
Table 9. For a fair comparison, we test their execution time using loss
functions with random tensors. Our vectorized and semi-vectorized
implementations of $\mathcal{L}_{\textrm{Rank}}$ and
$\mathcal{L}_{\textrm{Sort}}$ are almost 100 and 90 times, respectively,
faster than their official implementations. However, when they are integrated
into models, we observe less performance increase due to other bottlenecks
such as forward/backward in models.

Implementations | Loss | Exec. Time (msec/img)
---|---|---
- | CE | 0.32
2-3[0.4pt/2pt] - | CE+DICE | 0.39
For-iterations |  RankED (${R}$) | 142.79
2-3[0.4pt/2pt] |  RankED (${R}$ \+ ${S}$) | 1530.22
Semi-vectorized |  RankED (${R}$) | 2.90
2-3[0.4pt/2pt] |  RankED (${R}$ \+ ${S}$) | 16.73
Vectorized |  RankED (${R}$) | 1.00
2-3[0.4pt/2pt] |  RankED (${R}$ \+ ${S}$) | 14.09

Table 9: Execution time comparisons (avg. of 100 runs). | Input
---
|  |
| Ground-truth
---
|  |
|  HED [49]
---
|  |
|  BDCN [20]
---
|  |
|  PiDiNet [44]
---
|  |
|  EDTER [39]
---
|  |
| RankED
---
(Ours)
|  |
Figure 3: Qualitative results on BSDS dataset. All outputs are obtained after
the post-processing step. Red: OIS scores.

##  5 Conclusion

We presented a ranking-based solution for object contour detection, or as
widely referred to as edge detection, to address both the imbalance problem
between positive (edge) and negative (non-edge) classes and uncertainty
arising from disagreement between different annotators. Our solution contained
two novel loss functions: One for ranking positive edge pixels over negatives,
and another for sorting positive pixels with respect to their edge
certainties. Our extensive experiments show the efficiency of our proposal on
NYUD-v2, BSDS, and Multicue datasets. RankED can train an edge detector
maximize AP (via its the ranking loss). Our paper paves the way for optimizing
other objectives and making all post processing steps as part of the training
process.

## Acknowledgements

We gratefully acknowledge the computational resources provided by METU-ROMER,
Center for Robotics and Artificial Intelligence, Middle East Technical
University. Dr. Akbas is supported by the “Young Scientist Awards Program
(BAGEP)” of Science Academy, Türkiye.

## References

  * Arbelaez et al. [2010] Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik.  Contour detection and hierarchical image segmentation.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 33(5):898–916, 2010.
  * Bertasius et al. [2015a] Gedas Bertasius, Jianbo Shi, and Lorenzo Torresani.  Deepedge: A multi-scale bifurcated deep network for top-down contour detection.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 4380–4389, 2015a.
  * Bertasius et al. [2015b] Gedas Bertasius, Jianbo Shi, and Lorenzo Torresani.  High-for-low and low-for-high: Efficient boundary detection from deep object features and its applications to high-level vision.  In _Int. Conf. Comput. Vis._ , pages 504–512, 2015b.
  * Canny [1986] John Canny.  A computational approach to edge detection.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 8(6):679–698, 1986.
  * Chen et al. [2019] Kean Chen, Jianguo Li, Weiyao Lin, John See, Ji Wang, Lingyu Duan, Zhibo Chen, Changwei He, and Junni Zou.  Towards accurate one-stage object detection with ap-loss.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 5119–5127, 2019.
  * Chen et al. [2020] Kean Chen, Weiyao Lin, Jianguo Li, John See, Ji Wang, and Junni Zou.  AP-loss for accurate one-stage object detection.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 43(11):3782–3798, 2020.
  * Contributors [2020] MMSegmentation Contributors.  MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark.  https://github.com/open-mmlab/mmsegmentation, 2020.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 248–255. IEEE, 2009.
  * Deng and Liu [2020] Ruoxi Deng and Shengjun Liu.  Deep structural contour detection.  In _Proceedings of the 28th ACM international conference on multimedia_ , pages 304–312, 2020.
  * Deng et al. [2018] Ruoxi Deng, Chunhua Shen, Shengjun Liu, Huibing Wang, and Xinru Liu.  Learning to predict crisp boundaries.  In _Eur. Conf. Comput. Vis._ , pages 562–578, 2018.
  * Deng et al. [2021] Ruoxi Deng, Shengjun Liu, Jinxin Wang, Huibing Wang, Hanli Zhao, and Xiaoqin Zhang.  Learning to decode contextual information for efficient contour detection.  In _Proceedings of the 29th ACM International Conference on Multimedia_ , pages 4435–4443, 2021.
  * Deng et al. [2023] Ruoxi Deng, Zhao-Min Chen, Huiling Chen, and Jie Hu.  Learning to refine object boundaries.  _Neurocomputing_ , 557:126742, 2023.
  * Dollár and Zitnick [2013] Piotr Dollár and C. Lawrence Zitnick.  Structured forests for fast edge detection.  In _Int. Conf. Comput. Vis._ , 2013.
  * Dollár and Zitnick [2014] Piotr Dollár and C Lawrence Zitnick.  Fast edge detection using structured forests.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 37(8):1558–1570, 2014.
  * Elharrouss et al. [2023] Omar Elharrouss, Youssef Hmamouche, Assia Kamal Idrissi, Btissam El Khamlichi, and Amal El Fallah-Seghrouchni.  Refined edge detection with cascaded and high-resolution convolutional network.  _Pattern Recognition_ , 138:109361, 2023.
  * Everingham et al. [2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman.  The pascal visual object classes (voc) challenge.  _Int. J. Comput. Vis._ , 88:303–338, 2010.
  * Gupta et al. [2013] Saurabh Gupta, Pablo Arbelaez, and Jitendra Malik.  Perceptual organization and recognition of indoor scenes from rgb-d images.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 564–571, 2013.
  * Hallman and Fowlkes [2015] Sam Hallman and Charless C Fowlkes.  Oriented edge forests for boundary detection.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 1732–1740, 2015.
  * Harakeh et al. [2020] Ali Harakeh, Michael Smart, and Steven L Waslander.  Bayesod: A bayesian approach for uncertainty estimation in deep object detectors.  In _2020 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 87–93. IEEE, 2020.
  * He et al. [2019] Jianzhong He, Shiliang Zhang, Ming Yang, Yanhu Shan, and Tiejun Huang.  Bi-directional cascade network for perceptual edge detection.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 3828–3837, 2019.
  * Hornauer and Belagiannis [2022] Julia Hornauer and Vasileios Belagiannis.  Gradient-based uncertainty for monocular depth estimation.  In _Eur. Conf. Comput. Vis._ , pages 613–630, 2022.
  * Huang et al. [2018] Po-Yu Huang, Wan-Ting Hsu, Chun-Yueh Chiu, Ting-Fan Wu, and Min Sun.  Efficient uncertainty estimation for semantic segmentation in videos.  In _Eur. Conf. Comput. Vis._ , pages 520–535, 2018.
  * Kelm et al. [2019] André Peter Kelm, Vijesh Soorya Rao, and Udo Zölzer.  Object contour and edge detection with refinecontournet.  In _Computer Analysis of Images and Patterns: 18th International Conference_ , pages 246–258, 2019.
  * Kendall and Gal [2017] Alex Kendall and Yarin Gal.  What uncertainties do we need in bayesian deep learning for computer vision?  _Adv. Neural Inform. Process. Syst._ , 30, 2017.
  * Kittler [1983] Josef Kittler.  On the accuracy of the sobel edge detector.  _Image and Vision Computing_ , 1(1):37–42, 1983.
  * Kokkinos [2015] Iasonas Kokkinos.  Pushing the boundaries of boundary detection using deep learning.  _arXiv preprint arXiv:1511.07386_ , 2015.
  * Kraus and Dietmayer [2019] Florian Kraus and Klaus Dietmayer.  Uncertainty estimation in one-stage object detection.  In _2019 ieee intelligent transportation systems conference (itsc)_ , pages 53–60. IEEE, 2019.
  * Liu and Lew [2016] Yu Liu and Michael S Lew.  Learning relaxed deep supervision for better edge detection.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 231–240, 2016.
  * Liu et al. [2017] Yun Liu, Ming-Ming Cheng, Xiaowei Hu, Kai Wang, and Xiang Bai.  Richer convolutional features for edge detection.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 3000–3009, 2017.
  * Liu et al. [2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _Int. Conf. Comput. Vis._ , pages 10012–10022, 2021.
  * Maninis et al. [2016] Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Pablo Arbeláez, and Luc Van Gool.  Convolutional oriented boundaries.  In _Eur. Conf. Comput. Vis._ , pages 580–596, 2016.
  * Marr and Hildreth [1980] David Marr and Ellen Hildreth.  Theory of edge detection.  _Proceedings of the Royal Society of London. Series B. Biological Sciences_ , 207(1167):187–217, 1980.
  * Mély et al. [2016] David A Mély, Junkyung Kim, Mason McGill, Yuliang Guo, and Thomas Serre.  A systematic comparison between visual cues for boundary detection.  _Vision Research_ , 120:93–107, 2016.
  * Oksuz et al. [2020a] Kemal Oksuz, Baris Can Cam, Emre Akbas, and Sinan Kalkan.  A ranking-based, balanced loss function unifying classification and localisation in object detection.  _Adv. Neural Inform. Process. Syst._ , 33:15534–15545, 2020a.
  * Oksuz et al. [2020b] Kemal Oksuz, Baris Can Cam, Sinan Kalkan, and Emre Akbas.  Imbalance problems in object detection: A review.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 43(10):3388–3415, 2020b.
  * Oksuz et al. [2021] Kemal Oksuz, Baris Can Cam, Emre Akbas, and Sinan Kalkan.  Rank & sort loss for object detection and instance segmentation.  In _Int. Conf. Comput. Vis._ , pages 3009–3018, 2021.
  * Poggi et al. [2020] Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mattoccia.  On the uncertainty of self-supervised monocular depth estimation.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 3227–3237, 2020.
  * Pu et al. [2021] Mengyang Pu, Yaping Huang, Qingji Guan, and Haibin Ling.  Rindnet: Edge detection for discontinuity in reflectance, illumination, normal and depth.  In _Int. Conf. Comput. Vis._ , pages 6879–6888, 2021.
  * Pu et al. [2022] Mengyang Pu, Yaping Huang, Yuming Liu, Qingji Guan, and Haibin Ling.  Edter: Edge detection with transformer.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 1402–1412, 2022.
  * Qian et al. [2020] Qi Qian, Lei Chen, Hao Li, and Rong Jin.  DR loss: Improving object detection by distributional ranking.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 12164–12172, 2020.
  * Rolínek et al. [2020] Michal Rolínek, Vít Musil, Anselm Paulus, Marin Vlastelica, Claudio Michaelis, and Georg Martius.  Optimizing rank-based metrics with blackbox differentiation.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 7620–7630, 2020.
  * Shen et al. [2015] Wei Shen, Xinggang Wang, Yan Wang, Xiang Bai, and Zhijiang Zhang.  Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 3982–3991, 2015.
  * Silberman et al. [2012] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus.  Indoor segmentation and support inference from rgbd images.  In _Eur. Conf. Comput. Vis._ , pages 746–760. Springer, 2012.
  * Su et al. [2021] Zhuo Su, Wenzhe Liu, Zitong Yu, Dewen Hu, Qing Liao, Qi Tian, Matti Pietikäinen, and Li Liu.  Pixel difference networks for efficient edge detection.  In _Int. Conf. Comput. Vis._ , pages 5117–5127, 2021.
  * Sudre et al. [2017] Carole H Sudre, Wenqi Li, Tom Vercauteren, Sebastien Ourselin, and M Jorge Cardoso.  Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations.  In _Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Third International Workshop, DLMIA 2017, and 7th International Workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, Proceedings 3_ , pages 240–248. Springer, 2017.
  * Wang et al. [2021] Le Wang, Meng Han, Xiaojuan Li, Ni Zhang, and Haodong Cheng.  Review of classification methods on unbalanced data sets.  _IEEE Access_ , 9:64606–64628, 2021.
  * Wang et al. [2017] Yupei Wang, Xin Zhao, and Kaiqi Huang.  Deep crisp boundaries.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 3892–3900, 2017.
  * Xiaofeng and Bo [2012] Ren Xiaofeng and Liefeng Bo.  Discriminatively trained sparse code gradients for contour detection.  _Adv. Neural Inform. Process. Syst._ , 25, 2012.
  * Xie and Tu [2015] Saining Xie and Zhuowen Tu.  Holistically-nested edge detection.  In _Int. Conf. Comput. Vis._ , pages 1395–1403, 2015.
  * Xu et al. [2017] Dan Xu, Wanli Ouyang, Xavier Alameda-Pineda, Elisa Ricci, Xiaogang Wang, and Nicu Sebe.  Learning deep structured multi-scale features using attention-gated crfs for contour prediction.  _Adv. Neural Inform. Process. Syst._ , 30, 2017.
  * Xuan et al. [2022] Wenjie Xuan, Shaoli Huang, Juhua Liu, and Bo Du.  Fcl-net: Towards accurate edge detection via fine-scale corrective learning.  _Neural Networks_ , 145:248–259, 2022.
  * Yang et al. [2016] Jimei Yang, Brian Price, Scott Cohen, Honglak Lee, and Ming-Hsuan Yang.  Object contour detection with a fully convolutional encoder-decoder network.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 193–202, 2016.
  * Zhang et al. [2023] Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng.  Deep long-tailed learning: A survey.  _IEEE Trans. Pattern Anal. Mach. Intell._ , 2023.
  * Zheng and Yang [2021] Zhedong Zheng and Yi Yang.  Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation.  _Int. Conf. Comput. Vis._ , 129(4):1106–1120, 2021.
  * Zhou et al. [2023] Caixia Zhou, Yaping Huang, Mengyang Pu, Qingji Guan, Li Huang, and Haibin Ling.  The treasure beneath multiple annotations: An uncertainty-aware edge detector.  In _IEEE Conf. Comput. Vis. Pattern Recog._ , pages 15507–15517, 2023.


$\left\\{\mathbf{y}_{-0}^{\top}{\boldsymbol{\varphi}}(r)\right\\}^{2}w(r)\,\tilde{\phi}(r)=0$.
In particular, over $\Delta=\\{r\in[0,R]:\tilde{\phi}(r)>0\\}$,
$\left\\{\mathbf{y}_{-0}^{\top}{\boldsymbol{\varphi}}(r)\right\\}^{2}w(r)=0$.
If we integrate this over $\Delta$ which by condition 4 is such that
$[0,R]\setminus\Delta$ has zero Lebesgue measure, we obtain

$\displaystyle\int_{\Delta}\left\\{\mathbf{y}_{-0}^{\top}{\boldsymbol{\varphi}}(r)\right\\}^{2}w(r)\mathrm{d}r$
$\displaystyle=\int_{0}^{R}\left\\{\mathbf{y}_{-0}^{\top}{\boldsymbol{\varphi}}(r)\right\\}^{2}w(r)\mathrm{d}r=\|\mathbf{y}_{-0}\|^{2}=0.$

Hence $\mathbf{y}=0$, which ends the proof of (iii).

  4. (iv)

From the GNZ formula (2.3), we have
$\mathbb{E}\,\left[\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]=0$
whereby we deduce that
$\mathbb{E}\left[\mathbf{y}_{K_{n}}^{\top}\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]=0$.
Let
$\boldsymbol{\Sigma}_{K_{n}}(\boldsymbol{\theta}^{\star})=\mathbf{A}_{K_{n}}(\boldsymbol{\theta}^{\star})+\mathbf{B}_{K_{n}}(\boldsymbol{\theta}^{\star})$.
Since by Lemma 2 for $n$ large enough,
${\displaystyle|W_{n}|^{-1}\mathbf{y}_{K_{n}}^{\top}\boldsymbol{\Sigma}_{K_{n}}(\boldsymbol{\theta}^{\star})\,\mathbf{y}_{K_{n}}\geq\frac{1}{2}\inf_{\mathbf{y}\in\mathcal{S}_{m}}\mathbf{y}^{\top}\widebar{\boldsymbol{\Sigma}}(\boldsymbol{\theta}^{\star})\,\mathbf{y}}$,
we deduce from Lemma 3(iii)
${\liminf_{n\to\infty}\;|W_{n}|^{-1}\mathbf{y}_{K_{n}}^{\top}\boldsymbol{\Sigma}_{K_{n}}(\boldsymbol{\theta}^{\star})\,\mathbf{y}_{K_{n}}>0}$.
We can now apply a central limit theorem (CLT) to
$\mathbf{y}_{K_{n}}^{\top}\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$
proved by [22] for innovation type statistics (condition 2 is a particular
case and implies that the sequences $\alpha_{n}$ and $R_{n}$ in [22] are such
that $\alpha_{n}=R_{n}=R+\delta$).

  5. (v)

Let
$\boldsymbol{\Psi}_{K_{n}}(\mathbf{X})=\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta})-\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\prime})$.
We have

$\displaystyle\boldsymbol{\Psi}_{K_{n}}(\mathbf{X})={\int_{W_{n}}\mathbf{S}_{K_{n}}(u,\mathbf{X})\mathbf{S}_{K_{n}}(u,\mathbf{X})^{\top}\left\\{\lambda(u,\mathbf{X};\boldsymbol{\theta})-\lambda(u,\mathbf{X};\boldsymbol{\theta}^{\prime})\right\\}\mathrm{d}u}.$

Using first-order Taylor expansion, there exists $s\in(0,1)$ such that

$\lambda(u,\mathbf{X};\boldsymbol{\theta})-\lambda(u,\mathbf{X};\boldsymbol{\theta}^{\prime})=(\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime})^{\top}\mathbf{S}_{K_{n}}(u,\mathbf{X})\lambda(u,\mathbf{X};\check{\boldsymbol{\theta}})$

where
$\check{\boldsymbol{\theta}}=\boldsymbol{\theta}+s(\boldsymbol{\theta}^{\prime}-\boldsymbol{\theta})$.
Hence,

$\displaystyle\boldsymbol{\Psi}_{K_{n}}(\mathbf{X})={\int_{W_{n}}\mathbf{S}_{K_{n}}(u,\mathbf{X})\mathbf{S}_{K_{n}}(u,\mathbf{X})^{\top}(\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime})^{\top}\mathbf{S}_{K_{n}}(u,\mathbf{X})\lambda(u,\mathbf{X};\check{\boldsymbol{\theta}})\mathrm{d}u}.$

By (LS),

$\displaystyle\|\boldsymbol{\Psi}_{K_{n}}(\mathbf{X})\|\leq
c\|\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}\|{\int_{W_{n}}\|\mathbf{S}_{K_{n}}(u,\mathbf{X})\|^{3}\mathrm{d}u}.$

Using Lemma 1 and the ergodic theorem (see e.g. proof of Lemma 2) we have

$\displaystyle{\int_{W_{n}}\|\mathbf{S}_{K_{n}}(u,\mathbf{X})\|^{3}\mathrm{d}u}=O_{a.s.}(K_{n}^{m+3/2}\,|W_{n}|)$

whereby we deduce that
$\|\boldsymbol{\Psi}_{K_{n}}(\mathbf{X})\|=O_{\mathrm{P}}(K_{n}^{m+3/2}|W_{n}|\,\|\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}\|)$.

  6. (vi)

Following the proof of Lemma 2(i), we could also prove that

$\xi_{n}(\mathbf{X}):=|W_{n}|^{-1}\sup_{\begin{subarray}{c}\mathbf{y}\in\,\mathcal{S}_{m}\\\
\|\mathbf{y}\|=1\end{subarray}}\left\\{\mathbf{y}^{\top}\left({\mathbf{A}}(\mathbf{X};\boldsymbol{\theta}^{\star})-{\mathbf{A}}(\boldsymbol{\theta}^{\star})\right)\mathbf{y}\right\\}\to
0$

almost surely as $n\to\infty$.

∎

##  Appendix C Consistency of the estimating equation
$\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})$

######  Proposition 1.

Let

$\displaystyle\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})=\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})+\boldsymbol{\delta}_{K_{n}}(\mathbf{X},\boldsymbol{\theta}^{\star})$

where

$\boldsymbol{\delta}_{K_{n}}(\mathbf{X},\boldsymbol{\theta}^{\star})=\int_{W_{n}}\mathbf{S}_{K_{n}}(u,\mathbf{X})\lambda(u,\mathbf{X};\boldsymbol{\theta}^{\star})r_{K_{n}}(u,\mathbf{X})\mathrm{d}u$
(C.1)

and where $r_{K_{n}}$ is given by (B.2). Under the assumptions of Theorem
1(i), as $n\to\leavevmode\nobreak\ \infty$

$\displaystyle\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$
$\displaystyle=O_{\mathrm{P}}({K_{n}^{1/2}|W_{n}|^{1/2}})$
$\displaystyle\boldsymbol{\delta}_{K_{n}}(\mathbf{X},\boldsymbol{\theta}^{\star})$
$\displaystyle=O_{\mathrm{P}}(a_{K_{n}}K_{n}^{1/2}|W_{n}|+a_{K_{n}}K_{n}^{(m+1)/2}|W_{n}|^{1/2})$

whereby we deduce that

$\displaystyle\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})=O_{\mathrm{P}}\left({K_{n}^{(m\vee
1)/2}|W_{n}|^{1/2}}+a_{K_{n}}K_{n}^{1/2}|W_{n}|\right).$ (C.2)

In the following $c$ may stand for a generic non-negative constant independent
of $\boldsymbol{\theta},\boldsymbol{\theta}^{\star},\mathbf{X},n,\dots$ and
which may vary from line to line.

###### Proof.

We split the proof into two parts focusing on the control of
$\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$ and
$\boldsymbol{\delta}_{K_{n}}(\mathbf{X},\boldsymbol{\theta}^{\star})$.

Control of $\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$.
The random vector
$\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$, which is a
specific innovation statistic sudied in particular by [7, 24], is centered
with variance given by
$\mathrm{Var}[\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})]=|W_{n}|(\mathbf{\bar{A}}_{K_{n}}(\boldsymbol{\theta}^{\star})+\mathbf{\bar{B}}_{K_{n}}(\boldsymbol{\theta}^{\star}))$.
From the stationarity of the Gibbs point process (LS), the boundedness of
$\phi_{\boldsymbol{\theta}}$ for any $\boldsymbol{\theta}$ and Lemma 1

$\displaystyle\|\mathbf{\bar{A}}_{K_{n}}(\boldsymbol{\theta}^{\star})\|$
$\displaystyle\leq\bar{\lambda}\,\mathbb{E}\left[\|\mathbf{S}_{K_{n}}(0,\mathbf{X})\|^{2}\right]=O(K_{n})$
$\displaystyle\|\mathbf{\bar{B}}_{K_{n}}(\boldsymbol{\theta}^{\star})\|$
$\displaystyle\leq
c\int_{\mathcal{B}(0)}\mathbb{E}\left[\|\mathbf{S}_{K_{n}}(0,\mathbf{X})\|\|\mathbf{S}_{K_{n}}(v,\mathbf{X})\|\right]\mathrm{d}v+c\int_{\mathcal{B}(0)}\|\tilde{\boldsymbol{\varphi}}(\|v\|-\delta)\|^{2}\mathrm{d}v$
$\displaystyle=O\left(\mathbb{E}\left[\|\mathbf{S}_{K_{n}}(0,\mathbf{X})\|^{2}\right]\right)+O\left(\sum_{k=1}^{K_{n}}\int_{0}^{R}\varphi_{k}(r)^{2}w(r)\frac{(r+\delta)^{d-1}}{w(r)}\mathrm{d}r\right)=O(K_{n})$

whereby we deduce that
$\mathrm{Var}[\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})]=O(K_{n}|W_{n}|)$
and that
$\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})=O_{\mathrm{P}}(\sqrt{K_{n}|W_{n}|})$.

Control of
$\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$. First,
$\mathbb{E}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]=|W_{n}|\mathbb{E}\left[\mathbf{S}_{K_{n}}(0,\mathbf{X})r_{K_{n}}(0,\mathbf{X})\lambda(0,\mathbf{X};\boldsymbol{\theta}^{\star}\right]$.
From (LS), Cauchy-Schwarz inequality and Lemma 1, we have

$\displaystyle\|\mathbb{E}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]\|$
$\displaystyle\leq
c|W_{n}|\mathbb{E}\left[\|\mathbf{S}_{K_{n}}(0,\mathbf{X})\|\,|r_{K_{n}}(0,\mathbf{X})|\right]$
$\displaystyle\leq
c|W_{n}|\mathbb{E}\left[\|\mathbf{S}_{K_{n}}(0,\mathbf{X})\|^{2}\right]^{1/2}\mathbb{E}\left[r_{K_{n}}(0,\mathbf{X})^{2}\right]^{1/2}$
$\displaystyle=O(|W_{n}|a_{K_{n}}K_{n}^{1/2}).$ (C.3)

Second,
$\mathrm{Var}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]=T_{1}-T_{2}$
with
$T_{1}=\mathbb{E}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})^{\top}\right]$
and
$T_{2}=\mathbb{E}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]\mathbb{E}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]^{\top}$.
Equation (C.3) yields
$\|T_{2}\|=O(\|\mathbb{E}[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})]\|^{2})=O(|W_{n}|^{2}a_{K_{n}}^{2}K_{n})$.
Regarding the term $T_{1}$, using the stationarity of the Gibbs point process
(FR), we have for $n$ large enough (ensuring that $\mathcal{B}(0)\subset
W_{n}$)

$\displaystyle T_{1}$
$\displaystyle=\mathbb{E}\left[\int_{W_{n}}\int_{W_{n}}\mathbf{S}_{K_{n}}(u,\mathbf{X})\mathbf{S}_{K_{n}}(v,\mathbf{X})^{\top}r_{K_{n}}(u,\mathbf{X})r_{K_{n}}(v,\mathbf{X})\lambda(u,\mathbf{X};\boldsymbol{\theta}^{\star})\lambda(v,\mathbf{X};\boldsymbol{\theta}^{\star})\right]\mathrm{d}u\mathrm{d}v$
$\displaystyle=|W_{n}|\int_{\mathcal{B}(0)}\mathbb{E}\left[\mathbf{S}_{K_{n}}(0,\mathbf{X})\mathbf{S}_{K_{n}}(v,\mathbf{X})^{\top}r_{K_{n}}(0,\mathbf{X})r_{K_{n}}(v,\mathbf{X})\lambda(0,\mathbf{X};\boldsymbol{\theta}^{\star})\lambda(v,\mathbf{X};\boldsymbol{\theta}^{\star})\right]\mathrm{d}v.$

Now, using Hölder’s inequality and again the stationarity of the Gibbs point
process

$\displaystyle\|T_{1}\|$ $\displaystyle\leq
c|W_{n}|\mathbb{E}\left[\|\mathbf{S}_{K_{n}}(0,\mathbf{X}\|^{4}\right]^{1/2}\mathbb{E}\left[r_{K_{n}}(0,\mathbf{X})^{4}\right]^{1/2}$
$\displaystyle=O(|W_{n}|K_{n}^{(2m+2)/2}(a_{K_{n}}^{4})^{1/2})=O(|W_{n}|a_{K_{n}}^{2}K_{n}^{m+1}).$

Hence

$\mathrm{Var}\left[\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right]=O\left(a_{K_{n}}^{2}K_{n}|W_{n}|^{2}+a_{K_{n}}^{2}K_{n}^{m+1}|W_{n}|\right).$

The control in probability of
$\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})$ is
easily deduced whereby we deduce that

$\displaystyle\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star}_{K_{n}})$
$\displaystyle=O_{\mathrm{P}}\left(K_{n}^{1/2}|W_{n}|^{1/2}+a_{K_{n}}K_{n}^{1/2}|W_{n}|+a_{K_{n}}K_{n}^{(m+1)/2}|W_{n}|^{1/2}\right)$
$\displaystyle=O_{\mathrm{P}}\left(|W_{n}|\left(\left(\frac{K_{n}}{|W_{n}|}\right)^{1/2}+a_{K_{n}}K_{n}^{1/2}+a_{K_{n}}K_{n}^{1/2}\left(\frac{K_{n}^{m}}{|W_{n}|}\right)^{1/2}\right)\right)$
$\displaystyle=O_{\mathrm{P}}\left(K_{n}^{(m\vee
1)/2}|W_{n}|^{1/2}+a_{K_{n}}K_{n}^{1/2}|W_{n}|\right)$

if $a_{K_{n}}K_{n}^{1/2}\to 0$ and $K_{n}^{m\vee 1}|W_{n}|^{-1}\to 0$ as
$n\to\infty$. ∎

##  Appendix D Proof of Theorem 1

###### Proof.

  1. (i)

Using first-order Taylor expansion, there exists $t\in(0,1)$ such that

$\displaystyle\boldsymbol{e}_{K_{n}}\big{(}\mathbf{X};\hat{\boldsymbol{\theta}}_{K_{n}}\big{)}-\boldsymbol{e}_{K_{n}}\big{(}\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star}\big{)}=\boldsymbol{e}_{K_{n}}^{(1)}\big{(}\mathbf{X};\widetilde{\boldsymbol{\theta}}_{K_{n}}\big{)}\big{(}\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\big{)},$

where
$\widetilde{\boldsymbol{\theta}}_{K_{n}}=\boldsymbol{\theta}_{K_{n}}^{\star}+\,t\big{(}\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\big{)}$.
Since
$\hat{\boldsymbol{\theta}}_{K_{n}}=\underset{\boldsymbol{\theta}_{K_{n}}\in\mathcal{S}_{m}}{\text{argmax}}\,\mathrm{LPL}_{n}(\mathbf{X};\boldsymbol{\theta}_{K_{n}})$
and
$\boldsymbol{e}_{K_{n}}^{(1)}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})=-\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})$,
we therefore have

$\displaystyle\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})=\mathbf{A}_{K_{n}}(\mathbf{X};\widetilde{\boldsymbol{\theta}}_{K_{n}})\big{(}\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\big{)}.$
(D.1)

Rewrite the last equation as $L=R$ where

$\displaystyle L=$
$\displaystyle\,\mathbf{t}_{n}^{\top}\mathbf{A}_{K_{n}}(\mathbf{X};\widetilde{\boldsymbol{\theta}}_{K_{n}})\,\mathbf{t}_{n}\cdot\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|^{2}$
$\displaystyle R=$
$\displaystyle\,(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})^{\top}\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})$

with
$\mathbf{t}_{n}=(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})/\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|$.
We have that $\mathbf{t}_{n}\in\mathcal{S}_{m}$ and $\|\mathbf{t}_{n}\|=1$.
Note that

$\displaystyle|W_{n}|^{-1}\mathbf{t}_{n}^{\top}\mathbf{A}_{K_{n}}(\mathbf{X};\widetilde{\boldsymbol{\theta}}_{K_{n}})\,\mathbf{t}_{n}\geq\underset{\begin{subarray}{c}\mathbf{y}_{K_{n}},\boldsymbol{\theta}\in\mathcal{S}\\\
\|\mathbf{y}_{K_{n}}\|=1\end{subarray}}{\inf}\,|W_{n}|^{-1}\mathbf{y}_{K_{n}}^{\top}\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta})\mathbf{y}_{K_{n}}.$

By Lemma 2(i)-(ii), for $n$ large enough a.s.

$\displaystyle\underset{\begin{subarray}{c}\mathbf{y}_{K_{n}},\boldsymbol{\theta}\in\mathcal{S}_{m}\\\
\|\mathbf{y}_{K_{n}}\|=1\end{subarray}}{\inf}\,|W_{n}|^{-1}\mathbf{y}_{K_{n}}^{\top}\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta})\mathbf{y}_{K_{n}}$
$\displaystyle\geq\frac{1}{2}\,\underset{\begin{subarray}{c}\mathbf{y},\boldsymbol{\theta}\in\mathcal{S}_{m}\\\
\|\mathbf{y}\|=1\end{subarray}}{\inf}\,\mathbf{y}^{\top}\widebar{\mathbf{A}}(\boldsymbol{\theta})\,\mathbf{y}$
$\displaystyle\geq\frac{1}{2}\,\underset{\begin{subarray}{c}\mathbf{y}\in\mathbb{R}^{\mathbb{N}},\|\mathbf{y}\|=1\\\
\boldsymbol{\theta}\in\mathcal{S}_{m}\end{subarray}}{\inf}\,\mathbf{y}^{\top}\widebar{\mathbf{A}}(\boldsymbol{\theta})\,\mathbf{y}=:\frac{1}{c}>0$

where the latter inequality ensues from Lemma 3(i). Hence, for $n$ large
enough $L\geq
c^{-1}\,|W_{n}|\,\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|^{2}$
almost surely. On the other hand, we have
$|R|\leq\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|\,\|\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})\|$
whereby we deduce that

$\displaystyle\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|\leq
c\,|W_{n}|^{-1}\,\|\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})\|.$

The results are easily deduced from Proposition 1.

(ii) We start first with the following equation

$\mathbf{A}_{K_{n}}(\boldsymbol{\theta}_{K_{n}}^{\star})(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})-\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})=\sum_{i=1}^{4}T_{i}$

where

$\displaystyle T_{1}$
$\displaystyle=(\mathbf{A}_{K_{n}}(\boldsymbol{\theta}_{K_{n}}^{\star})-\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})-)(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})$
$\displaystyle T_{2}$
$\displaystyle=(\mathbf{A}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})-\mathbf{A}_{K_{n}}(\mathbf{X};\widetilde{\boldsymbol{\theta}}_{K_{n}}))(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})$
$\displaystyle T_{3}$
$\displaystyle=\mathbf{A}_{K_{n}}(\mathbf{X};\widetilde{\boldsymbol{\theta}}_{K_{n}})(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})-\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star}_{K_{n}})$
$\displaystyle T_{4}$
$\displaystyle=\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star}_{K_{n}})-\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star}).$

From Lemma 3(vi),
$\|T_{1}\|=o_{\mathrm{P}}(|W_{n}|\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star})\|)=o_{\mathrm{P}}(a_{K_{n}}K_{n}^{1/2}|W_{n}|+K_{n}^{(m\vee
1)/2}|W_{n}|^{1/2})$. Lemma 3(v) and Theorem 1(i) show that

$\displaystyle T_{2}$
$\displaystyle=O_{\mathrm{P}}\left(|W_{n}|K_{n}^{m+3/2}\left(K_{n}^{(m\vee
1)/2}{|W_{n}|}^{-1/2}+a_{K_{n}}{K_{n}}^{1/2}\right)^{2}\right)$
$\displaystyle=O_{\mathrm{P}}\left(K_{n}^{m+m\vee
1+3/2}+a_{K_{n}}^{2}K_{n}^{m+5/2}|W_{n}|\right).$

By definition $T_{3}=0$ and by Proposition 1,
$T_{4}=\boldsymbol{\delta}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})=O_{\mathrm{P}}(a_{K_{n}}K_{n}^{1/2}|W_{n}|)$.
Hence,

$\sum_{i}T_{i}=O_{\mathrm{P}}\left(K_{n}^{(m\vee
1)/2}|W_{n}|^{1/2}+K_{n}^{m+m\vee
1+3/2}+a_{K_{n}}K_{n}^{1/2}|W_{n}|+a_{K_{n}}^{2}K_{n}^{m+5/2}|W_{n}|\right).$

Hence, using Lemma 3(ii) we obtain

$\displaystyle\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}=$
$\displaystyle\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})+O_{\mathrm{P}}(x_{n})$
(D.2)

with

$\displaystyle x_{n}$ $\displaystyle=\frac{K_{n}^{(m\vee
1)/2}}{|W_{n}|^{1/2}}+\frac{K_{n}^{m+m\vee
1+3/2}}{|W_{n}|}+a_{K_{n}}K_{n}^{1/2}+a_{K_{n}}^{2}K_{n}^{m+5/2}$
$\displaystyle=O\left(\frac{K_{n}^{m+m\vee
1+3/2}}{|W_{n}|}+a_{K_{n}}K_{n}^{1/2}+a_{K_{n}}^{2}K_{n}^{m+5/2}\right).$
(D.3)

Now, let $r\in(\delta,R+\delta)$. Using (D.2) and since
$\|\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)\|=\sqrt{K_{n}}$ , we
have

$\displaystyle\hat{g}_{n}(r;K_{n})-g(r)=$
$\displaystyle\sum_{k=1}^{K_{n}}(\hat{\theta}_{k,n}-\theta_{k}^{\star})\varphi_{k}(r-\delta)-\sum\limits_{k>K_{n}}\theta_{k}^{\star}\varphi_{k}(r-\delta)$
$\displaystyle=$
$\displaystyle\,\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)^{\top}\left(\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\right)-\sum\limits_{k>K_{n}}\theta_{k}^{\star}\varphi_{k}(r-\delta)$
$\displaystyle=$
$\displaystyle\,\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)^{\top}\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})+O_{\mathrm{P}}(\sqrt{K_{n}}x_{n})+O(a_{K_{n}})$

Let
$\mathbf{z}=\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)^{\top}\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})$.
From Lemma 3(ii) and Cauchy-Schwarz inequality, we have

$\sum_{k=0}^{K_{n}}k^{m}\,|z_{k}|\leq K_{n}^{m+1/2}\|\mathbf{z}\|\leq
c\|\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)^{\top}\|\,\|\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\|=O(K_{n}^{m+1}|W_{n}|^{-1})=o(1)$

by assumption. Hence, $\mathbf{z}\in\mathcal{S}_{m}$ and by Lemma 3(iv),

$s_{n}^{-1}(r)\,\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)^{\top}\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\xrightarrow{d}\mathcal{N}(0,1)$

where $s_{n}(r)$ is given by (4.7). Now, take
$\mathbf{y}_{K_{n}}=\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\,\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)^{\top}$.
We then deduce from Lemma 3(iii) that
${\displaystyle\liminf_{n\to\infty}\;|W_{n}|^{-1}\mathrm{Var}\left(\mathbf{y}_{K_{n}}^{\top}\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\right)>0}$,
i.e. ${\liminf_{n\to\infty}\;|W_{n}|^{-1}s_{n}^{2}(r)>0}$ which implies that
$s_{n}^{-1}(r)=O(|W_{n}|^{-1/2})$. Finally

$\displaystyle s_{n}^{-1}(r)\sqrt{K_{n}}x_{n}$
$\displaystyle=\frac{K_{n}^{m+m\vee
1+2}}{|W_{n}|^{3/2}}+a_{K_{n}}K_{n}^{1/2}\left(\frac{K_{n}}{|W_{n}|}\right)^{1/2}+a_{K_{n}}^{2}\frac{K_{n}^{m+3}}{|W_{n}|^{1/2}}.$

The second term tends to 0 by assumptions of Theorem 1(i). The first and third
one tend to 0 by assumption of Theorem 1(ii). Finally, using Slutsky’s
Theorem, we conclude that

$s_{n}^{-1}(r)\left\\{\hat{g}_{n}(r;K_{n})-g(r)\right\\}\xrightarrow{d}\mathcal{N}(0,1).$

∎

##  Appendix E Version of Theorem 1 when
$\sup_{r\in[0,R]}(r+\delta)^{d-1}/w(r)=\infty$

As mentioned in the discussion of Condition 3, the point (iii) of this
condition is used in Lemma 1 to prove that
$\int_{0}^{R}\varphi_{k}(r)^{2}\mathrm{d}r=O(1)$. If
$\sup_{r\in[0,R]}(r+\delta)^{d-1}/w(r)=\infty$ (a situation which occurs with
the Fourier-Bessel basis and $\delta>0$), then we only have
$\int_{0}^{R}\varphi_{k}(r)^{2}\mathrm{d}r=O(k^{m})$. We claim that in this
situation Theorem 1 remains true but with different assumptions. Here are the
changes. The proofs are omitted. In Lemma 1,
$\mathbb{E}(S_{k}(0,\mathbf{X})^{p})=O(k^{mp})$ and
$\mathbb{E}(\|\mathbf{S}_{K_{n}}(0,\mathbf{X})\|^{p})=O(k^{mp+p/2})$. Lemmas
2-3 remain unchanged. In Proposition 1,
$\mathbf{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}_{K_{n}}^{\star})=O_{\mathrm{P}}\big{(}K_{n}^{m+1/2}|W_{n}|^{1/2}+a_{K_{n}}K_{n}^{m+1/2}|W_{n}|\big{)}$.
Theorem 1 is reformulated as follows
(i) if $K_{n}^{m+1/2}|W_{n}|^{-1/2}\to 0$ and $a_{K_{n}}K_{n}^{m+1/2}\to 0$ as
$n\to\infty$

$\displaystyle\mathrm{ISE}(\hat{g}_{n}(\cdot\,;K_{n}))=O_{\mathrm{P}}\left(\|{\hat{\boldsymbol{\theta}}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|^{2}\right)=O_{\mathrm{P}}\left({\frac{K_{n}^{2m+1}}{|W_{n}|}}+a_{K_{n}}^{2}K_{n}^{2m+1}\right)$

(ii) (4.7) is true if $a_{K_{n}}K_{n}^{m+1/2}\to 0$,
$a_{K_{n}}^{2}K_{n}^{6m+3}|W_{n}|^{-1/2}\to 0$ and
$K_{n}^{10m/3+2}|W_{n}|^{-1}\to 0$ as $n\to\infty$.

##  Appendix F Pointwise convergence of $\hat{g}_{n}(r;K_{n})$

######  Corollary 2.

Assume the conditions 1-3 hold. As $n\to\infty$, assume also that
$a_{K_{n}}K_{n}\to 0$ and that $K_{n}^{2}/|W_{n}|\to 0$. Then, for
$r\in(\delta,R+\delta)$

$\hat{g}_{n}(r;K_{n})-g(r)=O_{\mathrm{P}}\left(\sqrt{\frac{K_{n}^{2}}{|W_{n}|}}+a_{K_{n}}K_{n}\right).$

###### Proof.

For $r\in(\delta,R+\delta)$, we have

$\displaystyle|\hat{g}_{n}(r;K_{n})-g(r)|\leq\left|\sum_{k=1}^{K_{n}}(\hat{\theta}_{k,n}-\theta_{k}^{\star})\varphi_{k}(r-\delta)\right|+\left|\sum\limits_{k>K_{n}}\theta_{k}^{\star}\varphi_{k}(r-\delta)\right|$

which implies by Cauchy-Schwarz inequality and since
$\|\hat{\boldsymbol{\theta}}_{-0,K_{n}}-\boldsymbol{\theta}_{-0K_{n}}^{\star}\|\leq\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|$
that

$|\hat{g}_{n}(r;K_{n})-g(r)|\leq\|\hat{\boldsymbol{\theta}}_{K_{n}}-\boldsymbol{\theta}_{K_{n}}^{\star}\|\times\|\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)\|+a_{K_{n}}.$

The result is deduced from Theorem 1(i) and Lemma 1 which ensures that
$\|\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r-\delta)\|=O(\sqrt{K_{n}})$. ∎

##  Appendix G A multivariate extension of Theorem 1$\mathrm{(ii)}$

The next result is a multivariate extension of Theorem 1(ii). It may be used
for instance to build a statistical test of a specific parametric form for the
pairwise interaction function. We did not investigate this in this paper.

######  Proposition 2.

Let
$\hat{\mathbf{g}}_{n}(\mathbf{r};K_{n})=(\hat{g}_{n}(r_{i};K_{n}))_{i=1,\dots,q}$
and $\mathbf{g}(\mathbf{r})=(g(r_{i}))_{i=1,\dots,q}$ where
$\mathbf{r}=(r_{1},\dots,r_{q})^{\top}\in\mathbb{R}^{q}$ and
$r_{i}\in(\delta,R+\delta)$, $1\leq i\leq q$. Suppose that the assumptions of
Theorem 1(ii) are satisfied. Then,

$\displaystyle\boldsymbol{\Sigma}_{K_{n}}^{-1/2}(\mathbf{r})\Big{\\{}\hat{\mathbf{g}}_{n}(\mathbf{r};K_{n})-\mathbf{g}(\mathbf{r})\Big{\\}}\xrightarrow{d}\mathcal{N}(\mathrm{0},\mathbf{I}_{q})$

where

$\displaystyle\boldsymbol{\Sigma}_{K_{n}}(\mathbf{r})=$
$\displaystyle\widetilde{\boldsymbol{\varphi}}_{K_{n}}^{\top}(\mathbf{r}-\delta)\,\boldsymbol{\Pi}_{K_{n}}\,\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta),$
$\displaystyle\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta)=$
$\displaystyle\left(\widetilde{\boldsymbol{\varphi}}_{K_{n}}(r_{i}-\delta)\right)_{1\leq
i\leq q}.$ (G.1)

Note that the $(K_{n}+1,q)$ matrix
$\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta)$ can be
equivalently written in the following form

$\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta)=\left[{\begin{array}[]{cccc}0&0&\cdots&0\\\
\varphi_{1}(r_{1}-\delta)&\varphi_{1}(r_{2}-\delta)&\cdots&\varphi_{1}(r_{q}-\delta)\\\
\vdots&\vdots&&\vdots\\\
\varphi_{K_{n}}(r_{1}-\delta)&\varphi_{K_{n}}(r_{2}-\delta)&\cdots&\varphi_{K_{n}}(r_{q}-\delta)\\\
\end{array}}\right].$

###### Proof.

Let $\mathbf{t}\in\mathbb{R}^{d}$. We have

$\displaystyle\mathbf{t}^{\top}\Big{\\{}\hat{\mathbf{g}}_{n}(\mathbf{r};K_{n})-\mathbf{g}(\mathbf{r})\Big{\\}}=$
$\displaystyle\,\sum_{i=1}^{q}t_{i}\,\Big{\\{}\hat{g}_{n}(r_{i};K_{n})-g(r_{i})\Big{\\}}$
$\displaystyle=$
$\displaystyle\,\mathbf{t}^{\top}\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta)^{\top}\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})+O_{\mathrm{P}}(x_{n})$

where $x_{n}$ is given by (i). We may prove similarly to the proof of Theorem
1(ii) that
$\mathbf{z}=\mathbf{t}^{\top}\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta)^{\top}\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\in\mathcal{S}_{m}$.
Hence, by Lemma 3(iv)

$\displaystyle\sigma_{\mathbf{t}}^{-1}\,\mathbf{t}^{\top}\widetilde{\boldsymbol{\varphi}}_{K_{n}}(\mathbf{r}-\delta)^{\top}\mathbf{A}_{K_{n}}^{-1}(\boldsymbol{\theta}_{K_{n}}^{\star})\boldsymbol{e}_{K_{n}}(\mathbf{X};\boldsymbol{\theta}^{\star})\xrightarrow{d}\mathcal{N}(0,1)$

where
$\sigma_{\mathbf{t}}=\Big{(}\mathbf{t}^{\top}\boldsymbol{\Sigma}_{K_{n}}(\mathbf{r})\,\mathbf{t}\Big{)}^{1/2}$
(note that $\boldsymbol{\Sigma}_{K_{n}}(\mathbf{r})$ is given by (G.1)). Using
Lemma 3(iii), we have
${\displaystyle\liminf_{n\to\infty}\;|W_{n}|^{-1}\sigma_{\mathbf{t}}^{2}>0}$
whereby we deduce that $\sigma_{\mathbf{t}}^{-1}=O(|W_{n}|^{-1/2})$. With
similar arguments used in the proof of Theorem 1(ii), we have that
$\sigma_{t}K_{n}^{1/2}x_{n}=o(1)$. Hence, by Slutsky’s Theorem

$\displaystyle\sigma_{\mathbf{t}}^{-1}\,\mathbf{t}^{\top}\Big{\\{}\hat{\mathbf{g}}_{n}(\mathbf{r};K_{n})-\mathbf{g}(\mathbf{r})\Big{\\}}\xrightarrow{d}\mathcal{N}(0,1).$

Finally, using Lemma 2.1 of [15], we deduce that

$\boldsymbol{\Sigma}_{K_{n}}^{-1/2}(\mathbf{r})\Big{\\{}\hat{\mathbf{g}}_{n}(\mathbf{r};K_{n})-\mathbf{g}(\mathbf{r})\Big{\\}}\xrightarrow{d}\mathcal{N}(\mathrm{0},\mathbf{I}_{d}).$

∎

##  Appendix H Additional Figures

Figure 5: For $l=1,\ldots,4$, theoretical logarithm of pairwise interaction
functions $g_{l}=\log\phi_{l}$ (dashed black curves), Monte Carlo means of
$\hat{g}_{l}(\cdot,\hat{K}_{\mathrm{cAIC}}^{\star})$ in $W_{j}$ ($j=1,2,3$).
Envelopes correspond to 95% Monte-Carlo pointwise confidence intervals. Figure
6: For $l=1,\ldots,4$, theoretical logarithm of pairwise interaction functions
$g_{l}=\log\phi_{l}$ (dashed black curves), Monte Carlo means of
$\hat{g}_{l}(\cdot,\hat{K}_{\mathrm{cAIC}}^{\star})$ (red curves) using the
Fourier-Bessel basis in $W_{2}$ when $\delta$ and $R$ are known (first
column), estimated (second column), or when $R+\delta$ is set to
$R_{\max}=0.12$ (third column). Envelopes correspond to 95% Monte-Carlo
pointwise confidence intervals. Figure 7: For $l=1,\ldots,4$, curves of
$s_{n}^{-1}(r_{0})\left\\{\hat{g}_{l}(r_{0};\hat{K}_{\mathrm{cAIC}}^{\star})-g_{l}(r_{0})\right\\}$
at $r_{0}=0.035$ in $W_{j}$ ($j=1,2,3$). The black curve is the standard
normal distribution. Figure 8: For $l=1,\ldots,4$, curves of
$s_{n}^{-1}(r_{0})\left\\{\hat{g}_{l}(r_{0};\hat{K}_{\mathrm{cAIC}}^{\star})-g_{l}(r_{0})\right\\}$
at $r_{0}=0.05$ in $W_{j}$ ($j=1,2,3$). The black curve is the standard normal
distribution.