# Sample and Computation Redistribution for Efficient Face Detection

Jia Guo * 2 Jiankang Deng * 1,2 Alexandros Lattas1 Stefanos Zafeiriou1
1Imperial College 2InsightFace
<EMAIL_ADDRESS>{j.deng16<EMAIL_ADDRESS>
###### Abstract

Although tremendous strides have been made in uncontrolled face detection,
efficient face detection with a low computation cost as well as high precision
remains an open challenge. In this paper, we point out that training data
sampling and computation distribution strategies are the keys to efficient and
accurate face detection. Motivated by these observations, we introduce two
simple but effective methods (1) Sample Redistribution (SR), which augments
training samples for the most needed stages, based on the statistics of
benchmark datasets; and (2) Computation Redistribution (CR), which reallocates
the computation between the backbone, neck and head of the model, based on a
meticulously defined search methodology. Extensive experiments conducted on
WIDER FACE demonstrate the state-of-the-art efficiency-accuracy trade-off for
the proposed SCRFD family across a wide range of compute regimes. In
particular, SCRFD-34GF outperforms the best competitor, TinaFace, by $3.86\%$
(AP at hard set) while being more than _3 $\times$ faster_ on GPUs with VGA-
resolution images. We also release our code to facilitate future research.
https://github.com/deepinsight/insightface/tree/master/detection/scrfd

††* Equal contributions. InsightFace is a nonprofit face project.

##  1 Introduction

Face detection is a long-standing problem in computer vision with many
applications, such as face alignment [2], face reconstruction [9], face
attribute analysis [36, 26], and face recognition [28, 6]. Following the
pioneering work of Viola-Jones [32], numerous face detection algorithms have
been designed. Among them, the single-shot anchor-based approaches [25, 39,
30, 15, 24, 7, 23, 41] have recently demonstrated the most promising
performance. In particular, on the most challenging face detection dataset
WIDER FACE [34], the average precision (AP) on its Hard test set has been
boosted to $92.4\%$ by TinaFace [41].

Figure 1: Performance-computation trade-off on the WIDER FACE validation hard
set for different face detectors. Flops and APs are reported by using the VGA
resolution ($640\times 480$) during testing. The proposed SCRFD outperforms a
range of state-of-the-art open-sourced methods by using much fewer flops.

Even though TinaFace [41] achieves impressive results on unconstrained face
detection, it employs large-scale (_e.g_. $1,650$ pixels) testing, which
consumes huge amounts of computational cost (as shown in Tab. 1). In addition,
TinaFace is designed based on a generic object detector (_i.e_. RetinaNet
[19]), directly taking the classification network as the backbone, tiling
dense anchors on the multi-scale feature maps (_i.e_. P2 to P7 of neck) and
adopting heavy head designs. Without considering the prior of faces, the
backbone, neck and head design of TinaFace is thus redundant and sub-optimal.

Since directly taking the backbone of the classification network for object
detection is sub-optimal, the recent CR-NAS [17] reallocates the computation
across different resolutions. It is based on the observation that the
allocation of computation across different resolutions has a great impact on
the Effective Receptive Field (ERF) and affects the detection performance. In
BFbox [22], a meaningful phenomenon has been observed that the same backbone
performs inconsistently between the task of general object detection on COCO
[20] and face detection on WIDER FACE [34], due to the huge gap of scale
distribution. Based on this observation, a face-appropriate search space is
designed in BFbox [22], including a joint optimisation of backbone and neck.
In ASFD [35], another interesting phenomenon has been observed that some
previous Feature Enhance Modules (FAE) perform well in generic object
detection but fail in face detection. To this end, a differential architecture
search is employed in ASFD [35] to discover optimised feature enhance modules
for efficient multi-scale feature fusion and context enhancement.

Even though the works [22, 35] have realised the limitation of directly
applying general backbone, neck and head settings to face detection, CR-NAS
[17] only focuses the optimisation on backbone, BFbox [22] neglects the
optimisation of head, and ASFD [35] only explores the best design for neck.

In this paper, we explore efficient face detection under a fixed VGA
resolution (_i.e_. $640\times 480$) instead of using large-scale for testing
[41] to reduce the computation cost. Under this scale setting, most of the
faces ($78.93\%$) in WIDER FACE are smaller than $32\times 32$ pixels, and
thus they are predicted by shallow stages. To obtain more training samples for
these shallow stages, we first propose a Sample Redistribution (SR) method
through a large cropping strategy.

Moreover, as the structure of a face detector determines the distribution of
computation and is the key in determining its accuracy and efficiency, we
further discover principles of computation distribution under different flop
regimes. Inspired by [27], we control the degrees of freedom and reduce the
search space. More specifically, we randomly sample model architectures with
different configurations on backbone (stem and four stages), neck and head.
Based on the statistics of these models, we follow [27] to compute the
empirical bootstrap [8] and estimate the likely range in which the best models
fall. To further decrease the complexity of the search space, we divide the
computation ratio estimation for backbone and the whole detector into two
steps.

To sum up, this paper makes the following contributions:

  * •

We explore the efficiency of face detection under VGA resolution, and propose
a sample redistribution strategy (SR) that helps to obtain more training
samples for shallow stages.

  * •

We design a simplified search space for computation redistribution across
different components (backbone, neck and head) of a face detector. The
proposed two-step computation redistribution method can easily gain insight on
computation allocation.

  * •

Extensive experiments conducted on WIDER FACE demonstrate the significantly
improved accuracy and efficiency trade-off of the proposed SCRFD across a wide
range of compute regimes as shown in Fig. 1.

##  2 Related Work

Face Detection. To deal with extreme variations (_e.g_. scale, pose,
illumination and occlusion) in face detection [34], most of the recent single-
shot face detectors focus on improving the anchor sampling/matching or feature
enhancement. SSH [25] builds detection modules on different feature maps with
a rich receptive field. S3FD [39] introduces an anchor compensation strategy
by offsetting anchors for outer faces. PyramidBox [30] formulates a data-
anchor-sampling strategy to increase the proportion of small faces in the
training data. DSFD [15] introduces small faces supervision signals on the
backbone, which implicitly boosts the performance of pyramid features. Group
sampling [24] emphasizes the importance of the ratio for matched and unmatched
anchors. RetinaFace [7] employs deformable context modules and additional
landmark annotations to improve the performance of face detection. HAMBox [23]
finds that many unmatched anchors in the training phase also have strong
localization ability and proposes an online high-quality anchor mining
strategy to assign high-quality anchors for outer faces. BFbox [22] employs a
single-path one-shot search method [11] to jointly optimise the backbone and
neck for face detector. ASFD [35] explores a differential architecture search
to discover optimised feature enhance modules for efficient multi-scale
feature fusion and context enhancement. All these methods are either designed
by expert experience or partially optimised on backbone, neck and head. By
contrast, we search for computation redistribution across different components
(backbone, neck and head) of a face detector across a wide range of compute
regimes, which is a global optimisation.

Neural Architecture Search. Given a fixed search space of possible networks,
Neural Architecture Search (NAS) automatically finds a good model within the
search space. DetNAS [4] adopts the evolution algorithm for the backbone
search to boost object detection on COCO [20]. By contrast, CR-NAS [17]
reallocates the computation across different stages within the backbone to
improve object detection. NAS-FPN [10] uses reinforcement learning to search
the proper FPN for general object detection. As there is an obvious
distribution gap between COCO [20] and WIDER FACE [34], the experience in the
above methods is not directly applicable for face detection but gives us an
inspiration that the backbone, neck and head can be optimised to enhance the
performance of face detection. Inspired by RegNet [27], we optimise the
computation distribution on backbone, neck and head based on the statistics
from a group of randomly sampled models. We successfully reduce the search
space and find the stable computation distribution under a particular complex
regime, which significantly improves the model’s performance.

##  3 TinaFace Revisited

(a) Different Testing Scales

(b) Computation Distribution of TinaFace@$640\times 480$

Figure 2: (a) Precision-recall curves of TinaFace-ResNet50 on the WIDER FACE
hard validation subset, under different testing scales. (b) Computation
distribution of TinaFace on backbone, neck and head with $640\times 480$ as
the testing scale.

Testing Scale | AP |  AP@Precision $>98\%$ | #Flops(G)
---|---|---|---
Multi-scale | 0.934 | 0.746 | 42333.64
1650 | 0.930 | 0.750 | 1021.82
Original(1024) | 0.914 | 0.706 | 508.47
640 | 0.814 | 0.539 | 172.95

Table 1: Performance and computation comparisons of TinaFace under different
testing scales. The average scale of original images is around $882\times
1024$.

Based on RetinaNet [18], TinaFace [41] employs ResNet-50 [12] as backbone, and
Feature Pyramid Network (FPN) [18] as neck to construct the feature extractor.
For the head design, TinaFace first uses a feature enhancement module on each
feature pyramid to learn surrounding context through different receptive
fields in the inception block [29]. Then, four consecutive $3\times 3$
convolutional layers are appended on each feature pyramid. Focal loss [19] is
used for the classification branch, DIoU loss [40] for the box regression
branch and cross-entropy loss for the IoU prediction branch.

To detect tiny faces, TinaFace tiles anchors of three different scales, over
each level of the FPN (_i.e_.
$\\{2^{4/3},2^{5/3},2^{6/3}\\}\times\\{4,8,16,32,64,128\\}$, from level
$P_{2}$ to $P_{7}$). The aspect ratio is set as $1.3$. During training, square
patches are cropped from the original image and resized to $640\times 640$,
using a scaling factor randomly sampled from $[0.3,0.45,0.6,0.8,1.0]$,
multiplied by the length of the original image’s short edge. During testing,
TinaFace employs single scale testing, when the short and long edge of the
image do not surpass $[1100,1650]$. Otherwise, it employs with short edge
scaling at $[500,800,1100,1400,1700]$, shift with directions
$[(0,0),(0,1),(1,0),(1,1)]$ and horizontal filp.

As shown in Fig. 2(a) and Tab. 1, we compare the performance of TinaFace under
different testing scales. For the multi-scale testing, TinaFace achieves an
impressive AP of $93.4\%$, which is the current best performance on the WIDER
FACE leader-board. For large single-scale testing ($1650$), the AP slightly
drops at $93.0\%$ but the computation significantly decreases to $1021.82$
Gflops. On the original scale (~$1024$), the performance of TinaFace is still
very high, obtaining an AP of $91.4\%$ with $508.47$ Gflops. Moreover, when
the testing scale decreases to VGA level ($640$), the AP significantly reduces
to $81.4\%$, with the computation further decreasing at $172.95$ Gflops.

In Fig. 2(b), we illustrate the computation distribution of TinaFace on the
backbone, neck and head components with a testing scale of $640$. From the
view of different scales of the feature pyramid, the majority of the
computational costs (about $68\%$) are from stride 4, as the resolution of the
feature map is quite large ($120\times 160$). From the view of the different
components of the face detector, most of the computational costs (about
$79\%$) are from the head, since the backbone structure is directly borrowed
from the ImageNet classification task [5], without any modification.

Even though TinaFace achieves state-of-the-art performance on tiny face
detection, the heavy computational cost renders it unsuitable for real-time
applications. In addition, real-world face detection systems always require
high precision (_e.g_. $>98\%$), in order to avoid frequent false alarms. As
shown in Tab. 1, the APs of TinaFace significantly drop, when the threshold of
detection scores is increased to meet the requirement of high precision.

##  4 Methodology

Based on the above analysis of TinaFace, and the following meticulous
experimentation, we propose the following efficiency improvements on the
design of face detection, conditioned on (1) the testing scale bounded at the
VGA resolution ($640$), and (2) there is no anchor tiled on the feature map of
stride 4. In particular, we tile anchors of $\\{16,32\\}$ at the feature map
of stride 8, anchors of $\\{64,128\\}$ at stride 16, and anchors of
$\\{256,512\\}$ at stride 32. Since our testing scale is smaller, most of the
faces will be predicted on stride 8. Therefore, we first investigate the
redistribution of positive training samples across different scales of feature
maps (Sec. 4.1). Then, we explore the computation redistribution across
different scales of feature maps, as well as across different components
(_i.e_. backbone, neck and head), given a pre-defined computation budget (Sec.
4.2).

###  4.1 Sample Reallocation

Figure 3: Cumulative face scale distribution on the WIDER FACE validation
dataset (Easy $\subset$ Medium $\subset$ Hard). When the long edge is fixed as
$640$ pixels, most of the easy faces are larger than $32\times 32$, and most
of the medium faces are larger than $16\times 16$. For the hard track, 78.93%
faces are smaller than $32\times 32$, 51.85% faces are smaller than $16\times
16$, and 13.36% faces are smaller than $8\times 8$.

(a) Ground-truth Distribution

(b) Positive Anchor Distribution

Figure 4: Ground-truth and positive anchor distribution within one epoch. The
baseline method employs a random size from the set $[0.3,1.0]$, while our
method uses a random size from the set $[0.3,2.0]$. The number of small faces
($<32\times 32$) significantly increases after the large cropping strategy is
used.

We find that the feature map of stride 8 is most important in our setting.
This is evident in Fig. 3, where we draw the cumulative face scale
distribution on the WIDER FACE validation dataset. When the test scale is
fixed as $640$ pixels, 78.93% faces are smaller than $32\times 32$.

During training data augmentation, square patches are cropped from the
original images with a random size from the set $[0.3,1.0]$ of the short edge
of the original images. To generate more positive samples for stride 8, we
enlarge the random size range from $[0.3,1.0]$ to $[0.3,2.0]$. When the crop
box is beyond the original image, average RGB values fill the missing pixels.
As illustrated in Fig. 4(a), there are more faces below the scale of $32$
after the proposed large cropping strategy is used. Moreover, even though
there will be more extremely tiny faces (_e.g_. $<4\times 4$) under the large
cropping strategy, these ground-truth faces will be neglected during training
due to unsuccessful anchor matching. As shown in Fig. 4(b), positive anchors
within one epoch significantly increase from 72.3K to 118.3K at the scale of
$16$, and increase from 95.9K to 115.1K at the scale of $32$. With more
training samples redistributed to the small scale, the branch to detect tiny
faces can be trained more adequately.

###  4.2 Computation Redistribution

Directly utilising the backbone of a classification network for scale-specific
face detection can be sub-optimal. Therefore, we employ network structure
search [27] to reallocate the computation on the backbone, neck and head,
under a wide range of flop regimes. We apply our search method on RetinaNet
[18], with ResNet [12] as backbone, Path Aggregation Feature Pyramid Network
(PAFPN) [21] as the neck and stacked $3\times 3$ convolutional layers for the
head. While the general structure is simple, the total number of possible
networks in the search space is unwieldy. In the first step, we explore the
reallocation of the computation within the backbone parts (_i.e_. stem, C2,
C3, C4, and C5), while fixing the neck and head components. Based on the
optimised computation distribution on the backbone that we find, we further
explore the reallocation of the computation across the backbone, neck and
head. By optimising in both manners, we achieve the final optimised network
design for face detection.

Computation search space reduction. Our goal is to design better networks for
efficient face detection, by redistributing the computation. Given a fixed
computation cost, as well as the face scale distribution presented in Fig. 3,
we explore the relationship between computation distribution and performance
from populations of models.

Following RegNet [27], we explore the structures of face detectors, assuming
fixed standard network blocks (_i.e_., basic residual or bottleneck blocks
with a fixed bottleneck ratio of $4$). In our case, the structure of a face
detector includes: (1) the backbone stem, three $3\times 3$ convolutional
layers with $w_{0}$ output channels [13], (2) the backbone body, four stages
operating at progressively reduced resolution, with each stage consisting of a
sequence of identical blocks. For each stage $i$, the degrees of freedom
include the number of blocks $d_{i}$ (_i.e_. network depth) and the block
width $w_{i}$ (_i.e_. number of channels). The structure of the face detector
also includes: (3) the neck, a multi-scale feature aggregation module by a
top-down path and a bottom-up path with $n_{i}$ channels [21], (4) the head,
with $h_{i}$ channels of $m$ blocks to predict face scores and regress face
boxes.

As the channel number of the stem is equal to the block width of the first
residual block in C2, the degree of freedom of the stem can be merged into
$w_{1}$. In addition, we employ a shared head design for three-scale of
feature maps and fix the channel number for all $3\times 3$ convolutional
layers within the heads. Therefore, we reduce the degrees of freedom to three
within our neck and head design: (1) output channel number $n$ for neck, (2)
output channel number $h$ for head, and (3) the number of $3\times 3$
convolutional layers $m$. We perform uniform sampling of $n\leq 256$, $h\leq
256$, and $m\leq 6$ (both $n$ and $h$ are divisible by 8).

The backbone search space has 8 degrees of freedom as there are 4 stages and
each stage $i$ has 2 parameters: the number of blocks $d_{i}$ and block width
$w_{i}$. Following RegNet [27], we perform uniform sampling of $d_{i}\leq 24$
and $w_{i}\leq 512$ ($w_{i}$ is divisible by 8). As state-of-the-art backbones
have increasing widths [27], we also shrink the search space constrained to
the principle of $w_{i+1}\geq w_{i}$.

With the above simplifications, our search space becomes more straightforward.
We repeat the random sampling in our search space until we obtain $320$ models
in our target complexity regime, and train each model on the WIDER FACE
training set for $80$ epochs. Then, we test the AP of each model on the
validation set. Based on these $320$ pairs of model statistics
$(x_{i},AP_{i})$, where $x_{i}$ is the computation ratio of a particular
component and $AP_{i}$ the corresponding performance, we follow [27] to
compute the empirical bootstrap [8] to estimate the likely range in which the
best models fall.

Finally, to further decrease the complexity of search space, we divide our
network structure search into the following two steps:

  * •

SCRFD1: search the computational distribution for the backbone only, while
fixing the settings of neck and head to the default configuration.

  * •

SCRFD2: search the computational distribution over the whole face detector
(_i.e_. backbone, neck and head), with the computational distribution within
the backbone following the optimised SCRFD1.

Here, we use SCRFD constrained to 2.5 Gflops (SCRFD-2.5GF) as an example to
illustrate our two-step searching strategy.

(a) Stem $\sim(10\%,20\%)$

(b) C2 $\sim(24\%,39\%)$

(c) C3 $\sim(26\%,47\%)$

(d) C4 $\sim(4\%,15\%)$

(e) C5 $\sim(1\%,16\%)$

Figure 5: Computation redistribution on the backbone (stem, C2, C3, C4 and C5)
with fixed neck and head under the constraint of 2.5 Gflops. For each
component within the backbone, the range of computation ratio in which the
best models may fall is estimated by the empirical bootstrap.

(a) Stem+C2+C3 $\sim(72\%,91\%)$

(b) C4+C5 $\sim(9\%,28\%)$

Figure 6: Computation redistribution between the shallow and deep stages of
the backbone under the constraint of 2.5 Gflops.

Computation redistribution on backbone. Since the backbone performs the bulk
of the computation, we first focus on the structure of the backbone, which is
central in determining the network’s computational cost and accuracy. For
SCRFD1-2.5GF, we fix the output channel of the neck at $32$ and use two
stacked $3\times 3$ convolutions with $96$ output channels. As the neck and
head configurations do not change in the whole search process of SCRFD1, we
can easily find the best computation distribution of the backbone. As
described in Fig. 5, we show the distribution of $320$ model APs (on the WIDER
FACE hard validation set) versus the computation ratio over each component
(_i.e_. stem, C2, C3, C4 and C5) of backbone. After applying an empirical
bootstrap [8], a clear trend emerges showing that the backbone computation is
reallocated to the shallow stages (_i.e_. C2 and C3). In Fig. 6, we present
the computation ratio between the shallow (_i.e_. stem, C2, and C3) and deep
stages (_i.e_. C4 and C5) of the backbone. Based on the observation from these
search results, it is obvious that around $80\%$ of the computation is
reallocated to the shallow stages.

(a) 2.5GF

(b) SCRFD-2.5GF

(c) 10GF

(d) SCRFD-10GF

(e) 34GF

(f) SCRFD-34GF

Figure 7: Computation redistribution (left column) and the searched network
structures (right column) under different computation constraints (2.5GF,
10GF, and 34GF). Network diagram legends in the second row contain all
information required to implement the SCRFD models that we have optimised the
computation across stages and components.

(a) 0.5GF

(b) SCRFD-0.5GF

Figure 8: Computation redistribution (left) and the searched network
structures (right) for the mobile regime (0.5 Gflops).

(a) Backbone $\sim(67\%,88\%)$

(b) Neck $\sim(1\%,7\%)$

(c) Head $\sim(10\%,26\%)$

(d) Architecture Sketches

Figure 9: Computation redistribution and architecture sketches under the
constraint of 2.5 Gflops. The computation distribution of SCRFD-2.5GF within
backbone follows Fig. 5. Please refer to Tab. 2 for the network configuration
of ResNet-2.5GF. In (d), the yellow rectangles in C2 to C5 represents the
basic residual block. The width of rectangles corresponds to the computation
cost. After computation redistribution, more computations are allocated to
shallow stages (_i.e_. C2 and C3).

Computation redistribution on backbone, neck and head. After we find the
optimised computation distribution within the backbone under a specific
computational constraint of 2.5 Gflops, we search for the best computation
distribution over the backbone, neck and head. In this step, we only keep the
randomly generated network configurations whose backbone settings follow the
computation distribution from SCRFD1 as shown in Fig. 5.

Now there are another three degrees of freedom (_i.e_. output channel number
$n$ for neck, output channel number $h$ for head, and the number of $3\times
3$ convolutional layers $m$ in head). We repeat the random sampling in our
search space, until we obtain $320$ qualifying models in our target complexity
regime (_i.e_. 2.5 Gflops). As evident in Fig. 9, most of the computation is
allocated in the backbone, with the head following and the neck having the
lowest computation ratio. Fig. 9(d) depicts the comparison of the model
architecture, under the constraint of 2.5 Gflops. The network configuration of
the baseline (ResNet-2.5GF) is introduced in Tab. 2.

By employing the proposed two-step computation redistribution method, we find
a large amount of capacity is allocated to the shallow stages, resulting in an
AP improvement from $74.47\%$ to $77.87\%$ on the WIDER FACE hard validation
set.

Higher compute regimes and mobile regime. Besides the complexity constraint of
2.5 Gflops, we also utilise the same two-step computation redistribution
method to explore the network structure optimisation for higher compute
regimes (_e.g_. 10 Gflops and 34 Gflops) and low compute regimes (_e.g_. 0.5
Gflops). In Fig. 7 and Fig. 8, we show the computation redistribution and the
optimised network structures under different computation constraints.

Our final architectures have almost the same flops as the baseline network.
From these redistribution results, we can draw the following conclusions: (1)
more computation is allocated in the backbone and the computation on the neck
and head is compressed; (2) more capacity is reallocated in shallow stages for
the 2.5 Gflops, 10 Gflops and 34 Gflops regimes due to the specific scale
distribution on WIDER FACE; (3) for the high compute regime (_e.g_. 34
Gflops), the explored structure utilises the bottleneck residual block and we
observe significant depth scaling, instead of width scaling in shallow stages.
Scaling the width is subject to over-fitting due to the larger increase in
parameters [1]. By contrast, scaling the depth, especially in the earlier
layers, introduces fewer parameters compared to scaling the width; (4) for the
mobile regime (0.5 Gflops), allocating the limited capacity in the deep stage
(_e.g_. C5) for the discriminative features captured in the deep stage, can
benefit the shallow small face detection by the top-down neck pathway.

Name | Backbone | Neck | Head
---|---|---|---
ResNet-2.5GF | ResNet34x0.25 | 48 | [96,96]
ResNet-10GF | ResNet34x0.5 | 128 | [160,160]
ResNet-34GF | ResNet50 | 128 | [256,256]
MobileNet-0.5GF | MobileNetx0.25 | 32 | [80,80]
Table 2: Network configurations for baselines across different compute regimes. $\times 0.25$ and $\times 0.5$ in backbone refer to decreasing the channel number by $0.25$ and $0.5$ compared to the original designs [12, 14]. Basic residual blocks are used in ResNet-2.5GF and ResNet-10GF, while bottleneck residual blocks are used in ResNet-34GF. For MobileNet-0.5GF, depth-wise convolution is used in both backbone and head. Method | Easy | Medium | Hard
---|---|---|---
ResNet-2.5GF | 91.87 | 89.49 | 67.32
SR + ResNet-2.5GF | 93.21 | 91.11 | 74.47
CR@backbone | 92.32 | 90.25 | 69.78
CR@detector | 92.61 | 90.74 | 70.98
CR@two-step | 92.66 | 90.72 | 71.37
SR+CR@two-step | 93.78 | 92.16 | 77.87
Table 3: Ablation experiments of SCRFD-2.5GF (_i.e_. SR+CR@two-step) on the
WIDER FACE validation subset. The baseline model is ResNet-2.5GF as introduced
in Tab. 2. “SR” and “CR” denote the proposed sample and computation
redistribution, respectively.

Method | Backbone | Easy | Medium | Hard | #Params(M) | #Flops(G) | Infer(ms)
---|---|---|---|---|---|---|---
DSFD [15] (CVPR19) | ResNet152 | 94.29 | 91.47 | 71.39 | 120.06 | 259.55 | 55.6
RetinaFace [7] (CVPR20) | ResNet50 | 94.92 | 91.90 | 64.17 | 29.50 | 37.59 | 21.7
HAMBox [23] (CVPR20) | ResNet50 | 95.27 | 93.76 | 76.75 | 30.24 | 43.28 | 25.9
TinaFace [41] (Arxiv20) | ResNet50 | 95.61 | 94.25 | 81.43 | 37.98 | 172.95 | 38.9
ResNet-34GF | ResNet50 | 95.64 | 94.22 | 84.02 | 24.81 | 34.16 | 11.8
SCRFD-34GF | Bottleneck Res | 96.06 | 94.92 | 85.29 | 9.80 | 34.13 | 11.7
ResNet-10GF | ResNet34x0.5 | 94.69 | 92.90 | 80.42 | 6.85 | 10.18 | 6.3
SCRFD-10GF | Basic Res | 95.16 | 93.87 | 83.05 | 3.86 | 9.98 | 4.9
ResNet-2.5GF | ResNet34x0.25 | 93.21 | 91.11 | 74.47 | 1.62 | 2.57 | 5.4
SCRFD-2.5GF | Basic Res | 93.78 | 92.16 | 77.87 | 0.67 | 2.53 | 4.2
RetinaFace [7] (CVPR20) | MobileNet0.25 | 87.78 | 81.16 | 47.32 | 0.44 | 0.802 | 7.9
FaceBoxes [38] (IJCB17) | - | 76.17 | 57.17 | 24.18 | 1.01 | 0.275 | 2.5
MobileNet-0.5GF | MobileNetx0.25 | 90.38 | 87.05 | 66.68 | 0.37 | 0.507 | 3.7
SCRFD-0.5GF | Depth-wise Conv | 90.57 | 88.12 | 68.51 | 0.57 | 0.508 | 3.6

Table 4: Accuracy and efficiency of different methods on the WIDER FACE
validation set. The size of test images is bounded by $640$. The proposed
sample redistribution is also employed by ResNet-34GF, ResNet-10GF,
ResNet-2.5GF and MobileNet-0.5GF for fair comparisons.#Params and #Flops
denote the number of parameters and multiply-adds. “Infer” refers to network
inference latency with VGA resolution ($640\times 480$) images on NVIDIA
2080TI.

(a) Easy

(b) Medium

(c) Hard

(d) Hard@Precision$>98\%$

(e) Visualisation Results of SCRFD-2.5GF

Figure 10: Precision-recall curves and qualitative results on the validation
set of the WIDER FACE dataset. Yellow rectangles show the detection results by
the proposed SCRFD-2.5GF and brightness encodes the detection confidence.

##  5 Experiments

###  5.1 Implementation Details

Training. For the baseline methods, square patches are cropped from the
original images with a random size from $[0.3,0.45,0.6,0.8,1.0]$ and then
these patches are resized to $640\times 640$ for training. For the proposed
Sample Redistribution (SR), we enlarge the random size set by adding
$[1.2,1.4,1.6,1.8,2.0]$ scales. Besides scale augmentation, the training data
are also augmented by color distortion and random horizontal flipping, with a
probability of $0.5$. For the anchor setting, we tile anchors of
$\\{16,32\\}$, $\\{64,128\\}$, and $\\{256,512\\}$ on the feature maps of
stride 8, 16, and 32, respectively. The anchor ratio is set as $1.0$. In this
paper, we employ Adaptive Training Sample Selection (ATSS) [37] for positive
anchor matching. In the detection head, weight sharing and Group Normalisation
[33] are used [31]. The losses of classification and regression branches are
Generalised Focal Loss (GFL) [16] and DIoU loss [40], respectively.

Our experiments are implemented in PyTorch, based on the open-source
mmdetection [3]. We adopt the SGD optimizer (momentum 0.9, weight decay 5e-4)
with a batch size of $8\times 4$ and train on four Tesla V100. The initial
learning rate is set to $0.00001$, linearly warming up to $0.01$ within the
first $3$ epochs. During network search, the learning rate is multiplied by
$0.1$ at the $56$-th, and $68$-th epochs. The learning process terminates on
the $80$-th epoch. For re-training of both baselines and the best
configurations after search, the learning rate decays by a factor of 10 at the
$440$-th and $544$-th epochs, and the learning process terminates at the
$640$-th epoch. For the computation constrained baselines in Tab. 2, we also
employ the same training schedule. All the models are trained from scratch
without any pre-training.

Testing. We only employ single-scale testing, bounded by the VGA resolution
($640\times 480$). The results of DSFD [15], RetinaFace [7], TinaFace [41] and
Faceboxes [38] are reported by testing the released models, while the HAMBox
[23] model is shared from the author.

###  5.2 Ablation Study

In Tab. 3 we present the AP performance of models by gradually including the
proposed sample and computation redistribution methods. Our baseline model
(_i.e_. ResNet-2.5GF as introduced in Tab. 2) gets $91.87\%$, $89.49\%$, and
$67.32\%$ in the three settings on the validation subset. By adding sample
redistribution, the hard set AP significantly increases to $74.47\%$,
indicating the benefit from allocating more training samples on the feature
map of stride 8.

After separately employing the proposed computation redistribution on the
backbone and the whole detector, the AP on the hard set improves to $69.78\%$
and $70.98\%$. This indicates that (1) the network structure directly
inherited from the classification task is sub-optimal for the face detection
task, and (2) joint computation reallocation on the backbone, neck and head
outperforms computation optimisation applied only on the backbone.
Furthermore, the proposed two-step computation redistribution strategy
achieves AP of $71.37\%$, surpassing one-step computation reallocation on the
whole detector by $0.39\%$. As we shrink the whole search space by the
proposed two-step strategy and our random model sampling number is fixed at
$320$, the two-step method is possible to find better network configurations
from the large search space. Since sample redistribution and computation
redistribution are orthogonal, their combination (_i.e_. SCRFD-2.5GF) shows a
further improvement, achieving AP of $77.87\%$ on the hard track.

###  5.3 Accuracy and Efficiency on WIDER FACE

As shown in Tab. 4 and Fig. 10, we compared the proposed method with other
state-of-the-art face detection algorithms (_e.g_. DSFD [15], RetinaFace [7],
HAMBox [23] and TinaFace [41]). Since we fix the testing scale at $640$, the
flops and inference time directly correspond to the reported AP. The proposed
SCRFD-34GF outperforms all these state-of-the-art methods on the three
subsets, especially for the hard track, which contains a large number of tiny
faces. More specifically, SCRFD-34GF surpasses TinaFace by $3.86\%$ while
being more than _3 $\times$ faster_ on GPUs. In addition, the computation cost
of SCRFD-34GF is only around $20\%$ of TinaFace. As SCRFD-34GF scales the
depth in the earlier layers, it also introduces fewer parameters, resulting in
a much smaller model size ($9.80M$).

Overall, all of the proposed SCRFD models (_e.g_. SCRFD-34GF, SCRFD-10GF,
SCRFD-2.5GF and SCRFD-0.5GF) provide considerable improvements compared to the
baseline models (listed in Tab. 2), by optimising the network structure alone
(_i.e_. computation redistribution), across a wide range of compute regimes.
For the low-compute regimes, SCRFD-0.5GF significantly outperforms
RetinaFaceM0.25 by $21.19\%$ on the hard AP, while consuming only $63.34\%$
computation and $45.57\%$ inference time.

For real-world face detection system, high precision (_e.g_. $>98\%$) is
required to avoid frequent false alarms. As shown in Fig. 10(d), SCRFD-2.5GF
obtains comparable AP ($53.7\%$) compared to TinaFace ($53.9\%$) when the
threshold score is improved to achieve prevision higher than $98\%$, while the
computation cost is only $1.46\%$ and the inference time is only $10.8\%$.
Fig. 10(e) shows qualitative results generated by SCRFD-2.5GF. As can be seen,
our face detector works very well in both indoor and outdoor crowded scenes
under different conditions (_e.g_. appearance variations from pose, occlusion
and illumination). The impressive performance across a wide range of scales
indicate that SCRFD-2.5GF has a very high recall and can detect faces
accurately even without large scale testing.

##  6 Conclusions

In this work, we present a sample and computation redistribution paradigm for
efficient face detection. Our results show significantly improved accuracy and
efficiency trade-off by the proposed SCRFD across a wide range of compute
regimes, when compared to the current state-of-the-art.

## References

  * [1] Irwan Bello, William Fedus, Xianzhi Du, Ekin Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, and Barret Zoph.  Revisiting resnets: Improved training and scaling strategies.  arXiv:2103.07579, 2021.
  * [2] Adrian Bulat and Georgios Tzimiropoulos.  How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks).  In ICCV, 2017.
  * [3] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al.  Mmdetection: Open mmlab detection toolbox and benchmark.  arXiv:1906.07155, 2019.
  * [4] Yukang Chen, Tong Yang, Xiangyu Zhang, Gaofeng Meng, Chunhong Pan, and Jian Sun.  Detnas: Neural architecture search on object detection.  arXiv:1903.10979, 2019.
  * [5] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In CVPR, 2009.
  * [6] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.  Arcface: Additive angular margin loss for deep face recognition.  In CVPR, 2019.
  * [7] Jiankang Deng, Jia Guo, Yuxiang Zhou, Jinke Yu, Irene Kotsia, and Stefanos Zafeiriou.  Retinaface: Single-stage dense face localisation in the wild.  In CVPR, 2020.
  * [8] Bradley Efron and Robert J Tibshirani.  An introduction to the bootstrap.  CRC press, 1994.
  * [9] Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, and Xi Zhou.  Joint 3d face reconstruction and dense alignment with position map regression network.  In ECCV, 2018.
  * [10] Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le.  Nas-fpn: Learning scalable feature pyramid architecture for object detection.  In CVPR, 2019.
  * [11] Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, and Jian Sun.  Single path one-shot neural architecture search with uniform sampling.  arXiv:1904.00420, 2019.
  * [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2016.
  * [13] Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li.  Bag of tricks for image classification with convolutional neural networks.  In CVPR, 2019.
  * [14] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  arXiv:1704.04861, 2017.
  * [15] Jian Li, Yabiao Wang, Changan Wang, Ying Tai, Jianjun Qian, Jian Yang, Chengjie Wang, Jilin Li, and Feiyue Huang.  Dsfd: dual shot face detector.  In CVPR, 2019.
  * [16] Xiang Li, Wenhai Wang, Lijun Wu, Shuo Chen, Xiaolin Hu, Jun Li, Jinhui Tang, and Jian Yang.  Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection.  arXiv:2006.04388, 2020.
  * [17] Feng Liang, Chen Lin, Ronghao Guo, Ming Sun, Wei Wu, Junjie Yan, and Wanli Ouyang.  Computation reallocation for object detection.  In ICLR, 2020.
  * [18] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In CVPR, 2017.
  * [19] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In ICCV, 2017.
  * [20] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In ECCV, 2014.
  * [21] Shu Liu, Lu Qi, Haifang Qin, Jianping Shi, and Jiaya Jia.  Path aggregation network for instance segmentation.  In CVPR, 2018.
  * [22] Yang Liu and Xu Tang.  Bfbox: Searching face-appropriate backbone and feature pyramid network for face detector.  In CVPR, 2020.
  * [23] Yang Liu, Xu Tang, Xiang Wu, Junyu Han, Jingtuo Liu, and Errui Ding.  Hambox: Delving into online high-quality anchors mining for detecting outer faces.  CVPR, 2020.
  * [24] Xiang Ming, Fangyun Wei, Ting Zhang, Dong Chen, and Fang Wen.  Group sampling for scale invariant face detection.  In CVPR, 2019.
  * [25] Mahyar Najibi, Pouya Samangouei, Rama Chellappa, and Larry S Davis.  Ssh: Single stage headless face detector.  In ICCV, 2017.
  * [26] Hongyu Pan, Hu Han, Shiguang Shan, and Xilin Chen.  Mean-variance loss for deep age estimation from a face.  In CVPR, 2018.
  * [27] Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár.  Designing network design spaces.  In CVPR, 2020.
  * [28] Florian Schroff, Dmitry Kalenichenko, and James Philbin.  Facenet: A unified embedding for face recognition and clustering.  In CVPR, 2015.
  * [29] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In CVPR, 2015.
  * [30] Xu Tang, Daniel K Du, Zeqiang He, and Jingtuo Liu.  Pyramidbox: A context-assisted single shot face detector.  In ECCV, 2018.
  * [31] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  Fcos: Fully convolutional one-stage object detection.  In ICCV, 2019.
  * [32] Paul Viola and Michael J Jones.  Robust real-time face detection.  IJCV, 2004.
  * [33] Yuxin Wu and Kaiming He.  Group normalization.  In ECCV, 2018.
  * [34] Shuo Yang, Ping Luo, Chen-Change Loy, and Xiaoou Tang.  Wider face: A face detection benchmark.  In CVPR, 2016.
  * [35] Bin Zhang, Jian Li, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, Yili Xia, Wenjiang Pei, and Rongrong Ji.  Asfd: Automatic and scalable face detector.  arXiv:2003.11228, 2020.
  * [36] Feifei Zhang, Tianzhu Zhang, Qirong Mao, and Changsheng Xu.  Joint pose and expression modeling for facial expression recognition.  In CVPR, 2018.
  * [37] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li.  Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection.  In CVPR, 2020.
  * [38] Shifeng Zhang, Xiangyu Zhu, Zhen Lei, Hailin Shi, Xiaobo Wang, and Stan Z Li.  Faceboxes: A cpu real-time face detector with high accuracy.  In IJCB, 2017.
  * [39] Shifeng Zhang, Xiangyu Zhu, Zhen Lei, Hailin Shi, Xiaobo Wang, and Stan Z Li.  S3fd: Single shot scale-invariant face detector.  In ICCV, 2017.
  * [40] Zhaohui Zheng, Ping Wang, Wei Liu, Jinze Li, Rongguang Ye, and Dongwei Ren.  Distance-iou loss: Faster and better learning for bounding box regression.  In AAAI, 2020.
  * [41] Yanjia Zhu, Hongxiang Cai, Shuhan Zhang, Chenhao Wang, and Yichao Xiong.  Tinaface: Strong but simple baseline for face detection.  arXiv:2011.13183, 2020.


or convective. The demand for energy conservation is implemented through an
iterative scheme that searches for a solution that is consistent with the
boundary conditions (see Paper I; Mordasini et al. (2012c)).

Initially, the gaseous envelope around all planets transitions smoothly into
the nebular gas, the so-called attached phase (see panel (d) in Fig. 1). In
the attached phase gas from the nebular disk flows into the planet to
compensate for planetary contraction. Planets contract as they cool down by
radiating away the energy gained through accretion. The surface pressure and
temperature of the planet are balanced with those of disk mid-plane When a
planet reaches a critical threshold mass, large radiative losses cannot be
balanced by accretional energy, and further contraction of the envelope
ensues. This results in even more gas accretion, further increasing the
accretional energy, and runaway accretion of gas is inevitable.
Consequentially, planets gain a massive envelope and very rapidly become giant
planets.

For planetary cores massive enough to undergo runaway gas accretion, the rate
of gas accretion may exceed the ability of the disk to supply gas (the maximum
gas accretion rate). Then the envelope detaches from the gas disk and the
planet continues to accrete in this detached phase. In the detached phase gas
accretion does not depend on the planetary internal structure but on the
protoplanetary gas disk. The planet’s radius contracts very rapidly to $\sim
R_{\text{J}}$ as it adjusts to the new boundary conditions.

The last phase of planetary evolution, which is common to all planets, is the
isolated phase, which occurs after the gas disk has dissipated. Gas accretion
comes to a halt and planets will now contract as they cool down.

##  Appendix C KOBE: Additional details

###  C.1 KOBE-Shadows

Figure 15: The geometry of transiting planets as implemented in KOBE. Left:
Transit geometry for a single star–planet system. The dashed red arrow
represents the initial location of an observer $O_{i}(\theta,\phi)$, and the
solid red arrow gives the rotated location of same observer $O_{i}(f,\alpha)$
(see text for details). Right: Example of transit shadow bands for a 24-planet
system (star and planets are not shown). KOBE-Shadows calculates whether an
observer is present inside the full transit shadow band of any planet in this
system. The innermost planet has $P_{\mathrm{tra}}=12\%$ and is detectable by
observers marked in blue. The second ($P_{\mathrm{tra}}=6\%$) and the third
innermost planet ($P_{\mathrm{tra}}=2\%$) are detectable by observers (in
orange and green, respectively). The probability to potentially detect any
four planets simultaneously via transit for this system is $\sim 8\%$, which
is halved for simultaneously detecting any nine planets.

Figure 16: Inputs required by KOBE to simulate the Kepler transit survey.
Left: Distribution of stellar noise, rms
$\mathrm{CDPP}_{$6\text{\,}\mathrm{h}$}$, as seen by Kepler for FGK solar-type
stars. Each individual KOBE system is assigned a CDPP value from this
distribution. Right: Two-dimensional histogram depicting completeness of
Kepler’s Robovetter, as calculated by KOBE-Vetter. To emulate high
reliability, a disposition score cut-off is placed. Labels on each bin
indicate the completeness (in $\%$) for that bin. This calculation is based on
the injected transit signals (Coughlin 2017). Completeness is not calculated
for bins with less than ten injected signals (not labelled). These bins are
assigned a completeness of $0\%$.

Whether a planet can transit for a given observer is determined by KOBE-
Shadows. For this the transit shadow band (TSB) is determined by examining
transit geometry, as shown in Fig. 15 (left). A planet orbiting a star will
intercept some of the starlight, casting its shadow on a celestial sphere
centred on this star. As the planet moves in its orbit the planet’s shadow
traces a shadow band on the celestial sphere. The area on the celestial sphere
which falls inside a planet’s shadow constitutes the TSB.

All observers inside a planet’s TSB can potentially detect this planet via its
transit. This planet will not appear to be transiting to any observer who is
outside the planet’s TSB. KOBE-Shadows utilizes this distinction between
transiting and non-transiting planets to detect the former.

To compare with Kepler, simulating $\sim 200\,000$ stars in NGPPS is
computationally expensive. KOBE-Shadows offers a convenient solution. Using
transits, a single observer may never find all the planets in a planetary
system. This could occur either when the TSBs from all the planets never
overlap or when the location of an observer is outside the TSB of some
planets. These are the basic geometrical limitations of the transit method.
When the same system is observed by another observer located elsewhere, they
may find different transiting planets than those found by the first observer.
This means that two observers can view two different subsets of the same
planetary system. By analysing the transit geometry of a planetary system for
several observers, KOBE-Shadows generates multiple subsets of transiting
systems (one from each observer). Subsequent modules in KOBE treat these
subsystems, KOBE systems, as independent planetary systems. This allows KOBE
to emulate observations of $200\,000$ stars from $1\,000$ NGPPS systems.
Additional steps are taken in other modules to ensure that these system are
treated independently252525 KOBE systems, being subsets of the NGPPS systems,
are not truly independent. In KOBE a single observer observing $1\,000$
synthetic planetary systems may detect $\sim 100$ transiting planets around
$\sim 50$ stars. The strategy of using multiple observers in KOBE grew out of
a need to (a) find more transiting planets, (b) imprint the effect of transit
probability (via TSB calculation), and (c) systematize KOBE’s approach. . KOBE
systems with at least one transiting planet form the KOBE-Shadows catalogue,
which is analysed in subsequent modules.

To compute the TSB for multiple planets in a system, KOBE-Shadows requires
three things: First, the stellar and the planetary radii,
$\left(R_{\star},R_{\mathrm{planet}}\right)$. These are provided by the Bern
Model and they evolve with time. Second, the orbital elements
$\left(a,e,i,\Omega,\omega,\overline{f}\right)$. The six orbital elements are
semi-major axis $a$, eccentricity $e$, inclination $i$, longitude of ascending
node $\Omega$, argument of periapse $\omega$, and true anomaly $\overline{f}$.
The first two elements describe the 2D shape of an elliptical orbit. The next
three orbital elements define the relative orientation of an orbit in 3D
space. The last element gives the position of a planet on its orbit with
respect to the periastron262626A planet’s position on the orbit,
$\overline{f}$, is not required to calculate its TSB because the calculation
is done for virtually all positions on the orbit.. The Bern Model provides all
of these elements for all planets. However, an orbit’s inclination as
calculated in the model is with reference to the original protoplanetary disk,
as opposed to any observer’s reference sky-plane. These values are calculated
only during the N-body stage. A planet’s period $P$ is calculated as
$P^{2}=\tfrac{4\pi^{2}}{GM_{\star}}a^{3}$. Finally, third, the location of an
observer, $O_{i}(\theta,\phi)$. The location of an observer is specified in a
standard coordinate system $(X,Y,Z)$. The origin is at the star (see Fig. 15,
left). The $X$-axis is along a reference line and $(X,Y)$ define a reference
plane ($Z$ is perpendicular to this plane). An observer’s location is
specified in polar co-ordinates by an azimuth angle $\theta\in[0,2\pi]$ and a
polar angle $\phi\in[0,\pi]$. The subscript $i$ is used to distinguish between
multiple observers, and two observers are not allowed to be at the same
location. The Bern Model does not provide the location of any special
observer. It is assumed that the location of observers does not evolve with
time.

The calculation procedure implemented in KOBE-Shadows is as follows:

  1. 1.

The number of observers, $n_{\mathrm{obs}}$, is fixed by the number of stars
observed by a survey, $n_{\star,\mathrm{survey}}$, and the number of synthetic
systems (at time $t$) in a population, $n_{\mathrm{system}}(t)$. The relation
is

$n_{\mathrm{obs}}=\frac{n_{\star,\mathrm{survey}}}{n_{\mathrm{system}}(t)}.$
(12)

Fixing $n_{\star,\mathrm{survey}}$ to $200\,000$ for Kepler, a population with
$n_{\mathrm{system}}(t)=1\,000,$ would require $200$ observers. Although all
three populations simulate $1\,000$ systems, $n_{\mathrm{system}}(t)$ may
diminish with time. For the nominal populations, at $4\text{\,}\mathrm{Gyr}$,
NG74 has 998, NG75 has 999, and NG76 has $1\,000$ systems.

  2. 2.

The locations of the observers are distributed uniformly and homogeneously
around the celestial sphere in the $(X,Y,Z)$ coordinate system. This is done
by generating two different random numbers $u_{1},u_{2}\in[0,1)$ and using
inverse transform sampling to obtain polar coordinates:

$\theta=2\pi u_{1},\hskip 10.00002pt\text{and}\hskip
10.00002pt\phi=\mathrm{cos}^{-1}(1-2u_{2}).$ (13)

This initial location of observers remains fixed for one NGPPS system.
Spherical coordinates are converted to Cartesian giving $\@vec{v}_{(X,Y,Z)}$,
the unit-normed vectorial location of an observer.

  3. 3.

To align the transit geometry, the Bern Model gives the position of a planet
and its orbit in a different coordinate system, $(x,y,z)$ with origin at the
star. Here, $x$ is along the periastron, $(x,y)$ are in the orbital plane, and
$z$ points along the angular momentum vector of the planet. To proceed, the
two coordinate systems must be aligned. This is done by rotating the initial
location of all observers (given in $X,Y,Z$ coordinates) with respect to each
planet’s orbit (given in $x,y,z$ coordinates) via three rotations (for details
see Murray & Correia (2010))272727Equivalently, the planet’s orbit could be
rotated, keeping the location of observers fixed.:

$\@vec{v^{\prime}}_{(x,y,z)}=R^{-1}_{Z}(\omega)\ R^{-1}_{X}(i)\
R^{-1}_{Z}(\Omega)\ \ \@vec{v}_{(X,Y,Z)}.$ (14)

Here, $R^{-1}_{X},R^{-1}_{Y}$, and $R^{-1}_{Z}$ are the inverses of the
standard SO(3) rotation matrices. $R^{-1}_{Z}(\Omega)$ aligns the line of
nodes with the $X$-axis. $R^{-1}_{X}(i)$ aligns the reference plane with the
orbital plane. $R^{-1}_{Z}(\omega)$ rotates the reference plane such that the
$X$-axis points along the periastron point of the orbit. The new location of
the observer is given by the vector $\@vec{v^{\prime}}_{(x,y,z)}$ and in polar
coordinates $O^{\prime}_{i}(f,\alpha)$. In Fig. 15 (left) an example of such
rotation is shown.

Now the location of observers and the planetary orbits is known in the same
coordinate system. This step ensures that the relative orientation between
different planets is maintained, and information coming from all orbital
elements is used.

  4. 4.

Next, we estimate the angular width of the TSB. The semi-cone angle $\psi$,
subtended by the planet’s shadow as it blocks starlight at the azimuthal
location of observer $O^{\prime}_{i}(f,\alpha),$ is given by (Winn 2010)

$\begin{split}\mathrm{sin}(\psi)&=\left(\frac{R_{\star}\pm
R_{\mathrm{planet}}}{r_{\mathrm{planet}}}\right),\hskip
10.00002pt\begin{cases}+\hskip 10.00002pt\text{include grazing transits}\\\
-\hskip 10.00002pt\text{exclude grazing transits}\end{cases}\\\
r_{\mathrm{planet}}&=\frac{a\leavevmode\nobreak\ (1-e^{2})}{1+e\
\mathrm{cos}(f)}.\end{split}$ (15)

Here, $r_{\mathrm{planet}}$ is the star–planet distance when the planet’s true
anomaly is $\overline{f}=f$282828For this paper, grazing transits are
excluded.. Finally, it is checked whether the observer
$O^{\prime}_{i}(f,\alpha)$ is inside the TSB through the following condition:

$\begin{split}\alpha\in\left[\frac{\pi}{2}-\psi,\frac{\pi}{2}+\psi\right]&\rightarrow\text{Inside
TSB},\\\ \text{otherwise}&\rightarrow\text{Outside TSB}.\end{split}$ (16)

This information is now stored for a planet at the original location of the
observer $O_{i}(\theta,\phi)$.

  5. 5.

Repeat from step 4 for all $n_{\mathrm{obs}}$ observers.

  6. 6.

Repeat from step 3 for all planets in a system.

  7. 7.

Repeat from step 2 for all systems in a population.

Figure 15 (right) displays the result from KOBE-Shadows for a system with 24
planets in NG76. This calculation, done using $10^{6}$ observers and
considering only full transits, takes about 1s.

These calculations undergo two major consistency checks. Firstly, the area
under a planet’s TSB over the area of the celestial sphere gives the transit
probability for this planet. Here, the number of observers found inside a TSB
over $n_{\mathrm{obs}}$ gives a numerical proxy for the same. The analytical
expression for transit probability, $P_{\mathrm{tra}}$, of a planet can be
easily derived (Barnes 2007):

$\begin{split}P_{\mathrm{tra}}&=\frac{R_{\star}\pm R_{\mathrm{planet}}}{a\
(1-e^{2})}\hskip 10.00002pt\begin{cases}+\hskip 10.00002pt\text{include
grazing transits}\\\ -\hskip 10.00002pt\text{exclude grazing
transits}\end{cases}\end{split}.$ (17)

The numerical and analytical values of $P_{\mathrm{tra}}$, with varying
$n_{\mathrm{obs}}$ from ${10}^{5}{10}^{8}$, are found to be in good agreement.
In addition, the transit probability for multiple planets is available for
free via the numerical recipe described above. Finding the same through
analytical approaches is a difficult problem.

Secondly, the impact parameter of a transiting planet for all observers inside
the TSB should be less than $1+(R_{\mathrm{planet}}/R_{\star})$. The impact
parameter, $b$, is the sky-projected star–planet distance expressed in units
of stellar radii, and is given by

$b=\frac{r_{\mathrm{planet}}}{R_{\star}}\ \mathrm{cos}(i).$ (18)

The impact parameter for an observer $O^{\prime}_{i}(f,\alpha)$, is calculated
by identifying $i=\alpha$. This condition is satisfied by all observers that
are inside the TSB, for all planets, for all systems, in all three
populations.

The above procedure produces the KOBE-Shadows catalogue, which consists of
KOBE systems containing at least one transiting planet. Although all of the
planets in this catalogue will transit, not all of them will be detected.
KOBE-Transits takes care of this problem.

###  C.2 KOBE-Transits

KOBE-Transits examines the transit signal for all transiting planets found by
KOBE-Shadows. To be detected a transiting planet has to produce a signal that
is strong enough to be detected by observers. To check this, KOBE-Transits
calculates the transit S/N. An estimate of the noise as seen by an observer is
required for this.

In KOBE-Transits noise for a KOBE System is sampled from the distribution of
rms combined differential photometric precision (CDPP). Produced by the Kepler
pipeline (DR25) for individual target stars, CDPP is an empirical measure of
the stellar photometric noise (Christiansen et al. 2012)292929This data is
available in tabular format from the NASA Exoplanet Archive.. Three cuts are
placed on this distribution to ensure that FGK solar-type stars are sampled.
These are on mass $M_{\star}\in[0.7,1.3]M_{\odot}$, on radius
$R_{\star}\leq$5\text{\,}\mathrm{R}_{\odot}$$, and on stellar temperature
$T_{\star}\in[3880,7200]$\mathrm{K}$$ (Pecaut & Mamajek 2013).

The amount of stellar flux blocked by a planet is proportional to its area.
The transit signal generated by a single transit is
$\delta=(R_{\mathrm{planet}}^{2}/R_{\star}^{2})$. The single transit S/N is

$\begin{split}S/N&=\frac{\delta}{\mathrm{CDPP}_{\mathrm{eff}}},\hskip
10.00002pt\text{single transit}\\\
\mathrm{CDPP}_{\mathrm{eff}}&=\mathrm{CDPP}_{t_{\mathrm{trial}}}\Bigg{(}\frac{t_{\mathrm{trial}}}{t_{\mathrm{dur}}}\Bigg{)}^{\frac{1}{2}}.\end{split}$
(19)

Here, $\mathrm{CDPP}_{\mathrm{eff}}$ is the effective stellar noise seen by an
observer during a transit of duration $t_{\mathrm{dur}}$.
$\mathrm{CDPP}_{t_{\mathrm{trial}}}$ is the rms CDPP calculated by the Kepler
pipeline for different trials of transit durations (varying from
$1.5\text{\,}\mathrm{h}15\text{\,}\mathrm{h}$). Following W18, here
$t_{\mathrm{trial}}=$6\text{\,}\mathrm{h}$$. Figure 16 (left) shows the
distribution of rms $\mathrm{CDPP}_{$6\text{\,}\mathrm{h}$}$ after placing the
above cuts. To enhance the independent treatment of KOBE systems, a noise
value is drawn randomly from the rms $\mathrm{CDPP}_{$6\text{\,}\mathrm{h}$}$
distribution for every star in the KOBE-Shadows catalogue.

The transit duration for a planet is estimated by the time taken by a planet
to cross the stellar disk. Following W18, circular orbits and $b=0$ are
assumed. This gives

$t_{\mathrm{dur}}=\frac{2R_{\star}}{\big{(}\frac{2\pi
a}{P}\big{)}}=\frac{R_{\star}P}{\pi a}.$ (20)

When a planet’s transit is observed $n_{\mathrm{tra}}$ times, the multi-
transit S/N improves by a factor of $\sqrt{n_{\mathrm{tra}}}$. For a planet
with period $P$ and transit survey of duration $t_{\mathrm{survey}}$, the
average number of transits can be estimated as
$n_{\mathrm{tra}}=t_{\mathrm{survey}}/P$. Then the multi-transit S/N generated
by a transiting planet for
$t_{\mathrm{survey}}=t_{\mathrm{kepler}}=$3.5\text{\,}\mathrm{yr}$$ is

$S/N=\underbrace{\bigg{(}\frac{R_{\mathrm{planet}}}{R_{\star}}\bigg{)}^{2}}_{=\delta}\
\underbrace{\bigg{(}\frac{R_{\star}}{a}\bigg{)}^{\frac{1}{2}}}_{=P_{\mathrm{tra}}}\
\bigg{[}\bigg{(}\frac{t_{\mathrm{kepler}}}{\pi\
t_{\mathrm{trial}}}\bigg{)}^{\frac{1}{2}}\
\frac{1}{\mathrm{CDPP}_{t_{\mathrm{trial}}}}\bigg{]}.$ (21)

This equation shows that the transit $S/N$ is given by the transit signal
generated by a planet, multiplied by a factor $\sqrt{\frac{R_{\star}}{a}}$,
scaled with instrument- and survey-related constants. Thus, a large planet
closely orbiting a small quiet star will produce a high $S/N$. These are some
of the detection biases of the transit method.

In KOBE-Transits transiting planets that have $S/N\geq 7.1$ and
$n_{\mathrm{tra}}\geq 2$ constitute the KOBE-periodic threshold crossing event
(pTCE) catalogue. For the Kepler pipeline, the threshold for detection of a
TCE was multiple event statistic (MES, analogous to multi-transit S/N) $\geq
7.1\sigma$, and $n_{\mathrm{tra}}\geq 3$ (Twicken et al. 2016, 2018;
Christiansen et al. 2012; Thompson et al. 2018). Following W18, the minimum
$n_{\mathrm{tra}}$ is fixed to 2.

###  C.3 KOBE-Vetter

In Thompson et al. (2018), the DR25 TCE are further reviewed by an automatic
program, the Robovetter. The Robovetter examines several metrics, and
identifies consistent TCEs as Kepler objects of interest (KOIs). Through
further analysis, the Robovetter eventually vets KOIs as either planet
candidates or false positives. However, not all candidates are true exoplanets
and some false positives may have been genuine signals.

The catalogue produced by the Robovetter was characterized for completeness
and reliability. Completeness measures the fraction of true planets that are
absent in the catalogue, and reliability measures the fraction of planetary
candidates that are truly exoplanets (Coughlin 2017). These measurements are
done by injecting simulated transit signals in the Robovetter. The Robovetter
completeness is given by the fraction of injected transit signals that are
characterized as planetary candidates. The confidence of vetting a TCE as a
planetary candidate is expressed with a disposition score (ranging from $01$,
a higher value implying higher confidence that a TCE is a candidate).
Selecting only high scoring candidates produces a catalogue that has less
completeness, but is highly reliable.303030Thompson et al. (2018) suggested
using a cut on disposition score for occurrence rate studies. This approach
has been used by Mulders et al. (2018) and Hsu et al. (2018), and extensively
studied by Bryson et al. (2020).

KOBE-Vetter calculates the Robovetter completeness (for disposition score
$\geq 0.9$) using the results of these injection tests (Coughlin
2017)313131The Robovetter Disposition Table is available from the NASA
Exoplanet Archive.. Completeness is calculated as a function of planetary
radius (bin size $2\text{\,}\mathrm{R}_{\oplus}$) and period (bin size
$50\text{\,}\mathrm{d}$). Figure 16 (right) shows a 2D histogram of
completeness (including reliability). These values are applied to the pTCE
catalogue in a straightforward manner. If a 2D bin has completeness of $C\%$,
then for all planets in the pTCE catalogue that fall in this bin $C\%$ are
randomly vetted as candidates and make the planetary candidates catalogue. The
rest are rejected as false positives. For example, if there are 100 planets
with $R_{\mathrm{planet}}\in[0,2)R_{\oplus}$ and
$P\in[300,350)\leavevmode\nobreak\ $\mathrm{d}$$ in the pTCE catalogue, then
33 planets will be randomly marked as candidates and the remaining 66 planets
are vetted as false positives.