# The Impact of Partial Occlusion on Pedestrian Detectability

Shane Gilroy1,2, Darragh Mullins1, Edward Jones1, Ashkan Parsi1 and Martin
Glavin1
1National University of Ireland, Galway, Ireland
2Atlantic Technological University, Ireland

###### Abstract

Robust detection of vulnerable road users is a safety critical requirement for
the deployment of autonomous vehicles in heterogeneous traffic. One of the
most complex outstanding challenges is that of partial occlusion where a
target object is only partially available to the sensor due to obstruction by
another foreground object. A number of leading pedestrian detection benchmarks
provide annotation for partial occlusion, however each benchmark varies
greatly in their definition of the occurrence and severity of occlusion.
Recent research demonstrates that a high degree of subjectivity is used to
classify occlusion level in these cases and occlusion is typically categorized
into 2-3 broad categories such as “partially” and “heavily” occluded. This can
lead to inaccurate or inconsistent reporting of pedestrian detection model
performance depending on which benchmark is used. This research introduces a
novel, objective benchmark for partially occluded pedestrian detection to
facilitate the objective characterization of pedestrian detection models.
Characterization is carried out on seven popular pedestrian detection models
for a range of occlusion levels from 0-99%. Results demonstrate that
pedestrian detection performance degrades, and the number of false negative
detections increase as pedestrian occlusion level increases. Of the seven
popular pedestrian detection routines characterized, CenterNet has the
greatest overall performance, followed by SSDlite. RetinaNet has the lowest
overall detection performance across the range of occlusion levels.

##  1 Introduction

Accurate and robust pedestrian detection systems are an essential requirement
for the safe navigation of autonomous vehicles in heterogeneous traffic. The
SAE J3016 standard [1] defines levels of driving automation ranging from Level
0, where the vehicle contains zero automation and the human driver is in
complete control, to level 5 where the vehicle is solely responsible for all
perception and driving tasks in all scenarios. The progression from automation
levels 3-5 requires a significant increase in assumption of responsibility by
the vehicle, placing progressively increasing demands on the performance of
pedestrian detection systems to inform efficient path planning and to ensure
the safety of vulnerable road users. Despite recent improvements in pedestrian
detection systems, many challenges still exist before we reach the object
detection capabilities required for safe autonomous driving. One of the most
complex and persistent challenges is that of partial occlusion, where a target
object is only partially available to the sensor due to obstruction by another
foreground object. The frequency and variety of occlusion types in the
automotive environment is large and diverse as pedestrians navigate between
vehicles, buildings, traffic infrastructure, other road users. Pedestrians can
be occluded by static or dynamic objects, may inter-occlude (occlude one
another) such as in crowds, and self-occlude - where parts of a pedestrian
overlap. Leading pedestrian detection systems claim a detection performance of
approximately 65%-75% of partially and heavily occluded pedestrians
respectively using current benchmarks [2][3][4][5]. However, recent research
[6] demonstrates that the definition of the occurrence and severity of
occlusion varies greatly, and a high degree of subjectivity is used to
categorize pedestrian occlusion level in each benchmark. In addition,
occlusion is typically split into 2-3 broad, loosely defined, categories such
as “partially” or “heavily” occluded [7][8][9]. A knowledge gap exists for
objective, detailed occlusion level analysis for pedestrian detection across
the complete spectrum of occlusion levels. Use of an objective, fine grained
occlusion specific benchmark will result in more objective, consistent and
detailed analysis of pedestrian detection algorithms for partially occluded
pedestrians.

This research uses a novel, objective benchmark for partially occluded
pedestrian detection to facilitate the objective characterization of
pedestrian detection models. Objective characterization of occluded pedestrian
detection performance is carried out for seven popular pedestrian detection
routines for a range of occlusion levels from 0-99%. The contributions of this
research are: 1\. A novel, objective, test benchmark for partially occluded
pedestrian detection is presented. 2\. Objective characterization of
pedestrian detection performance is carried out for seven popular pedestrian
detection routines.

Table 1: Overview of Pedestrian Detection Models. Model | Classifier | Training Data | Weights Source | Performance (mAP)
---|---|---|---|---
FasterRCNN [10] | ResNet-50 FPN | COCO | Voxel51[11] | 0.398
MaskRCNN [12] | ResNet-50 FPN | COCO | Voxel51[11] | 0.411
R-FCN [13] | ResNet-101 | COCO | Voxel51[11] | 0.411
SSD [14] | VGG16 | COCO | Torchvision[15] | 0.412
SSDlite [16] [17] | MobileNetV3 Large | COCO | Torchvision[15] | 0.464
RetinaNet [18] | ResNet-50 FPN | COCO | Voxel51[11] | 0.361
CenterNet [19] | Hourglass-104 | COCO | Voxel51[11] | 0.533
Figure 1: Dataset Statistics. The number of pedestrian instances per occlusion
level. The custom dataset contains 820 pedestrian instances under progressive
levels of occlusion from 0-99%. Figure 2: Dataset Sample. An example of
dataset images for each level of occlusion. The custom dataset contains 820
pedestrian instances containing a wide range of pedestrian poses and occluding
objects. All images are compiled from publicly available sources.

##  2 Related Work

A number of popular pedestrian detection benchmarks provide annotation of
pedestrian occlusion level to determine the relative detection performance for
partially occluded pedestrians. Dollar et al [20] provides analysis on
occluded pedestrians based on the Caltech Pedestrian Dataset [21]. Caltech
Pedestrian estimates the occlusion ratio of pedestrians by annotating 2
bounding boxes, one for the visible pedestrian area and one for the annotators
estimate of the total pedestrian area. Pedestrians are categorised into 2
occlusion categories, “partially occluded”, defined as 1-35% occluded and
“heavily occluded”, defined as 35-80% occluded. Any pedestrians suspected to
be more that 80% occluded are labelled as fully occluded. Analysis of the
frequency of occlusion on the Caltech Pedestrian Dataset demonstrated that
over 70% of pedestrians were occluded in at least one frame, highlighting the
frequency of occurrence of pedestrian occlusion in the automotive environment.
The Eurocity Persons [9] Dataset categorizes pedestrians according to three
occlusion levels: low occlusion (10%-40%), moderate occlusion (40%-80%), and
strong occlusion (larger than 80%). Classification is carried out by human
annotators in a similar manner to the Caltech Pedestrian Dataset. The full
extent of the occluded pedestrian is estimated, and the approximate level of
occlusion is then estimated to be within one of the three defined categories.
Citypersons [8] calculate occlusion levels by drawing a line from the top of
the head to the middle of the two feet of the occluded pedestrian. Human
annotators are required to estimate the location of the head and feet if these
are not visible. A bounding box is then generated for the estimated full
pedestrian area using a fixed aspect ratio of 0.41(width/height). This is then
compared to the visible area bounding box to denote occlusion level. These
estimates of occlusion level are then categorized into two levels,
“reasonable” ($<$=35% occluded) and “heavy occlusion” (35%-75%). Similar
approaches are taken in [22][23][24][25] [26]. The Kitti Vision Benchmark [7]
and Multispectral Pedestrian Dataset [27] tasked human annotators with marking
each pedestrian bounding box as “visible”, “semi-occluded”, “fully-occluded”.
Although these methods are useful for the relative comparison of detection
performance on specific datasets, the occlusion categories used are broad
(usually 2 to 3 categories), are inconsistent from benchmark to benchmark, and
involve a high degree of subjectivity by the human annotator [6]. A knowledge
gap exists for a detailed, objective benchmark to compare pedestrian detection
performance for partially occluded pedestrians. Many pedestrian detection
analysis papers [20][22][28][29][30][31][32][33][34][35] and occlusion-
specific survey papers [2][3][5][36][37] highlight the outstanding challenges
posed by occluded pedestrians, however, no known objective characterization of
pedestrian detection performance spanning the spectrum of occlusion levels has
been carried out to date.

Gilroy et al [38] describes an objective method of occlusion level annotation
and visible body surface area estimation of partially occluded pedestrians.
Keypoint detection is applied to identify semantic body parts and findings are
cross-referenced with a visibility score and the pedestrian mask in order to
confirm the presence or occlusion of each semantic part. A novel method of 2D
body surface area estimation based on the ”Wallace rule of Nines” [6][39] is
then used to quantify the total occlusion level of pedestrians.

(a)

(b)

Figure 3: Detection Performance by Occlusion Level. Pedestrian detection
performance of seven popular pedestrian detection models is displayed for
images containing progressive levels of occlusion. Pedestrian detection
performance declines in a linear manner as the level of pedestrian occlusion
is increased, (a). CenterNet[19] is the highest performing detection model for
pedestrians up to 80% occluded, (b). Figure 4: Overall Detection Performance.
CenterNet has the greatest overall performance with a mAP of 0.533, followed
by SSDlite (mAP = 0.464). RetinaNet has the lowest overall performance on the
test dataset with a mAP of 0.361.

(a)

(b)

(c)

(d)

(e)

Figure 5: True Positives, False Positives and False Negatives. (a) displays
the percentage of true positive detections by occlusion level for seven
popular pedestrian detection models. (b), (c) and (d) display the number of
false positives per occlusion level for each model. Note the differences in
the Y-axis for each graph. (c) displays the number of false negatives by
occlusion level. Results demonstrate that the number of false negative
detections increase as the pedestrian occlusion level increases, and the
percentage of true positive detections reduce significantly for pedestrians
over 50% occluded.

##  3 Methodology

A novel occluded pedestrian test dataset, containing 820 person instances in
724 images, has been created in order to characterize pedestrian detection
performance across a range of occlusion levels from 0 to 99% occluded. A
diverse mix of images are used ensure that a wide variety of target
pedestrians, pedestrian poses, backgrounds, and occluding objects are
represented. The dataset is sourced from three main categories of images: 1)
The ”occluded body” subset of the partial re-identification dataset ”Partial
ReID” provided by Zheng et al [40], 2) The Partial ReID ”whole body” subset
[40] with custom superimposed occlusions and 3) Images collated from publicly
available sources including [6][8][9][41]. All images are annotated using the
objective occlusion level classification method described in [38]. Complex
cases at very high occlusion rates were manually verified using the method of
2D body surface area estimation presented in [6]. Each occlusion level
contains a minimum of 55 pedestrian instances. Dataset statistics by occlusion
level and a sample of the test dataset can be seen in Figure 1 and Figure 2
respectively.

###  3.1 Pedestrian Detection Models

Performance characterization was carried out on seven popular pedestrian
detection models. All models use publicly available pretrained weights from
two popular model zoos [15][11] and are trained using the COCO “train 2017”
dataset [42]. An overview of the pedestrian detection models can be seen in
Table 1. The pedestrian detection models chosen for characterization can be
divided into 3 categories: Two-Stage Frameworks, One-Stage Frameworks and
Keypoint Estimation. Two-stage frameworks such as FasterRCNN[10], MaskRCNN[12]
and R-FCN[13] apply two separate networks to perform classification. One
network is used to propose regions of interest and a dedicated second network
performs object detection [43]. One-stage frameworks such as RetinaNet[18],
SSD[14] and SSDLite[16][17] attempt to reduce computation and increase speed
by performing object detection using a single feed forward convolutional
network that does not interact with a region proposal module. RetinaNet also
implements a novel method of “focal loss” which is used to reduce the
imbalance between foreground and background classes during training with a
view to increasing detection precision. CenterNet[19] takes an alternative
approach based on keypoint estimation. Objects are represented as a single
point at their bounding box center identified by a heat map generated using a
fully convolutional network. Other object features such as object size,
orientation and pose are then regressed directly from the image features at
the center location.

###  3.2 Experiments

Detection performance is analyzed for the complete test dataset, and for each
occlusion range from 0-9% to 90-99%, for pedestrian detection models to assess
the impact of progressive levels of occlusion on the detectability of
pedestrians. Experiments are carried out with an intersection over union (IoU)
of 0.5. Analysis is carried out using Voxel51[44] and COCO style evaluation
metrics. A summary of the results are shown in Figure 3, Figure 4 and Figure
5.

##  4 Results and Analysis

Results demonstrate that pedestrian detection performance (mAP) declines as
the level of pedestrian occlusion increases, Figure 3(a). The number of false
negative detections increase as occlusion level increases, Figure 5(e) and in
general, the number of true positive detections begin to significantly
decrease as occlusion level increases for pedestrians more than 50% occluded,
Figure 5(a). As shown in Figure 4, of the seven popular pedestrian detection
models analyzed, CenterNet [19] has the greatest overall detection performance
for partially occluded pedestrians with an overall mAP of 0.533, followed by
SSDLite [16][17] with a total dataset mAP of 0.464. MaskRCNN [12] has the
greatest detection performance for pedestrians occluded more than 80%, Figure
4(b). RetinaNet [18] is the lowest performing overall on the test data with a
mAP of 0.361. RetinaNet’s true positive detections begin to degrade in a
linear fashion once pedestrians are more than 30% occluded and this model has
the highest number of false negatives for pedestrians more than 30% occluded,
Figure 5(a) and 5(e). Single Shot Detectors, SSD [14] and SSDLite [16][17]
have the highest number of true positive detections at high levels of
occlusion, Figure 5(a), and maintain a very high level of true positive
detections up to 60% occlusion, however their false positive rate is in the
region of 100 times larger than popular two stage detectors such as FasterRCNN
[10] and RFCN [13], Figure 5(b), and approximately 16 times larger than
MaskRCNN [12], Figure 5(c). Unlike false negatives, the number of false
positives per image does not appear to be significantly impacted by the
occlusion level as these are not typically related to the target pedestrian in
an image. SSDlite [16][17] outperforms SSD [14] for almost all levels of
occlusion despite having a higher number of false positive detections.
MaskRCNN [12] has a higher percentage of true positives than Faster RCNN [10]
for pedestrians over 40% occluded, however, it has around 4 times more false
positive detections for the same data, Figure 5. Mask RCNN, RFCN and SSD all
have similar overall performance on the test dataset, however, MaskRCNN and
RFCN have a higher detection performance than SSD for pedestrians that are
more than 60% occluded, Figure 3(b).

Figure 6 compares the output from a two stage detector, FasterRCNN, with a one
stage detector, SSD, for an occluded pedestrian. Two stage detectors first
generate key regions of interest before applying object detection, one stage
detectors directly apply object detection to the entire image. Figure 6
demonstrates that for the same image, FasterRCNN produces 4 detection outputs
(1 true positive with 88% confidence and 3 false positives), Figures 6(b) and
6(c), whereas SSD produces 84 detection outputs (1 true positive with 20%
confidence and 83 false positives), Figures 6(d) and 6(e). This indicates that
all detection models must not be treated equally in the design of a pedestrian
detection system. The characteristics of the detection model used must be
taken into account further downstream in the object detection system, as some
model outputs may be less reliable than others for safety critical systems.

Figure 6: FasterRCNN vs. SSD. Detection performance is compared for a two
stage network, FasterRCNN vs. a one stage network, SSD for an occluded
pedestrian. The ground truth is shown in (a). FasterRCNN generates 4 proposals
(b), 1 true positive detection with 88% confidence (c), and 3 false positives.
SSD generates 84 detections (d), 1 true positive with 22% confidence (e), and
83 false positives.

###  4.1 Key Semantic Parts

Further analysis has been carried out to determine the impact that the
visibility of a pedestrian’s head has on the detection of occluded
pedestrians. The dataset was split into two subsets: 1) Only images where the
target pedestrian’s head is visible and 2) Only images where the target
pedestrian’s head is occluded. Of the 820 pedestrian instances, the target
pedestrian’s head is visible in 582 instances and is occluded in 252
instances. Figure 7(a) displays the percentage of pedestrian instances with
their head visible across each of the occlusion levels. Three pedestrian
detection models, FasterRCNN, RetinaNet and SSD were then tested on both data
subsets across the occlusion range. This experiment demonstrates that,
regardless of whether a pedestrians head is visible, a distinct declining
profile in detection performance is observed as pedestrian occlusion level
increases, Figures 7(b), 7(c) and 7(d). This indicates that the detection
models under test are not overly reliant on head visibility for the
classification of partially occluded pedestrians.

(a)

(b)

(c)

(d)

Figure 7: Analysis of data based on head visibility. (a) Dataset statistics
based on head visibility. 820 total instances, 568 instances with head
visible, 252 instances with head occluded. (b) Detection performance for
FasterRCNN, SSD and RetinaNet for all data. (c) Detection performance for only
images with head visible. Note, no occlusion level of more than 90% possible
with head visible. (d) Detection performance for only images with head
occluded. This analysis demonstrates that, in general, detection performance
degrades as occlusion level increases regardless of the presence of key
semantic features such as a pedestrian’s head.

##  5 Conclusion

The detection of partially occluded pedestrians remains a persistent challenge
for driver assistance systems and autonomous vehicles. Current methods of
characterizing detection performance for partially occluded pedestrians have
been broad, subjective, and inconsistent in their definition of the severity
of occlusion. This research presents a novel test benchmark for the detailed,
objective analysis of pedestrian detection models for partially occluded
pedestrians. Detection performance is characterized for seven popular
pedestrian detection models across a range of occlusion levels from 0-99%.
Results demonstrate that detection performance experiences a linear decline as
the occlusion level increases and the visibility of a pedestrian is reduced.
An increase in the number of false negative detections is observed as
occlusion level increases and the percentage of true positive detections
significantly degrade for pedestrians who are more than 50% occluded. Further
analysis demonstrates that not all pedestrian detection models should be
treated equally within an object detection system. The speed vs. accuracy
trade-off, encouraged by the near real-time requirements of autonomous
vehicles, can result in high levels of false positive detections and lower
detection confidence at progressive levels of pedestrian occlusion,
particularly when using single stage detection models. Thorough objective
characterization of pedestrian detection models at the design stage will
improve the performance of object detection systems by calibrating the
priority of detections in scenarios where known weaknesses can occur. System
improvements may be gained through the use of an occlusion-aware step in the
object detection pipeline to inform the priority of camera-based detections in
sensor fusion networks for SAE level 4 and level 5 autonomous vehicles. In
this manner, any reduction in performance at high occlusion levels can be
mitigated in the design of the overall system to increase the safety of
vulnerable road users and improve the efficiency of path planning based on
environment detection. Widespread use of the proposed benchmark will result in
more objective, consistent and detailed analysis of pedestrian detection
models for partially occluded pedestrians.

## References

  * [1] S. international, “Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles,” SAE, 2018.
  * [2] S. Gilroy, E. Jones, and M. Glavin, “Overcoming occlusion in the automotive environment-a review,” IEEE Transactions on Intelligent Transportation Systems, 2019.
  * [3] C. Ning, L. Menglu, Y. Hao, S. Xueping, and L. Yunhong, “Survey of pedestrian detection with occlusion,” Complex & Intelligent Systems, vol. 7, no. 1, pp. 577–587, 2021.
  * [4] J. Cao, Y. Pang, J. Xie, F. S. Khan, and L. Shao, “From handcrafted to deep features for pedestrian detection: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  * [5] Y. Xiao, K. Zhou, G. Cui, L. Jia, Z. Fang, X. Yang, and Q. Xia, “Deep learning for occluded and multi-scale pedestrian detection: A review,” IET Image Processing, 2021.
  * [6] S. Gilroy, M. Glavin, E. Jones, and D. Mullins, “Pedestrian occlusion level classification using keypoint detection and 2d body surface area estimation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3833–3839, 2021.
  * [7] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354–3361, IEEE, 2012.
  * [8] S. Zhang, R. Benenson, and B. Schiele, “Citypersons: A diverse dataset for pedestrian detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3221, 2017.
  * [9] M. Braun, S. Krebs, F. Flohr, and D. M. Gavrila, “Eurocity persons: A novel benchmark for person detection in traffic scenes,” IEEE transactions on pattern analysis and machine intelligence, vol. 41, no. 8, pp. 1844–1861, 2019\.
  * [10] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: towards real-time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 6, pp. 1137–1149, 2016\.
  * [11] B. E. Moore and J. J. Corso, “Fiftyone model zoo,” Online. https://voxel51.com/docs/fiftyone/user_guide/model_zoo, 2022.
  * [12] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 2961–2969, 2017.
  * [13] J. Dai, Y. Li, K. He, and J. Sun, “R-fcn: Object detection via region-based fully convolutional networks,” in Advances in neural information processing systems, pp. 379–387, 2016.
  * [14] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” in European conference on computer vision, pp. 21–37, Springer, 2016.
  * [15] Pytorch, “Torchvision model zoo,” Online. https://pytorch.org/vision/stable/models.html, 2022.
  * [16] A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan, et al., “Searching for mobilenetv3,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1314–1324, 2019.
  * [17] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4510–4520, 2018.
  * [18] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision, pp. 2980–2988, 2017.
  * [19] X. Zhou, D. Wang, and P. Krähenbühl, “Objects as points,” arXiv preprint arXiv:1904.07850, 2019.
  * [20] P. Dollar, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: An evaluation of the state of the art,” IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 4, pp. 743–761, 2011.
  * [21] P. Dollár, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: A benchmark,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 304–311, IEEE, 2009.
  * [22] S. Zhang, R. Benenson, M. Omran, J. Hosang, and B. Schiele, “How far are we from solving pedestrian detection?,” in Proceedings of the iEEE conference on computer vision and pattern recognition, pp. 1259–1267, 2016.
  * [23] Y. Pang, J. Cao, Y. Li, J. Xie, H. Sun, and J. Gong, “Tju-dhd: A diverse high-resolution dataset for object detection,” IEEE Transactions on Image Processing, vol. 30, pp. 207–219, 2020.
  * [24] S. Shao, Z. Zhao, B. Li, T. Xiao, G. Yu, X. Zhang, and J. Sun, “Crowdhuman: A benchmark for detecting human in a crowd,” arXiv preprint arXiv:1805.00123, 2018.
  * [25] C. Chi, S. Zhang, J. Xing, Z. Lei, S. Z. Li, and X. Zou, “Pedhunter: Occlusion robust pedestrian detector in crowded scenes,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10639–10646, 2020.
  * [26] X. Li, L. Li, F. Flohr, J. Wang, H. Xiong, M. Bernhard, S. Pan, D. M. Gavrila, and K. Li, “A unified framework for concurrent pedestrian and cyclist detection,” IEEE transactions on intelligent transportation systems, vol. 18, no. 2, pp. 269–281, 2016.
  * [27] S. Hwang, J. Park, N. Kim, Y. Choi, and I. So Kweon, “Multispectral pedestrian detection: Benchmark dataset and baseline,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1037–1045, 2015.
  * [28] S. Walk, N. Majer, K. Schindler, and B. Schiele, “New features and insights for pedestrian detection,” in 2010 IEEE Computer society conference on computer vision and pattern recognition, pp. 1030–1037, IEEE, 2010.
  * [29] R. N. Rajaram, E. Ohn-Bar, and M. M. Trivedi, “An exploration of why and when pedestrian detection fails,” in 2015 IEEE 18th International Conference on Intelligent Transportation Systems, pp. 2335–2340, IEEE, 2015.
  * [30] J. Mao, T. Xiao, Y. Jiang, and Z. Cao, “What can help pedestrian detection?,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3127–3136, 2017.
  * [31] S. Zhang, R. Benenson, M. Omran, J. Hosang, and B. Schiele, “Towards reaching human performance in pedestrian detection,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 973–986, 2017\.
  * [32] N. Ragesh and R. Rajesh, “Pedestrian detection in automotive safety: understanding state-of-the-art,” IEEE Access, vol. 7, pp. 47864–47890, 2019.
  * [33] J. Cao, Y. Pang, J. Han, B. Gao, and X. Li, “Taking a look at small-scale pedestrians and occluded pedestrians,” IEEE transactions on image processing, vol. 29, pp. 3143–3152, 2019.
  * [34] T. Toprak, B. A. Can, M. Ozcelikors, S. B. Tekin, and M. A. Selver, “Limitations of feature-classifier strategies on pedestrian detection for self driving cars,” in 2020 International Conference on Electrical, Communication, and Computer Engineering (ICECCE), pp. 1–6, IEEE, 2020.
  * [35] I. Hasan, S. Liao, J. Li, S. U. Akram, and L. Shao, “Generalizable pedestrian detection: The elephant in the room,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11328–11337, 2021\.
  * [36] H. Chandel and S. Vatta, “Occlusion detection and handling: a review,” International Journal of Computer Applications, vol. 120, no. 10, 2015.
  * [37] K. Saleh, S. Szénási, and Z. Vámossy, “Occlusion handling in generic object detection: A review,” in 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 000477–000484, IEEE, 2021.
  * [38] S. Gilroy, M. Glavin, E. Jones, and D. Mullins, “An objective method for pedestrian occlusion level classification,” arXiv preprint arXiv:2205.05412, 2022.
  * [39] A. Wallace, “The exposure treatment of burns,” The Lancet, vol. 257, no. 6653, pp. 501–504, 1951.
  * [40] W.-S. Zheng, X. Li, T. Xiang, S. Liao, J. Lai, and S. Gong, “Partial person re-identification,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 4678–4686, 2015.
  * [41] J. Zhuo, Z. Chen, J. Lai, and G. Wang, “Occluded person re-identification,” in 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6, IEEE, 2018.
  * [42] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision, pp. 740–755, Springer, 2014.
  * [43] L. Chen, S. Lin, X. Lu, D. Cao, H. Wu, C. Guo, C. Liu, and F.-Y. Wang, “Deep neural network based vehicle and pedestrian detection for autonomous driving: A survey,” IEEE Transactions on Intelligent Transportation Systems, 2021\.
  * [44] B. E. Moore and J. J. Corso, “Fiftyone,” GitHub. Note: https://github.com/voxel51/fiftyone, 2020.