# Multiple Object Detection and Tracking in Panoramic Videos for Cycling
Safety Analysis

Jingwei Guo  Department of Civil, Environmental, and Geomatic Engineering,
University College London (UCL), UK Meihui Wang  SpaceTimeLab, Department of
Civil, Environmental, and Geomatic Engineering, University College London
(UCL), UK  0000-0001-8420-2141 Ilya Ilyankou  SpaceTimeLab, Department of
Civil, Environmental, and Geomatic Engineering, University College London
(UCL), UK  0009-0008-7082-7122 Natchapon Jongwiriyanurak  SpaceTimeLab,
Department of Civil, Environmental, and Geomatic Engineering, University
College London (UCL), UK  0000-0002-6188-8432 Xiaowei Gao  SpaceTimeLab,
Department of Civil, Environmental, and Geomatic Engineering, University
College London (UCL), UK  0000-0003-3273-7499 Nicola Christie  Centre for
Transport Studies, Department of Civil, Environmental, and Geomatic
Engineering, University College London (UCL), UK  0000-0001-7152-5240
James Haworth  SpaceTimeLab, Department of Civil, Environmental, and Geomatic
Engineering, University College London (UCL), UK  Corresponding author:
<EMAIL_ADDRESS>0000-0001-9506-4266

###### Abstract

Panoramic cycling videos can record 360° views around the cyclists. Thus, it
is essential to conduct automatic road user analysis on them using computer
vision models to provide data for studies on cycling safety. However, the
features of panoramic data such as severe distortions, large number of small
objects and boundary continuity have brought great challenges to the existing
CV models, including poor performance and evaluation methods that are no
longer applicable. In addition, due to the lack of data with annotations, it
is not easy to re-train the models.

In response to these problems, the project proposed and implemented a three-
step methodology: (1) improve the prediction performance of the pre-trained
object detection models on panoramic data by projecting the original image
into 4 perspective sub-images; (2) introduce supports for boundary continuity
and category information into DeepSORT, a commonly used multiple object
tracking model, and set an improved detection model as its detector; (3) using
the tracking results, develop an application for detecting the overtaking
behaviour of the surrounding vehicles.

Evaluated on the panoramic cycling dataset built by the project, the proposed
methodology improves the average precision of YOLO v5m6 and Faster RCNN-FPN
under any input resolution setting. In addition, it raises MOTA and IDF1 of
DeepSORT by 7.6% and 9.7% respectively. When detecting the overtakes in the
test videos, it achieves the F-score of 0.88.

The code is available on GitHub at github.com/cuppp1998/360_object_tracking to
ensure the reproducibility and further improvements of results.

## 1\. Introduction

Cycling, as an active travel mode, has zero carbon emissions ([83]) and can
support reductions in traffic congestion ([59]) while improving the health of
urban residents ([101]). Therefore, many local governments and transport
authorities are implementing policies to encourage cycling as an alternative
to car travel. However, cycling is often perceived as a dangerous activity,
particularly in cities that lack high quality cycling infrastructure. In
London, cycle lanes account for only 6.4% of the total road length (excluding
highways) ([96]), meaning that cyclists often have to share lanes with motor
vehicles. According to Transport for London (TfL), in 2020, despite the mode
share of cyclists being only 3.4%, there were 868 killed or seriously injured
(KSI) incidents involving cyclists, accounting for 28% of all such casualties
in the city (Transport for London, 2021). This disproportionately high risk of
incidents can reduce people’s willingness to ride ([62]), which hinders the
uptake of cycling. Thus, it is necessary to take measures to improve urban
cycling safety, which may include improving road infrastructure ([90]),
implementing policies such as lower speed limits, or running public awareness
campaigns like Operation Close Pass ([102]). However, in order to gain
support, these improvements must be accompanied by evidence on their
effectiveness. One aspect of this is the impact on cycling safety.

Although incident data are available through hospital reports and official
statistical releases, such as the UK’s STATS19, the low mode share of cycling
makes it difficult to draw statistically significant conclusions at a
spatially and temporally granular level. Near-miss events, in contrast, are
more frequent and have provided a rich source of data for researchers in the
form of naturalistic studies ([70]). These studies have used a range of
sensors to collect data while people carry out routine activities on bikes,
with the rider indicating when they experience a near-miss. Alternatively,
sensors are used to collect objective data on a particular near-miss type,
such as a close pass ([57]). To date these studies have been limited in the
perspective of the near-miss that they capture. In recent years, with the
rapid development of street view imagery, action/dash cameras, virtual reality
(VR) and related technologies, panoramic photography is becoming widespread
and mature ([71]). Unlike traditional cameras, which can only capture content
within a limited field of view, a panoramic camera can capture 180-degree
views of the front and rear of the device at the same time with two fisheye
lenses, and then concatenate them into one high-resolution 360° image ([68]).
These consumer grade cameras are suitable to be mounted on helmets or
handlebars to record videos of the riders’ surroundings in all directions.
Importantly, the 360 view captures the relative movement of the rider and the
other people recorded in the scene. For example, in an overtaking manoeuvre, a
vehicle will first appear in the rear-view of the camera, before passing into
the side view (right or left depending on locality) and into the front view.
Computer vision based object detection and tracking technologies have the
potential to track these movements and categorise them into actions based on
interpretations of the relative movements.

The emergence of deep learning has had a transformative effect on a range of
machine learning tasks, especially computer vision ([75]). A range of
algorithms have been developed for implementing various visual tasks such as
object detection ([107]) and multiple object tracking (MOT) ([81]). However,
since most of these models and the attached pre-trained weights were designed
and trained for traditional images and videos, if applied directly to
panoramic videos in equirectangular projection, their performance will be
negatively affected ([104]). In addition, some models’ evaluation metrics are
not applicable to panoramic images. These problems exist because of the
following differences between panoramic images and standard images:

  * •

The acquisition of panoramic videos requires specific cameras. As a result,
the number of available annotated panoramic datasets is much lower than that
of standard cameras;

  * •

Due to the distortions introduced by the equirectangular projection, objects
near the camera may be seriously deformed, while objects far away from the
camera will be small;

  * •

Panoramic images in equirectangular projection have boundary continuity in
which the contents of the left and right sides of the image are continuous
([80]).

To overcome these limitations, this research develops a framework for applying
deep learning based computer vision models pre-trained on ‘traditional’
datasets to the task of object detection and tracking in panoramic videos. The
methodology is applied to a case study in cycling safety analysis; the
identification of overtaking manoeuvres.

The remainder of the paper is structured as follows: Following this
Introduction, we discuss related work on cycling safety, computer vision and
its application to panoramic data in Section 2. Section 3 describes the
collection, pre-processing and annotation process of the data used in this
project, and introduces their distribution. In Section 4, we introduce the
methodology, and then present and analyse the results in Section 5. In Section
6 we discuss the successes and limitations of our work, and propose some
future work to overcome the limitations. Finally, we present our conclusion in
Section 7.

## 2\. Related work

###  2.1 Challenges in Understanding Cycling Crash Risk

Urban transport networks are complex environments in which many agents
(pedestrians, cyclists and other road users) interact with each other while
using different types of infrastructure. In combination, these actions and
interactions within the environment contribute to crash risk. Traditionally,
crash risk is assessed using historical data on crash records, such as the
UK’s STATS19, combined with measures of exposure such as traffic flows, which
are combined in a multiple regression framework to identify risk factors
([55]). Existing cycling safety studies have heavily relied on crash and
conflict data from open government resources ([67]; [61]; [106]),
questionnaires surveys ([100]; [99]), and crowdsourced data reported by
volunteer cyclists ([88]; [64]). However, these data sources are often
subjective and incomplete, making it challenging to understand the underlying
causes of hazards. Furthermore, due to the low mode share of cycling in many
countries, records on bicycle crashes may be sparse and exposure data limited
to particular locations and times, or not collected at all.

As a complement to traditional methods, researchers have turned to
naturalistic studies to observe cyclists as they carry out their routine
activities ([70]). Typically this involves the rider using a suite of sensors
that passively collect data such as video, inertial measurement unit (IMU) and
location data. Early work relied on bespoke equipment for this purpose ([74]),
which limited its applicability beyond the scope of the study. Subsequent
studies produced sensors for particular purposes, such as detecting close
passes, enabling the collection of high quality, objective data ([57]). [70]
([]Ibrahim2021) provide a review of the recent work. While bespoke equipment
allows a high level of control over experimental design, the use of consumer
grade equipment provides an opportunity for larger scale data collection. For
example, many road users routinely use dashcams or action cameras to collect
data on their journeys to protect themselves if they are involved in a crash.
Recognising this, recent work, particularly in the field of autonomous
vehicles, has leveraged dashcam video data for crash detection. [93]
([]rocky_review_2024) provide a review of the current state of the art.
Similar methods have also been applied in cycling safety analysis. For
example, using data sourced from YouTube, [69] ([]ibrahim_cyclingnet_2021)
used a deep learning approach that harnessed the optical flow of video data to
detect near misses from video streams. Harnessing this data as a form of
crowdsourcing could provide a way to continuously monitor safety from the
cyclist’s perspective, and studies such as [92] ([]rick_cycling_2021) have
made use of popular devices such as GoPro cameras.

The key enabling computer vision technologies for understanding the risk
environment from video streams are object detection and multiple object
tracking (MOT). Object detection identifies and localises objects such a
vehicles and people within a video frame. Techniques such as You Only Look
Once (YOLO) and Faster Residual Convolutional Neural Network (R-CNN) have been
employed that achieve high accuracy in detecting different road users, such as
CyDet ([82]) and MoDeCla ([86]). MOT is a task that associates multiple
objects in different frames of a video into several trajectories ([72]). MOT
technologies have been widely used in the field of transport, particularly
autonomous vehicles. Various modifications have been made to improve car
tracking in poor light conditions ([95]), continuously track road users that
are sometimes occluded in the videos ([73]) and to track objects using the
views from multiple cameras ([72]).

Existing object detection and MOT algorithms are trained on video from cameras
with a limited field of view (FOV). This suits the data that has been
collected in naturalistic cycling studies to date, which contain only front or
rear facing views. However, risks may be present that cannot be captured in a
single view. For example, rear-end crashes have proven to result in a high
fatality rate and tailgating is a risk factor (Bureau, 2006). For a
comprehensive view of the risk environment, it is necessary to record video
both in front of and behind the rider. This can be accomplished using separate
cameras, but leads to problems of synchronisation and matching of objects
between the two video streams. Furthermore, the side views of the rider are
not captured, which are important in the context of certain events such as
close passes and car-dooring. As an alternative, panoramic cameras capture
360° views around the user, allowing seamless tracking of interactions between
the rider, the environment and other road users.

Existing CV models struggle with panoramic videos due to a lack of panoramic
training data, distorted and small objects in equirectangular projections, and
issues with boundary continuity. Various studies have attempted to solve these
issues. Some have created panoramic datasets to train existing models ([63];
[105]), while others have applied techniques to improve model performance
without retraining ([104]), but these do not fully address boundary continuity
or the challenge of long objects being split across sub-windows. For MOT, the
quality of object detection significantly affects tracking performance.
DeepSORT is a commonly used tracker that has been adapted for panoramic data.
However, it faces challenges such as object ID switching caused by boundary
continuity and inaccuracies in multi-category tracking (MCT). [80]
([]liu_2018) proposed extending input images to mitigate boundary continuity
issues, yet further improvements are needed to handle objects appearing
multiple times across frame boundaries and to enhance MCT.

To address these issues, our research intends to combine and extend these
approaches by adding support for boundary continuity and long objects in
detection, introducing category information to reduce tracking errors, and
enhancing boundary continuity implementation in DeepSORT for better multi-
object tracking.

## 3\. Data

In order to test the improving effect of the proposed methods on object
detection and MOT on panoramic cycling videos, this project built an annotated
dataset so that the performance of different models can be easily compared. In
addition, for presenting and evaluating the application results of the
improved models in recognising the overtakes of the surrounding vehicles, some
other video clips containing overtakes were prepared.

###  3.1 Data collection

The researcher attached a GoPro Max camera to their helmet with a metal mount
to take panoramic videos while riding. To ensure the authenticity of the data
collected, when and where to ride was not restricted — the researcher recorded
videos as they carried out routine activities such as commuting. The only
limitation was that the data collection had to be in good lighting conditions
so that all the objects would be recognisable in the videos.

Since the dataset with annotations was built only for evaluating the proposed
models rather than training a deep neural network from scratch, there was no
requirement for a large data volume. Therefore, 3 clips were chosen from the
collected videos, which are 10 to 15 seconds in length, as shown in Table 1.

Video |  Length (s) | FPS | Frames | Description | Preview
---|---|---|---|---|---
Video 1 | 14.0 | 30 | 420 | Taken on a road |
Video 2 | 11.1 | 30 | 333 |  Taken on a bicycle path |
Video 3 | 12.0 | 30 | 360 | Taken on a road |
Table 1: Video Clips in the Annotated Dataset

To evaluate the performance of the proposed model in detecting overtaking
behaviour, 3 further clips of 1 to 2 minutes taken by the researcher were
selected. 54 overtakes were extracted from these videos, as shown in Table 2.
To mitigate bias that may be introduced by using video from a single person in
a limited range of conditions, further validation is carried out on a set of
video clips filmed by 49 different cyclists, who were participants in UCL’s
100 Cyclists Project. The participants, located in London, UK, used the GoPro
Max camera to record their activities over a period of two weeks, and were
instructed to say ‘near-miss’ when experiencing a near-miss event. This
dataset provides a wider range of riding conditions and rider characteristics
on which to test the proposed method: 50 of the videos contain a single clear
overtaking event lasting several seconds (‘positive’ examples), and 50 do not
(‘negative’). To account for diverse cycling environments, selected videos
were captured during daytime (78), night-time (17), and at dawn or dusk (5).
24 videos featured glare from the sun and artificial lighting sources, such as
lampposts and headlights. While only one video was recorded in rain, a further
6 were recorded after recent rain and are slightly obfuscated by rain drops on
the camera lens.

Video |  Length (s) | FPS | Frames |  Num. Overtakes | Preview
---|---|---|---|---|---
Video 5 | 66.0 | 30 | 1,980 | 12 |
Video 6 | 108.0 | 30 | 3,240 | 17 |
Video 7 | 91.0 | 30 | 2,730 | 25 |
Table 2: Video Clips for Evaluating the Overtaking Behaviour Detection

###  3.2 Data Pre-processing & Annotation

After collecting videos and selecting relevant clips, GoPro Player ([65]) was
used to trim the clips from the original videos, which are in .360 format. The
‘Horizon Levelling’ functionality was applied, which uses the gyroscope data
to level the videos on the horizontal axis ([66]), as Figure 1 shows. After
that, the clips were exported as 5.6k (5,368 × 2,688) equirectangular videos
in MOV format.

Figure 1: An example of the images before (left) and after (right) turning on
the ‘Horizon Levelling’ functionality

To generate labels to evaluate object detection and MOT, the videos were
annotated using the Computer Vision Annotation Tool (CVAT) ([87]) developed by
Intel. The following 7 categories of objects were chosen for labelling, which
are both relevant to cycling safety and contained in the Microsoft COCO
dataset ([79]): person, bicycle, car, motorbike, bus, truck and traffic light.
In order to output the annotations in both COCO and MOT formats, ‘tracks’ were
created to associate the same objects in different image frames.

Following the standards of MOT format, CVAT only supports labelling each
object once in each frame. Hence, objects divided into two parts due to the
boundary continuity were labelled as two separate objects. For example, in
Figure 2, the two parts of the bus were labelled as ‘BUS 6’ and ‘BUS 11’,
respectively.

Figure 2: How objects across the boundaries were annotated in CVAT

After the annotations were output in both COCO and MOT formats, a method was
designed and implemented to merge these split objects in the MOT annotation
files without violating the rules of MOT format ([84]), whose workflow is
shown below:

Figure 3: Workflow of merging the split objects in MOT annotation files

After the ID modifications, FFmpeg ([85]), an open-source command line tool,
was used to split video 1, 2 and 3 into images frame by frame with the
following line of command: ffmpeg -i video1.mp4 -r 30 -start_number 0 -f
image2 frame_%06d.PNG. These images are used for the evaluation of object
detection models with the COCO annotation files.

As shown in Table 3, after pre-processing and annotation, the dataset has
24,454 labelled objects and 224 labelled tracks in total.

Video | Objects | Tracks
---|---|---
Video 1 | 9,338 | 75
Video 2 | 5,404 | 49
Video 3 | 9,712 | 100
Total | 24,454 | 224
Table 3: Numbers of Objects and Tracks in the Dataset for Object Detection and
MOT Evaluation

###  3.3 Exploratory Data Analysis

In order to better design the methodologies for improving the applicability of
the existing object detection and MOT models, and to better understand the
corresponding evaluation results, this project conducted some exploratory
analysis of the annotated dataset.

####  3.3.1 Position Distribution

To understand where the objects of interest are located, a matrix whose size
is the same as the videos (5,368 × 2,688) is used to count how many times each
pixel is within an annotated bounding box. Then, the matrix is plotted as a
heat map, as Figure 4 shows. As the top and bottom areas of an equirectangular
image represent the sky and the ground respectively, almost all the objects in
the dataset are distributed from -50° to 70° of the y axes.

Figure 4: Position distribution of the ground truth bounding boxes in the
dataset

####  3.3.2 Category Distribution

Figure 5: Category distribution of the ground truth bounding boxes in the
dataset

For the purpose of knowing whether the seven categories of the labelled
objects are balanced, a pie chart containing the number and proportion of the
bounding boxes of each class was plotted, as shown in Figure 5. The dataset is
imbalanced — the two categories with the largest quantities (car and person)
account for about 80% of the total number, while motorbikes, trucks and buses,
which appear less frequently in the selected videos, have less than 1,000
labelled instances.

####  3.3.3 Size Distribution

Detecting small objects is a problem for object detection models due to the
small number of features they include ([77]). Thus, it is pertinent to explore
the distribution of objects of different sizes.

When classifying the bounding boxes by their area, the project did not use
thresholds set by COCO, which are 322 and 962 ([79]) — since the resolution of
the images in this test set (5,368 × 2,688) is much higher than that of COCO
(640 × 480), objects should also contain more pixels. According to the ratio
of the number of pixels on the short side (5.6:1), the thresholds were defined
as 1282 and 3842, that is to say, objects with an area less than 1282 were
classified as ‘small objects’, objects larger than 3842 were classified as
‘large objects’, while the remaining ones were classified to be ‘medium-
sized’.

Figure 6: Size distribution of the ground truth bounding boxes in the dataset

From Figure 6, it can be seen that a large proportion of the annotated
bounding boxes are small ones. Contrastingly, ‘large objects’ account for less
than 10% of the annotations. Such size distribution is consistent with the
characteristics of equirectangular panoramic videos summarised previously.

## 4\. Methodology

Figure 7: Three steps of the methodology

In order to apply computer vision models to automated road user behaviour
analysis of panoramic cycling videos, this project designed and implemented a
three-step methodology, as shown in Figure 7.

As the first step of the methodology, an approach was designed to improve the
applicability of the existing pre-trained object detection models to
equirectangular images. Then, in the second step, the improved models in step
1 are set as the detector of DeepSORT and its modified structure is modified
according to the characteristics of panoramic videos. Finally, in step 3, a
simple application was designed and developed which can automatically detect
the overtaking behaviour of the surrounding vehicles in panoramic videos using
the tracking results from step 2.

###  4.1 Improving Object Detection on Panoramic Images

The proposed method, shown in Figure 8, consists of the following four
operations: 1) Project the equirectangular image into four sub-images of
perspective projection whose distortions are less severe; 2) Implement object
detection models pre-trained on the COCO dataset to identify objects in each
of the sub-images; 3) Reproject the detected bounding boxes in the sub-images
to the original image; 4) Since some long objects (such as the blue and red
cars in Figure 8) are divided into several parts and shown in different sub-
images, their bounding boxes are merged.

Figure 8: Workflow of the proposed method for improving object detection on
panoramic images

In the following subsections, each of these steps are explained in detail.
After that, the evaluation method is introduced.

####  4.1.1 Projection Transformation from Equirectangular to Perspective

As an approach to presenting a whole sphere as a 2D image, equirectangular
projection maps longitude lines and latitude circles to equally spaced
vertical and horizontal lines, respectively ([94]). In contrast, perspective
projection only maps part of a sphere (within the FOV) from the centre to a
tangent plane ([104]). Thus, essentially, the transformation from
equirectangular to perspective is a projection from a sphere to a plane.

Figure 9: Two cases of projection from a sphere to a plane: (a) towards the
z-axis (b) towards any direction

According to Lee (2021), as shown in Figure 9 (a), assuming that the
perspective projection is along the z-axis, the corresponding point P’
$(X,Y,Z)$ on the sphere of the point P $(u,v)$ on the tangent plane can be
calculated as:

$X=\frac{u-T}{\sqrt{u^{2}+v^{2}+2T^{2}-2uT-2vT+1}}$ (1)

$Y=\frac{-v+T}{\sqrt{u^{2}+v^{2}+2T^{2}-2uT-2vT+1}}$ (2)

$Z=\frac{1}{\sqrt{u^{2}+v^{2}+2T^{2}-2uT-2vT+1}}$ (3)

Where $T$ is the tangent of the half of the field of view of the projection,
i.e., $T=tan(\frac{FOV}{2})$.

If represented in a geographic coordinate system, P’ can be also represented
as longitude $\phi$ and latitude $\theta$ ([76]):

$\phi=tan^{-1}(-\frac{Z}{X})$ (4)

$\theta=sin^{-1}(\frac{Y}{\sqrt{X^{2}+Y^{2}+Z^{2}}})$ (5)

As Figure 9 (b) shows, if the projection is not towards the z-axis, when
calculating the coordinates of point P’ $(X^{\prime},Y^{\prime},Z^{\prime})$,
a rotation equation $R$ should be applied as follows ([76]):

$\begin{bmatrix}X^{\prime}\\\ Y^{\prime}\\\ Z^{\prime}\\\
\end{bmatrix}=R\begin{bmatrix}X\\\ Y\\\ Z\\\
\end{bmatrix}=\begin{bmatrix}cos\alpha&-sin\alpha sin\theta_{c}&sin\alpha
cos\theta_{c}\\\ 0&cos\theta_{c}&sin\theta_{c}\\\
-sin\alpha&-sin\theta_{c}cos\alpha&cos\alpha
cos\theta_{c}\end{bmatrix}\begin{bmatrix}X\\\ Y\\\ Z\end{bmatrix}$ (6)

Where $\alpha=\phi_{c}+\frac{\pi}{2}$, $\phi_{c}$ is the angle between the
projection direction and plane $xoz$, while $\theta_{c}$ is the angle between
the projection direction and plane $yoz$.

In this project, the GetEquirec() function of the Perspective-and-
Equirectangular library in Python ([97]) was called to realise this projection
transformation. It provides FOV, theta and phi parameters, which correspond to
$FOV$, $\theta_{c}$ and $\phi_{c}$ in Equations (1) to (6).

To identify these parameters, this research referred to the position
distribution analysed in subsection 3.3.1: since almost all the objects in the
videos are distributed from -50° to 70° vertically, FOV and phi were set to
120° and -10°, so that only the objects within this interval would be
projected and detected in the improved models.

Although three sub-images with a 120° FOV are enough to cover an
equirectangular image horizontally, because there are still some distortions
in the perspective images, objects near the edges of the sub-images may not be
detected. To solve this problem, this paper chose to project the original
image into 4 sub-windows along the lines of 0°, 90°, -180° and -90° longitude
(i.e., theta) — since there is an overlap of 30° between each pair of adjacent
sub-images, objects within 15° from an edge in one sub-image, such as the
circled person in Figure 10, must be shown closer to the centre in another
sub-image, which results in less severe distortions.

Figure 10: Projections of an object in different sub-images: The closer it is
to the edges of a sub-image, the greater distortions it may have

####  4.1.2 Object Detection with Pre-trained Models

Although any model can be used to predict bounding boxes for the sub-images,
in order to check whether the proposed method can improve the performance of
different types of pre-trained models, this paper chose YOLO v5 and Faster
RCNN, the representatives of one-stage and two-stage models, to perform object
detection.

Both Faster RCNN and YOLO v5 provide several models with different structures,
which are designed for different speed and accuracy requirements. In this
step, Faster RCNN-FPN ([89]) and YOLOv5m6 ([98]) pre-trained on the COCO
dataset were chosen for they are moderate in both speed and accuracy.

By default, the bounding boxes predicted by Faster RCNN are output in the
following format: $(X_{min},Y_{min},X_{max},Y_{max})$, where
$(X_{min},Y_{min})$ and $(X_{max},Y_{max})$ are the coordinates of the top-
left and bottom-right corners of a box, while YOLO outputs the boxes in
$(X,Y,W,H)$ format, where (X, Y) are the coordinates of the centre point, and
$W,H$ represent the width and height of a box. Thus, to unify the formats,
detection implemented by YOLO was converted with the following equations:

$X_{min}=X-W/2$ (7)

$X_{max}=X+W/2$ (8)

$Y_{min}=Y-H/2$ (9)

$Y_{max}=Y+H/2$ (10)

####  4.1.3 Reprojection of the Detection

When doing the projection transformation, the GetEquirec() function in the
Perspective-and-Equirectangular library creates two matrices called ‘lon_map’
and ‘lat_map’ ([97]), which determine where to locate the pixels in the
perspective images. Based on this process, the coordinates of the
corresponding point on the equirectangular image of a point $(p,q)$ on a
perspective sub-image can be represented as $(m,n)$, where $m$ and $n$ are the
values of $lon\\_map$ $(p,q)$ and $lat\\_map$ $(p,q)$, respectively, as shown
in Figure 11.

As Figure 12 shows, to map all the bounding boxes which are at least 15° away
from the edges back to the original image, every point on the borders of these
boxes was reprojected with the approach above. However, since this process
bends the borders, which makes it hard to output and evaluate the predictions,
this project used minimum bounding rectangles to represent the boxes.

Figure 11: Process of reprojecting a point from perspective to equirectangular
Figure 12: Process of reprojecting a bounding box from perspective to
equirectangular

Due to the boundary continuity, the detection crossing the centre lines of the
sub-images projected along 180° (i.e., Image 2 in Figure 10) are shown twice
near the left and the right edges respectively in equirectangular views. Thus,
when dealing with such detection, as illustrated in Figure 13, instead of
directly projecting the whole box, this study first divided each of them into
two parts along the centre line, and then projected them separately, so that
the detection outputs can be displayed and evaluated correctly.

Figure 13: Process of reprojecting an object which is across the boundaries

####  4.1.4 Merging the Reprojected Bounding Boxes

As the final step of the proposed workflow, for merging the bounding boxes of
the objects that cannot be fully displayed in a single sub-image, the boxes
with at least one side coinciding with a boundary of a sub-image were picked
out. According to Figure 14, first, such boxes that are entirely within the
30-degree-wide overlapped areas were removed, as they must be included in
another bounding box with smaller distortions. Then, for the boxes of the same
category which are tangent to consecutive boundaries, the project combined
them into their MBRs, while the score of each new bounding box was calculated
as the area-weighted average.

Figure 14: Method for processing the objects which are displayed in several
sub-images

####  4.1.5 Model Evaluation

For the evaluation of the improving effect of the proposed method, after
designing and implementing each step of the workflow, using the methods in the
COCOEvaluator class of Detectron2 ([89]), this paper tested the pre-trained
Faster RCNN-FPN and YOLO v5m6 before and after the method was applied on the
dataset built in Chapter 3 under different input resolution settings (640,
1280, 1920). These models’ mean average precision (mAPs) metrics and average
precision (APs) metrics of different sizes were recorded and compared, as
higher APs indicate better object detection performance.

In addition, to prove the effect of merging the bounding boxes, which is an
innovation compared to the method in [104] ([]yang2018object) ’s study, this
project also compared the models with and without combining the bounding boxes
in the same way.

###  4.2 Improving Multiple Object Tracking on Panoramic Cycling Videos

Figure 15: Workflow of DeepSORT ([56]) and the proposed method for improving
its performance on panoramic cycling videos

The second step of the proposed methodology is to improve the applicability of
DeepSORT ([103]), the most commonly-used MOT model, to panoramic cycling
videos. As shown in Figure 15, the following modifications were done: First,
one of the enhanced object detection models was set as the detector; Then,
supports for category information and boundary continuity were introduced into
the process of data association, according to the characteristics of panoramic
videos.

####  4.2.1 DeepSORT

According to Figure 15, tracks in DeepSORT ([56]) have two states: ‘confirmed’
and ‘unconfirmed’ — only if a track gets matched in 3 consecutive frames will
its state be switched to confirmed. For each frame in a video, DeepSORT sends
$n$ detections and $m$ existing confirmed tracks into a matching cascade
module. The module extracts features from each detection pair and tracks with
a pre-trained ReID model and calculates the cosine and Mahalanobis distances
between them, whose weighted averages are then used to form an $m\times n$
cost matrix. After that, the Hungarian algorithm is applied to associate the
tracks and detection according to the values in the cost matrix, and the
matched tracks are then updated by Kalman filters using the information of the
corresponding detection.

For the tracks and detections that are unmatched in the matching cascade
module, DeepSORT creates another cost matrix for them according to their IOUs,
and the Hungarian algorithm is used again to do the data association. After
that, for each unmatched detection, DeepSORT creates new tracks for them,
treating them as objects new to the video; while for the unmatched tracks, if
they are ‘unconfirmed’ or have not been matched for more than a certain number
of frames, DeepSORT removes them from the track list.

####  4.2.2 Applying the Improved Object Detection Models

For testing on the data that are annotated in COCO format, the object
detection models improved in subsection 4.1 output the predictions in the
following format: $(X_{min}$, $Y_{min}$, $X_{max}$, $Y_{max})$. Since DeepSORT
only supports inputs in $(X,Y,W,H)$ format, before being input into DeepSORT,
each prediction from the detector was converted with the following equations:

$X=(X_{min}+X_{max})/2$ (11)

$Y=(Y_{min}+Y_{max})/2$ (12)

$W=X_{max}-X_{min}$ (13)

$H=Y_{max}-Y_{min}$ (14)

Where $X_{min},X_{max},Y_{min},Y_{max},W,H$ have the same definition as
previously.

####  4.2.3 Introducing the Support for Category Information

As mentioned above, when associating the detections with tracks using cosine,
Mahalanobis and IOU distances, DeepSORT ignores the category information of
the detection, treating objects of different categories equally. Thus, tracks
and detections can be wrongly matched when their corresponding values in the
cost matrices are small. In fact, if applying the original model to panoramic
cycling videos, such wrong associations can be seen when objects of different
categories alternate at a short distance. As shown in Figure 16, since the
bike in the right image appears not far away from where person 89 left the
video, the track representing the person is wrongly matched with the bike.

Figure 16: An example of wrong associations between tracks and detections

In response to the problem, a new attribute called ‘_class’ was added to the
Track class of DeepSORT. Detection categories were also input into DeepSORT.
In that way, when a track is initiated, its ‘_class’ attribute will be set as
the category of the corresponding detection.

Figure 17: Process of category filtering on a cost matrix

In addition, whenever the cost matrices were formed, as shown in Figure 17,
category filtering was performed to set all the values representing the
distances between objects and detections of different classes to $10^{5}$, so
that the existing tracks can only get matched with the detections of the same
categories.

####  4.2.4 Introducing the Support for Boundary Continuity

To track the detections which are divided into two parts (i.e., objects
crossing the centre lines of the sub-images projected along 180°), this
research proposes an approach referring to the annotation method for panoramic
MOT designed in subsection 3.2: As shown in Figure 18, the first step is to
move each of the left bounding boxes to the right and merge it with the right
box; Then, input the combined bounding boxes into DeepSORT directly, despite
the fact they are beyond the boundaries.

Figure 18: Method for tracking the detections which are shown twice in one
frame

In addition, to avoid ID switching caused by objects that leave from one side
of the image and return from the other side, the distance calculation process
of DeepSORT was also improved. As Figure 18 shows, for calculating the
Mahalanobis or IOU distance between a predicted track and a detection, the
proposed method first duplicates the detection twice on two virtual images
extended from both sides of the original image; Then, it calculates the
distances from the track to these three bounding boxes respectively and
selects the shortest one to fill the cost matrix.

Figure 19: Preparations for the improved distance calculation process
(suitable for Mahalanobis distance and IOU distance)

####  4.2.5 Model Evaluation

To evaluate the proposed method for improving MOT on panoramic cycling videos,
the research used DeepSORT before and after an improved detection model was
applied, and also before and after the support for category information and
boundary continuity were introduced, to track the objects of interest in the 3
annotated videos. Then, by comparing their results with the ground truth using
py-motmetrics ([60]), the project also calculated the evaluation metrics of
the models, which can be used to analyse and compare the MOT performance of
different models.

In addition, for qualitative analysis of the tracking effects, the results
were also exported as videos marked with bounding boxes, track ids, categories
and confidence scores.

###  4.3 Automated Overtaking Behaviour Detection

In the last step of the proposed methodology, an application was designed and
developed, which can use the tracking results to detect the overtaking
behaviour of the vehicles. As shown in Figure 20, for each image frame, the
application first classifies each vehicle into ‘forwards’ or ‘backwards’
according to the movement of its track; After that, overtaking behaviour is
detected among the vehicles moving forwards, and the detected overtakes are
divided into two states: confirmed (completed) and unconfirmed (ongoing) — For
the confirmed overtakes, the application outputs their start and end frames
with the track IDs; while for the unconfirmed ones, if they are close to the
camera, the application can warn the users of their current directions, so
that the cyclists can react to the potential dangers in advance. It should be
noted that overtaking is used as a test case to demonstrate the model and
different actions can be detected by changing the type, direction and location
(left or right of the rider) of the tracked objects.

Figure 20: Workflow of the application which can detect the overtaking
behaviour of the surrounding vehicles

In the following subsections, each module of the application is described in
detail and an experiment is carried out to evaluate the detection accuracy of
the application.

####  4.3.1 Movement Direction Recognition

According to the definition of equirectangular projection ([94]), in a
panoramic video, objects behind the camera are shown on the left and right
sides, while objects in the front are shown in the middle. Therefore, if an
object’s current position is closer to the centre line compared to its
position in the last frame, it is moving forwards relative to the camera.
Similarly, if an object moves away from the centre, it is moving backwards.

However, in practice, judging the moving directions based on only two frames
is not rigorous: In some frames where the camera moves slightly with the
rotation of the cyclists’ head, some objects may incorrectly move in the
opposite direction. Thus, to judge the movement of an object, this project
calculated its movement in each of the last 5 frames relative to the previous
frame. If more than 3 of them were forward, then it was classified as
‘forwards’. Likewise, if more than 3 of the movements were backwards, then the
object was classified as ‘backwards’.

####  4.3.2 Overtaking Behaviour Detection

The project defined overtaking behaviour as the act of a motor vehicle moving
forwards and completely passing the cyclist. In an equirectangular video, this
definition can be abstracted as the occasion in which the front and rear of a
vehicle on the left (or right) side of the image cross the -90° (or 90°)
longitude line in order.

Figure 21: Flow chart of the overtaking behaviour detection for a specific
object

Figure 21 shows how the application was designed to realise the overtaking
behaviour detection on a specific object in the panoramic videos: Take a
vehicle on the left side of the video as an example, if the right side of its
bounding box crosses the -90° longitude line for the first time as it is
moving forwards, it will be assigned with an unconfirmed (ongoing) overtake.
The overtake will remain unconfirmed until the left boundary of the bounding
box passes the -90° line or the right border goes back to the left of the
line, which means the overtake is finished (confirmed), and the overtake is
failed, respectively.

####  4.3.3 Application Evaluation

To evaluate how well the application can detect the overtaking behaviour of
the vehicles, this project used the application to perform detections on Video
4, 5 and 6 trimmed in Section 3. After all the detections had finished, the
corresponding bounding boxes in the period of each overtake were marked in
red. Then, according to these marked objects, this project manually counted
the number of TP (correctly detected overtakes), FP (wrong detections) and FN
(undetected overtakes) and calculated the precision and recall using Formula 1
and 2. In addition, to explore how the improved MOT and object detection can
affect the performance of overtaking detection, the project also compared the
applications using models without the proposed modifications.

## 5\. Results & Analysis

After implementing the proposed methodology, in this chapter, the project will
present and analyse the results obtained in each step systematically.

###  5.1 Evaluation of the Model for Improving Object Detection on Panoramic
Cycling Images

The experiments designed in subsection 4.1.5 were implemented on a server
equipped with an NVIDIA Tesla P100 GPU and an Intel Xeon E5-2630 processor,
whose results are shown in 4.

As the baseline models, the original pre-trained Faster RCNN-FPN and YOLO v5m6
are first compared. From Table 4, it can be seen that: the average precision
of both models increase with the growth of input resolution, and according to
their APs, APm and APl, this increase is mainly due to the better prediction
performance on objects smaller than $128^{2}$; In addition, although YOLO v5m6
is a one-stage model, since it is newly released, under the same input
resolution settings, it can always achieve an accuracy similar to, or even
better than that of Faster RCNN at twice the speed.

Then, the modified models are compared with the original ones. According to
Table 4, thanks to the smaller distortions of perspective projection, under
the same input resolution settings, the proposed method can significantly
improve the detection performance for small (0, $128^{2}$) and medium
[$128^{2}$, $384^{2}$) objects of both models. However, since multiple
projections split some long objects into several parts, before applying the
operation of merging the bounding boxes, the proposed models’ precision
metrics for large objects are much lower than those of the original models.

Merging the bounding boxes can effectively mitigate this problem because, from
the table, it is obvious that no matter what the input resolution is, it can
always raise the APl by at least 3. But, the improved precision for large
objects are still slightly smaller than those of the original models.

Nevertheless, since as Figure 6 shows, more than 90% of the objects in the
collected videos are small or medium ones, in most cases, no matter whether
merging the boxes or not, the mean APs of the proposed models exceed those of
the original models by 5, which are even close to the results of the original
models under higher input resolution settings.

In terms of inference speed, due to the requirement for sequential detection
on 4 sub-images, the improved models are slower than the original ones.
However, since even the speeds of the original models are far from real-time
detection (30 FPS), this project considers this decrease in speed acceptable.

Pre-Trained Model |  Input size | Model | AP | APs** | APm** | APl** | FPS
---|---|---|---|---|---|---|---
Faster RCNN-FPN | 640 | Original | 8.098 | 2.342 | 19.343 | 40.450 | 1.52
Proposed (No Merging) | 15.209 | 10.097 | 31.639 | 34.194 | 0.76
Proposed (Merging) | 16.019 | 10.097 | 31.640 | 38.823 | 0.83
1280 | Original | 17.332 | 10.497 | 34.224 | 47.976 | 1.46
Proposed (No Merging) | 20.303 | 15.771 | 35.439 | 34.608 | 0.43
Proposed (Merging) | 20.990 | 15.772 | 35.490 | 38.397 | 0.42
1920 | Original | 20.586 | 14.431 | 35.546 | 45.880 | 1.40
Proposed (No Merging) | 20.326 | 15.830 | 35.634 | 34.446 | 0.25
Proposed (Merging) | 21.046 | 15.830 | 35.694 | 38.345 | 0.24
YOLO v5m6 | 640 | Original | 8.271 | 1.808 | 19.761 | 47.092 | 3.60
Proposed (No Merging) | 15.108 | 10.210 | 33.975 | 32.804 | 0.86
Proposed (Merging) | 16.107 | 10.210 | 34.062 | 38.292 | 0.85
1280 | Original | 16.910 | 9.588 | 34.943 | 49.679 | 3.37
Proposed (No Merging) | 24.442 | 20.789 | 37.419 | 35.875 | 0.46
Proposed (Merging) | 25.428 | 20.789 | 37.462 | 39.456 | 0.45
1920 | Original | 23.211 | 16.861 | 37.705 | 49.184 | 3.02
Proposed (No Merging) | 29.265 | 26.171 | 40.000 | 32.590 | 0.26
Proposed (Merging) | 30.144 | 26.171 | 40.050 | 36.326 | 0.25
Table 4: Average Precision Metrics of Faster RCNN-FPN and YOLO v5m6 before and
after the Proposed Methods are Applied
Bolded values represent the best results
** APs, APm and APl are the average precision metrics of the small
$(0,128^{2})$, medium $[128^{2},384^{2})$, and large $[384^{2},+\infty)$
objects.

###  5.2 Evaluation of the Model for Improving Multiple Object Tracking on
Panoramic Cycling Videos

The experiments designed in subsection 4.2.5 were implemented on the same
server. Since according to Table 4, YOLO v5m6 gets a high accuracy at a
moderate speed when the input size is 1280, it was used as the detector of
DeepSORT in these experiments.

First, DeepSORT models before and after the improved YOLO is applied are
compared, whose results are shown in Table 5. Since comparing some metrics
such as ID switch is meaningless when the detectors are different, only IDF1,
MT, PT, ML, FP, FN and MOTA are compared here.

Model | IDF1 | GT | MT | PT | ML | FP | FN | MOTA
---|---|---|---|---|---|---|---|---
Original YOLO \+ Original DeepSORT | 57.0% | 218 | 34 | 100 | 84 | 1513 | 11479 | 45.3%
Improved YOLO \+ Original DeepSORT | 65.3% | 218 | 131 | 63 | 24 | 7246 | 4892 | 48.7%
Table 5: Evaluation Metrics of Original DeepSORT before and after an Improved
Detection Model (YOLO v5m6, Input Size: 1280) is Applied. Best results in bold

From the Table, it can be seen that the improved detector significantly
reduces the number of false negatives. As a result, according to the values of
MT (Mostly Tracked), PT (Partially Tracked) and ML (Mostly Lost), much more
ground truths are successfully tracked. In addition, the proposed detector
also raises IDF1 by 8.3%, which shows the model using the improved YOLO can
track objects more continuously. Although the proposed detector also increases
the number of FP, in terms of MOTA, it effectively improves the MOT
performance of DeepSORT.

Then, using the improved YOLO as detectors, models before and after the
implementations of supports for category information and boundary continuity
are compared, whose results are shown in Table 6.

Model | IDF1 | FP | FN | IDs | FM | MOTA | MOTP |  Time (s)
---|---|---|---|---|---|---|---|---
Improved YOLO \+ Original DeepSORT | 65.3% | 7246 | 4892 | 181 | 392 | 48.7% | 0.231 | 0.056
Improved YOLO \+ DeepSORT (Category Support) | 64.7% | 7235 | 4878 | 185 | 394 | 49.8% | 0.231 | 0.061
Improved YOLO \+ DeepSORT (Category and Boundary Support) | 66.7% | 6344 | 4795 | 179 | 392 | 52.9% | 0.231 | 0.066
Table 6: Evaluation Metrics of DeepSORT before and after the Supports for
Category Information and Boundary Continuity are Introduced. Best results in
bold

From the table, the support for category information reduces FP and FN. As a
result, MOTA rises from 48.7% to 49.8%. Although the improving effect on
metrics seems relatively slight, through observing the output videos of the
models with and without the support for category information, it can be seen
that some incorrect data matching has been successfully avoided, as shown in
Figure 22.

Figure 22: Examples of how category information avoids incorrect data matching
(Left: Original DeepSORT; Right: DeepSORT with Support for Category
Information)

On the basis of the introduction of category information, support for boundary
continuity further enhances the accuracy of data association, for it increases
MOTA by 3%. In addition, according to the higher IDF1 score and the fewer ID
switches, it can be seen that the tracking continuity of DeepSORT on panoramic
cycling videos is enhanced.

These enhancements are also evident in the output tracking videos: With the
support for boundary continuity, DeepSORT can now treat the two parts of the
same object (such as car 3 in Figure 23) as a whole; In addition, using the
improved distance calculation method, DeepSORT is now capable of keeping the
ids of the objects (such as human 11 in Figure 24) that leave from one side of
the image and return from the other side.

Figure 23: Tracking results of the objects which are shown in two parts before
and after the support for boundary continuity is introduced

In terms of tracking speed, for each image frame, support for category
information and boundary continuity only take DeepSORT 0.01 seconds longer to
finish the data association. Considering the significant performance
improvements provided by the proposed method, such a small drop in speed
acceptable.

Figure 24: Tracking results of the objects which leave from one side of the
image and return from the other side before and after the support for boundary
continuity is introduced

###  5.3 Evaluation of the Automated Overtaking Behaviour Detection

To evaluate the performance of overtaking behaviour detection, two validation
tests are carried out. The first test compares the performance of the proposed
model against the Original YOLO v5m6 + Original DeepSORT to assess the
relative performance of each model. The second test applies the proposed model
to a range of videos collected under different riding conditions in London,
UK.

In the first experiment, under the input resolution setting of 1280, two
combinations of models were tried: the Original YOLO v5m6 + Original DeepSORT,
and the Improved YOLO v5m6 + Improved DeepSORT. Their evaluation results are
shown in Table 7.

Model | TP | FP | FN | Recall | Precision | FPS
---|---|---|---|---|---|---
Original YOLO v5m6 \+ Original DeepSORT | 53 | 0 | 1 | 98.15% | 100% | 2.75
Improved YOLO v5m6 \+ Improved DeepSORT | 54 | 0 | 0 | 100% | 100% | 0.43
Table 7: Evaluation Results of Overtaking Behaviour Detection - Experiment 1

When being used for automated overtaking behaviour detection, the application
using the tracking results of the original models correctly detects 53 of 54
overtakes recorded in Video 3, 4 and 5, while on the basis of that, the one
using the improved models successfully identifies the missed overtake, thus
obtains a recall and precision of 100%.

Figure 25: The instance which gets different detection results in different
applications

By observing the only difference between the results of the two applications,
as shown in Figure 25, it can be seen that in the application using the
original models, since there is no support for multiple categories, the
vehicle is incorrectly matched with a traffic light, thus its movement is
ignored. As a comparison, the proposed models effectively eliminate wrong
associations, which results in the successful detection of the overtaking
behaviour of this vehicle.

Although the proposed models bring higher detection accuracy, due to the lower
inference speed of the proposed object detection model, the overall FPS of the
application using the improved models is 2.3 frames slower than that using the
original models.

|  Predicted Positive |  Predicted Negative
---|---|---
Actual Positive | 44 | 6
Actual Negative | 6 | 44
Table 8: Evaluation Results of Overtaking Behaviour Detection - Experiment 2

In the second experiment, we tested the improved algorithm on one hundred
15-second $360^{\circ}$ video clips filmed by 49 different cyclists, who were
participants in UCL’s 100 Cyclists Project. The participants used the GoPro
Max camera to record their activities over a period of two weeks, and were
instructed to say ‘near-miss’ when experiencing a near-miss event. The test
results are presented as a confusion matrix in Table 8.

It can be seen that the algorithm demonstrates both specificity and
sensitivity of $44\div 50=0.88$, and the F-score of $(2\times 44)\div(2\times
44+6+6)=0.88$. All 6 videos classed as positive by the algorithm that do not
contain overtaking have very quick ‘flash’ classifications lasting one or
several frames and are due to the occasional incorrect object detection by
YOLO. An example of a bus being erroneously detected, which is in fact a shop
front, is depicted in Figure 26. Applying a minimum time threshold for the
overtaking manoeuvre, such as 0.5 or 1 seconds, would eliminate all false
positives in our training set. To solve this problem during real-time
overtaking detection, one could introduce a lag of several frames and only
label the behaviour as overtaking after that point.

Figure 26: Example of a ghost bus, which in fact is a shop front that is
classed as a bus in a single frame, leads to an erroneous overtaking detection

Among the 6 videos classed as false negatives (where overtaking is undetected
by the algorithm), in 3 videos the overtaking vehicle moves fully or mostly in
a lane adjacent to the cyclist, which may or may not be viewed as overtaking.
In one exported video, the cyclist’s camera is not properly centred and moves
during overtaking. In another video, the cyclist’s food delivery-style
backpack, as well as rain drops, partly obstruct the manoeuvre as shown in
Figure 27.

Figure 27: The cyclist’s food delivery-style backpack, as well as rain drops,
partly obstruct the overtaking manoeuvre, preventing the algorithm from
detecting the overtaking behaviour

## 6\. Discussion

###  6.1 Improving Object Detection

Although Table 4 indicates that the performance of the proposed method can be
also achieved by the original models with larger input resolution, once the
input resolution is as large as that of the original image, the effect can not
be replaced anymore. In other words, it increases the upper limit of accuracy
that an object detection model can achieve on a panoramic image.

Conversely, since the prediction performance of the proposed model is always
similar to (or even better than) that of the original models with higher input
resolution, for the same accuracy requirements, the proposed model consumes
fewer GPU resources. Thus, it can achieve competitive accuracy even on low-end
GPUs ([104]).

Additionally, according to Table 5, the improved object detection models can
effectively enhance the performance of MOT models (such as DeepSORT) as more
objects can be tracked correctly, which is consistent with [58]
([]bewley_2016) ’s opinion.

###  6.2 Improving Multiple Object Tracking

First, the proposed method realises the annotation and results output of the
objects which go across the boundaries in an equirectangular image without
violating the MOT16 annotation rules ([84]). In that way, the existing MOT
metrics can be still used for evaluating the tracking results on panoramic
videos.

What’s more, on the basis of the performance improvement brought by the
proposed object detection models, the introduction of category information and
support for boundary continuity further enhances the applicability of DeepSORT
to panoramic videos by reducing the number of mismatches and ID switches.

###  6.3 Overtaking Behaviour Detection

The proposed method can detect the completed overtaking behaviour of vehicles
in panoramic videos with a high degree of accuracy and can output the start
and end frames of each overtake with the corresponding track ID. This precise
and automated analysis can provide abundant conflict data for studies on
cycling safety in the future.

###  6.4 Limitations and Future Work

The proposed methodology has some limitations, which will be addressed in
future work. Firstly, as multiple projection transformations divide long
objects into several parts, according to Table 4, even when bounding box
combination has been applied, the proposed models’ performance on large
objects (size $>$ 3842 pixels) is inferior that of the original models. As
shown in Figure 28, this can be explained by some parts of large objects (such
as the blue car in sub-image 1) not being detected at all, preventing them
from being merged. This issue may be mitigated by increasing the number of
sub-images to explore whether more overlapping areas can achieve better
detection performance on large objects.

Figure 28: Example of unsuccessful detection of the proposed model on objects
across the sub-images

Furthermore, as predictions on sub-images are done sequentially, the inference
speed is lower than the original models. This affects the operating speed of
MOT and overtaking behaviour detection, which use its results. Multi-
processing using, for example, Torch.Multiprocessing ([91]) will be used to
address this in future work. This will allow inferences on sub-images in
parallel.

As an MOT model originally designed for pedestrian tracking, DeepSORT uses the
ReID (Re-Identification) network trained on pedestrian datasets ([103]).
Although [108] ([]zuraimi2021vehicle) ’s research has demonstrated that it is
possible to use DeepSORT to realise MOT on other objects such as vehicles
without changing its feature extraction network, the results presented here
show that when the angle of a vehicle with respect to the camera changes, some
ID switching may occur. As Figure 29 shows, using the current ReID network,
the matching cascade module in DeepSORT fails in matching the front and rear
of these two cars. Referring to [78] ([]Li2021) , this research plans to
construct a vehicle image dataset and re-train a ReID network with it, which
may improve the performance of DeepSORT in associating the vehicles according
to their appearance.

Figure 29: Examples of ID switching when the angle of a vehicle toward the
camera changes

Currently, the moving directions of the vehicles are estimated relative to the
cyclists’ movement. Since generally, cyclists are unable to move backwards, it
is reasonable to use relative forward movements to detect overtakes. However,
for the ‘backward’ objects (those moving from in front of to behind the
rider), there are 3 possibilities for their actual movements: moving
backwards, being stationary or moving forwards at a slower pace than the
cyclist. Thus, if there is no other information such as the speed of the
cyclist, other road user behaviours such as braking or reversing may be
difficult to detect. Fortunately, the GoPro Max camera, along with many
consumer grade action cameras, collects high quality IMU and GPS data, which
includes the speed, position, acceleration and orientation of the user. These
data will be combined with the video to detect other behaviours associated
with cycling risk, such as passing parked vehicles, filtering through traffic
or entering the blind spot of heavy goods vehicles (HGVs).

## 7\. Conclusion

In the field of cycling safety, most of the existing studies applied computer
vision models to ‘traditional’ videos with a limited field of view for
automated road user behaviour analysis ([70]). To enable analysis of the
complete environment surrounding the rider, this research develops methods for
object detection and MOT in panoramic equirectangular videos. For object
detection, existing pre-trained object detection models are applied to four
perspective sub-images projected from equirectangular images. When evaluated
on the annotated panoramic dataset, the AP of YOLO v5m6 and Faster RCNN-FPN
are improved under any input resolution setting. For MOT, a method the
improved YOLO is used as the detector and support for boundary continuity and
category information is implemented in DeepSORT, improving MOTA by 7.6% and
IDF1 by 9.7%. The tracking results of the improved DeepSORT are implemented in
the case of automated overtaking detection, which achieves high recall and
precision on the testing videos.

Although the proposed methodology has been applied to overtaking detection,
the framework is capable of detecting and tracking any objects defined as
classes in a pre-trained object detection model, such as pedestrians,
bicycles, buses and motorcyles. Therefore, in future work, methods will be
defined to detect other types of interactions based on the relative motions
and positions of the rider and the objects in their surroundings.

If deployed in real-time, these algorithms will enable early warning of
potential incidents. For example, the rider may be alerted to a vehicle
preparing to overtake, or may be advised that they are riding in the ‘car-
dooring zone’ next to parked vehicles. Delivery of this information will
necessitate the development of lightweight versions of the algorithms that can
be deployed on edge devices. Given sufficient penetration of action cameras
among cyclists, the technology could be used for crowdsourced risk mapping,
whereby risky interactions extracted from video are mapped to the transport
network. These could then be correlated with crash locations from official
data sources to gain a better understanding of overall risk.

## References

  * [1] Jiří Ambros, Chris Jurewicz, Shane Turner and Mariusz Kieć “An international review of challenges and opportunities in development and use of crash prediction models” In _European Transport Research Review_ 10.2 Springer ScienceBusiness Media LLC, 2018 DOI: 10.1186/s12544-018-0307-7
  * [2] Danilo Avola et al. “MS-Faster R-CNN: Multi-Stream Backbone for Improved Faster R-CNN Object Detection and Aerial Tracking from UAV Images” In _Remote Sensing_ 13.9, 2021 DOI: 10.3390/rs13091670
  * [3] Ben Beck et al. “How much space do drivers provide when passing cyclists? Understanding the impact of motor vehicle and infrastructure characteristics on passing distance” In _Accident Analysis & Prevention_ 128, 2019, pp. 253–260 DOI: https://doi.org/10.1016/j.aap.2019.03.007
  * [4] Alex Bewley et al. “Simple online and realtime tracking” In _2016 IEEE International Conference on Image Processing (ICIP)_ , 2016, pp. 3464–3468 DOI: 10.1109/ICIP.2016.7533003
  * [5] J. Brunsing “PUBLIC TRANSPORT AND CYCLING: EXPERIENCE OF MODAL INTEGRATION IN GERMANY. FROM THE GREENING OF URBAN TRANSPORT” Wiley (John)Sons, Limited, 1997
  * [6] cheind “py-motmetrics” Accessed 23 August 2022, 2019 URL: https://github.com/cheind/py-motmetrics
  * [7] Stijn Daniels, Erik Nuyts and Geert Wets “The effects of roundabouts on traffic safety for bicyclists: An observational study” In _Accident Analysis & Prevention_ 40.2 Elsevier BV, 2008, pp. 518–526 DOI: 10.1016/j.aap.2007.07.016
  * [8] Eric M. Delmelle and Elizabeth Cahill Delmelle “Exploring spatio-temporal commuting patterns in a university environment” In _Transport Policy_ 21 Elsevier BV, 2012, pp. 1–9 DOI: 10.1016/j.tranpol.2011.12.007
  * [9] Fucheng Deng, Xiaorui Zhu and Jiamin Ren “Object detection on panoramic images based on deep learning” In _2017 3rd International Conference on Control, Automation and Robotics (ICCAR)_ , 2017, pp. 375–380 DOI: 10.1109/ICCAR.2017.7942721
  * [10] Jaimy Fischer, Trisalyn Nelson, Karen Laberee and Meghan Winters “What does crowdsourced data tell us about bicycling injury? A case study in a mid-sized Canadian city” In _Accident Analysis & Prevention_ 145 Elsevier, 2020, pp. 105695
  * [11] GoPro “Player + ReelSteady.” Accessed 25 August 2022, 2021 URL: https://gopro.com/en/us/info/gopro-player
  * [12] GoPro “What Is Horizon Leveling? GoPro support.” Accessed 21 August 2022, 2021 URL: https://community.gopro.com/s/article/what-is-horizon-leveling
  * [13] Tove Hels and Ivanka Orozova-Bekkevold “The effect of roundabout design features on cyclist accident rate” In _Accident Analysis & Prevention_ 39.2 Elsevier BV, 2007, pp. 300–307 DOI: 10.1016/j.aap.2006.07.008
  * [14] Hong Hua and N. Ahuja “A high-resolution panoramic camera” In _Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001_ 1, 2001, pp. I–I DOI: 10.1109/CVPR.2001.990634
  * [15] Mohamed R. Ibrahim, James Haworth, Nicola Christie and Tao Cheng “CyclingNet: Detecting cycling near misses from video streams in complex urban scenes with deep learning” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1049/itr2.12101 In _IET Intelligent Transport Systems_ 15.10, 2021, pp. 1331–1344 DOI: 10.1049/itr2.12101
  * [16] Mohamed R. Ibrahim et al. “Cycling near misses: a review of the current methods, challenges and the potential of an AI-embedded system” In _Transport Reviews_ 41.3 Informa UK Limited, 2021, pp. 304–328 DOI: 10.1080/01441647.2020.1840456
  * [17] Sunghoon Im et al. “All-Around Depth from Small Motion with a Spherical Panoramic Camera” In _Lecture Notes in Computer Science_ Springer International Publishing, 2016, pp. 156–172 DOI: 10.1007/978-3-319-46487-9_10
  * [18] Diego M. Jiménez-Bravo et al. “Multi-object tracking in traffic environments: A systematic literature review” In _Neurocomputing_ 494 Elsevier BV, 2022, pp. 43–55 DOI: 10.1016/j.neucom.2022.04.087
  * [19] Jean-Philippe Jodoin, Guillaume-Alexandre Bilodeau and Nicolas Saunier “Urban Tracker: Multiple object tracking in urban mixed traffic” In _IEEE Winter Conference on Applications of Computer Vision_ , 2014, pp. 885–892 DOI: 10.1109/WACV.2014.6836010
  * [20] Marilyn Johnson, Judith Charlton, Jennifer Oxley and Stuart Newstead “Naturalistic cycling study: identifying risk factors for on-road commuter cyclists” Annual Conference of the Association for the Advancement of Automotive Medicine (AAAM) 2010; Conference date: 01-01-2010 In _Annals of Advances in Automotive Medicine_ 54, 2010, pp. 275–283
  * [21] Yann LeCun, Yoshua Bengio and Geoffrey Hinton “Deep learning” In _Nature_ 521.7553 Springer ScienceBusiness Media LLC, 2015, pp. 436–444 DOI: 10.1038/nature14539
  * [22] Geon-Won Lee and Jong-Ki Han “Viewport Rendering Algorithm with a Curved Surface for a Wide FOV in 360° Images” In _Applied Sciences_ 11.3 MDPI AG, 2021, pp. 1133 DOI: 10.3390/app11031133
  * [23] Jianan Li et al. “Perceptual Generative Adversarial Networks for Small Object Detection” In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ IEEE, 2017 DOI: 10.1109/cvpr.2017.211
  * [24] Jiangeng Li, Zhibo Zang, Haizheng Xie and Guangsheng Wang “Research on Distribution Strategy of Region of Interest in Panoramic Video Based on Improved Deepsort” In _2021 33rd Chinese Control and Decision Conference (CCDC)_ IEEE, 2021 DOI: 10.1109/ccdc52312.2021.9601917
  * [25] Tsung-Yi Lin et al. “Microsoft COCO: Common Objects in Context” In _Computer Vision-ECCV 2014_ Cham: Springer International Publishing, 2014, pp. 740–755
  * [26] Keng-Chi Liu, Yi-Ting Shen and Liang-Gee Chen “Simple online and realtime tracking with spherical panoramic camera” In _2018 IEEE International Conference on Consumer Electronics (ICCE)_ , 2018, pp. 1–6 DOI: 10.1109/ICCE.2018.8326132
  * [27] Wenhan Luo et al. “Multiple object tracking: A literature review” In _Artificial Intelligence_ 293 Elsevier BV, 2021, pp. 103448 DOI: 10.1016/j.artint.2020.103448
  * [28] Alexerand Masalov et al. “CyDet: Improving Camera-based Cyclist Recognition Accuracy with Known Cycling Jersey Patterns” In _2018 IEEE Intelligent Vehicles Symposium (IV)_ Changshu, Suzhou, China: IEEE Press, 2018, pp. 2143–2149 DOI: 10.1109/IVS.2018.8500668
  * [29] Roel Massink et al. “The Climate Value of Cycling” In _Natural Resources Forum_ 35.2, 2011, pp. 100–111 DOI: https://doi.org/10.1111/j.1477-8947.2011.01345.x
  * [30] Anton Milan et al. “MOT16: A Benchmark for Multi-Object Tracking”, 2016 arXiv: https://arxiv.org/abs/1603.00831
  * [31] Mkver “FFmpeg” Accessed 18 August 2022, 2014 URL: https://github.com/FFmpeg/FFmpeg
  * [32] Risto Ojala, Jari Vepsalainen and Kari Tammi “Motion detection and classification: ultra-fast road user detection” In _Journal of Big Data_ 9.1 Springer, 2022, pp. 28
  * [33] OpenCV “Computer Vision Annotation Tool (CVAT)” Accessed 15 August 2022, 2019 URL: https://github.com/opencv/cvat
  * [34] Roslyn G Poulos et al. “Exposure-based cycling crash, near miss and injury rates: The Safer Cycling Prospective Cohort Study protocol” In _Injury prevention_ 18.1 BMJ Publishing Group Ltd, 2012, pp. e1–e1
  * [35] ppwwyyxx “Detectron2 Model Zoo and Baselines” Accessed 21 August 2022, 2021 URL: https://github.com/facebookresearch/detectron2
  * [36] John Pucher and Ralph Buehler “Safer cycling through improved infrastructure” In _American Journal of Public Health_ 106.12 American Public Health Association, 2016, pp. 2089–2091
  * [37] PyTorch “MULTIPROCESSING PACKAGE - TORCH.MULTIPROCESSING” Accessed 25 August 2022, 2022 URL: https://pytorch.org/docs/stable/multiprocessing.html
  * [38] Oliver J.. Rick and Jacob J. Bustad “Cycling in the Flattened City: Urban Assemblages and Digital Visual Research” Publisher: Edinburgh University Press In _Somatechnics_ 11.2, 2021, pp. 246–264 DOI: 10.3366/soma.2021.0354
  * [39] Arash Rocky, Qingming Jonathan Wu and Wandong Zhang “Review of Accident Detection Methods Using Dashcam Videos for Autonomous Driving Vehicles” Conference Name: IEEE Transactions on Intelligent Transportation Systems In _IEEE Transactions on Intelligent Transportation Systems_ , 2024, pp. 1–19 DOI: 10.1109/TITS.2024.3354852
  * [40] John P Snyder “Flattening the earth: two thousand years of map projections” University of Chicago Press, 1997
  * [41] Mohamed Taha, Hala H. Zayed, Taymoor Nazmy and M.. Khalifa “Multi Objects Tracking in Nighttime Traffic Scenes” In _The 7th International Conference on Information Technology_ , ICIT 2015 Al-Zaytoonah University of Jordan, 2015 DOI: 10.15849/icit.2015.0002
  * [42] Caroline Tait, Roger Beecham, Robin Lovelace and Stuart Barber “Is cycling infrastructure in London safe and equitable? Evidence from the cycling infrastructure database” In _Journal of Transport & Health_ 26 Elsevier, 2022, pp. 101369
  * [43] timt90022 “Perspective-and-Equirectangular” Accessed 20 August 2022, 2018 URL: https://github.com/timy90022/Perspective-and-Equirectangular
  * [44] Ultralytics “yolov5” Accessed 17 August 2022, 2020 URL: https://github.com/ultralytics/yolov5
  * [45] Sergio A Useche et al. “Cross-culturally approaching the cycling behaviour questionnaire (CBQ): evidence from 19 countries” In _Transportation research part F: traffic psychology and behaviour_ 91 Elsevier, 2022, pp. 386–400
  * [46] Cheng Wang et al. “Exploring factors influencing the risky cycling behaviors of young cyclists aged 15–24 years: A questionnaire-based study in China” In _Risk analysis_ 40.8 Wiley Online Library, 2020, pp. 1554–1570
  * [47] Miriam Wanner et al. “Active transport, physical activity, and body weight in adults: a systematic review” In _American journal of preventive medicine_ 42.5 Elsevier, 2012, pp. 493–502
  * [48] Warwickshire Police “Operation targets motorists who ignore Highway Code rules by passing cyclists too close”, 2023 URL: https://www.warwickshire.police.uk/news/warwickshire/news/2023/september/operation-targets-motorists-who-ignore-highway-code-rules-by-passing-cyclists-too-close/
  * [49] Nicolai Wojke, Alex Bewley and Dietrich Paulus “Simple online and realtime tracking with a deep association metric” In _2017 IEEE international conference on image processing (ICIP)_ , 2017, pp. 3645–3649 IEEE
  * [50] Wenyan Yang et al. “Object detection in equirectangular panorama” In _2018 24th international conference on pattern recognition (icpr)_ , 2018, pp. 2190–2195 IEEE
  * [51] Yiming Zhang, Xiangyun Xiao and Xubo Yang “Real-Time object detection for 360-degree panoramic image using CNN” In _2017 international conference on virtual reality and visualization (icvrv)_ , 2017, pp. 18–23 IEEE
  * [52] Yingheng Zhang, Haojie Li and Gang Ren “Analyzing the injury severity in single-bicycle crashes: An application of the ordered forest with some practical guidance” In _Accident Analysis & Prevention_ 189 Elsevier, 2023, pp. 107126
  * [53] Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu and Xindong Wu “Object detection with deep learning: A review” In _IEEE transactions on neural networks and learning systems_ 30.11 IEEE, 2019, pp. 3212–3232
  * [54] Muhammad Azhad Bin Zuraimi and Fadhlan Hafizhelmi Kamaru Zaman “Vehicle detection and tracking using YOLO and DeepSORT” In _2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE)_, 2021, pp. 23–29 IEEE

## References

  * [55] Jiří Ambros, Chris Jurewicz, Shane Turner and Mariusz Kieć “An international review of challenges and opportunities in development and use of crash prediction models” In _European Transport Research Review_ 10.2 Springer ScienceBusiness Media LLC, 2018 DOI: 10.1186/s12544-018-0307-7
  * [56] Danilo Avola et al. “MS-Faster R-CNN: Multi-Stream Backbone for Improved Faster R-CNN Object Detection and Aerial Tracking from UAV Images” In _Remote Sensing_ 13.9, 2021 DOI: 10.3390/rs13091670
  * [57] Ben Beck et al. “How much space do drivers provide when passing cyclists? Understanding the impact of motor vehicle and infrastructure characteristics on passing distance” In _Accident Analysis & Prevention_ 128, 2019, pp. 253–260 DOI: https://doi.org/10.1016/j.aap.2019.03.007
  * [58] Alex Bewley et al. “Simple online and realtime tracking” In _2016 IEEE International Conference on Image Processing (ICIP)_ , 2016, pp. 3464–3468 DOI: 10.1109/ICIP.2016.7533003
  * [59] J. Brunsing “PUBLIC TRANSPORT AND CYCLING: EXPERIENCE OF MODAL INTEGRATION IN GERMANY. FROM THE GREENING OF URBAN TRANSPORT” Wiley (John)Sons, Limited, 1997
  * [60] cheind “py-motmetrics” Accessed 23 August 2022, 2019 URL: https://github.com/cheind/py-motmetrics
  * [61] Stijn Daniels, Erik Nuyts and Geert Wets “The effects of roundabouts on traffic safety for bicyclists: An observational study” In _Accident Analysis & Prevention_ 40.2 Elsevier BV, 2008, pp. 518–526 DOI: 10.1016/j.aap.2007.07.016
  * [62] Eric M. Delmelle and Elizabeth Cahill Delmelle “Exploring spatio-temporal commuting patterns in a university environment” In _Transport Policy_ 21 Elsevier BV, 2012, pp. 1–9 DOI: 10.1016/j.tranpol.2011.12.007
  * [63] Fucheng Deng, Xiaorui Zhu and Jiamin Ren “Object detection on panoramic images based on deep learning” In _2017 3rd International Conference on Control, Automation and Robotics (ICCAR)_ , 2017, pp. 375–380 DOI: 10.1109/ICCAR.2017.7942721
  * [64] Jaimy Fischer, Trisalyn Nelson, Karen Laberee and Meghan Winters “What does crowdsourced data tell us about bicycling injury? A case study in a mid-sized Canadian city” In _Accident Analysis & Prevention_ 145 Elsevier, 2020, pp. 105695
  * [65] GoPro “Player + ReelSteady.” Accessed 25 August 2022, 2021 URL: https://gopro.com/en/us/info/gopro-player
  * [66] GoPro “What Is Horizon Leveling? GoPro support.” Accessed 21 August 2022, 2021 URL: https://community.gopro.com/s/article/what-is-horizon-leveling
  * [67] Tove Hels and Ivanka Orozova-Bekkevold “The effect of roundabout design features on cyclist accident rate” In _Accident Analysis & Prevention_ 39.2 Elsevier BV, 2007, pp. 300–307 DOI: 10.1016/j.aap.2006.07.008
  * [68] Hong Hua and N. Ahuja “A high-resolution panoramic camera” In _Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001_ 1, 2001, pp. I–I DOI: 10.1109/CVPR.2001.990634
  * [69] Mohamed R. Ibrahim, James Haworth, Nicola Christie and Tao Cheng “CyclingNet: Detecting cycling near misses from video streams in complex urban scenes with deep learning” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1049/itr2.12101 In _IET Intelligent Transport Systems_ 15.10, 2021, pp. 1331–1344 DOI: 10.1049/itr2.12101
  * [70] Mohamed R. Ibrahim et al. “Cycling near misses: a review of the current methods, challenges and the potential of an AI-embedded system” In _Transport Reviews_ 41.3 Informa UK Limited, 2021, pp. 304–328 DOI: 10.1080/01441647.2020.1840456
  * [71] Sunghoon Im et al. “All-Around Depth from Small Motion with a Spherical Panoramic Camera” In _Lecture Notes in Computer Science_ Springer International Publishing, 2016, pp. 156–172 DOI: 10.1007/978-3-319-46487-9_10
  * [72] Diego M. Jiménez-Bravo et al. “Multi-object tracking in traffic environments: A systematic literature review” In _Neurocomputing_ 494 Elsevier BV, 2022, pp. 43–55 DOI: 10.1016/j.neucom.2022.04.087
  * [73] Jean-Philippe Jodoin, Guillaume-Alexandre Bilodeau and Nicolas Saunier “Urban Tracker: Multiple object tracking in urban mixed traffic” In _IEEE Winter Conference on Applications of Computer Vision_ , 2014, pp. 885–892 DOI: 10.1109/WACV.2014.6836010
  * [74] Marilyn Johnson, Judith Charlton, Jennifer Oxley and Stuart Newstead “Naturalistic cycling study: identifying risk factors for on-road commuter cyclists” Annual Conference of the Association for the Advancement of Automotive Medicine (AAAM) 2010; Conference date: 01-01-2010 In _Annals of Advances in Automotive Medicine_ 54, 2010, pp. 275–283
  * [75] Yann LeCun, Yoshua Bengio and Geoffrey Hinton “Deep learning” In _Nature_ 521.7553 Springer ScienceBusiness Media LLC, 2015, pp. 436–444 DOI: 10.1038/nature14539
  * [76] Geon-Won Lee and Jong-Ki Han “Viewport Rendering Algorithm with a Curved Surface for a Wide FOV in 360° Images” In _Applied Sciences_ 11.3 MDPI AG, 2021, pp. 1133 DOI: 10.3390/app11031133
  * [77] Jianan Li et al. “Perceptual Generative Adversarial Networks for Small Object Detection” In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ IEEE, 2017 DOI: 10.1109/cvpr.2017.211
  * [78] Jiangeng Li, Zhibo Zang, Haizheng Xie and Guangsheng Wang “Research on Distribution Strategy of Region of Interest in Panoramic Video Based on Improved Deepsort” In _2021 33rd Chinese Control and Decision Conference (CCDC)_ IEEE, 2021 DOI: 10.1109/ccdc52312.2021.9601917
  * [79] Tsung-Yi Lin et al. “Microsoft COCO: Common Objects in Context” In _Computer Vision-ECCV 2014_ Cham: Springer International Publishing, 2014, pp. 740–755
  * [80] Keng-Chi Liu, Yi-Ting Shen and Liang-Gee Chen “Simple online and realtime tracking with spherical panoramic camera” In _2018 IEEE International Conference on Consumer Electronics (ICCE)_ , 2018, pp. 1–6 DOI: 10.1109/ICCE.2018.8326132
  * [81] Wenhan Luo et al. “Multiple object tracking: A literature review” In _Artificial Intelligence_ 293 Elsevier BV, 2021, pp. 103448 DOI: 10.1016/j.artint.2020.103448
  * [82] Alexerand Masalov et al. “CyDet: Improving Camera-based Cyclist Recognition Accuracy with Known Cycling Jersey Patterns” In _2018 IEEE Intelligent Vehicles Symposium (IV)_ Changshu, Suzhou, China: IEEE Press, 2018, pp. 2143–2149 DOI: 10.1109/IVS.2018.8500668
  * [83] Roel Massink et al. “The Climate Value of Cycling” In _Natural Resources Forum_ 35.2, 2011, pp. 100–111 DOI: https://doi.org/10.1111/j.1477-8947.2011.01345.x
  * [84] Anton Milan et al. “MOT16: A Benchmark for Multi-Object Tracking”, 2016 arXiv: https://arxiv.org/abs/1603.00831
  * [85] Mkver “FFmpeg” Accessed 18 August 2022, 2014 URL: https://github.com/FFmpeg/FFmpeg
  * [86] Risto Ojala, Jari Vepsalainen and Kari Tammi “Motion detection and classification: ultra-fast road user detection” In _Journal of Big Data_ 9.1 Springer, 2022, pp. 28
  * [87] OpenCV “Computer Vision Annotation Tool (CVAT)” Accessed 15 August 2022, 2019 URL: https://github.com/opencv/cvat
  * [88] Roslyn G Poulos et al. “Exposure-based cycling crash, near miss and injury rates: The Safer Cycling Prospective Cohort Study protocol” In _Injury prevention_ 18.1 BMJ Publishing Group Ltd, 2012, pp. e1–e1
  * [89] ppwwyyxx “Detectron2 Model Zoo and Baselines” Accessed 21 August 2022, 2021 URL: https://github.com/facebookresearch/detectron2
  * [90] John Pucher and Ralph Buehler “Safer cycling through improved infrastructure” In _American Journal of Public Health_ 106.12 American Public Health Association, 2016, pp. 2089–2091
  * [91] PyTorch “MULTIPROCESSING PACKAGE - TORCH.MULTIPROCESSING” Accessed 25 August 2022, 2022 URL: https://pytorch.org/docs/stable/multiprocessing.html
  * [92] Oliver J.. Rick and Jacob J. Bustad “Cycling in the Flattened City: Urban Assemblages and Digital Visual Research” Publisher: Edinburgh University Press In _Somatechnics_ 11.2, 2021, pp. 246–264 DOI: 10.3366/soma.2021.0354
  * [93] Arash Rocky, Qingming Jonathan Wu and Wandong Zhang “Review of Accident Detection Methods Using Dashcam Videos for Autonomous Driving Vehicles” Conference Name: IEEE Transactions on Intelligent Transportation Systems In _IEEE Transactions on Intelligent Transportation Systems_ , 2024, pp. 1–19 DOI: 10.1109/TITS.2024.3354852
  * [94] John P Snyder “Flattening the earth: two thousand years of map projections” University of Chicago Press, 1997
  * [95] Mohamed Taha, Hala H. Zayed, Taymoor Nazmy and M.. Khalifa “Multi Objects Tracking in Nighttime Traffic Scenes” In _The 7th International Conference on Information Technology_ , ICIT 2015 Al-Zaytoonah University of Jordan, 2015 DOI: 10.15849/icit.2015.0002
  * [96] Caroline Tait, Roger Beecham, Robin Lovelace and Stuart Barber “Is cycling infrastructure in London safe and equitable? Evidence from the cycling infrastructure database” In _Journal of Transport & Health_ 26 Elsevier, 2022, pp. 101369
  * [97] timt90022 “Perspective-and-Equirectangular” Accessed 20 August 2022, 2018 URL: https://github.com/timy90022/Perspective-and-Equirectangular
  * [98] Ultralytics “yolov5” Accessed 17 August 2022, 2020 URL: https://github.com/ultralytics/yolov5
  * [99] Sergio A Useche et al. “Cross-culturally approaching the cycling behaviour questionnaire (CBQ): evidence from 19 countries” In _Transportation research part F: traffic psychology and behaviour_ 91 Elsevier, 2022, pp. 386–400
  * [100] Cheng Wang et al. “Exploring factors influencing the risky cycling behaviors of young cyclists aged 15–24 years: A questionnaire-based study in China” In _Risk analysis_ 40.8 Wiley Online Library, 2020, pp. 1554–1570
  * [101] Miriam Wanner et al. “Active transport, physical activity, and body weight in adults: a systematic review” In _American journal of preventive medicine_ 42.5 Elsevier, 2012, pp. 493–502
  * [102] Warwickshire Police “Operation targets motorists who ignore Highway Code rules by passing cyclists too close”, 2023 URL: https://www.warwickshire.police.uk/news/warwickshire/news/2023/september/operation-targets-motorists-who-ignore-highway-code-rules-by-passing-cyclists-too-close/
  * [103] Nicolai Wojke, Alex Bewley and Dietrich Paulus “Simple online and realtime tracking with a deep association metric” In _2017 IEEE international conference on image processing (ICIP)_ , 2017, pp. 3645–3649 IEEE
  * [104] Wenyan Yang et al. “Object detection in equirectangular panorama” In _2018 24th international conference on pattern recognition (icpr)_ , 2018, pp. 2190–2195 IEEE
  * [105] Yiming Zhang, Xiangyun Xiao and Xubo Yang “Real-Time object detection for 360-degree panoramic image using CNN” In _2017 international conference on virtual reality and visualization (icvrv)_ , 2017, pp. 18–23 IEEE
  * [106] Yingheng Zhang, Haojie Li and Gang Ren “Analyzing the injury severity in single-bicycle crashes: An application of the ordered forest with some practical guidance” In _Accident Analysis & Prevention_ 189 Elsevier, 2023, pp. 107126
  * [107] Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu and Xindong Wu “Object detection with deep learning: A review” In _IEEE transactions on neural networks and learning systems_ 30.11 IEEE, 2019, pp. 3212–3232
  * [108] Muhammad Azhad Bin Zuraimi and Fadhlan Hafizhelmi Kamaru Zaman “Vehicle detection and tracking using YOLO and DeepSORT” In _2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE)_, 2021, pp. 23–29 IEEE