# MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera
Configurations

Mike Wong1, Murali Ramanujam1, Guha Balakrishnan2, Ravi Netravali1 1Princeton
University 2Rice University

###### Abstract.

Camera orientations (i.e., rotation and zoom) govern the content that a camera
captures in a given scene, which in turn heavily influences the accuracy of
live video analytics pipelines. However, existing analytics approaches leave
this crucial adaptation knob untouched, instead opting to only alter the way
that captured images from fixed orientations are encoded, streamed, and
analyzed. We present MadEye, a camera-server system that automatically and
continually adapts orientations to maximize accuracy for the workload and
resource constraints at hand. To realize this using commodity pan-tilt-zoom
(PTZ) cameras, MadEye embeds (1) a search algorithm that rapidly explores the
massive space of orientations to identify a fruitful subset at each time, and
(2) a novel knowledge distillation strategy to efficiently (with only camera
resources) select the ones that maximize workload accuracy. Experiments on
diverse workloads show that MadEye boosts accuracy by 2.9-25.7% for the same
resource usage, or achieves the same accuracy with 2-3.7$\times$ lower
resource costs.

##  1\. Introduction

Building on the steady growth in camera deployments and advances in deep
neural networks (DNNs) for vision tasks (e.g., classification or detection)
(Li et al., 2015; Lin et al., 2017; Cai et al., 2015; He et al., 2017; pyt,
2021), live video analytics pipelines have become prevalent. These pipelines
operate by continually streaming live video feeds from cameras to processing
servers (either edge (Padmanabhan et al., 2022; eky, 2022; Ananthanarayanan et
al., 2019; Wang et al., 2019a; Microsoft Azure, 2021) or cloud (Zhang et al.,
2017a; Li et al., 2020; Du et al., 2020a; Jiang et al., 2018a)), where DNNs
are run on incoming frames to produce low latency and highly accurate results
for different application queries, i.e., combinations of task, DNN, and
object(s) of interest. Key use cases include autonomous driving, footfall
tracking, traffic coordination, business analytics, among others (Cassel,
[n.d.]; Rijas, [n.d.]; Anderson, [n.d.]; Datondji et al., 2016; for, 2022;
Ghodgaonkar et al., 2020; tra, 2020; Rizzoli, 2022; Grand View Research,
[n.d.]; Collins, [n.d.]).

Given their practical importance, much research has been devoted to improving
both the resource efficiency and accuracy of live video analytics pipelines.
Existing solutions include accuracy-aware tuning of inference configuration,
encoding, or appearance knobs (Jiang et al., 2018b; Zhang et al., 2017a; Du et
al., 2022; Paul et al., 2023), filtering out redundant content (Du et al.,
2020a; Li et al., 2020; Chen et al., 2015; Jain et al., 2020), using cheaper
model variants (Romero et al., 2021; eky, 2022), improving job scheduling
(Zhang et al., 2017a; Shen et al., 2019; Padmanabhan et al., 2022), and so on.
However, all of these works assume that the content observable by cameras is
unchangeable, and instead can only be encoded, streamed, or analyzed
differently. In essence, they focus on optimizing _fixed, preset_ camera
deployments.

Unfortunately, the deployment of cameras for analytics is itself a daunting
task for operators. Subject to practical constraints (e.g., mounts, power
sources), for a scene of interest, operators must determine the number of
cameras to deploy and the orientation (i.e., combination of rotation and zoom
factor) to use for each. There exist many possible orientations, and altering
these decisions requires manual intervention. Yet we find that doing so can be
highly fruitful: across different workloads and scenes, dynamically adapting
orientations over time can yield accuracy improvements of 21.3-35.3% (without
inflating resource usage) compared to even the _best_ fixed-orientation
scheme. Further, these wins cannot be reaped by simply deploying more fixed
cameras to simultaneously cover more orientations: most orientations are
‘best’ for short total periods of time (median of 6 sec for each 10-min
video), drastically hindering the efficiency of such an approach, especially
in the resource-constrained settings where video analytics are run (eky, 2022;
Shi et al., 2016; Microsoft Azure, 2021; Li et al., 2021).

An alternative strategy is to leverage PTZ (pan-tilt-zoom) cameras that offer
software libraries for tuning orientations, thereby providing a logical
approach to capturing the above wins. Indeed, despite existing for nearly two
decades, PTZ camera popularity has surged in recent years (global market value
of $3 billion in 2020 (Business Research Insights, [n.d.])) largely due to
declining price points that now rival fixed-camera costs (Honey Optics,
[n.d.]; SecurityBros, [n.d.]). However, multiple challenges complicate their
use for live analytics (§2.3). First, queries are highly sensitive, in
different ways, to orientation knobs due to their diverse goals (e.g., tasks),
inherent model biases (how models perceive scenes and objects), and scene
dynamism (where objects are located) – optimizing orientation tuning for one
workload can forego up to 25.1% of the potential median accuracy wins for
another. Second, the ‘grid’ of orientations is large, but the selection space
is sparse, with steep accuracy drops from the best orientation(s) to other at
any time. Third, the best orientation changes rapidly, e.g., 85% of changes
occur in $\leq$1 sec since the last change.

To overcome these issues, we present MadEye, a camera-server system that
automatically and continually adapts PTZ camera orientations to maximize
analytics accuracy for the scene and workload at hand. The key insight behind
MadEye is that the speed at which commodity PTZ cameras can change
orientations (i.e., upwards of 600° per sec with near-instantaneous digital
zoom) far outpaces the rate at which applications require analytics results
(typically 1-30 frames per second (fps), i.e., every 33-1000 ms). This, in
turn, allows MadEye to eschew typical non-stationary multi-armed bandit
strategies (Kshitija Taywade, 2022; Vidit Saxena, Joakim Jaldén, Joseph E.
Gonzalez, Mats Bengtsson, Hugo Tullberg, Ion Stoica, 2019; Omar Besbes,
Yonatan Gur, Assaf Zeevi, 2014) that rely purely on previous explorations to
determine orientation importance, in favor of a more informed strategy based
on _current_ scene content. Concretely, in each timestep (33 ms for 30 fps)
and subject to network/compute resource availability, MadEye cameras explore
multiple orientations and quickly determine which will maximize workload
accuracy and warrant transmission to the backend for full inference. However,
realizing this strategy in practice involves addressing several technical
challenges.

First, to enable fast camera-side evaluation of the importance of different
orientations, MadEye adopts a custom knowledge distillation (Hinton et al.,
2015) strategy with edge-grade, ultra-compressed NN models. To cope with their
potentially limited predictive power, we task them with modeling query
sensitivities only to the point of accurately _ranking_ orientations in terms
of impact on workload accuracy – precise results are left to backend servers.
Even with this relaxed framing, MadEye must employ several optimizations to
achieve sufficient rank accuracy. Most notably, MadEye trains edge models
using a common abstraction – detection for objects of interest – that reflects
the minimum information needed to capture sensitivities and biases for popular
tasks. Task-specific semantics need not be baked into edge models, and instead
can be incorporated by post processing the generated results.

Edge models are _continually_ trained on MadEye’s backend using both the
latest and historical workload results, with the goal of mitigating data skew
towards recently-selected orientations (given uncertainties in what will be
selected next). Importantly, to balance resource costs and accuracy, each edge
model covers only a single query but all orientations. The intuition is that,
while model results can exhibit substantial divergence (Agarwal and Netravali,
[n.d.]; Balakrishnan et al., 2021; Du et al., 2020b; Khosla et al., 2012),
feature-level variance between orientations for the same scene is considerably
narrower, often smaller than that in typical pre-training datasets (Lin et
al., 2014). Accordingly, MadEye freezes pre-trained feature extraction layers
across queries, caching those weights on cameras, thereby lowering retraining
and (downlink) model update overheads.

Second, we develop a novel, on-camera search strategy to explore orientations
with the goal of capturing the best one (accuracy-wise) at each timestep.
Three key empirical observations guide our search: (1) despite rapid temporal
shifts, transitions between best orientations move slowly in the spatial
dimension, (2) the best orientations are typically spatially clustered, and
(3) neighboring orientations (with overlapping regions) exhibit highly
correlated trends in efficacy.

Building on these observations, MadEye explores a flexible shape of contiguous
orientations at each timestep, and considers shifting only towards neighboring
orientations whose efficacy can be robustly predicted. Decisions to
keep/remove orientations are governed by both response rates (and the
corresponding time budgets) and _relative_ comparisons of recent edge model
results. For the former, MadEye uses an efficient heuristic to determine path
feasibility in the time budget (a variant of the NP-Hard Traveling Salesman
Problem (Held and Karp, 1970)). For the latter, MadEye gracefully trades off
exploration (i.e., shape size) for network usage (i.e., sending more
orientations for backend inference) to bound the effects of edge model errors
and maximize accuracy for the required response rate.

To evaluate MadEye, we developed the first (to our knowledge) dataset that
supports tuning rotation and zoom at each time instant by splicing out scenes
of interest from publicly available 360° videos. Using this dataset, we
evaluated MadEye on a variety of network conditions and workloads that
incorporate multiple vision DNNs and query tasks: classification, counting
(per-frame and aggregate), and detection. Across these settings, MadEye boosts
accuracy by 2.9-25.7% compared to an oracle fixed-orientation strategy without
inflating resource usage; these wins are within 1.8-13.9% of the oracle
dynamic strategy. Framed differently, MadEye achieves those accuracy boosts
with 2-3.7$\times$ lower resource footprints than the best strategy of using
(multiple) fixed-orientation cameras. Moreover, MadEye outperforms recent PTZ
tracking algorithms (Shubham Jain, Viet Nguyen, Marco Gruteser, Paramvir Bahl,
2017; SCW, [n.d.]) (by 2.0-3.8$\times$) and multi-armed bandit solutions
(Vidal, 2021) (by 5.8$\times$). We will release MadEye and our datasets. This
work does not raise any ethical issues.

##  2\. Background and Motivation

We start with an overview of live video analytics deployments (§2.1). We then
show measurements highlighting the importance of dynamically adapting camera
orientations to workloads and scenes (§2.2), and the challenges associated
with realizing those benefits in practice (§2.3).

###  2.1. Overview of Live Video Analytics

In a live video analytics deployment, one or more cameras continually stream
their video frames to servers for processing. Servers can range from distant
(but powerful) cloud machines (Zhang et al., 2017a; Shen et al., 2019) to
nearby (but weaker) edge boxes (Padmanabhan et al., 2022; eky, 2022; Microsoft
Azure, 2021), and are tasked with running queries on the incoming frames to
support different applications. Queries most often involve running deep neural
network (DNN) inference on individual frames, with the goals of locating and
characterizing various objects in the scene, e.g., an intersection. Moreover,
the queries for different applications can vary in terms of the tasks they
perform, the objects they consider, the DNNs they use (different architectures
and weights), and the response rates they require. For instance, footfall
tracking for business analytics will count people passing through an area,
with response rates at 1 fps or less (Anderson, [n.d.]). In contrast, smart
driving or sports analytics applications will detect the specific locations of
cars or people, with response rates upwards of 30 fps (Rizzoli, 2022).

In this paper, we focus on the following four query tasks (and their
corresponding accuracy metrics) that have been prevalent in recent literate
(Kang et al., 2019; Kang et al., 2017; Cangialosi et al., 2022; Li et al.,
2020; Du et al., 2020a) and real-world deployments (Padmanabhan et al., 2022;
Lu et al., 2016). We note that these query types also serve as the building
blocks for complex applications and other tasks, e.g., tracking queries rely
on object detections.

  * $\bullet$

Binary classification: asks if any objects of interest are present in a frame.
Accuracy across the video is measured as the fraction of frames with the
correct binary decision.

  * $\bullet$

Counting: counts the number of objects of interest in each frame. Accuracy for
each frame is measured as the percent difference between the returned and
ground truth counts.

  * $\bullet$

Detection: finds the precise bounding box coordinates for objects of interest
in a frame. Accuracy per frame is measured using mAP (Everingham et al.,
2010), which evaluates the overlap between each returned box and its ground
truth counterpart.

  * $\bullet$

Aggregate counting: counts the _unique_ objects of interest that appear in a
scene. Accuracy per video is the percent difference between the returned and
ground truth counts.

Over time, an analytics deployment will face diverse workloads to run on the
video feeds it manages, each varying in query composition and size
(Padmanabhan et al., 2022; Agarwal and Netravali, [n.d.]). Yet, the
overarching goals persist: subject to resource constraints, deliver low-
latency results (at the desired response rate) with maximal accuracy.

###  2.2. Opportunities with Tuning Camera Orientations

Existing optimizations for video analytics (§6) assume that a stationary
camera’s orientation (rotation and zoom), and thus what it ingests from the
target scene, is fixed and incapable of being adapted. To quantify the
significance of this restriction, we run experiments on our 50-video dataset
and workloads that incorporate 4 model architectures, the 4 tasks from §2.1,
and people/cars; §5.1 details both. Each video supports tuning of rotations
(150° horizontally by 30°, 75° vertically by 15°) and zoom (1-3$\times$); we
consider other granularities in §5.4.

For each video, we obtained per-frame (15 fps here) results for each workload
by running its queries on all 75 orientations. We then define accuracy
relative to the _best_ orientation for each frame, i.e., the orientation that
maximized per-frame accuracy for the workload. For instance, for counting, an
orientation’s accuracy at any time is its object of interest count divided by
the max count across all orientations at that time. Using this methodology, we
compare three schemes: (1) _one time fixed_ which selects the best orientation
at time=0 and keeps it throughout the video, (2) _best fixed_ which uses
oracle knowledge to pick the best single orientation that maximizes average
workload accuracy for the video, and (3) _best dynamic_ which selects the best
orientation per frame in the video.

Figure 1. Accuracy for 5 representative workloads when using varying degrees
of orientation adaptation. Bars list results for the median video, with error
bars spanning 25-75th percentiles. Figure 2. Accuracy wins from adapting
orientations (compared to _best fixed_) grow as query specificity grows. Bars
list median videos, with error bars for 25-75th percentiles. We exclude agg.
counting+cars due to limits of multi-object trackers (§5.1).

As shown in Figure 1, adapting camera orientations brings substantial accuracy
improvements without inflating resource usage, i.e., the same number of frames
are transmitted and processed: median boosts with _best dynamic_ are
30.4-46.3% over _one time fixed_ and 21.3-35.3% over the _best fixed_ scheme
that is an upper bound for any fixed-orientation approach. Figure 2 breaks
down these results by query task. Notably, the importance of adapting
orientations grows as query types become more specific. For instance, for
YOLOv4 and cars, median accuracy improvements over _best fixed_ are 1.2%,
13.4%, and 16.4% for binary classification, counting, and detection,
respectively. The reason is that coarser queries mask certain differences
across orientations, e.g., if many objects of interest are present in the
scene, any orientation that catches a single object will deliver max accuracy
for binary classification; counting, on the other hand, will favor the
orientation with the most objects.

Primer on PTZ cameras. Pan-tilt-zoom (PTZ) cameras present an intuitive
mechanism to realize such adaptation. PTZ cameras come in two forms,
traditional (E et al., 2023; PTZ Optics, [n.d.]a) and electronic (ePTZ) (PTZ
Optics, [n.d.]b; HuddleCamHD, [n.d.]), both of which support software tuning
of pan (horizontal rotation), tilt (vertical rotation), and zoom. The key
difference between the two variants is in their tuning mechanisms. Traditional
PTZ cameras embed physical motors to rotate at well over 360°-per-second and
optically zoom (i.e., without reducing resolutions). In contrast, ePTZ cameras
capture wide field-of-views and employ near-instantaneous digital rotation and
zoom to focus on specific parts of the scene. ePTZ cameras change orientations
faster and are cheaper, but also cover smaller rotation areas (150° vs. 360°)
and degrade image quality by using digital zoom. PTZ cameras rival traditional
ones in on-board compute resources, with recent offerings housing edge-grade
GPUs (NVIDIA, 2021).

###  2.3. Challenges

Despite the potential benefits of adapting camera orientations using PTZ
cameras, three fundamental challenges complicate this approach in practice. We
describe them in turn.

Figure 3. Shifts in the best orientation are frequent. Results list a PDF
(binned by 1 sec) of time between switches in best orientation across all
videos and workloads.

C1: rapid changes in best orientation over time. As shown in Figure 3, due to
the dynamic nature of video content, switches in best orientation are
frequent: 85% of switches occur in $\leq$1 sec since the last switch.

Figure 4. Workloads exhibit different sensitivity to orientations. Results
apply the best orientations for workload $X$ (legend) to workload $Y$ (x
axis), and plot the accuracy wins (over best fixed for $Y$) that are lost from
not using the best orientations for workload $Y$. Bars list medians; error
bars for 25-75th percentiles.

C2: diverse workload sensitivities to zoom and rotation at each time. At any
point in time, the best orientation can vary across individual queries and
workloads. Figure 4 illustrates this, showing that adapting orientations to
maximize accuracy for one workload can result in foregoing 3.2-25.1% of the
potential (median) accuracy wins for other workloads.

Figure 5 highlights this at a query level, showing that different models,
objects, and tasks can all influence orientation selections. Model
discrepancies influence what can be discerned in the scene during inference
and under what orientations. For instance, with people counting, selecting
best orientations for a query using YOLOv4 will miss out on 26.3% median
accuracy wins for the same task using SSD (even when trained on the same
dataset). In contrast, tasks dictate the specificity needed in the collected
results, e.g., optimizing for counting people (with YOLOv4) rather than
aggregate people counting with the same model foregoes 10.2% of potential
wins. Lastly, objects of interest govern the importance of regions based on
object densities, as well as the features used for and difficulty in detecting
relevant objects (smaller objects are typically tougher to discern (Redmon and
Farhadi, 2018)). Thus, unsurprisingly, optimizing for a YOLOv4 people counting
query would forego 13.3% of wins if the query considered cars instead.

Figure 6 provides example screenshots to illustrate the benefits and harm of
changing orientations. Importantly, tuning orientations does not simply bring
new objects into field of view, and instead plays a large role in a model’s
ability to detect objects that were already visible.

Figure 5. Applying the best orientations for a base query of {YOLOv4,
counting, people} to a query $Y$ that modifies a single element in the base
query; we compare the accuracy wins (over best fixed) to those when using the
best orientations for $Y$. Bars list medians; error bars for 25-75th
percentiles. Figure 6. Screenshots showing the (diverse) impact of rotation
and zoom for different queries. Each column shows two images from the same
time instant that use either different rotation or zoom. On the bottom row,
green arrows show newly captured objects, while red arrows show objects that
are newly missed after the orientation change. Left: rotation brings a new
object into the scene, helps detect 2 previously-visible objects, but loses a
previously-detected object. Middle: zooming in helps detect new people. Right:
after switching models, the same zoom from the middle column actually reduces
the number of detected people. Figure 7. Most orientations are best for short
total times in each video. Results consider all orientation-video pairs per
workload.

C3: massive (but sparse) search space. The orientation space exhibits
substantial sparsity in the spatial and temporal dimensions. For the former,
among the 75 orientations at any time, only 1 (or several, with ties) is best,
with steady dropoff in accuracy to the others, e.g., median dips of 4.8% and
20.7% from the best to 2nd and 5th best. For the latter, most orientations are
best for short total times in each video, with median durations of 5-6 sec
across workloads (Figure 7).

##  3\. Design

Figure 8. Overview of MadEye’s end-to-end workflow.

Figure 8 shows the end-to-end operation of MadEye. The main insight behind
MadEye is to leverage fast PTZ rotation speeds to explore many orientations in
each timestep (i.e., between when results are needed for an fps), and then
select, based on their _current_ content, the one(s) that maximize workload
accuracy under resource constraints. The idea is to limit the “guess work”
compared to prior search algorithms that rely only on past orientation
efficacy (§5.3).

As in other video analytics systems (Padmanabhan et al., 2022; Jiang et al.,
2018b; Li et al., 2020; Du et al., 2020a), users register queries with a
backend agent (on an edge or cloud server), specifying a target scene, as well
as a model to use, object(s) of interest, and a task. To operate under camera
compute constraints, MadEye then trains edge-compatible (i.e., highly
compressed) models (§3.1), not to replace the original (more accurate) query
models (as in typical knowledge distillation (Hinton et al., 2015)), but
instead to _approximately_ extract information of importance in a frame for
each query. In other words, approximation models are explicitly designed to
estimate the inherent sensitivities of each query (C2 from §2.3).

To cope with the large space of orientations and rapid shifts in best
orientations (C1 and C3 from §2.3), MadEye employs an efficient on-camera
search strategy (§3.3) that explores as many potentially fruitful orientations
as possible while avoiding fps violations for results. The camera then runs
approximation models on all captured orientations in each timestep and uses
the results to (1) _rank_ the orientations in terms of their likelihood to
maximize overall workload accuracy, and (2) determine the set of orientations
to consider in the next time step. The highest ranked orientations that the
network can support are sent to the backend for full workload inference; new
results are used to continually adapt approximation models to the current
scene (§3.2).

###  3.1. Designing Approximation Models

The primary objective of MadEye’s approximation models is to quantify the
_relative_ importance of orientations for the queries in a workload. However,
this requires capturing the sensitivity of each query to different orientation
and scene dynamics, subject to camera compute constraints. Given the potential
complexity of workload queries, we eschew noisy (and limited) vision features
based on local gradients (Dalal and Triggs, 2005; Lowe, 2004) in favor of
knowledge distillation with compressed models (Hinton et al., 2015). However,
we alter this approach in several ways to favorably balance ranking accuracy
and resource efficiency.

We design approximation models using a common abstraction that reflects the
minimum amount of information needed to sufficiently rank orientations. The
key idea is that the core elements of query sensitivity pertain to how models
find and characterize objects, rather than how tasks post-process those
results. Thus, MadEye’s approximation models are structured purely as ultra-
lightweight detectors for objects of interest; this strategy also avoids
tricky development of compressed models per task. Concretely, we use the
smallest variant of the edge EfficientDet family (Tan et al., 2019),
EfficientDet-D0 (3.9M parameters, $>$150 fps on a Jetson edge GPU). More
complex detectors could be used, but cameras possess limited GPU memory
(Padmanabhan et al., 2022; Li et al., 2020), and inference delays negatively
influence the degree to which MadEye can explore orientations (§3.3).

Why a detector? Two alternatives we considered for the approximation models
are to directly estimate object counts in an image, and to directly output
rank orderings across multiple images. However, we empirically observed high
error rates with both. This is largely because such approaches can only relate
the presence of features to objects via a global regression over an entire
image (or multiple images), failing to leverage local regressions via bounding
box predictions to boost precision. While image-level DNN object counters do
exist (Sindagi and Patel, 2017; Yang et al., 2020; Zhang et al., 2019; Zhao et
al., 2019), they focus on large crowds of people. In contrast, there are often
few objects of interest in an orientation at any time (§2.3), making rank
orderings extremely sensitive to small errors in count prediction.

MadEye uses one approximation model per query, rather than per workload or per
object. Though more efficient, we avoid per-workload and per-object
approximation models as we (like others (Agarwal and Netravali, [n.d.])) find
that different DNNs can exhibit wildly varying response profiles to even the
same object classes due to object-independent factors like scale and
resolution (Huang et al., 2017). Moreover, DNNs trained on very different
datasets are known to inherit different algorithmic biases (Balakrishnan et
al., 2021; Du et al., 2020b; Khosla et al., 2012; Oksuz et al., 2020; Steed
and Caliskan, 2021; Wang et al., 2020).

However, each approximation model is configured to support _all_ orientations
for two reasons. First, the number of orientations is large (§2.3), making
per-orientation approximation models impractical with on-camera GPUs. Second,
neighboring orientations exhibit substantial overlap, and since we only
consider orientations for a given scene, divergence in background content,
lighting, shadows, etc. are minimal. Indeed, we measured the perceptual
distance (Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, Oliver
Wang, 2018) of images (LPIPS) from different orientations in the same scene to
be 0.30. For context, the same value for the popular MS-COCO and Pascal VOC
datasets used to successfully pre-train many vision models (including
EfficientDet) are 0.46 and 0.41.

Estimating workload accuracies. MadEye post-processes the generated bounding
boxes from all approximation models to compute _predicted workload accuracies_
for orientation ranking. To do this, MadEye follows the per-task accuracy
metrics from §2.1, but computes per-orientation predicted accuracy in a
relative manner compared to the other orientations under test. For instance,
counting computes the ratio of object counts between each orientation and the
max among the set of explored orientations at that timestep, while detection
expands this to incorporate object area sizes (as per mAP score). Lastly,
aggregate counting modulates count scores to favor less explored orientations
(that may have unseen objects).

###  3.2. Continually Training Approximation Models

MadEye servers train a new approximation model for each new query, with the
goals of being fast (since training blocks deployment) and accurate (in
ranking orientation importance). Initial training uses a small set of 1000
historical images from the target scene that is then labeled (online) using
the DNN in the registered query; label generation takes 7-90 sec depending on
the DNN. However, to accelerate this process, MadEye begins with a version of
EfficientDet that is pre-trained on Pascal VOC, and freezes both the backbone
network and the BiFPN layers responsible for feature extraction and fusion.
Only weights for the final 3 bounding box and class prediction layers are
fine-tuned to mimic the target query’s behavior. The rationale is that model
features progressively move from general (e.g., textures, gradients) to task-
specific (e.g., object prediction) as a function of layer depth (Bau et al.,
2017; Zeiler and Fergus, 2014; Yosinski et al., 2014). Initial fine-tuning
lasts for 40 epochs ($\approx$25 mins).

Even after initial fine-tuning, approximation models may fail to generalize to
changing scene dynamics (Suprem et al., 2020), leading to degrading accuracy.
To cope with such data drift, MadEye employs continual learning (every 400-500
ms) to update the model’s weights using the latest query results on
orientations sent to the server for full workload inference. While continual
learning has been applied to edge video analytics (eky, 2022; Mehrdad Khani,
Ganesh Ananthanarayanan, Kevin Hsieh, Junchen Jiang, Ravi Netravali , Yuanchao
Shu, Mohammad Alizadeh , Victor Bahl, 2023), MadEye requires several
alterations from prior efforts. The main challenge is that within each
retraining window, samples are only available for the orientations that
MadEye’s camera-side component recently visited and deemed worthy of backend
inference. Since orientations are typically best for short total times (§2.3),
there is often severe imbalance in the orientations covered by new training
samples. For instance, with perfect rankings, the average 2-minute window sees
only 9.3% of orientations get sent to the backend. This can result in
overfitting to certain orientations, and catastrophic forgetting (Kemker et
al., 2018) for others that may soon be ranked highly.

To deal with this, MadEye retrieves the most recent historical training
samples from each orientation and uses this to balance the dataset. As we will
discuss in §3.3, we find that orientation shifts are often spatially
localized, with changes to distant orientations happening over longer
timescales. Thus, MadEye pads the data samples for neighboring orientations
(up to 3 away from the latest one) to match the count for the most popular
orientation in the retraining window. The remaining orientations use an
exponentially declining number of samples based on their distance from the
latest orientation.

###  3.3. Exploring and Ranking Orientations

The primary goal of MadEye’s on-camera component is to efficiently explore (a
subset of) the large orientation space to capture the best orientation for
each timestep. Realizing this is challenging for three reasons. First, MadEye
only has visibility into the orientations that it has recently explored, but
other orientations can change in content and importance at any time. Second,
even among recently explored orientations, MadEye only has access to coarse
results from approximation models (i.e., that accurately capture only relative
importance) for most. Third, each timestep is not only dedicated to
exploration, but also (1) running approximation models on explored
orientations, (2) encoding and shipping select orientations to the server, and
(3) running the workload on shipped images.

Figure 9. Spatial distance between successive best orientations is small, with
most transitions between neighboring orientations. Results aggregate across
all videos and workloads for 15 fps. Figure 10. Top ranked orientations are
often spatially clustered. Results use 15 fps, are aggregated across all
workloads and videos, and show the max distance between orientations in the
top $k$ ranked orientations at each timestep.

Rather than relying on previous (and potentially stale) observations at each
orientation (§5.3), MadEye opts for a more informed strategy guided by 3
empirical observations.

  * $\bullet$

Although best orientations change rapidly over time (§2.3), those changes are
far slower in the spatial dimension. Figure 9 illustrates this, showing that
the median and 90th percentile spatial distance between successive best
orientations are 30° and 63.5°, which pertains to shifts spanning only 1 or 2
orientations in our default grid (§5.1).

  * $\bullet$

The best performing orientations (accuracy-wise) at any time are often
spatially clustered (Figure 10). Concretely, across our dataset, the 75th
percentile distance separating orientations in the top $k$ at each timestep is
1 and 2 orientations for $k$ values of 2 and 6.

  * $\bullet$

Accuracy for neighboring orientations often shift in tandem. Indeed, as shown
in Figure 11, the correlation coefficient for accuracy changes in direct
neighbors is 0.83; intuitively, this value shrinks to 0.75 when considering
neighbors 2-hops away (that exhibit less content overlap).

Taken together, these findings motivate a search strategy that considers a
flexible shape of contiguous orientations at each timestep, and swaps out
underperforming orientations in the previous shape only for neighboring ones
whose trends we can robustly predict for the next timestep. We start with a
description of the algorithm that does not account for zoom or resource
constraints and later incorporate those elements. Common themes are: only
relative comparisons of approximation model results are used, we leverage all
outputs from those models (including bounding boxes), and search decisions are
entirely local (i.e., on cameras) to remain rapid.

Figure 11. Correlation in accuracy changes across orientations separated by
$N$ hops. Results list Pearson Correlation Coefficients and cover 3
representative videos and workloads (15 fps).

MadEye begins with a rectangular seed shape that reflects the largest
coverable area in the time budget, thereby maximizing early exploration; we
reset to this shape any time 0 objects of interest are found in a shape. The
corresponding orientations are captured and analyzed with approximation models
to compute a predicted workload accuracy for each (§3.1). After sending the
top $k$ orientations to the server for workload inference, MadEye must use
these prior results to determine the set of orientations to explore in the
next timestep.

To do this, MadEye labels each orientation from the last timestep with a value
that indicates the likelihood of being fruitful in the _next_ timestep.
Concretely, we combine the exponentially weighted moving averages from recent
(10) timesteps for (1) any computed predicted accuracy values, and (2) the
deltas between those values. Weighted averages are used to remain robust to
inconsistencies in DNN results across consecutive frames (Agarwal and
Netravali, [n.d.]; Paul et al., 2021), which is especially pronounced with
MadEye’s compressed approximation models.

Using those labels, MadEye must now determine which orientations to remove and
add for the upcoming timestep. For this, MadEye sorts orientations into an
ordered list based on their label values. Using pointers at the head $H$
(largest label) and tail $T$ (smallest label) of the list, MadEye iteratively
compares orientations by asking: should we remove the orientation at $T$ in
favor of adding a neighbor to $H$? Concretely, MadEye computes the ratio of
label values for $H$/$T$. If (1) that ratio exceeds a threshold (indicating a
substantial disparity in the potential of $H$ and $T$), (2) $H$ has neighbors
not already in the shape, and (3) removing $T$ would not break contiguity, we
remove the orientation at $T$ and increment the pointer. The process repeats
by considering the addition of another neighbor for $H$, this time using a
larger threshold to account for the additional uncertainty of adding more
neighbors. $H$ is decremented when a neighbor cannot be added, and the process
ends when even one neighbor for $H$ cannot be added.

For each iteration that results in a neighbor addition for $H$, MadEye selects
among $H$’s neighbors by analyzing the bounding boxes that its approximation
models generated in the last timestep. For each candidate neighbor, we compute
the ratio of two values: normal distances to the center of $H$ and to the
centroid of all bounding boxes in $H$. Values $<$1 indicate lower chances of
$H$’s objects moving to the candidate in the next timestep. We repeat this
process for all other orientations in the last shape that the candidate
exhibits any non-zero overlap with. Candidate neighbor scores are computed as
the weighted sum of these ratios (weights according to degree of overlap), and
the candidate with the max score is selected.

Reachability and path selection. The search algorithm thus far ignores whether
a PTZ camera can sufficiently cover the selected shape in a given time budget.
Formally, the shape of orientations can be represented as a fully-connected
undirected graph with edge weights pertaining to the time taken to move
between two adjacent orientations (given a rotation speed). Our goal is to
determine whether the shape is coverable in a given time budget, and if so,
what is the shortest path. The paths between orientations satisfy the triangle
inequality property (Tversky and Gati, 1982), so this can be modeled as a
variant of the NP-Hard Traveling Salesman Problem (TSP) (Bender and Chekuri,
1999). Given our tight time budgets, MadEye employs the Minimum Spanning Tree
(MST) heuristic (Held and Karp, 1970), but optimizes it to minimize online
delays. In particular, since our orientation grid is static, we precompute
pairwise distances and the entire MST ahead of time. Online, for a given
shape, we quickly extract and perform a preorder walk on the corresponding
subgraph to get the shortest path. This reduces the heuristic to linear
complexity (in orientations); each path computation takes 14 µ$s$, and the
resultant paths are within 92% of optimal. Upon failure, MadEye greedily
removes the orientation with the lowest potential (that does not break
contiguity) and rechecks reachability.

Balancing search size and network/compute delays. MadEye pipelines its
exploration through orientations with the running of approximation models on
each one. However, network transmission to and workload inference on the
backend do not overlap with orientation exploration. The reason is that
transmissions are governed by global ranks across _all_ orientations explored
in each timestep. Thus, in each timestep, we face a tradeoff between exploring
more orientations and sending more orientations to the backend.

MadEye resolves this tension based on the expected difficulty for its
approximation models to accurately rank the considered orientations, which in
turn governs the risk associated with exploring more orientations (and sending
fewer to get ground truth results). Intuitively, scenarios where the
considered orientations are projected to contribute similar accuracies pose
the biggest difficulty for approximation models (as the gaps between ranks
shrinks). MadEye determines the right balance by first selecting a target
number of frames to send according to the training accuracy for approximation
models (provided by the backend) and the variance in predicted accuracy values
in the last timestep, e.g., 85% training accuracy and 25% variance results in
sending at least 2 frames. MadEye then computes a target shape size for
exploration, accounting for network transmission delays (harmonic mean of past
5 transfers (Yin et al., 2015)), backend compute delays, camera rotation
speeds, and approximation model inference delays.

Handling zoom. After selecting the set of orientations to visit, the search
algorithm must determine the zoom factor to use for each one. The challenge is
that past accuracies are insufficient for determining zoom fidelity as MadEye
cannot know what objects are being missed by not zooming in/out. Instead, we
rely on bounding boxes from approximation models to determine the risk of
zooming in. When an orientation is added to the shape, we start at the lowest
zoom factor to gain visibility into its whole content. At each timestep, we
compute the average distance between each bounding box and the centroid of all
boxes; smaller distances indicate more clustering and less risk of zooming in.
These values are compared with the area covered by each zoom factor to select
one, and MadEye automatically zooms out after 3 seconds to avoid missing newly
entering objects in the orientation.

Transmitting images. At the end of each timestep, MadEye must transmit select
images to the server for workload inference. Unlike standard streaming, MadEye
sends disjoint sets of images from each orientation’s video stream. To keep
bandwidth costs low, MadEye maintains a list of the last image shared for each
orientation, and employs a functional encoder (Fouladi et al., 2018) that
computes deltas relative to that image.

##  4\. Implementation

MadEye’s core components are written in 9.1k lines of Python code, with all
training and inference tasks across the backend and camera run in PyTorch. We
use TensorRT (ten, 2021) to accelerate inference on the backend, and a variant
of Nexus (Shen et al., 2019) as a round-robin scheduler for approximation
model inference on cameras. Orientations are first represented as rotational
values, projected onto a 360° space, and then converted using an in-house
equirectangular-to-rectilinear image converter (written in C++) to match the
APIs offered by recent PTZ cameras (Beaufort, [n.d.]). For ground truth
accuracy computations (§5.1) that require a global (i.e., across all
orientations) perspective on object locations and uniqueness, atop the
ByteTrack multi-object tracker (Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu,
Zehuan Yuan, Ping Luo, Wenyu Liu, Xinggang Wang, 2021) that links objects
across an orientation’s video, we use cv2 and scikit-image to extract image
features (e.g., SIFT) that link objects across orientations.

##  5\. Evaluation

We evaluated MadEye across diverse workloads, network settings, and videos.
Our key findings are:

  * $\bullet$

MadEye increases median workload accuracies by 2.9-25.7% compared to an oracle
fixed-orientation strategy (while using the same amount of resources); wins
are within 1.8-13.9% of the oracle dynamic strategy.

  * $\bullet$

Achieving MadEye’s accuracy wins with 1 PTZ camera would require the best 4-6
fixed-orientation cameras, which comes with a 2-3.7$\times$ inflation in
resource costs.

  * $\bullet$

MadEye outperforms prior PTZ algorithms by 2.0-5.8$\times$, providing 31.1%,
46.8%, and 52.7% higher accuracy than Panoptes (Shubham Jain, Viet Nguyen,
Marco Gruteser, Paramvir Bahl, 2017), tracking (SCW, [n.d.]), and multi-armed
bandits (Vidal, 2021).

  * $\bullet$

MadEye gracefully balances on-camera exploration and transmission of
orientations to maximize accuracy even as resources shrink and response rates
rise.

###  5.1. Methodology

Video dataset for PTZ analysis. To the best of our knowledge, there does not
exist a public video dataset for PTZ cameras that enables users to tune
rotation and zoom knobs; instead, existing PTZ datasets reflect pre-determined
knob decisions. Thus, to evaluate MadEye, we generate our own dataset. To
construct our dataset, we begin with the abundance of 360° datasets.
Concretely, we use 50 360-degree videos from YouTube that incorporate scenes
resembling those from prior work (Li et al., 2020; Du et al., 2020a; eky,
2022), e.g., traffic intersections, walkways, shopping centers. Each video
lasts 5-10 minutes.

From each video, we carve out scenes of interest as regions spanning 150°
horizontally and 75° vertically. We then subdivide those regions into grids of
orientations to mimic recent PTZ offerings (HuddleCamHD, [n.d.]) (30° and 15°
granularities for pan and tilt; we explore other grids in §5.4), and extract a
full video per orientation. For zoom, since we operate on pre-captured videos,
we employ digital zoom (1-3$\times$) by cropping images and scaling back the
dimensions to match the original image.

Models and workloads. We consider 4 popular architectures for vision tasks:
SSD (Liu et al., 2016) and Faster RCNN (Shaoqing Ren, Kaiming He, Ross
Girshick, Jian Sun, 2015) with ResNet-50 backbones, YOLOv4 and Tiny-YOLOv4
(Wang et al., 2021) with CSPDarknet53 backbones. We consider two versions of
each model trained on Pascal VOC and MS-COCO, but show results for the latter
as the trends were similar. To construct queries, we follow the same
methodology from recent work (based on production deployments) (Padmanabhan et
al., 2022). Each model can perform any of the four tasks from §2.1 with a
focus on either people or cars. We enumerate all possible workloads sized
between 2-20 queries and pick 10 randomly. The appendix details each workload.
We run workloads on all videos and consider response rates from 1-30 fps.

Hardware and networks. On-camera computations run on an edge-grade Jetson Nano
(NVIDIA, 2021) equipped with a 128-core Maxwell GPU, quad-core ARM CPU with
1.43 GHz clock speed, and 4 GB of memory. We consider default camera rotation
speeds of 400° per second; we study this parameter in §5.4. Workload inference
and training of approximation models run on a server with an NVIDIA GTX 1080
GPU (8 GB RAM) and 18-core Intel Xeon 5220 CPU (2.2 GHz; 125 GB RAM). Camera
and server components are connected with emulated Mahimahi networks (Netravali
et al., 2015) using fixed-capacity (24-60 Mbps; 5-20 ms) and real-world mobile
traces.

Metrics. Our primary evaluation metric is average workload accuracy per video.
For each frame, following the accuracy definitions from §2.1, we compute per-
orientation accuracy for each query relative to the orientation that delivers
the max accuracy at that time. Per-query accuracies at each time are averaged
to compute per-frame workload accuracies, which in turn are averaged to
compute workload accuracy for a video.

While computing these values for binary classification and counting are
straightforward, detections and aggregate counting require slight alterations.
For detections, mAP scores depend on bounding box coordinates for specific
objects and thus cannot be measured by comparing results directly across
orientations. Thus, we consolidate the bounding boxes across orientations into
a global view, and employ de-duplication (Pan and Lyu, 2010) to eliminate
redundant objects in overlapping regions. We then compute each orientation’s
mAP score relative to the global scene, and assign per-orientation accuracies
as the ratio of its mAP score to the max one across orientations.

Aggregate counting queries are directly evaluated across the entire video (not
per frame). Thus, we compute the ratio of unique objects across the
orientations that a system selects compared to the total number of unique
objects in the video. Note that ByteTrack (§4) was unable to robustly support
car tracking, so we exclude aggregate counting for cars.

###  5.2. Overall Results

(a) 1 fps

(b) 15 fps

(c) 30 fps

Figure 12. Comparing MadEye with the best possible fixed- and adaptive-
orientation schemes across all videos and workloads with a {24 Mbps, 20 ms}
network and varying fps. Bars list medians with errors bars spanning 25-75th
percentiles.

We first compare MadEye with the two baselines from §2.2, _best fixed_ and
_best dynamic_ , on different network and fps settings. Both baselines
impractically rely on oracle knowledge of video content and workload accuracy,
i.e., to pick the best orientation per video or per timestep, respectively,
that maximizes accuracy for the target workload-video. Nonetheless, they serve
as useful context for MadEye’s performance. Note that MadEye automatically
adapts the number of frames it explores and transmits based on network delays
and response rates (§3.3). For _best fixed_ , we leverage increasing network
speeds by adding more fixed cameras (i.e., best, 2nd best, etc.), rather than
simply capturing more (redundant) frames from 1 camera. _Best dynamic_ does
not change for any query other than aggregate counting, for which we send the
largest number of fruitful orientations that the network can support.

(a) Verizon LTE

(b) 24 Mbps; 20 ms

(c) 60 Mbps; 5 ms

Figure 13. Comparing MadEye with the best possible fixed- and adaptive-
orientation schemes across all videos and workloads with fixed fps (15) and
varying networks (improving from left to right). Bars list medians with errors
bars spanning 25-75th percentiles.

Our results are captured in Figures 12-13. Across these settings, MadEye
delivers median and 75th percentile accuracies that are 2.9-25.7% and
1.6-20.7% higher than _best fixed_ , and within 1.8-13.9% and 1.3-12.5% of
_best dynamic_. Digging deeper, our results show two key trends. First, as
frame rates decrease (for a fixed network), MadEye’s accuracies and wins over
_best fixed_ grow, e.g., for a {24 Mbps, 20 ms} network, median wins improve
from 5.8-13.3% to 12.4-25.7% as fps drops from 15 to 1. The reason is that
lower fps yields larger timesteps (e.g., 1 sec for 1 fps, 66.7 ms for 15 fps),
enabling more exploration and/or transmission. Second, as network speeds grow
(for fixed fps), the same trends persist (since each network transfer is
faster) but to a lesser extent, e.g., median 15 fps wins grow to 8.6-18.4% for
{60 Mbps, 5 ms}.

(a) People

(b) Cars

Figure 14. MadEye’s accuracy improvements (over best fixed) for different
query tasks and objects. Results consider all videos and models, and use 15
fps and {24 Mbps; 20 ms}.

Figure 14 breaks down MadEye’s wins over _best fixed_ by task and object.
Following the rationale from §2.2, accuracy boosts with MadEye grow as task
specificity grows: median wins grow from 8.6% to 13.3% to 22.1% as we move
from counting to detections to aggregate counting for people. We also observe
consistently larger accuracy wins for people queries (rather than cars) due to
their less structured motion patterns (more frequent and scattered orientation
switches), e.g., for detections, wins for cars shrink to 6.7%.

MadEye Variant | Median Accuracy (%) | # Fixed Cameras
---|---|---
MadEye-1 | 63.1 | 3.7
MadEye-2 | 66.3 | 5.5
MadEye-3 | 66.8 | 6.1
Table 1. Number of optimally-configured fixed cameras needed to match the
accuracy of MadEye. MadEye-$k$ refers to a version of MadEye that is
restricted to sending the top $k$ frames to the server for workload inference.
Results consider a {24 Mbps; 20 ms} network, 15 fps response rate, and all
video-workload pairs.

Results thus far focus on accuracy improvements. However, a key goal with
MadEye is to maximize accuracy for a given resource cost, i.e., network and
backend inference overheads. Table 1 lists the smallest number of optimally
configured fixed cameras that would be required to match the accuracies that
different versions of MadEye deliver, each of which sends a different number
of frames per timestep. As shown, it would take 3.7 fixed cameras to realize
the 63.1% accuracy that MadEye-1 achieves, implying a 3.7$\times$ reduction in
network and backend compute usage. MadEye-2 is matched by 5.5 fixed cameras;
here, however, the resource reduction factor is 2.8$\times$ since MadEye also
sends 2 frames per timestep.

###  5.3. Comparisons with State-of-the-Art

We compare MadEye with 3 alternate approaches for adaptive camera
orientations. Figure 15 shows results for a {24 Mbps; 20 ms} network and 15
fps; trends hold for all other scenarios.

First, we consider Panoptes (Shubham Jain, Viet Nguyen, Marco Gruteser,
Paramvir Bahl, 2017), a recent PTZ system that configures orientations for
workloads of applications, each explicitly concerned with specific
orientation(s). For orientations of relevance, Panoptes generates a static
round-robin schedule that is weighted according to how many queries an
orientation is of interest to and how much motion has been detected
historically in that orientation; higher weights indicate staying in an
orientation for longer. Panoptes then switches between orientations according
to this schedule with one exception: if motion gradients in the direction of
any overlapping orientation of interest exceed a threshold, Panoptes switches
there for several sec before resuming the round robin. Panoptes does not
specify a zoom strategy, so we consider the best zoom (accuracy-wise) for any
orientation it visits.

Figure 15. MadEye vs. 3 camera tuning strategies. Results are for all
workloads and videos, 15 fps, and {24 Mbps; 20 ms}.

We consider two versions of Panoptes, _Panoptes-all_ and _Panoptes-few_ , in
which each workload query is interested in all orientations or only its best
fixed orientation, respectively. Max accuracy in both cases is defined
relative to the best orientation among only the set of considered ones. As
shown in Figure 15, MadEye outperforms Panoptes-all by 3.8$\times$, with 46.8%
higher accuracy at the median. The reason is that Panoptes cycles through
orientations based on a pre-determined schedule and motion gradients _in the
current orientation_ , neither of which are sufficient indicators of
importance of other orientations at the current time, e.g., orientations are
suboptimal most of the time (§2.3). In contrast, MadEye considers many
orientations per timestep, ranking them based on current content. The wins
persist compared to Panoptes-few (not shown due to the different accuracy
metric), but are less pronounced (median of 40.5%) as there are fewer
unfruitful orientations for Panoptes to consider.

Next, we consider tracking algorithms that most PTZ cameras come equipped with
today (SCW, [n.d.]). This algorithm starts in a home region (best fixed in our
experiment), selects the largest object it finds, and tracks that object
continually across orientations aiming to keep it as centered as possible. The
algorithm resets to the home region upon losing the tracked object. We
consider a favorable variant in which all orientations explored in a timestep
are shared with the backend, which uses the one with the highest accuracy. As
shown, MadEye delivers 2.0$\times$ higher workload accuracies (31.1% more at
the median) compared to this tracking scheme. The main reason again is that
the presence of a large object is a poor indicator of accuracy importance as
it fails to capture more general scene properties and the sensitivity of the
queries under test. In contrast, MadEye directly estimates query sensitivity
to make workload-aware, informed orientation selections.

Finally, we consider the common UCB1 multi-armed bandit (MAB) algorithm
(Vidal, 2021). Each orientation is considered a lever with a weight set to the
average observed accuracy across all past visits (we seed this with historical
data). The algorithm continually selects an orientation to visit as the one
with the highest sum of weighted average and upper confidence bound (which
favors less-visited orientations). As with tracking, we send all visited
orientations to the backend, which selects the best one per timestep. MadEye
delivers 52.7% higher median accuracies than this scheme, i.e., a 5.8$\times$
win. Unlike the schemes above, MAB does factor in workload accuracies in
selecting orientations. However, its adaptation considers only historical
efficacy (not current content), and scene dynamics have shifted by the time it
updates its patterns.

System | Resource reduction | Median accuracy
---|---|---
Chameleon (Jiang et al., 2018b) |  2.4$\times$ | 46.3%
Chameleon + MadEye |  2.4$\times$ | 56.1%
Table 2. MadEye preserves resource savings of recent systems, while improving
accuracy. Results use 15 fps, {24 Mbps; 20 ms}.

Compatibility with other optimizations. By focusing on previously un-tuned
knobs (rotation and zoom) to boost accuracy, MadEye is largely compatible with
prior efforts that optimize resource overheads. To illustrate this, we
consider a variant of Chameleon (Jiang et al., 2018b) that dynamically tunes
pipeline knobs (resolution and frame rate) to lower network and backend
inference resource costs without harming accuracy; we brute force selections
per frame focused on the best fixed orientation. We then run MadEye atop the
fps and resolution selections that Chameleon makes, sending the same amount of
network data. As shown in Table 2, Chameleon lowers resource costs by
2.4$\times$ compared to the naive scheme that sends all frames at the highest
resolution; MadEye preserves these efficiency wins, while increasing accuracy
by 9.8%.

###  5.4. Deep Dive Results

Rotation speeds. We evaluated the impact of camera rotation speed on MadEye’s
performance by considering values of {200, 400, 500, infinite}° per second, a
fixed network ({24 Mbps; 20 ms}), and 15 fps. Intuitively, accuracy grows as
rotation speeds increase, e.g., jumping from 54.2% to 64.9% as rotation speed
grows from 200 to 500° per second. The reason is that faster rotations enable
the exploration of additional orientations or, in rarer instances, additional
transmissions. Importantly, benefits plateau since most queries (other than
aggregate counting) are fully satisfied accuracy-wise as long as MadEye finds
the best orientation at each timestep.

Grid granularity. To understand the effect of grid granularity (with other
settings fixed), we focus on the pan dimension (since it is wider) and
consider steps of {15, 30, 45, 60}°. Overall, MadEye’s accuracy benefits
shrink as grids become more fine-grained (with more orientations), e.g.,
median accuracies drop from 67.5% to 51.8% when pan steps drop from 45 to 15.
This is because, although exploration in a time budget is governed by rotation
speeds rather than grid granularity, the same distance (in °) of exploration
will warrant more approximation model inference on more orientations, thereby
shrinking each timestep’s exploration budget.

Overheads. On MadEye’s backend, the primary overheads are in initializing
approximation models and continually sharing model updates with the camera.
Across our workloads, we find median bootstrapping delays to be 27 mins
(including labeling and initial fine-tuning). Downlink streaming consumes 3.2
Mbps for the median experiment. Recall that both overheads are mitigated by
MadEye’s fine-tuning strategy (§3.2). On cameras, the main overheads are in
selecting orientations to explore and running approximation models; for the
median workload-video pair, per-timestep delays for each task were 17µ$s$ and
6.7 ms for 15 fps and {24 Mbps; 20 ms}. The former benefits from pre-computed
reachability analysis (§3.3).

Figure 16. Comparing different approximation model designs: MadEye’s
lightweight detection models and compressed counting models (Count CNN).
Results use all videos, {24 Mbps; 20 ms}, 15 fps, and list median rank
assigned to the best explored orientation at each timestep (error bars for
25-75th percentiles).

Microbenchmarks. MadEye’s performance is governed by two main tasks: (1)
ranking orientations with approximation models, and (2) selecting orientations
to explore to find the best one(s) per timestep. For the former, Figure 16
show that MadEye’s approximation models assign median ranks of 1.1-1.3 to the
best explored orientation at each timestep, significantly outperforming the
variant that relies on counting directly on images. For the latter, for the
median workload-video pair on {24 Mbps; 20 ms} and 15 fps, MadEye explores
best orientation 89.3% of the time, with 6.8% of errors coming from our
conservative zoom strategy (§3.3).

##  6\. Related Work

Adapting video analytics knobs. VideoStorm (Zhang et al., 2017b) selects an
input knob configuration (e.g., frame rate, resolution) per workload to lower
resource costs and facilitate job scheduling on backend servers. Chameleon
(Jiang et al., 2018b) extends such configuration tuning to be adaptive in
order to cope with ever-changing scene dynamics while keep resource costs low.
As shown in §5.3, by focusing on tuning camera orientations (and not backend
pipeline knobs), MadEye provides complementary benefits to these efforts,
boosting accuracies while preserving the resource efficiency wins they bring.
Other efforts focus on camera-side knobs as MadEye does. For example, CamTuner
(Paul et al., 2023) uses SARSA Reinforcement Learning to boost accuracy by
automatically tuning capture knobs that cameras do not usually auto-adjust,
e.g., brightness, contrast, and sharpness. AccMPEG (Du et al., 2022) predicts
the effects of macroblock encoding settings on server-side DNNs, and tunes
encoding to maximize accuracy. MadEye shares the same goal as these efforts –
tune camera knobs to boost workload accuracy – but focuses on complementary
knobs, i.e., camera orientations.

Frame filtering and result reuse. Many prior efforts exploit temporal
redundancies in video data by filtering out frames for network transfer and
processing, and reusing results accordingly (Chen et al., 2015; Li et al.,
2020; Canel et al., 2019; Zhang et al., 2015; Guo and Hu, 2018; Drolia et al.,
2017; Xu et al., 2018; Apicharttrisorn et al., 2019; Zhu et al., 2018; Guo et
al., 2018; Paul et al., 2021; Kumar et al., 2019; Crankshaw et al., 2017).
Spatula (Jain et al., 2020) extends this to multi-camera settings, selecting
among cameras in a network. These optimizations are logically similar to
MadEye, which also aims to maximize accuracy per network usage. However, the
techniques are largely complementary: filtering decisions could be made among
explored orientations to maximize new content in transfers.

Computation and network optimizations. Several efforts seek to lower compute
footprints either by identifying lightweight model variants (Romero et al.,
2021; Han et al., 2016; Hinton et al., 2015; Courbariaux et al., 2016; Jacob
et al., 2018; Zhu et al., 2016; Liu et al., 2017; Blalock et al., 2020),
sharing model layers during inference (Padmanabhan et al., 2022; Jiang et al.,
2018c), or using smarter job scheduling strategies (Shen et al., 2019; Zhang
et al., 2017b). Other systems target lower network overheads by intelligently
compressing transmitted frames in a manner that is recoverable on the server
or does not negatively impact accuracy (Du et al., 2020a; Wang et al., 2019b;
Emmons et al., 2019). MadEye is entirely complementary to both directions in
that it solely focuses on judiciously selecting images (i.e., orientations) to
process at any time for an application-provided model (which can be
compressed); MadEye is agnostic to the way that selected frames are
transmitted or processed on the backed.

Drone coordination. Numerous efforts aim to adapt drone flight plans (and thus
the content on-board cameras see) to maximize analytics accuracy or scene
coverage (He et al., 2020; Jha et al., 2021; Vasisht et al., 2017; Bastani et
al., 2021). However, these systems focus on identifying events of interest
(e.g., wildfires, objects) in a geographically dispersed area for a preset
application. In contrast, MadEye focuses on tuning camera orientations for a
single scene to cope with workload nuances and maximize accuracy.

##  7\. Conclusion

This paper presents MadEye, a system that continually tunes PTZ camera
orientations to maximize accuracy for a given analytics workload and resource
setting. Key to MadEye are a rapid algorithm that searches through the large
space of orientations at each time, and a new, approximate transfer learning
strategy that efficiently selects the most fruitful (accuracy-wise)
orientations from those explored. Across many videos, workloads, and resource
conditions, MadEye increases accuracy by 2.9-25.7% for the same resource
usage, or achieves the same accuracy with 2.0-3.7$\times$ lower resource
costs.

## References

  * (1)
  * tra (2020) 2020\.  Video Analytics Traffic Study Creates Baseline for Change.  https://www.govtech.com/analytics/Video-Analytics-Traffic-Study-Creates-Baseline-for-Change.html.
  * ten (2021) 2021\.  NVIDIA TensorRT.  https://developer.nvidia.com/tensorrt
  * pyt (2021) 2021\.  PyTorch-YOLOv3.  https://github.com/eriklindernoren/PyTorch-YOLOv3.
  * eky (2022) 2022\.  Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers. In _19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22)_. USENIX Association, Renton, WA, 119–135.  https://www.usenix.org/conference/nsdi22/presentation/bhardwaj
  * for (2022) 2022\.  Video Analytics Market.  https://www.fortunebusinessinsights.com/industry-reports/video-analytics-market-101114.
  * Agarwal and Netravali ([n.d.]) Neil Agarwal and Ravi Netravali. [n.d.].  Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics.
  * Ananthanarayanan et al. (2019) Ganesh Ananthanarayanan, Victor Bahl, Landon Cox, Alex Crown, Shadi Nogbahi, and Yuanchao Shu. 2019.  Video Analytics - Killer App for Edge Computing. In _Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services_ (Seoul, Republic of Korea) _(MobiSys ’19)_. Association for Computing Machinery, New York, NY, USA, 695–696.  https://doi.org/10.1145/3307334.3328589
  * Anderson ([n.d.]) Larry Anderson. [n.d.].  Video Analytics Applications In Retail - Beyond Security.  https://www.securityinformed.com/insights/co-2603-ga-co-2214-ga-co-1880-ga.16620.html/.
  * Apicharttrisorn et al. (2019) Kittipat Apicharttrisorn, Xukan Ran, Jiasi Chen, Srikanth V. Krishnamurthy, and Amit K. Roy-Chowdhury. 2019.  Frugal Following: Power Thrifty Object Detection and Tracking for Mobile Augmented Reality. In _Proceedings of the 17th Conference on Embedded Networked Sensor Systems_ (New York, New York) _(SenSys ’19)_. Association for Computing Machinery, New York, NY, USA, 96–109.  https://doi.org/10.1145/3356250.3360044
  * Balakrishnan et al. (2021) Guha Balakrishnan, Yuanjun Xiong, Wei Xia, and Pietro Perona. 2021\.  Towards causal benchmarking of biasin face analysis algorithms.  In _Deep Learning-Based Face Analytics_. Springer, 327–359.
  * Bastani et al. (2021) Favyen Bastani, Songtao He, Ziwen Jiang, Osbert Bastani, and Sam Madden. 2021.  SkyQuery: An Aerial Drone Video Sensing Platform. In _Proceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software_ (Chicago, IL, USA) _(Onward! 2021)_. Association for Computing Machinery, New York, NY, USA, 56–67.  https://doi.org/10.1145/3486607.3486750
  * Bau et al. (2017) David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017.  Network dissection: Quantifying interpretability of deep visual representations. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 6541–6549.
  * Beaufort ([n.d.]) Francois Beaufort. [n.d.].  Control camera pan, tilt, and zoom – Pan, tilt, and zoom features on cameras are finally controllable on the web.  https://web.dev/camera-pan-tilt-zoom/.
  * Bender and Chekuri (1999) Michael A Bender and Chandra Chekuri. 1999.  Performance guarantees for the TSP with a parameterized triangle inequality. In _Algorithms and Data Structures: 6th International Workshop, WADS’99 Vancouver, Canada, August 11–14, 1999 Proceedings 6_. Springer, 80–85.
  * Blalock et al. (2020) Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. 2020.  What is the state of neural network pruning?  _arXiv preprint arXiv:2003.03033_ (2020).
  * Business Research Insights ([n.d.]) Business Research Insights. [n.d.].  Global PTZ Camera Market Research Report 2020.  https://www.businessresearchinsights.com/market-reports/ptz-cameras-market-100130.
  * Cai et al. (2015) Zhaowei Cai, Mohammad Saberian, and Nuno Vasconcelos. 2015\.  Learning Complexity-Aware Cascades for Deep Pedestrian Detection. In _Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)_ _(ICCV ’15)_. IEEE Computer Society, Washington, DC, USA, 3361–3369.  https://doi.org/10.1109/ICCV.2015.384
  * Canel et al. (2019) Christopher Canel, Thomas Kim, Giulio Zhou, Conglong Li, Hyeontaek Lim, David G. Andersen, Michael Kaminsky, and Subramanya R. Dulloor. 2019.  Scaling Video Analytics on Constrained Edge Nodes. In _2nd SysML Conference_.
  * Cangialosi et al. (2022) Frank Cangialosi, Neil Agarwal, Venkat Arun, Junchen Jiang, Srinivas Narayana, Anand Sarwate, and Ravi Netravali. 2022.  Privid: Practical, Privacy-Preserving Video Analytics Queries. In _Proceedings of the 19th USENIX Conference on Networked Systems Design and Implementation_ (Renton, WA, USA) _(NSDI’22)_. USENIX Association, Berkeley, CA, USA.
  * Cassel ([n.d.]) David Cassel. [n.d.].  Are We Ready for AI-Powered Security Cameras?  https://thenewstack.io/are-we-ready-for-ai-powered-security-cameras/.
  * Chen et al. (2015) Tiffany Yu-Han Chen, Lenin Ravindranath, Shuo Deng, Paramvir Bahl, and Hari Balakrishnan. 2015.  Glimpse: Continuous, Real-Time Object Recognition on Mobile Devices. In _Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems_. 155–168.
  * Collins ([n.d.]) Mary Collins. [n.d.].  The Hudl Algorithm: Turning Video into Player Tracking Data.  https://www.maryecollins.com/hudl-tracking.
  * Courbariaux et al. (2016) Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016.  Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1.  _arXiv preprint arXiv:1602.02830_ (2016).
  * Crankshaw et al. (2017) Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. 2017\.  Clipper: A Low-Latency Online Prediction Serving System. In _14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17)_. USENIX Association, Boston, MA, 613–627.  https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/crankshaw
  * Dalal and Triggs (2005) Navneet Dalal and Bill Triggs. 2005.  Histograms of oriented gradients for human detection. In _2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05)_ , Vol. 1. Ieee, 886–893.
  * Datondji et al. (2016) Sokemi Rene Emmanuel Datondji, Yohan Dupuis, Peggy Subirats, and Pascal Vasseur. 2016.  A Survey of Vision-Based Traffic Monitoring of Road Intersections.  _Trans. Intell. Transport. Sys._ 17, 10 (Oct. 2016), 2681–2698.  https://doi.org/10.1109/TITS.2016.2530146
  * Drolia et al. (2017) Utsav Drolia, Katherine Guo, Jiaqi Tan, Rajeev Gandhi, and Priya Narasimhan. 2017.  Cachier: Edge-Caching for Recognition Applications. In _2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)_. 276–286.  https://doi.org/10.1109/ICDCS.2017.94
  * Du et al. (2020a) Kuntai Du, Ahsan Pervaiz, Xin Yuan, Aakanksha Chowdhery, Qizheng Zhang, Henry Hoffmann, and Junchen Jiang. 2020a.  Server-Driven Video Streaming for Deep Learning Inference. In _Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication_ (Virtual Event, USA) _(SIGCOMM ’20)_. Association for Computing Machinery, New York, NY, USA, 557–570.  https://doi.org/10.1145/3387514.3405887
  * Du et al. (2022) Kuntai Du, Qizheng Zhang, Anton Arapin, Haodong Wang, Zhengxu Xia, and Junchen Jiang. 2022\.  AccMPEG: Optimizing Video Encoding for Accurate Video Analytics. In _Proceedings of Machine Learning and Systems_ , D. Marculescu, Y. Chi, and C. Wu (Eds.), Vol. 4. 450–466.  https://proceedings.mlsys.org/paper/2022/file/98f13708210194c475687be6106a3b84-Paper.pdf
  * Du et al. (2020b) Mengnan Du, Fan Yang, Na Zou, and Xia Hu. 2020b.  Fairness in deep learning: A computational perspective.  _IEEE Intelligent Systems_ 36, 4 (2020), 25–34.
  * E et al. (2023) Jinlong E, Lin He, Zhenhua Li, and Yunhao Liu. 2023\.  WiseCam: Wisely Tuning Wireless Pan-Tilt Cameras for Cost-Effective Moving Object Tracking. In _IEEE INFOCOM 2023-IEEE Conference on Computer Communications_. IEEE.
  * Emmons et al. (2019) John Emmons, Sadjad Fouladi, Ganesh Ananthanarayanan, Shivaram Venkataraman, Silvio Savarese, and Keith Winstein. 2019.  Cracking Open the DNN Black-Box: Video Analytics with DNNs across the Camera-Cloud Boundary. In _Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges_ (Los Cabos, Mexico) _(HotEdgeVideo’19)_. Association for Computing Machinery, New York, NY, USA, 27–32.  https://doi.org/10.1145/3349614.3356023
  * Everingham et al. (2010) Mark Everingham, Luc Gool, Christopher K. Williams, John Winn, and Andrew Zisserman. 2010\.  The Pascal Visual Object Classes (VOC) Challenge.  _Int. J. Comput. Vision_ 88, 2 (June 2010), 303–338.  https://doi.org/10.1007/s11263-009-0275-4
  * Fouladi et al. (2018) Sadjad Fouladi, John Emmons, Emre Orbay, Catherine Wu, Riad S. Wahby, and Keith Winstein. 2018\.  Salsify: Low-Latency Network Video through Tighter Integration between a Video Codec and a Transport Protocol. In _Proceedings of the 15th USENIX Conference on Networked Systems Design and Implementation_ (Renton, WA, USA) _(NSDI’18)_. USENIX Association, USA, 267–282.
  * Ghodgaonkar et al. (2020) Isha Ghodgaonkar, Subhankar Chakraborty, Vishnu Banna, Shane Allcroft, Mohammed Metwaly, Fischer Bordwell, Kohsuke Kimura, Xinxin Zhao, Abhinav Goel, Caleb Tung, et al. 2020\.  Analyzing Worldwide Social Distancing through Large-Scale Computer Vision.  _arXiv preprint arXiv:2008.12363_ (2020).
  * Grand View Research ([n.d.]) Grand View Research. [n.d.].  Global Sports Analytics Market Size Report, 2021-2028.  https://www.grandviewresearch.com/industry-analysis/sports-analytics-market.
  * Guo et al. (2018) Peizhen Guo, Bo Hu, Rui Li, and Wenjun Hu. 2018\.  FoggyCache: Cross-Device Approximate Computation Reuse. In _Proceedings of the 24th Annual International Conference on Mobile Computing and Networking_ (New Delhi, India) _(MobiCom ’18)_. Association for Computing Machinery, New York, NY, USA, 19–34.  https://doi.org/10.1145/3241539.3241557
  * Guo and Hu (2018) Peizhen Guo and Wenjun Hu. 2018.  Potluck: Cross-Application Approximate Deduplication for Computation-Intensive Mobile Applications.  _SIGPLAN Not._ 53, 2 (mar 2018), 271–284.  https://doi.org/10.1145/3296957.3173185
  * Han et al. (2016) Seungyeop Han, Haichen Shen, Matthai Philipose, Sharad Agarwal, Alec Wolman, and Arvind Krishnamurthy. 2016\.  MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints. In _Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services_ (Singapore, Singapore) _(MobiSys ’16)_. Association for Computing Machinery, New York, NY, USA, 123–136.  https://doi.org/10.1145/2906388.2906396
  * He et al. (2017) Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. 2017.  Mask R-CNN.  _CoRR_ abs/1703.06870 (2017).  arXiv:1703.06870 http://arxiv.org/abs/1703.06870
  * He et al. (2020) Songtao He, Favyen Bastani, Arjun Balasingam, Karthik Gopalakrishna, Ziwen Jiang, Mohammad Alizadeh, Hari Balakrishnan, Michael Cafarella, Tim Kraska, and Sam Madden. 2020.  BeeCluster: Drone Orchestration via Predictive Optimization. In _Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services_ (Toronto, Ontario, Canada) _(MobiSys ’20)_. Association for Computing Machinery, New York, NY, USA, 299–311.  https://doi.org/10.1145/3386901.3388912
  * Held and Karp (1970) Michael Held and Richard M Karp. 1970.  The traveling-salesman problem and minimum spanning trees.  _Operations Research_ 18, 6 (1970), 1138–1162.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015\.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ (2015).
  * Honey Optics ([n.d.]) Honey Optics. [n.d.].  How much do PTZ cameras cost?  https://honeyoptics.com/how-much-do-ptz-cameras-cost/.
  * Huang et al. (2017) Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, et al. 2017\.  Speed/accuracy trade-offs for modern convolutional object detectors. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 7310–7311.
  * HuddleCamHD ([n.d.]) HuddleCamHD. [n.d.].  Understanding the difference between EPTZ and PTZ .  https://huddlecamhd.com/eptz-and-ptz/.
  * Jacob et al. (2018) Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018\.  Quantization and training of neural networks for efficient integer-arithmetic-only inference. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 2704–2713.
  * Jain et al. (2020) Samvit Jain, Xun Zhang, Yuhao Zhou, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Victor Bahl, and Joseph Gonzalez. 2020\.  Spatula: Efficient cross-camera video analytics on large camera networks. In _ACM/IEEE Symposium on Edge Computing (SEC 2020)_.
  * Jha et al. (2021) Sagar Jha, Youjie Li, Shadi Noghabi, Vaishnavi Ranganathan, Peeyush Kumar, Andrew Nelson, Michael Toelle, Sudipta Sinha, Ranveer Chandra, and Anirudh Badam. 2021\.  Visage: Enabling Timely Analytics for Drone Imagery. In _Proceedings of the 27th Annual International Conference on Mobile Computing and Networking_ (New Orleans, Louisiana) _(MobiCom ’21)_. Association for Computing Machinery, New York, NY, USA, 789–803.  https://doi.org/10.1145/3447993.3483273
  * Jiang et al. (2018c) Angela H. Jiang, Daniel L.-K. Wong, Christopher Canel, Lilia Tang, Ishan Misra, Michael Kaminsky, Michael A. Kozuch, Padmanabhan Pillai, David G. Andersen, and Gregory R. Ganger. 2018c.  Mainstream: Dynamic Stem-Sharing for Multi-Tenant Video Processing. In _2018 USENIX Annual Technical Conference (USENIX ATC 18)_. USENIX Association, Boston, MA, 29–42.  https://www.usenix.org/conference/atc18/presentation/jiang
  * Jiang et al. (2018a) Junchen Jiang, Ganesh Ananthanarayanan, Peter Bodik, Siddhartha Sen, and Ion Stoica. 2018a.  Chameleon: Scalable Adaptation of Video Analytics. In _Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication_ (Budapest, Hungary) _(SIGCOMM ’18)_. ACM, New York, NY, USA, 253–266.  https://doi.org/10.1145/3230543.3230574
  * Jiang et al. (2018b) Junchen Jiang, Ganesh Ananthanarayanan, Peter Bodik, Siddhartha Sen, and Ion Stoica. 2018b.  Chameleon: Scalable Adaptation of Video Analytics. In _Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication_ (Budapest, Hungary) _(SIGCOMM ’18)_. Association for Computing Machinery, New York, NY, USA, 253–266.  https://doi.org/10.1145/3230543.3230574
  * Kang et al. (2019) Daniel Kang, Peter Bailis, and Matei Zaharia. 2019\.  BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics.  _Proc. VLDB Endow._ 13, 4 (Dec. 2019), 533–546.  https://doi.org/10.14778/3372716.3372725
  * Kang et al. (2017) Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017.  NoScope: Optimizing Neural Network Queries over Video at Scale.  _Proc. VLDB Endow._ 10, 11 (Aug. 2017), 1586–1597.  https://doi.org/10.14778/3137628.3137664
  * Kemker et al. (2018) Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. 2018.  Measuring Catastrophic Forgetting in Neural Networks.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 32, 1 (Apr. 2018).  https://doi.org/10.1609/aaai.v32i1.11651
  * Khosla et al. (2012) Aditya Khosla, Tinghui Zhou, Tomasz Malisiewicz, Alexei A Efros, and Antonio Torralba. 2012\.  Undoing the damage of dataset bias. In _Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part I 12_. Springer, 158–171.
  * Kshitija Taywade (2022) Judy Goldsmith Kshitija Taywade, Brent Harrison. 2022.  USING NON-STATIONARY BANDITS FOR LEARNING IN REPEATED COURNOT GAMES WITH NON-STATIONARY DEMAND.  _arXiv preprint arXiv:2201.00486_ (2022).
  * Kumar et al. (2019) Adarsh Kumar, Arjun Balasubramanian, Shivaram Venkataraman, and Aditya Akella. 2019.  Accelerating Deep Learning Inference via Freezing. In _11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 19)_. USENIX Association, Renton, WA.  https://www.usenix.org/conference/hotcloud19/presentation/kumar
  * Li et al. (2015) H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua. 2015.  A convolutional neural network cascade for face detection. In _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 5325–5334.
  * Li et al. (2020) Yuanqi Li, Arthi Padmanabhan, Pengzhan Zhao, Yufei Wang, Guoqing Harry Xu, and Ravi Netravali. 2020\.  Reducto: On-Camera Filtering for Resource-Efficient Real-Time Video Analytics. In _Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication_ (Virtual Event, USA) _(SIGCOMM ’20)_. Association for Computing Machinery, New York, NY, USA, 359–376.  https://doi.org/10.1145/3387514.3405874
  * Li et al. (2021) Zhuqi Li, Yuanchao Shu, Ganesh Ananthanarayanan, Longfei Shangguan, Kyle Jamieson, and Victor Bahl. 2021.  Spider: A Multi-Hop Millimeter-Wave Network for Live Video Analytics. In _ACM/IEEE Symposium on Edge Computing_. ACM/IEEE.  https://www.microsoft.com/en-us/research/publication/spider-a-multi-hop-millimeter-wave-network-for-live-video-analytics/
  * Lin et al. (2014) Michael Lin, Tsung-Yiand Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014.  Microsoft COCO: Common Objects in Context. In _Computer Vision – ECCV 2014_. 740–755.
  * Lin et al. (2017) T. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie. 2017\.  Feature Pyramid Networks for Object Detection. In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 936–944.  https://doi.org/10.1109/CVPR.2017.106
  * Liu et al. (2016) Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. 2016.  SSD: Single Shot MultiBox Detector. In _Computer Vision – ECCV 2016_. Springer International Publishing, 21–37.
  * Liu et al. (2017) Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. 2017\.  Learning efficient convolutional networks through network slimming. In _Proceedings of the IEEE International Conference on Computer Vision_. 2736–2744.
  * Lowe (2004) David G Lowe. 2004\.  Distinctive image features from scale-invariant keypoints.  _International journal of computer vision_ 60 (2004).
  * Lu et al. (2016) Yao Lu, Aakanksha Chowdhery, and Srikanth Kandula. 2016\.  Optasia: A Relational Platform for Efficient Large-Scale Video Analytics. In _Proceedings of the Seventh ACM Symposium on Cloud Computing_ (Santa Clara, CA, USA) _(SoCC ’16)_. ACM, New York, NY, USA, 57–70.  https://doi.org/10.1145/2987550.2987564
  * Mehrdad Khani, Ganesh Ananthanarayanan, Kevin Hsieh, Junchen Jiang, Ravi Netravali , Yuanchao Shu, Mohammad Alizadeh , Victor Bahl (2023) Mehrdad Khani, Ganesh Ananthanarayanan, Kevin Hsieh, Junchen Jiang, Ravi Netravali , Yuanchao Shu, Mohammad Alizadeh , Victor Bahl. 2023.  RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics. In _20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23)_. USENIX Association, Boston, MA.
  * Microsoft Azure (2021) Microsoft Azure. 2021\.  Azure Stack Edge.  https://azure.microsoft.com/en-us/services/databox/edge/
  * Netravali et al. (2015) R. Netravali, A. Sivaraman, K. Winstein, S. Das, A. Goyal, J. Mickens, and H. Balakrishnan. 2015.  Mahimahi: Accurate Record-and-Replay for HTTP _(Proceedings of ATC ’15)_. USENIX.
  * NVIDIA (2021) NVIDIA. 2021.  NVIDIA JetsonNano.  https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-nano/product-development/
  * Oksuz et al. (2020) Kemal Oksuz, Baris Can Cam, Sinan Kalkan, and Emre Akbas. 2020\.  Imbalance problems in object detection: A review.  _IEEE transactions on pattern analysis and machine intelligence_ 43, 10 (2020).
  * Omar Besbes, Yonatan Gur, Assaf Zeevi (2014) Omar Besbes, Yonatan Gur, Assaf Zeevi. 2014\.  Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards . In _NeurIPS_.
  * Padmanabhan et al. (2022) Arthi Padmanabhan, Neil Agarwal, Anand Iyer, Ganesh Ananthanarayanan, Yuanchao Shu, Nikolaos Karianakis, Guoqing Harry Xu, and Ravi Netravali. 2022.  GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge.  https://doi.org/10.48550/ARXIV.2201.07705
  * Pan and Lyu (2010) Xunyu Pan and Siwei Lyu. 2010.  Detecting image region duplication using SIFT features. In _2010 IEEE International Conference on Acoustics, Speech and Signal Processing_. IEEE, 1706–1709.
  * Paul et al. (2021) Sibendu Paul, Utsav Drolia, Y. Charlie Hu, and Srimat T. Chakradhar. 2021.  AQuA: Analytical Quality Assessment for Optimizing Video Analytics Systems.  https://doi.org/10.48550/ARXIV.2101.09752
  * Paul et al. (2023) Sibendu Paul, Kunal Rao, Giuseppe Coviello, Murugan Sankaradas, Oliver Po, Y. Charlie Hu, and Srimat Chakradhar. 2023.  Enhancing Video Analytics Accuracy via Real-Time Automated Camera Parameter Tuning. In _Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems_ (Boston, Massachusetts) _(SenSys ’22)_. Association for Computing Machinery, New York, NY, USA, 291–304.  https://doi.org/10.1145/3560905.3568527
  * PTZ Optics ([n.d.]a) PTZ Optics. [n.d.]a.  PTZ Optics PTZ cameras.  https://ptzoptics.com/products/.
  * PTZ Optics ([n.d.]b) PTZ Optics. [n.d.]b.  What is ePTZ and how does it compare with true PTZ?  https://ptzoptics.com/what-is-eptz/.
  * Redmon and Farhadi (2018) Joseph Redmon and Ali Farhadi. 2018.  Yolov3: An incremental improvement.  _arXiv preprint arXiv:1804.02767_ (2018).
  * Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, Oliver Wang (2018) Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, Oliver Wang. 2018.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In _CVPR_.
  * Rijas ([n.d.]) Mohammed Rijas. [n.d.].  Powering The Edge With AI In An IoT World.  https://www.forbes.com/sites/forbestechcouncil/2020/04/06/powering-the-edge-with-ai-in-an-iot-world/.
  * Rizzoli (2022) Alberto Rizzoli. 2022\.  7 Game-Changing AI Applications in the Sports Industry.  https://www.v7labs.com/blog/ai-in-sports.
  * Romero et al. (2021) Francisco Romero, Qian Li, Neeraja J. Yadwadkar, and Christos Kozyrakis. 2021.  INFaaS: Automated Model-less Inference Serving. In _2021 USENIX Annual Technical Conference (USENIX ATC 21)_. USENIX Association, 397–411.  https://www.usenix.org/conference/atc21/presentation/romero
  * SCW ([n.d.]) SCW. [n.d.].  PTZ Auto-Tracking Explained.  https://www.getscw.com/knowledge-base/auto-tracking-explained.
  * SecurityBros ([n.d.]) SecurityBros. [n.d.].  Which Cheap Outdoor WiFi PTZ IP Camera is Best? Boavision vs Inqmega.  https://securitybros.com/which-cheap-outdoor-wifi-ptz-ip-camera-is-best-boavision-vs-inqmega/.
  * Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (2015) Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun. 2015.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In _NeurIPS_.
  * Shen et al. (2019) Haichen Shen, Lequn Chen, Yuchen Jin, Liangyu Zhao, Bingyu Kong, Matthai Philipose, Arvind Krishnamurthy, and Ravi Sundaram. 2019.  Nexus: A GPU Cluster Engine for Accelerating DNN-Based Video Analysis. In _Proceedings of the 27th ACM Symposium on Operating Systems Principles_ (Huntsville, Ontario, Canada) _(SOSP ’19)_. Association for Computing Machinery, New York, NY, USA, 322–337.  https://doi.org/10.1145/3341301.3359658
  * Shi et al. (2016) Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016.  Edge computing: Vision and challenges.  _IEEE internet of things journal_ 3, 5 (2016), 637–646.
  * Shubham Jain, Viet Nguyen, Marco Gruteser, Paramvir Bahl (2017) Shubham Jain, Viet Nguyen, Marco Gruteser, Paramvir Bahl. 2017.  Panoptes: Servicing Multiple Applications Simultaneously using Steerable Cameras. In _IPSN_.
  * Sindagi and Patel (2017) Vishwanath A Sindagi and Vishal M Patel. 2017.  Generating high-quality crowd density maps using contextual pyramid cnns. In _Proceedings of the IEEE international conference on computer vision_. 1861–1870.
  * Steed and Caliskan (2021) Ryan Steed and Aylin Caliskan. 2021.  Image representations learned with unsupervised pre-training contain human-like biases. In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_. 701–713.
  * Suprem et al. (2020) Abhijit Suprem, Joy Arulraj, Calton Pu, and Joao Ferreira. 2020.  ODIN: Automated Drift Detection and Recovery in Video Analytics.  _Proc. VLDB Endow._ 13, 12 (July 2020), 2453–2465.  https://doi.org/10.14778/3407790.3407837
  * Tan et al. (2019) Mingxing Tan, Ruoming Pang, and Quoc V. Le. 2019.  EfficientDet: Scalable and Efficient Object Detection.  _CoRR_ abs/1911.09070 (2019).  arXiv:1911.09070 http://arxiv.org/abs/1911.09070
  * Tversky and Gati (1982) Amos Tversky and Itamar Gati. 1982.  Similarity, separability, and the triangle inequality.  _Psychological review_ 89, 2 (1982), 123.
  * Vasisht et al. (2017) Deepak Vasisht, Zerina Kapetanovic, Jong-ho Won, Xinxin Jin, Ranveer Chandra, Ashish Kapoor, Sudipta N. Sinha, Madhusudhan Sudarshan, and Sean Stratman. 2017.  Farmbeats: An IoT Platform for Data-Driven Agriculture. In _Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation_ (Boston, MA, USA) _(NSDI’17)_. USENIX Association, USA, 515–528.
  * Vidal (2021) Lluís Martínez; Margarita Cabrera-Bean; Josep Vidal. 2021.  A Multi-Armed Bandit Model for Non-Stationary Wireless Network Selection. In _2021 IEEE Globecom Workshops_. Ieee.
  * Vidit Saxena, Joakim Jaldén, Joseph E. Gonzalez, Mats Bengtsson, Hugo Tullberg, Ion Stoica (2019) Vidit Saxena, Joakim Jaldén, Joseph E. Gonzalez, Mats Bengtsson, Hugo Tullberg, Ion Stoica. 2019\.  Contextual Multi-Armed Bandits for Link Adaptation in Cellular Networks. In _NetAI_.
  * Wang et al. (2021) Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2021\.  Scaled-YOLOv4: Scaling Cross Stage Partial Network.  arXiv:2011.08036 [cs.CV]
  * Wang et al. (2019a) Junjue Wang, Ziqiang Feng, Shilpa George, Roger Iyengar, Padmanabhan Pillai, and Mahadev Satyanarayanan. 2019a.  Towards Scalable Edge-Native Applications. In _Proceedings of the 4th ACM/IEEE Symposium on Edge Computing_ (Arlington, Virginia) _(SEC ’19)_. Association for Computing Machinery, New York, NY, USA, 152–165.  https://doi.org/10.1145/3318216.3363308
  * Wang et al. (2019b) Yiding Wang, Weiyan Wang, Junxue Zhang, Junchen Jiang, and Kai Chen. 2019b.  Bridging the Edge-Cloud Barrier for Real-time Advanced Vision Analytics. In _11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 19)_. USENIX Association, Renton, WA.  https://www.usenix.org/conference/hotcloud19/presentation/wang
  * Wang et al. (2020) Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky. 2020.  Towards fairness in visual recognition: Effective strategies for bias mitigation. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 8919–8928.
  * Xu et al. (2018) Mengwei Xu, Mengze Zhu, Yunxin Liu, Felix Xiaozhu Lin, and Xuanzhe Liu. 2018.  DeepCache: Principled Cache for Mobile Deep Vision. In _Proceedings of the 24th Annual International Conference on Mobile Computing and Networking_ (New Delhi, India) _(MobiCom ’18)_. Association for Computing Machinery, New York, NY, USA, 129–144.  https://doi.org/10.1145/3241539.3241563
  * Yang et al. (2020) Yifan Yang, Guorong Li, Dawei Du, Qingming Huang, and Nicu Sebe. 2020.  Embedding perspective analysis into multi-column convolutional neural network for crowd counting.  _IEEE Transactions on Image Processing_ 30 (2020), 1395–1407.
  * Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Zehuan Yuan, Ping Luo, Wenyu Liu, Xinggang Wang (2021) Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Zehuan Yuan, Ping Luo, Wenyu Liu, Xinggang Wang. 2021\.  ByteTrack: Multi-Object Tracking by Associating Every Detection Box.  arXiv:2110.06864 [cs.CV]
  * Yin et al. (2015) Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015.  A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP.  _SIGCOMM Comput. Commun. Rev._ 45, 4 (aug 2015), 325–338.  https://doi.org/10.1145/2829988.2787486
  * Yosinski et al. (2014) Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014\.  How transferable are features in deep neural networks?. In _Advances in Neural Information Processing Systems_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc.  https://proceedings.neurips.cc/paper/2014/file/375c71349b295fbe2dcdca9206f20a06-Paper.pdf
  * Zeiler and Fergus (2014) Matthew D Zeiler and Rob Fergus. 2014.  Visualizing and understanding convolutional networks. In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13_. Springer, 818–833.
  * Zhang et al. (2019) Anran Zhang, Lei Yue, Jiayi Shen, Fan Zhu, Xiantong Zhen, Xianbin Cao, and Ling Shao. 2019.  Attentional neural fields for crowd counting. In _Proceedings of the IEEE/CVF international conference on computer vision_. 5714–5723.
  * Zhang et al. (2017a) Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J. Freedman. 2017a.  Live Video Analytics at Scale with Approximation and Delay-tolerance. In _Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation_ (Boston, MA, USA) _(NSDI’17)_. USENIX Association, Berkeley, CA, USA, 377–392.  http://dl.acm.org/citation.cfm?id=3154630.3154661
  * Zhang et al. (2017b) Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, and Michael J. Freedman. 2017b.  Live Video Analytics at Scale with Approximation and Delay-Tolerance. In _14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17)_. USENIX Association, Boston, MA, 377–392.  https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/zhang
  * Zhang et al. (2015) Tan Zhang, Aakanksha Chowdhery, Paramvir Bahl, Kyle Jamieson, and Suman Banerjee. 2015.  The Design and Implementation of a Wireless Video Surveillance System. 426–438.  https://doi.org/10.1145/2789168.2790123
  * Zhao et al. (2019) Kun Zhao, Bin Liu, Luchuan Song, Weihai Li, and Nenghai Yu. 2019.  Cascaded residual density network for crowd counting. In _2019 IEEE International Conference on Image Processing (ICIP)_. IEEE, 2199–2203.
  * Zhu et al. (2016) Chenzhuo Zhu, Song Han, Huizi Mao, and William J Dally. 2016\.  Trained ternary quantization.  _arXiv preprint arXiv:1612.01064_ (2016).
  * Zhu et al. (2018) Yuhao Zhu, Anand Samajdar, Matthew Mattina, and Paul Whatmough. 2018.  Euphrates: Algorithm-SoC Co-Design for Low-Power Mobile Continuous Vision. In _Proceedings of the 45th Annual International Symposium on Computer Architecture_ (Los Angeles, California) _(ISCA ’18)_. IEEE Press, 547–560.  https://doi.org/10.1109/ISCA.2018.00052

##  Appendix A Appendix

###  A.1. Workloads

Model | Object | Type
---|---|---
SSD | people | aggregate count
Faster RCNN | cars | binary classification
SSD | people | count
YOLOv4 | people | detection
Faster RCNN | people | detection
Table 3. Workload 1 (W1) Model | Object | Type
---|---|---
YOLOv4 | people | aggregate count
Tiny YOLOv4 | people | aggregate count
Tiny YOLOv4 | people | detection
YOLOv4 | people | binary classification
Tiny YOLOv4 | people | aggregate count
Faster RCNN | people | count
Faster RCNN | people | detection
Faster RCNN | car | count
YOLOv4 | people | aggregate count
YOLOv4 | people | detection
YOLOv4 | people | count
Tiny YOLOv4 | people | aggregate count
YOLOv4 | car | count
YOLOv4 | car | detection
Tiny YOLOv4 | car | count
SSD | person | binary classification
Faster RCNN | car | count
SSD | car | count
Table 4. Workload 2 (W2) Model | Object | Type
---|---|---
SSD | car | binary classification
Faster RCNN | people | aggregate count
Faster RCNN | people | count
Tiny YOLOv4 | people | binary classification
Tiny YOLOv4 | people | binary classification
Tiny YOLOv4 | people | aggregate count
YOLOv4 | people | count
Faster RCNN | people | aggregate count
SSD | people | binary classification
Faster RCNN | car | count
SSD | car | count
Table 5. Workload 3 (W3) Model | Object | Type
---|---|---
Tiny YOLOv4 | car | count
Faster RCNN | car | detection
Faster RCNN | people | aggregate count
Table 6. Workload 4 (W4) Model | Object | Type
---|---|---
Tiny YOLOv4 | car | count
SSD | car | count
Faster RCNN | people | aggregate count
Table 7. Workload 5 (W5) Model | Object | Type
---|---|---
Tiny YOLOv4 | people | aggregate count
Tiny YOLOv4 | people | binary classification
SSD | car | count
YOLOv4 | people | aggregate count
Tiny YOLOv4 | people | count
Faster RCNN | car | binary classification
SSD | people | detection
Faster RCNN | car | detection
Faster RCNN | people | aggregate count
YOLOv4 | car | count
Tiny YOLOv4 | people | aggregate count
Faster RCNN | people | detection
SSD | people | aggregate count
YOLOv4 | car | detection
Table 8. Workload 6 (W6) Model | Object | Type
---|---|---
YOLOv4 | people | binary classification
SSD | people | detection
Tiny YOLOv4 | car | binary classification
Tiny YOLOv4 | people | detection
SSD | people | binary classification
SSD | people | aggregate count
Tiny YOLOv4 | people | detection
SSD | car | count
SSD | people | count
Faster RCNN | people | count
YOLOv4 | people | count
Faster RCNN | people | binary classification
Tiny YOLOv4 | people | aggregate count
Faster RCNN | people | aggregate count
Faster RCNN | car | count
YOLOv4 | car | binary classification
Table 9. Workload 7 (W7) Model | Object | Type
---|---|---
Faster RCNN | car | count
Tiny YOLOv4 | people | binary classification
YOLOv4 | people | aggregate count
YOLOv4 | car | count
Tiny YOLOv4 | people | aggregate count
Faster RCNN | people | aggregate count
YOLOv4 | people | aggregate count
Faster RCNN | car | count
SSD | car | count
Faster RCNN | car | count
SSD | car | binary classification
YOLOv4 | car | binary classification
SSD | car | binary classification
SSD | people | count
YOLOv4 | people | count
YOLOv4 | car | binary classification
Faster RCNN | person | aggregate count
SSD | car | detection
Table 10. Workload 8 (W8) Model | Object | Type
---|---|---
Tiny YOLOv4 | people | aggregate count
Faster RCNN | people | count
Faster RCNN | people | count
Tiny YOLOv4 | car | detection
Tiny YOLOv4 | people | binary clasification
YOLOv4 | people | detection
Faster RCNN | people | count
YOLOv4 | people | aggregate count
SSD | people | aggregate count
Table 11. Workload 9 (W9) Model | Object | Type
---|---|---
Faster RCNN | people | aggregate count
Faster RCNN | car | count
Faster RCNN | people | count
Table 12. Workload 10 (W10)