# Sampling Training Data for Continual Learning Between Robots and the Cloud

Sandeep Chinchali1∗, Evgenya Pergament1∗, Manabu Nakanoya2, Eyal Cidon1,
Edward Zhang3, Dinesh Bharadia4,
Marco Pavone1, and Sachin Katti1 * Co-Primary Authors1 Depts. of CS, EE, and
Aero-Astro, Stanford University, Stanford, USA {csandeep, evgenyap, mnakanoya,
ecidon, pavone<EMAIL_ADDRESS>NEC Corporation, Tokyo, Japan
nakanoya<EMAIL_ADDRESS>Dept. of ECE, UC San Diego, La
Jolla, USA<EMAIL_ADDRESS>
###### Abstract

Today’s robotic fleets are increasingly measuring high-volume video and LIDAR
sensory streams, which can be mined for valuable training data, such as rare
scenes of road construction sites, to steadily improve robotic perception
models. However, re-training perception models on growing volumes of rich
sensory data in central compute servers (or the “cloud”) places an enormous
time and cost burden on network transfer, cloud storage, human annotation, and
cloud computing resources. Hence, we introduce HarvestNet, an intelligent
sampling algorithm that resides on-board a robot and reduces system
bottlenecks by only storing rare, useful events to steadily improve perception
models re-trained in the cloud. HarvestNet significantly improves the accuracy
of machine-learning models on our novel dataset of road construction sites,
field testing of self-driving cars, and streaming face recognition, while
reducing cloud storage, dataset annotation time, and cloud compute time by
between $65.7-81.3\%$. Further, it is between $1.05-2.58\times$ more accurate
than baseline algorithms and scalably runs on embedded deep learning hardware.

##  I Introduction

Learning to identify rare events, such as traffic disruptions due to
construction or hazardous weather conditions, can significantly improve the
safety of robotic perception and decision-making models. While performing
their primary tasks, fleets of future robots, ranging from delivery drones to
self-driving cars, can passively collect rare training data they happen to
observe from their high-volume video and LIDAR sensory streams. As such, we
envision that robotic fleets will act as distributed, passive sensors that
collect valuable training data to steadily improve the robotic autonomy stack.

Despite the benefits of continually improving robotic models from field data,
they come with severe systems costs that are largely under-appreciated in
today’s robotics literature. Specifically, these systems bottlenecks (shown in
Fig. 1) stem from the time or cost of (1) network data transfer to compute
servers, (2) limited robot and cloud storage, (3) dataset annotation with
human supervision, and (4) cloud computing for training models on massive
datasets. Indeed, it is estimated that a single self-driving car will produce
upwards of 4 Terabytes (TB) of LIDAR and video data per day [1, 9], which we
contrast with our own calculations in Section II.

In this paper, we introduce a novel Robot Sensory Sampling Problem (Fig. 1),
which asks how a robot can maximally improve model accuracy via cloud re-
training, but with minimal end-to-end systems costs. To address this problem,
we introduce HarvestNet, a compute-efficient sampler that scalably runs on
even resource-constrained robots and reduces systems bottlenecks by
“harvesting” only task-relevant training examples for cloud processing.

Figure 1: The Robot Sensory Sampling Problem. What are the minimal set of
“interesting” training samples robots can send to the cloud to periodically
improve perception model accuracy while minimizing system costs of network
transfer, cloud storage, human annotation, and cloud-computing?

In general, the problem of intelligently sampling useful or anomalous training
data is of extremely wide scope, since field robotic data contains both known
weakpoints of a model as well as unknown weakpoints that have yet to even be
discovered. A concrete example of a known weakpoint is shown in Fig. 2, where
a fleet operator identifies that construction sites are common errors of a
perception model and wishes to collect more of such images, perhaps because
they are insufficiently represented in a training dataset. In contrast,
unknown weakpoints could be rare visual concepts that are not even anticipated
by a roboticist.

To provide a focused contribution, this paper addresses the problem of how
robotic fleets can source more training data for known weakpoints to maximize
model accuracy, but with minimal end-to-end systems costs. Though of tractable
scope, this problem is still extremely challenging since a robot must harvest
rare, task-relevant training examples from growing volumes of robotic sensory
data. A principal benefit of addressing known model weakpoints is that the
cloud can directly use knowledge of task-relevant image classes, which require
more training data, to guide and improve a robot’s sampling process. Our key
technical insight is to keep robot sampling logic simple, but to leverage the
power of the cloud to continually re-train models and annotate uncertain
images with a human-in-the-loop, thereby using cloud feedback to direct
robotic sampling. Moreover, we discuss how insights from HarvestNet can be
extended to collecting training data for unknown weakpoints in future work.

Literature Review: HarvestNet is closely aligned with cloud robotics [8, 18,
17] , where robots send video or LIDAR to the cloud to query compute-intensive
models or databases when they are uncertain. Robots can face significant
network congestion while transmitting high-datarate sensory streams to the
cloud, leading to work that selectively offloads only uncertain samples [13]
or uses low-latency “fog computing” nodes for object manipulation and tele-
operation [27, 28]. As opposed to focusing solely on network latency for real-
time control, HarvestNet addresses overall system cost (from network transfer
to human annotation time) and instead focuses on the open problem of continual
learning from cleverly-sampled batches of field data.

Our work is inspired by traditional active learning [29, 24, 14, 16], novelty
detection [20, 12], and few-shot learning [32, 30, 22], but differs
significantly since we use novel compute-efficient algorithms, distributed
between the robot and cloud, that are scalable to run on even resource-
constrained robots. Many active learning algorithms [29, 24, 14] use compute-
intensive acquisition functions to decide which limited samples a learner
should label to maximize model accuracy, which often involve Monte Carlo
Sampling to estimate uncertainty over a new image. Indeed, the most similar
work to ours [16] uses Bayesian Deep Neural Networks (DNNs), which have a
distribution over model parameters, to take several stochastic forward passes
of a neural network to empirically quantify uncertainty and decide whether an
image should be labeled. Such methods are infeasible for resource-constrained
robots, which might not have the time to take several forward passes of a DNN
for a single image or capability to efficiently run a distribution over model
parameters. Indeed, we show in Section V how our sampler uses a hardware DNN
accelerator, specifically the Google Edge Tensor Processing Unit (TPU), to
evaluate an on-board vision DNN, with one fixed set of pre-compiled model
parameters, in $<16$ms and decides to sample an image in only $\approx 0.3$ms!
More broadly, our work differs since we (1) address sensory streams (as
opposed to a pool of unlabeled data), (2) use a compute-efficient model to
guide sampling, (3) direct sampling with task-relevant cloud feedback, and (4)
also address storage, network, and computing costs.

Statement of Contributions: In light of prior work, our contributions are
four-fold. First, we collect months of novel video footage to quantify the
systems bottlenecks of processing growing sensory data. Second, we empirically
show that large improvements in perception model accuracy can be realized by
automatically sampling task-relevant training examples from vast sensory
streams. Third, we design a sampler that intelligently delegates compute-
intensive tasks to the cloud, such as adaptively tuning key parameters of
robot sampling policies via cloud feedback. Finally, we show that our
perception models run efficiently on the state-of-the-art Google Edge TPU [7]
for resource-constrained robots.

Organization: This paper is organized as follows. Section II motivates the
Robot Sensory Sampling Problem by quantifying our system costs for collecting
three novel perception datasets. Then, Section III formulates a general
sampling problem and introduces the HarvestNet system architecture (Fig. 3).
Next, Sections IV and V quantify HarvestNet’s accuracy improvements for
perception models, key reductions in system bottlenecks, and performance on
the Edge TPU [7]. Finally, we conclude with future applications in Section VI.

Figure 2: HarvestNet Re-Trained Vision Models. From only a minimal set of
training examples, HarvestNet adapts default vision models trained on the
standard COCO dataset [19] (top images) to better, domain-specific models
(bottom images). We flexibly fine-tune mobile-optimized models, such as
MobileNet 2, as well as compute-intensive, but more accurate models such as
Faster R-CNN [21].

##  II Motivation

We now describe the costs and benefits of continual learning in the cloud by
collecting and processing three novel perception datasets.

###  II-A Costs and Benefits of Continual Learning in the Cloud

#### Benefits

A prime benefit of continual learning in the cloud is to correct weak points
of existing models or specialize them to new domains. As shown in Fig. 2
(right, top), standard, publicly-available vision deep neural networks (DNNs)
pre-trained on benchmark datasets like MS-COCO [19] can misclassify traffic
cones as humans but also miss the large excavator vehicle, which neither looks
like a standard truck nor a car. However, as shown in Fig. 2 (right, bottom),
a specialized model re-trained in the cloud using automatically sampled, task-
relevant field data can correct such errors. More generally, field robots that
“crowd-source” interesting or anomalous training data could automatically
update high-definition (HD) road maps.

#### Costs

The above benefits of cloud re-training, however, often come with understated
costs, which we now quantify. Table II estimates monthly systems costs for a
small fleet of 10 self-driving cars, each of which is assumed to capture 4
hours of driving data per day. In the first scenario, called “Dashcam-ours”,
we assume a single car has only one dashcam capturing H.264 compressed video,
as in our collected dataset. In the second scenario, called “Multi-Sensor”, we
base our calculations off of an Intel report [1] that estimates each SDV will
generate 4 TB of video and LIDAR data in just 1.5 hours.

To be conservative, we assumed all field data is uploaded over fast 10 Gbps
ethernet, which might be realistic for SDVs, but is much slower for network-
limited robots. Most importantly, we assume only 1% of image data is
interesting and needs to be annotated, specifically by the state-of-the-art
Google Data Labeling service [4], which has standard rates for object
detection bounding boxes and (costlier) segmentation masks (referred to as “B.
Box” and “Mask” in Table II). Further, to roughly estimate annotation time, we
hand-labeled 1000 bounding boxes in 1.4 hours. Since we did not hand-label nor
train perception models for semantic segmentation, the time estimates are
marked as “NA”. Though conservative, our estimates, which are further detailed
in supplementary information, show significant costs, especially for
annotation. Indeed, we confirmed the sampling problem’s relevance by talking
with robotics startups that face high systems costs.

Bottleneck | Multi-Sensor | Dashcam-ours
---|---|---
B. Box | Mask | B. Box | Mask
Storage | $31,200 | $31,200 | $172.50 | $172.50
Transfer Time (hr) | 8.88 | 8.88 | 0.05 | 0.05
Annotation Cost | $95,256 | $1,652,400 | $15,876 | $275,400
Annot. Time (days) | 22 | NA | 18 | NA
TABLE I: Systems Costs: Monthly storage/annotation cost and daily network
upload time are significant for self-driving car data even if we annotate 1%
of video frames.

###  II-B Novel Datasets we Collected

To quantify systems costs first-hand, we collected over three months of field
data, comprising over 10 GB of compressed video clips from a dashboard iPhone
6 camera (“dashcam”). While modest in size, it contains 346,788 frames of
video, which is challenging to automatically mine for training examples. The
following datasets, as well as re-trained vision models, are available at
https://sites.google.com/view/harvestnet:

1\. Identifying Construction Sites: Our three months of construction footage
shows safety-critical scenarios and model drift, including roads that were
first excavated by tractors (Fig. 2), filled with tar, and rolled back to
normal by “compactor” machines and human workers.

2\. Identifying Self-driving Vehicles (SDVs): We captured rare footage of SDVs
being tested on public roads over 3 months, predominantly of the Waymo van
(Fig. 2). This example is a proof-of-concept of HarvestNet’s ability to sift
rare, task-relevant training samples from large volumes of regular car
footage, and is easily generalizable to other image classes.

3\. Streaming Face Recognition (FaceNet): To test HarvestNet’s robustness to
domains outside of road scenes, we use a video dataset of streaming faces of
consenting volunteers from a prior robotics paper [13]. We expressly do not
advocate privacy-infringing surveillance missions, and simply test a search-
and-rescue scenario where a sampler must use face embeddings, such as from the
de-facto FaceNet [23, 11] DNN, to sift faces of interest from large volumes of
video. Our results on all three datasets, henceforth referred to as Waymo,
Construction, and FaceNet, are shown in Section V.

##  III Problem Statement

Figure 3: System Architecture: Our core contribution is a robot sensory
sampler in green, which uses predictions from an on-board vision DNN and
cloud-selected image targets.

We now formulate a general Robot Sensory Sampling Problem that is applicable
to diverse field robots. Examples include self-driving cars, low-power drones,
and even mining robots which might only have network access when they surface.
As a common theme, these robots must prioritize sensory samples due to limits
in some combination of on-board storage, network connectivity, human
annotation, and/or cloud-computing time that preclude naïvely using all field
data. We now define general abstractions for the sampling problem.

1\. Sensory Input: A robot measures a stream of sensory inputs
$\\{x^{t,i}\\}_{t=0}^{T}$, where each $x^{t,i}$ is a new image or LIDAR point
cloud, on a day or learning “round” $i$. At the end of a day’s horizon of $T$
steps, it can upload samples to the cloud for downstream annotation and model
learning. In practice, time horizon $T$ can be the duration of a day’s field
test when a subterranean robot surfaces and connects to a network. Since the
robot is caching images for model-learning, as opposed to real-time control,
it can sub-sample the sensory input at a sustainable rate, such as $\Delta
t=30s$, by skipping frames.

2\. Robot Perception Models: Vision DNN $f_{\mathrm{DNN}}$ maps an image
$x^{t}$ to a prediction $\hat{y}^{t}$ (e.g. image class or object detections)
and associated confidence $\mathtt{conf}^{t}$, typically from softmax layers
in the DNN. Further, the DNN also provides embeddings $\mathtt{emb}^{t}$,
which can be used to compare the similarity between images as shown in Fig. 4
(right). The DNN outputs a vector of predictions, confidences, and embeddings,
defined as:

$\displaystyle\boldsymbol{y}^{t}$
$\displaystyle=[\hat{y}^{t},\mathtt{conf}^{t},\mathtt{emb}^{t}]=f_{\mathrm{DNN}}(x^{t}).$
(1)

We denote the perception DNN learned after round $i$ as
$f_{\mathrm{DNN}}^{i}$, where $f_{\mathrm{DNN}}^{0}$ is the initial DNN the
robot is equipped with before re-training. Our goal is to make
$f_{\mathrm{DNN}}^{i}$ as accurate as possible for increasing $i$ with the
fewest samples sent to the cloud. Fixed model $f_{\mathrm{DNN}}^{i}$ is not
re-trained within a round $i$ of sample collection since the samples must
first be annotated.

3\. Target Image Set: As stated in the introduction, the scope of this paper
is to collect more examples of known weakpoints of a model, such as the mis-
labeled construction sites in Fig. 2. In practice, these weakpoints, which we
refer to as task-relevant image classes, can be identified by a fleet operator
who might nightly review model performance on historical data. The set of
task-relevant image classes, such as traffic cones, excavators, and LIDAR
units (Fig. 2), are denoted by $\mathtt{TargetClassSet}$. Importantly, we
assume the cloud has a few initial examples of images belonging to
$\mathtt{TargetClassSet}$, denoted as $\mathtt{TargetImageSet}$. This is a
practical assumption, since $\mathtt{TargetImageSet}$ can be obtained from a
few examples of mis-labeled field data, like Fig. 2, that initially prompted
the problem, or even collated from Google Images. Equally importantly, the
cloud dataset can already have sufficient training data for well-understood
scenes from previous field tests or public datasets like ImageNet [15], to
ensure model performance does not degrade on such data.

4\. Sampling Policy: Our goal is to develop a lightweight sampling policy
$\pi_{\mathrm{sample}}$ (green unit in Fig. 3) that decides whether or not to
store a new image $x^{t}$ based on predictions from the on-board perception
DNN $\boldsymbol{y}^{t}=f_{\mathrm{DNN}}^{i}(x^{t})$ and the desired target-
set $\mathtt{TargetClassSet}$ of images to look for:

$\displaystyle
a^{t}=\pi_{\mathrm{sample}}(x^{t},\boldsymbol{y}^{t},\mathtt{TargetClassSet},\theta_{\mathrm{sampler}}^{i}),$
(2)

where $\theta_{\mathrm{sampler}}^{i}$ are sampler hyper-parameters that, in
general, can be adapted per round $i$. Here, $a^{t}=1$ indicates storing the
image to later send to the cloud and $a^{t}=0$ indicates dropping it, such as
for common, well-understood scenes. Ideally, to work on resource-constrained
robots as well as not interfere with real-time decision-making,
$\pi_{\mathrm{sample}}$’s logic, including use of hyper-parameters
$\theta_{\mathrm{sampler}}^{i}$, should be compute-efficient.

5\. Limited Robot Sample Cache: In round $i$, the robot saves sampled images
in a small cache denoted by
$\mathtt{Cache}^{i}=\\{x^{\tau}\\}_{\tau=0}^{N_{\mathrm{cache}}}$, where
$0\leq\tau\leq T$. These images, limited by a size $N_{\mathrm{cache}}\ll T$,
are small enough to store on-board a robot and will be uploaded to the cloud
at the end of round $i$.

6\. Annotated Cloud Dataset: When samples in $\mathtt{Cache}^{i}$ are uploaded
to the cloud, a human, or trusted machine, provides ground-truth annotations
$y_{\mathrm{GroundTruth}}^{t}$ for the selected images, which could be
different from the robot’s predictions $\hat{y}^{t}$. In the cloud, we store a
dataset $\mathcal{D}^{i}=\\{x^{\tau},y^{\tau}\\}$ that grows with rounds $i$
and comprises images $x^{\tau}$ and ground-truth labels $y^{\tau}$:

$\displaystyle\mathcal{D}^{i}=\mathcal{D}^{i-1}~{}\cup~{}\\{(x^{\tau},y_{\mathrm{GroundTruth}}^{\tau})~{},~{}\forall
x^{\tau}\in\mathtt{Cache}^{i}\\}.$ (3)

Initial dataset $\mathcal{D}^{0}$ has very few examples of the target images
that we care for, such as self-driving cars or road disruptions. However,
initial dataset $\mathcal{D}^{0}$ can be seeded with a large set of “common”
training examples, such as from ImageNet [15]. In the cloud, we split full
dataset $\mathcal{D}^{i}$ into train, validation, and test components denoted
by $\mathcal{D}_{\mathrm{train}}^{i}$, $\mathcal{D}_{\mathrm{val}}^{i}$, and
$\mathcal{D}_{\mathrm{test}}^{i}$.

7\. Independent Test Dataset $\mathcal{D}_{\mathrm{test,~{}final}}$: Model
accuracy is always reported on an independent, held-out test set denoted
$\mathcal{D}_{\mathrm{test,~{}final}}$. To accurately assess the efficacy of a
sampler in accruing task-relevant data that yields steady performance gains on
the target classes, $\mathcal{D}_{\mathrm{test,~{}final}}$ should have
sufficient representation, or even be biased towards, target classes. The rest
of the images can come from cloud datasets like ImageNet, if we wish to ensure
performance does not degrade on these classes. In the extreme case where we
only desire a specialized model, such as for road scene classes only,
$\mathcal{D}_{\mathrm{test,~{}final}}$ can also fully contain only target
class images.

In practice, $\mathcal{D}_{\mathrm{test,~{}final}}$ can be sourced from
designated robots in a fleet tasked with collecting only test data. If special
test robots are scarce, $\mathcal{D}_{\mathrm{test,~{}final}}$ can be
accumulated from a random subset of data annotated by a human per round $i$.
In both scenarios, we ensure test data is independently verified by annotators
and never used for training.

8\. Model Re-training and Evaluation: Once the new annotated cache
$\mathtt{Cache}^{i}$ is added to the dataset $\mathcal{D}^{i}$, model re-
training function $g_{\mathrm{retrain}}$ takes current robot model
$f_{\mathrm{DNN}}^{i}$ and the updated annotated dataset $\mathcal{D}^{i+1}$
and outputs an updated model $f_{\mathrm{DNN}}^{i+1}$. The training objective
is to minimize loss function
$\mathcal{L}(\hat{y}^{t},y_{\mathrm{GroundTruth}}^{t})$ penalizing prediction
errors, such as the standard cross-entropy loss for image classification or
mean average precision (mAP) for object detection [6]. Then, evaluation
function $g_{\mathrm{evaluate}}$ quantifies the accuracy improvements of newly
trained model $f_{\mathrm{DNN}}^{i+1}$ given a validation dataset,
$\mathcal{D}_{\mathrm{val}}^{i}$, or test set
$\mathcal{D}_{\mathrm{test,~{}final}}$. The model re-train and evaluation
functions are denoted by:

$\displaystyle f_{\mathrm{DNN}}^{i+1}\leftarrow
g_{\mathrm{retrain}}(f_{\mathrm{DNN}}^{i},\mathcal{D}_{\mathrm{train}}^{i+1})$
(4)
$\displaystyle\mathcal{L}^{i}=g_{\mathrm{evaluate}}(f_{\mathrm{DNN}}^{i},\mathcal{D}_{\mathrm{test,~{}final}}).$
(5)

Finally, new model $f_{\mathrm{DNN}}^{i+1}$ is downloaded back to the robot
over a network link. Vision DNN models are typically less than a few GB, which
are very small, in contrast to raw sensory data, to download daily even over
bandwidth-limited networks.

###  III-A The Robot Sensory Sampling Problem (Fig. 1)

We now formally define the abstract sampling problem.

######  Problem 1 (Robot Sensory Sampling Problem)

Given a target set of image classes $\mathtt{TargetClassSet}$, initial cloud
dataset $\mathcal{D}^{0}$, and initial robot model $f_{\mathrm{DNN}}^{0}$,
find the best sampling policy $\pi_{\mathrm{sample}}^{*}$ that minimizes full
held-out test set loss $\mathcal{D}_{\mathrm{test,~{}final}}$ for the final
robot DNN $f_{\mathrm{DNN}}^{N_{\mathrm{round}}}$ trained over
$N_{\mathrm{round}}$ days:

$\displaystyle\pi_{\mathrm{sample}}^{*}\in\mathop{\rm
argmin}_{\pi_{\mathrm{sample}}}\mathcal{L}(f_{\mathrm{DNN}}^{N_{\mathrm{round}}},\mathcal{D}_{\mathrm{test,~{}final}}),$
(6)

subject to the constraint that a robot can only sample $N_{\mathrm{cache}}$
images daily.

Sampling Problem Complexity: Deriving an optimal sampling policy that solves
Problem 1 is extremely challenging for several reasons. First, it is hard to
analytically model the sequence of high-dimensional sensory inputs
$\\{x^{t,i}\\}_{t=0}^{T}$ a robot can possibly measure on a day $i$. Second,
even if such an observation model exists, it is extremely challenging to
decide which limited subset of $N_{\mathrm{cache}}$ samples, chosen from $T$
possibilities, will lead to the best model performance of a complex DNN
evaluated on a final test-set at the end of several rounds
$N_{\mathrm{round}}$. Indeed, even for a small drone that measures video for
only 15 minutes at a slow frame-rate of 5 frames per second with a cache of
$N_{\mathrm{cache}}=2000$ daily images, there is a combinatorial explosion of
$\binom{T}{N_{\mathrm{cache}}}=4.21\times 10^{1340}$ images to choose from for
downstream model learning!

Wide Practicality of the Sensory Sampling Problem: Despite its complexity, the
sampling problem is built on simple, but general, abstractions of (a) a robot
perception model, (b) limited on-board storage, (c) occasional network access
to upload samples, and (d) centralized annotation and compute resources. These
abstractions apply to a wide variety of robots where the rate of sensory
measurements outpaces that of human annotation and downstream cloud
processing.

##  IV The HarvestNet Intelligent Sampler

While obtaining a provably optimal solution to Problem 1 is infeasible, one
can envision a myriad of possibly well-performing heuristics. To ensure
HarvestNet applies to a wide diversity of robots, we enforce a practical
requirement - the sampler should be sufficiently compute-efficient to run on
even a resource-constrained robot, equipped with, for example, a Raspberry Pi
and Google Edge TPU as shown in Fig. 8. A minimalistic on-board sampler that
does not interfere with real-time control is extremely useful for larger
robots as well, such as SDVs. We now describe three empirical insights,
evidenced in Fig. 4, that constrain the sampler design space and directly lead
to HarvestNet.

###  IV-A Empirical Insights Behind the HarvestNet Sampler

Figure 4: Empirical insights guiding HarvestNet: (Left) For relatively static
targets, classification test accuracy saturates with more images used to fine-
tune a vision DNN. (Right) We use either embedding distances or softmax
confidence scores to signal if a new image is likely a target of interest.

Insight 1: Use transfer learning for high model accuracy with few sampled
images.
Many pre-trained CNNs’ initial network layers serve as general-purpose feature
extractors that yield finite-dimension embeddings [25, 31, 2]. Embeddings can
be used to fine-tune final linear classification or object detection layers to
recognize new classes of interest, such as autonomous cars or construction
sites, that a CNN was not initially trained upon.

HarvestNet exploits fine-tuning by our empirical observation that only a
limited set of images are necessary to adapt a pre-trained vision DNN to new
domains, especially if the new domains’ data does not change much with time.
Fig. 4 (left) shows the full dataset test accuracy of our three separate
datasets (y-axis) versus the percentage of total training data provided to the
learning model (x-axis). Clearly, for the relatively static Waymo and FaceNet
domains, the final model accuracy saturates as more training images are
acquired.

In contrast, for the dynamic task of identifying construction sites, the
accuracy steadily increases as more data is acquired, even after months of
data. This is because construction sites are constantly changing, with new
tractors and signs being updated daily. Indeed, HarvestNet can be used for
continual re-training, allowing robots to update HD maps in the cloud.

Insight 2: Filter task-relevant target images using DNN embeddings and softmax
confidences.

#### Embedding distances

Many, but not all, CNNs have the property that the $L_{2}$ distance between
embeddings of two images is a proxy for image similarity. In particular,
FaceNet DNN embeddings typically have a lower embedding-distance for two
pictures of the same person than unrelated faces, since the DNN is optimized
with a triplet loss [11, 23]. We test this property with our FaceNet data in
Fig. 4 (right), allowing HarvestNet to exploit embedding distance as a
valuable indication of whether a new image is of the same desired class as
training data we wish to acquire.

#### Softmax confidence thresholds

Softmax confidence scores yield a probability distribution over CNN class
predictions. We empirically note that, while the softmax probabilities are not
necessarily a formal probability measure, images that are indeed truly a class
(“true positives”) have a higher softmax confidence element for that class of
interest compared to “true negatives”. For example, as shown in Fig. 4
(right), images that are indeed of Waymo cars (red) have a higher confidence
score for the Waymo class output compared to negatives (in blue), showing that
confidence scores above a separating threshold are a useful signal to store a
sample of interest.

Crucially, embeddings and softmax confidence scores are readily produced
simply by running a DNN forward-pass for inference. Thus, when a robot’s
perception model is already being run for a high-level task, such as motion
planning, the sampler obtains the embeddings and confidence scores “for free”.
Then, these two metrics can be used in a lightweight, data-driven filter to
discern if a new image is likely a target of interest for caching. We further
quantify in Sec. V-B how we can run HarvestNet’s models on the Edge TPU [7].

Insight 3: Use prioritized human feedback in the cloud to refine the target
image filter.
The final insight behind HarvestNet, shown in Fig. 5, is that the DNN softmax
confidence distributions between target images (red) and irrelevant images
(blue) become more differentiated as more human-annotated images are added to
the cloud dataset at every round $i$. Crucially, this allows the cloud to
better signal to the robot’s sampler how to identify task-relevant target
images from background noise, thereby directly building on insight 2. A
broader insight behind HarvestNet is that we can delegate compute-intensive
tasks of training to the cloud, but provide simple feedback to a robot to
improve its target image filter.

Figure 5: Role of Cloud Feedback: After a human annotates more samples on each
round $i$, the softmax confidence distribution difference between target (red)
and irrelevant (blue) images becomes sharper, which HarvestNet uses to improve
sampling by adapting the confidence threshold
$\mathtt{conf}_{\mathrm{thresh}}^{i}$.

###  IV-B Practical application of the HarvestNet insights

We now formalize how, together, insights 1-3 directly lead to our novel
HarvestNet sampler, introduced in Algorithm 1. Importantly, HarvestNet is
markedly distinct from today’s active learning and data selection algorithms,
due to its use of a compute-efficient robot model and adaptive feedback from
cloud computing resources.

#### Application of Insight 1 (Fine-Tuning)

Insight 1, which governs how to fine-tune a DNN with minimal images (Fig. 4,
left), is addressed by using pre-trained feature extraction layers for initial
robot model $f_{\mathrm{DNN}}^{0}$, such as from publicly available models,
and only re-training final layers with transfer learning for new target image
classes. Further, we limit the sampled cache size by $N_{\mathrm{cache}}$.

#### Application of Insight 2 (Simple Target Image Filter)

Insight 2, which governs how to use DNN softmax confidences or embedding
distances to build a compute-efficient on-board target image filter (Fig. 4,
right), is now formalized in two variants of sampling policies
$\pi_{\mathrm{sample}}$ (Eq. 2).

Variant 1. Embedding Distance Based Sampling: Two pictures of the same person
$x^{t_{1}}$ and $x^{t_{2}}$ typically have a lower embedding-distance
$\rho(x^{t_{1}},x^{t_{2}})=\left\|\mathtt{emb}^{t_{1}}-\mathtt{emb}^{t_{2}}\right\|^{2}_{2}$
than unrelated faces [23]. Using this insight, we compare the embedding
distance between a robot’s new sensory input $x^{t}$ to each of the target
images in $\mathtt{TargetImageSet}$, and cache the sample if the median
distance is below an empirical threshold $\rho_{\mathrm{thresh}}^{i}$,
indicating similarity to a face we want to learn. Embedding distance threshold
$\rho_{\mathrm{thresh}}^{i}$ serves as the sampler hyper-parameter
$\theta_{\mathrm{sampler}}^{i}=\\{\rho_{\mathrm{thresh}}^{i}\\}$ in Eqn. 2,
yielding:

$\displaystyle a^{t}$
$\displaystyle=\pi_{\mathrm{sample}}(x^{t},\boldsymbol{y}^{t},\mathtt{TargetImageSet},\theta_{\mathrm{sampler}}^{i})$
$\displaystyle=\mathbb{1}\left(\textsc{Median}\left\\{\rho(x^{t},x^{\mathrm{targ.}}),~{}\forall
x^{\mathrm{targ.}}\in\mathrm{TargImSet}\right\\}<\rho_{\mathrm{thresh}}^{i}\right).$

In this variant of the sampling policy, the cloud sends the
$\mathtt{TargetImageSet}$ of images and their embeddings to the robot to
compare with the embedding of a new image $x^{t}$. In practice, it is trivial
to store a limited set of target images on-board, such as 18 images which
sufficed in our experiments.

Variant 2. Classification Confidence Based Sampling:  In cases where embedding
distances are not optimized for similarity between two input images, we can
directly use confidence scores $\mathtt{conf}^{t}$ to determine if a new image
is likely to be a useful representative of the desired target. We rely on the
fact that initial robot DNN $f_{\mathrm{DNN}}^{0}$, as well as successive
versions $f_{\mathrm{DNN}}^{i}$, are trained with a few examples of target
images and can predict their target class $\mathtt{TargetClass}$ in the DNN
model architecture. Then, given a new image $x^{t}$, we see if the confidence
element for a target class, denoted by
$\mathtt{conf}[\mathtt{TargetClassSet}]$, is above an empirically determined
threshold $\mathtt{conf}_{\mathrm{thresh}}^{i}$ (Fig. 5), which serves as the
sampler hyper-parameter
$\theta_{\mathrm{sampler}}^{i}=\\{\mathtt{conf}_{\mathrm{thresh}}^{i}\\}$ in
Eqn. 2. Thus, the sampling policy becomes:

$\displaystyle a^{t}$
$\displaystyle=\pi_{\mathrm{sample}}(x^{t},\boldsymbol{y}^{t},\mathtt{TargetClassSet},\theta_{\mathrm{sampler}}^{i})$
$\displaystyle=\mathbb{1}\left(\mathtt{conf}^{t}[\mathtt{TargetClass}]>\mathtt{conf}_{\mathrm{thresh}}^{i}\right).$

Unlike the embedding-distance variant of the sampling policy, the softmax
confidence version simply needs to know the desired target classes
$\mathtt{TargetClassSet}$ to harvest more images of, which does not require
storing any target images $\mathtt{TargetImageSet}$ on-board. As measured in
Sec. V, it is trivial to do a comparison of the output softmax confidences of
a DNN with a fixed threshold efficiently on embedded-devices.

#### Application of Insight 3 (Cloud Feedback)

Our key insight, depicted in Fig. 5, is to use the new cloud dataset and
improved model at the end of round $i$, $\mathcal{D}^{i+1}$ and
$f_{\mathrm{DNN}}^{i+1}$, to best adapt the sampler hyper-parameters
$\theta_{\mathrm{sampler}}^{i+1}=\\{\mathtt{conf}_{\mathrm{thresh}}^{i+1},\rho_{\mathrm{thresh}}^{i+1}\\}$
for the next round. This is achieved by the $\mathtt{AdaptThresh}$ function:

$\displaystyle\theta_{\mathrm{sampler}}^{i+1}\leftarrow\mathtt{AdaptThresh}(\mathcal{D}^{i+1},f_{\mathrm{DNN}}^{i+1}).$
(7)

In practice, $\mathtt{AdaptThresh}$ can be implemented using any learning
algorithm that creates a simple decision boundary between task-relevant
images’ scores or embeddings and those of non-target images, such as logistic
regression or a support vector machine (SVM). In Sec. V, we show that cloud-
feedback on thresholds outperforms non-adaptive schemes.

###  IV-C The lightweight HarvestNet sampling algorithm

Having explained insights 1-3, we can now present HarvestNet’s sampler in
Algorithm 1, which serves as a practical heuristic solution to Problem 1. In
each learning round $i$, a robot has a fixed perception model
$f_{\mathrm{DNN}}^{i}$ and observes a stream of sensory inputs
$\\{x^{t,i}\\}_{t=0}^{T}$. Sampling policy $\pi_{\mathrm{sample}}$ stores
sampled images in cache $\mathtt{Cache}^{i}$, which is uploaded to the cloud,
annotated with ground-truth labels, appended to cloud dataset
$\mathcal{D}^{i}$, and finally used to re-train perception model
$f_{\mathrm{DNN}}^{i+1}$. This new model and an updated set of sampling
parameters is then downloaded to the robot. Learning repeats for a fixed set
of rounds $N_{\mathrm{round}}$, until DNN validation accuracy saturates. Key
steps of Algorithm 1 are colored.

Assemble Targets $\mathtt{TargetClassSet}$, Cloud Dataset $\mathcal{D}^{0}$

Pre-train Initial Robot Perception DNN $f_{\mathrm{DNN}}^{0}$ on
$\mathcal{D}^{0}$

Init.
$\\{\mathtt{conf}_{\mathrm{thresh}}^{0},\rho_{\mathrm{thresh}}^{0}\\}\leftarrow\mathtt{AdaptThresh}(\mathcal{D}^{0},f_{\mathrm{DNN}}^{0})$

for  _Learning Round $i~{}\leftarrow 0$ to $N_{\mathrm{round}}$_ do

$\mathtt{Cache}^{i}\leftarrow\\{\emptyset\\}$

// Init Empty Cache

for  _$t~{}\leftarrow 0$ to $T$_ do

$\boldsymbol{y}^{t}\leftarrow[\hat{y}^{t},\mathtt{conf}^{t},\mathtt{emb}^{t}]=f_{\mathrm{DNN}}^{i}(x^{t})$

$a^{t}\leftarrow\pi_{\mathrm{sample}}(x^{t},\boldsymbol{y}^{t},\mathtt{TargSet},\\{\mathtt{conf}_{\mathrm{thr.}}^{i},\rho_{\mathrm{thr.}}^{i}\\})$

$\mathtt{Cache}^{i}.\textsc{Append}(x^{t})~{}\mathbf{IFF}~{}a^{t}=1$

end for

$\\{(x^{\tau},y_{\mathrm{GroundTruth}}^{\tau})\\}\leftarrow\textsc{Annotate}(\mathtt{Cache}^{i})$

$\mathcal{D}^{i+1}\leftarrow\mathcal{D}^{i}~{}\cup~{}\\{(x^{\tau},y_{\mathrm{GroundTruth}}^{\tau})\\},~{}~{}\forall
x^{\tau}\in\mathtt{Cache}^{i}$

$\mathcal{D}_{\mathrm{train}}^{i+1},\mathcal{D}_{\mathrm{val}}^{i+1}\leftarrow\textsc{TrainValSplit}(\mathcal{D}^{i+1})$

$f_{\mathrm{DNN}}^{i+1}\leftarrow
g_{\mathrm{retrain}}(f_{\mathrm{DNN}}^{i},\mathcal{D}_{\mathrm{train}}^{i+1})$

// Re-train

$\\{\mathtt{conf}_{\mathrm{thresh}}^{i+1},\rho_{\mathrm{thresh}}^{i+1}\\}\leftarrow\mathtt{AdaptThresh}(\mathcal{D}^{i+1},f_{\mathrm{DNN}}^{i+1})$

end for

Result: Final DNN $f_{\mathrm{DNN}}^{N_{\mathrm{round}}}$, Cloud Dataset
$\mathcal{D}^{N_{\mathrm{round}}}$

Algorithm 1 HarvestNet Sensory Stream Sampler

###  IV-D Benchmark Sampling Algorithms

We compare HarvestNet with the following benchmarks:
1\. Random Sampling: This scheme randomly fills the limited cache from $T$
possible samples in round $i$.

2\. Non-adaptive cloud feedback: This scheme does not adapt thresholds using
Eq. 7 (yellow line in Alg. 1). Rather, this scheme caches an image based on a
probability that is proportional to the softmax confidence. For example, an
image with a softmax confidence of 90% is added to the cache with probability
$0.90$. Analogously, an image with the 10% quantile of embedding distances is
cached with probability $0.90$, since lower embedding distances are preferred.
Since this scheme does not calibrate itself using a threshold, it often misses
true positives which have low confidence scores at early rounds (Fig. 5,
left). We also experimented with an “arg-max” policy which stores an image if
the confidence element for the target-class was most probable, but this
performed poorly since the cutoff was too strict to store any task-relevant
images.

3\. HarvestNet-PriorityQueue: This scheme caches an image into a priority
queue ranked by either softmax confidence scores or embedding distances. The
first items that are dropped when the priority queue is full are the target
image with lowest softmax confidence or highest embedding distance.

4\. Oracle Sampling: This is an un-realizable upper bound since it perfectly
identifies images of the desired target at the robot. Since it is analytically
hard to compute the best set of limited images of size $N_{\mathrm{cache}}$ to
best improve the model, we instead approximate the upper bound by averaging
several samples of $N_{\mathrm{cache}}$ true target images, chosen without any
error. Importantly, the oracle takes the place of an “all-cloud” approach,
since uploading all the data to the cloud is infeasible for network-limited
robots like subterranean rovers.

##  V HarvestNet’s experimental performance

We now show HarvestNet’s ability to (1) achieve high model accuracy for a
minimal number of sampled images (Figs. 6, 7), and (2) produce compute-
efficient vision models. Every benchmark that we evaluate has a fixed cache
limit of $N_{\mathrm{cache}}$ and thus obeys the constraints of Problem 1.
Further, our results metric of high model accuracy is equivalent to minimizing
model loss, which is the objective of Problem 1.

Figure 6: FaceNet Results: We achieve steady accuracy gains with minimal face
examples, even matching the oracle performance for Face B, shown by the curve
overlap. Figure 7: Road-Scene Results: HarvestNet outperforms baselines that
sample randomly or without cloud feedback. Figure 8: Prototype: (Left) The
lightweight sampler runs on an embedded CPU, such as on the Raspberry Pi,
which uses an Edge TPU USB to accelerate DNN inference. Model Performance:
(Right) HarvestNet re-trained object-detectors flexibly run on the Edge TPU or
cloud GPUs.

###  V-A Performance on Diverse Video Streams

#### FaceNet

Our key result for the FaceNet scenario shows the final DNN classification
accuracy on all test images (y-axis) as more learning rounds $i$ progress
(x-axis). As shown in Fig. 6 (left) our sampler of HarvestNet, in orange,
achieves upto $1.68\times$ the final accuracy of benchmarks and no lower than
$92\%$ the accuracy of an upper-bound oracle solution in green. Fig. 6 also
shows HarvestNet can flexibly adapt to different target faces of interest,
since it shows steady model improvements for two randomly selected targets,
referred to as Face A and B in the plots. Importantly, the robot is only
provided $\mathtt{TargetSet}=18$ initial examples of the face of interest and
only caches a minimal set of images $N_{\mathrm{cache}}=10$ per learning round
$i$, much less than the thousands of frames it observes per video. We used the
widely-adopted OpenFace nn4.small2.v1 DNN for face embeddings [11], which were
passed to an SVM for face-classification.

#### Generalization to Dynamic Road Scenes

We now stress test HarvestNet’s ability to identify challenging road scenes of
self-driving cars and construction sites. Both domains showcase HarvestNet’s
flexibility in using softmax confidence scores to decide if an image is likely
a task-relevant training example to cache, unlike embedding distances in the
FaceNet scenario. In each round $i$, we fine-tuned the Inception V3 DNN [26]
in PyTorch on an Nvidia Tesla K80 cloud GPU for 20 epochs, which led to stable
convergence, using hyper-parameters outlined in our technical report.

Fig. 7 shows the steady increase of final test set accuracy of various
samplers (y-axis) for learning rounds $i$ in both the construction and self-
driving car domains. For Waymo, HarvestNet, in orange, achieves upto
$1.11\times$ the final accuracy of benchmarks and $93.5\%$ the accuracy of an
upper-bound oracle solution in green. The most challenging test of HarvestNet
comes from classifying whether a road scene has a construction site, since
these rapidly changed daily. Still, HarvestNet outperforms non-adaptive
sampling by over $1.38\times$ times and is within 87% of the oracle solution.

#### Comparison with Priority Queue Version

HarvestNet, which uses adaptive thresholds, convincingly outperforms
benchmarks of non-adaptive and random sampling. It also outperforms a variant
of HarvestNet that uses the exact same algorithm, but rather caches images in
a Priority Queue, dubbed $\mathrm{HNetPriorityQueue}$, albeit by a lower
margin such as in the Waymo domain. The priority queue version does worse when
the proportion of target images per day is highly non-uniform, especially on
days where there are no targets, leading it to cache irrelevant images below
the empirical threshold. Depending on how uniform the distribution of sensory
inputs are per day, a designer might consider using the
$\mathrm{HNetPriorityQueue}$ variant, which is also extremely compute-
efficient and simple.

#### Benefits of re-training with field data

To appreciate the accuracy gains of HarvestNet, we consider the final model
accuracy if we trained a CNN solely on high-quality public images of Waymo
cars and construction sites and tested on our realistic road dataset. We could
only find less than 200 Google Images for each scenario, which yielded an
extremely poor accuracy of only 41.6 % for Waymo and 0 % for construction.
Indeed, public images were often zoomed in, clear pictures from marketing
images, and were not representative of real road scenes with partially
occluded targets. Thus, our steady accuracy gains are substantial, especially
with minimally-sampled images. Unlike the rapid saturation of FaceNet and
Waymo accuracy, the construction example kept improving with time, serving as
key benefit of continual learning.

#### Systems Bottleneck Savings

The principal benefits of intelligent sampling derive from limiting the number
of images cached in each learning round (e.g $N_{\mathrm{cache}}=60$). Thus,
by re-training on between only 18-34% of possible images, we decrease storage,
annotation cost and time, and cloud-computing time by 65.7-81.3%, since costs
scaled linearly with the number of sampled images in our setup. These
estimates will vary considerably based on systems infrastructure.

###  V-B Prototype on Edge Tensor Processing Unit (TPU)

Finally, we show that training examples collected by HarvestNet can also be
used for re-training object detection CNNs, shown in Fig. 2, that scalably run
on compute-limited robots. Accordingly, we ran HarvestNet’s DNNs on Google’s
Edge TPU development board [7], where the TPU is a dedicated circuit for fast,
low-power TensorFlow [10] model inference. The relatively new Edge TPU board
supports, at best, a quantized MobileNet 2 (MN2) DNN and features a
complementary ARM Cortex embedded CPU, which runs DNNs without the aid of an
AI accelerator.

Fig. 8 (right) shows that HarvestNet can flexibly re-train mobile-optimized
models, such as MobileNet 2, or more accurate, but slower “cloud-grade” models
such as Faster R-CNN (contrasted in Fig. 2). Thus, one can trade-off speed,
size, and accuracy, quantified by the standard COCO mAP metric [6], for a wide
variety of robot compute resources.

For example, one can scalably run real-time vision algorithms on the TPU,
which readily provides embeddings and class confidences, and pass them to
HarvestNet’s sampler running on an embedded CPU with minimal overhead. This
configuration is shown on the robot we used in Fig. 8, where the Raspberry
Pi’s CPU can decide whether to cache a new image with a mean speed of just
$\approx.31$ms, which is an order of magnitude faster than DNN inference on
the Edge TPU!

#### Limitations of our sampler

HarvestNet might fail in scenarios with extremely rare visual data, such as
cases where we cannot obtain enough samples to seed the initial cloud dataset
and train the initial robot model on round $0$. In future work, we can
consider whether unknown weakpoints of a model can be cached by storing
examples with high confidence score entropy.

##  VI Discussion and Conclusions

The key contribution of this paper is to introduce a new, widely-applicable
sensory sampling problem to the robotics community and experimentally show the
gains of our compute-efficient sampler. Since we anticipate the volume of rich
robotic sensory data to grow, thereby making systems bottlenecks more acute,
we encourage the robotics community to significantly build upon our work. As
such, we have open-sourced our dataset, Edge TPU vision DNNs, and sampling
algorithm code at https://sites.google.com/view/harvestnet.

In the future, we hope to enhance HarvestNet by fusing data from several
cameras and LIDAR sensors on-board a robot. Further, the sampling algorithm
can be modified to select adversarial training examples for vision DNNs.
Lastly, HarvestNet’s sampler can be enhanced to explicitly not store “private”
training data by rejecting such samples. We believe HarvestNet is a vital
first step in keeping pace with surging sensory data in robotics by
insightfully filtering video streams for actionable training examples, using
insights from embedded AI, networking, and cloud robotics.

## References

  * int [2016] Data is the new oil in the future of automated driving.  https://newsroom.intel.com/editorials/krzanich-the-future-of-automated-driving/#gs.LoDUaZ4b, 2016\.  [Online; accessed 30-Jan.-2019].
  * CS2 [2019] Transfer learning.  http://cs231n.github.io/transfer-learning/, 2019.  [Online; accessed 08-Sep.-2019].
  * GCS [2019] Cloud storage pricing.  https://cloud.google.com/storage/pricing, 2019.  [Online; accessed 07-Sep.-2019].
  * clo [2019a] Ai platform data labeling service.  https://cloud.google.com/data-labeling/docs/, 2019a.
  * clo [2019b] Data labeling service pricing.  https://cloud.google.com/data-labeling/pricing, 2019b.
  * coc [2019] Coco: Common objects in context: Detection evaluation.  http://cocodataset.org/#detection-eval, 2019.
  * edg [2019] Edge tpu.  https://cloud.google.com/edge-tpu/, 2019.  [Online; accessed 01-Sep.-2019].
  * gol [2019] Cloud robotics and automation.  http://goldberg.berkeley.edu/cloud-robotics/, 2019.  [Online; accessed 02-Jan.-2019].
  * tux [2019] Autonomous and adas test cars produce over 11 tb of data per day.  https://www.tuxera.com/blog/autonomous-and-adas-test-cars-produce-over-11-tb-of-data-per-day/, 2019\.  [Online; accessed 08-Sep.-2019].
  * Abadi et al. [2016] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al.  Tensorflow: A system for large-scale machine learning.  In _Proceedings of the OSDI 2016. Savannah, Georgia, USA_ , 2016.
  * Amos et al. [2016] Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan.  Openface: A general-purpose face recognition library with mobile applications.  Technical report, CMU-CS-16-118, CMU School of Computer Science, 2016\.
  * Bodesheim et al. [2013] Paul Bodesheim, Alexander Freytag, Erik Rodner, Michael Kemmler, and Joachim Denzler.  Kernel null space methods for novelty detection.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3374–3381, 2013.
  * Chinchali et al. [2019] Sandeep Chinchali, Apoorva Sharma, James Harrison, Amine Elhafsi, Daniel Kang, Evgenya Pergament, Eyal Cidon, Sachin Katti, and Marco Pavone.  Network offloading policies for cloud robotics: a learning-based approach.  _arXiv preprint arXiv:1902.05703_ , 2019.
  * Cohn et al. [1996] David A Cohn, Zoubin Ghahramani, and Michael I Jordan.  Active learning with statistical models.  _Journal of artificial intelligence research_ , 4:129–145, 1996.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Gal et al. [2017] Yarin Gal, Riashat Islam, and Zoubin Ghahramani.  Deep bayesian active learning with image data.  In _International Conference on Machine Learning_ , 2017.
  * Kehoe et al. [2015] Ben Kehoe, Sachin Patil, Pieter Abbeel, and Ken Goldberg.  A survey of research on cloud robotics and automation.  _IEEE Trans. Automation Science and Engineering_ , 12(2):398–409, 2015.
  * Kuffner [2010] J Kuffner.  Cloud-enabled robots in: Ieee-ras international conference on humanoid robots.  _Piscataway, NJ: IEEE_ , 2010.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _European conference on computer vision_ , pages 740–755. Springer, 2014.
  * Perera and Patel [2019] Pramuditha Perera and Vishal M Patel.  Deep transfer learning for multiple class novelty detection.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 11544–11552, 2019.
  * Ren et al. [2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In _Advances in neural information processing systems_ , pages 91–99, 2015.
  * Sanakoyeu et al. [2019] Artsiom Sanakoyeu, Vadim Tschernezki, Uta Buchler, and Bjorn Ommer.  Divide and conquer the embedding space for metric learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 471–480, 2019.
  * Schroff et al. [2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin.  Facenet: A unified embedding for face recognition and clustering.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 815–823, 2015.
  * Settles [2009] Burr Settles.  Active learning literature survey.  Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009.
  * Sharif Razavian et al. [2014] Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson.  Cnn features off-the-shelf: an astounding baseline for recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , pages 806–813, 2014.
  * Szegedy et al. [2016] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2818–2826, 2016.
  * Tanwani et al. [2019] Ajay Kumar Tanwani, Nitesh Mor, John Kubiatowicz, Joseph E Gonzalez, and Ken Goldberg.  A fog robotics approach to deep robot learning: Application to object recognition and grasp planning in surface decluttering.  _arXiv preprint arXiv:1903.09589_ , 2019.
  * Tian et al. [2018] Nan Tian, Jinfa Chen, Mas Ma, Robert Zhang, Bill Huang, Ken Goldberg, and Somayeh Sojoudi.  A fog robotic system for dynamic visual servoing.  _arXiv preprint arXiv:1809.06716_ , 2018.
  * Tong [2001] Simon Tong.  _Active learning: theory and applications_ , volume 1.  Stanford University USA, 2001.
  * Wang et al. [2019] Xin Wang, Fisher Yu, Ruth Wang, Trevor Darrell, and Joseph E Gonzalez.  Tafe-net: Task-aware feature embeddings for low shot learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 1831–1840, 2019.
  * Yosinski et al. [2014] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson.  How transferable are features in deep neural networks?  In _Advances in neural information processing systems_ , pages 3320–3328, 2014.
  * Zhu et al. [2019] Pengkai Zhu, Hanxiao Wang, and Venkatesh Saligrama.  Generalized zero-shot recognition based on visually semantic embedding.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2995–3003, 2019.

##  Appendix A Explanation of Systems Costs Calculations

We now quantify system costs for cloud model re-training for a small fleet of
10 robots in Table II, each of which is assumed to capture 4 hours of driving
data per day. In the first scenario, called “Dashcam-ours”, we assume a single
car has only one dashcam capturing H.264 compressed video, as in our collected
dataset. In the second scenario, called “Multi-Sensor”, we base our
calculations off an Intel report [1] that estimates each self-driving car will
generate 4 TB of video and LIDAR data per day (1.5 hours of driving), of which
video data alone is about 432 GB.

To calculate monthly storage cost, we use the Google Cloud Storage rate of
$0.026 per GB of monthly data [3]. For network transfer time, we assume cars
upload data over state-of-the-art, extremely fast 10Gb/s ethernet, when being
serviced nightly at a garage. To be conservative, we assume only 1% of image
data is interesting and needs to be annotated, specifically by the state-of-
the-art Google Data Labeling service [4]. We consider two cases that are: (1)
these 1% of video frames need to be annotated with bounding boxes at a
standard rate of $49 per 1000 boxes [5], or (2) these 1% of frames need to be
labeled with segmentation masks for $850 per 1000 masks [5]. On average, we
assume each image has about 5 boxes or masks to be labeled. To roughly
estimate annotation time, we hand-labeled 1000 bounding boxes in 1.4 hours.
Using this rate, we estimate the number of days per month a team of 5
labellers will take in Table II. Since we did not train models for
segmentation, relevant estimates are marked as “NA”.

Overall, our estimates are conservative, since we assume a small fleet of 10
cars, extremely fast network transfer speeds, and pay for annotation of only
1% of video data because LIDAR annotation does not have easily quantifiable
costs from current data labelers. Nevertheless, Table II shows significant
costs, especially for dataset annotation.

Bottleneck | Multi-Sensor | Dashcam-ours
---|---|---
B. Box | Mask | B. Box | Mask
Storage | $31,200 | $31,200 | $172.50 | $172.50
Transfer Time (hr) | 8.88 | 8.88 | 0.05 | 0.05
Annotation Cost | $95,256 | $1,652,400 | $15,876 | $275,400
Annot. Time (days) | 22 | NA | 18 | NA
TABLE II: Systems Costs: Monthly storage/annotation cost and daily network
upload time are significant for self-driving car data even if we annotate 1%
of video frames.