# Characterizing Differentially-Private Techniques in the Era of Internet-of-
Vehicles

Yicun Duan*, Junyu Liu*, Wangkai Jin, Xiangjun Peng  User-Centric Computing
Group, University of Nottingham Ningbo Chinahttps://unnc-ucc.github.io

(2018)

###### Abstract.

Recent developments of advanced Human-Vehicle Interactions rely on the concept
Internet-of-Vehicles (IoV), to achieve large-scale communications and
synchronizations of data in practice. The concept of IoV is highly similar to
a distributed system, where each vehicle is considered as a node and all nodes
are grouped with a centralized server. In this manner, the concerns of data
privacy are significant since all vehicles collect, process and share personal
statistics (e.g. multi-modal, driving statuses and etc.). Therefore, it’s
important to understand how modern privacy-preserving techniques suit for IoV.

We present the most comprehensive study to characterize modern privacy-
preserving techniques for IoV to date. We focus on Differential Privacy (DP),
a representative set of mathematically-guaranteed mechanisms for both privacy-
preserving processing and sharing on sensitive data. The purpose of our study
is to demystify the tradeoffs of deploying DP techniques, in terms of service
quality. We first characterize representative privacy-preserving processing
mechanisms, enabled by advanced DP approaches. Then we perform a detailed
study of an emerging in-vehicle, Deep-Neural-Network-driven application, and
study the upsides and downsides of DP for diverse types of data streams. Our
study obtains 11 key findings and we highlight FIVE most significant
observations from our detailed characterizations. We conclude that there are a
large volume of challenges and opportunities for future studies, by enabling
privacy-preserving IoV with low overheads for service quality.

* stands for equal contribution.

††copyright: acmcopyright††journalyear: 2018††doi:
10.1145/1122445.1122456††conference: Woodstock ’18: ACM Symposium on Neural
Gaze Detection; June 03–05, 2018; Woodstock, NY††booktitle: Woodstock ’18: ACM
Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY††price:
15.00††isbn: 978-1-4503-9999-9/18/06

##  1\. Introduction

Techniques for Human-Vehicle Interactions naturally evolve from Vehicle Ad-hoc
Networks (VANETs) to Internet-of-Vehicles (IoV), to realize frequent, large-
scale and efficient communications and synchronizations of data in practice.
Unlike VANETs, IoV views multiple vehicles as nodes within a distributed
system, where groups of nodes are powered by a centralized server for rich
computational resources to achieve diverse types of usages. For instance,
transportation scheduling requires detailed vehicles information gathering
(e.g. coordinates, speed and etc.), and performs the scheduling according to
the collected statistics (Dandala et al., 2017; Chen and Chang, 2017). Similar
usages, via IoV, become more imminent and important with the emerging trends
of Autonomous Vehicles. However, though IoV provides extensive opportunities
for various novel techniques in Human-Vehicle Interaction (e.g. healthcare
applications (Hua et al., 2017), lane-changing warnings (Khatoun et al., 2015;
Panichpapiboon and Pattara-atikom, 2012) and etc. (Sharma and Kaushik, 2019)),
it also significantly raises the concerns of privacy in practice. Hereby, We
summarize them into the following two aspects.

First, sophisticated designs for better user experiences are usually composed
with high demands for computational resources (i.e. centralized servers in
IoV), and this results in a large amount of sensitive data being exposed
outside the vehicles. For instance, recent advances of Deep Neural Networks
provide rich functionalities (e.g. object detection (Zhao et al., 2019),
object recognition (Zhou et al., 2020) and etc.) but demand a huge amount of
computational resources (Sze et al., 2017). In IoV, sensitive data is possible
to be gathered in centralized servers for processing purposes, which incurs
the privacy leakage in potential.

Second, the design pre-requisites of several key applications, in the context
of IoV, demand all vehicles to share data within centralized servers. Such
action enforces all vehicles to submit sensitive data for centralized data
sharing and processing. For instance, recent developments take advantage of
dash recordings to identify the root causes of vehicle accidents, but require
all dash recordings to be gathered in centralized servers (Mehrish et al.,
2017). In this manner, gathered sensitive data are highly likely to become the
target of adversarial attacks, which would impair the end-users‘ privacy
integrity when disclosed to the untrusted parties.

Therefore, it’s important to introduce privacy-preserving techniques in the
era of IoV. However, since no prior works systematically to identify and
tackle this issue, it’s still unclear how modern privacy-preserving techniques
perform in the context of IoV. In this work, we focus on Differential Privacy
(DP), a representative set of mathematically-guaranteed mechanisms, because DP
can enable both privacy-preserving processing and sharing on sensitive data.

Our goal in this work is to investigate representative mechanisms of DP for
both processing and sharing on sensitive data, and experimentally characterize
the advantages and disadvantages in the context of IoV. To this end, we design
a rigorous methodology to understand the impacts of DP, for processing and
sharing respectively. For privacy-preserving processing in IoV, we abstract
the interactions between centralized servers and vehicle nodes and
characterize the effects from DP-protected processing of sensitive data; and
for privacy-preserving sharing in IoV, we select a representative in-vehicle
application and characterize the impacts of DP-enabled sharing of sensitive
data. (Section 3)

We first focus on DP-enabled processing for sensitive data, by abstracting the
interactions between vehicle nodes and centralized servers. For DP-enabled
processing, centralized servers perform queries to retrieve sensitive data
from vehicle nodes, and then perform the computation within servers. In our
characterizations, we select FOUR representative mechanisms for DP-enabled
processing, and design representative workloads according to these mechanisms.
Then we characterize the effects of these techniques and analyze the results
comprehensively, in terms of absolute/relative errors and service quality
(Section 4).

We then focus on DP-enabled sharing for sensitive data, by leveraging a
representative in-vehicle application (i.e., a state-of-the-art in-vehicle
predictor for drivers’ statistics taking facial expressions as the only input)
using Deep Neural Networks (DNN). In our characterizations, we consider both
centralized and distributed training of these DNN models, by enabling DP on
diverse types of input data streams and the trade-off between two DP
protection mechanisms (i.e. Global Differential Privacy and Local Differential
Privacy) with different privacy-preserving scales. Then we characterize the
effects of different parameter setting combinations of these techniques and
analyze the results comprehensively, with a particular focus on service
quality (Section 5).

Derived from the above two large-scale studies, we make 11 key observations
(i.e. distributed into separate sections) and highlight FIVE most significant
observations here: ➊ there are no one-size-fits-all mechanisms of DP for all
applications of IoV, in terms of Relative Error; ➋ there are no one-size-fits-
all mechanisms of DP for all applications of IoV, in terms of Absolute Error
as well; ➌ to strike a balance between data integrity and privacy preserving,
a preferred setting of $\epsilon$ might be located in the range between 0.01
and 1.2; ➍ LDP is a better solution, compared with GDP, when deploying DP in
the application-specific manner; and ➎ GDP and LDP protection on textual data
can potentially destroy the performance of the whole application. We summarize
most significant observations, and discuss key takeaways from our findings in
Section 6.

More specifically, we make three key contributions in this paper:

  * •

We address the challenges about preserving data privacy in the era of
Internet-of-Vehicles (Internet-of-Vehicles), and identify key opportunities to
tackle this challenge. We classify the applications of IoV, and outline the
key mechanism Differential Privacy to enable privacy-preserving features of
Internet-of-Vehicles in practice.

  * •

We design and characterize DP-enabled data processing and sharing on sensitive
data, in the context of IoV. We investigate, implement and evaluate the state-
of-the-art mechanisms from DP, over a state-of-the-art dataset for Human-
Vehicle Interaction Research.

  * •

Our characterizations derive FIVE most significant observations, selected from
our 11 key findings. As our work is the first step to experimentally
characterize the tradeoffs of privacy-preseving techniques for IoV, we believe
our works are significantly beneficial for future research practices in
privacy-preserving IoV.

The rest of this paper is organized as follows. Section 2 introduces the
background and motivation of our work. Section 3 gives an overview of our
study design regarding to the characterization of DP-enabled sensitive data
processing and sharing. Section 4 elaborates detailed characterization
methodologies, and comprehensively analyzes the results. Section 5 presents
the experimental methodology for a real-world application with extensive
results showcase and analysis. Section 6 discusses several takeaways and
future work.

##  2\. Background

In this section, we introduce background and motivation of our
characterizations. We first describe the background of Internet-of-Vehicles
(Section 2.1). Then we elaborate two major types of applications in IoV
(Section 2.2). Next, we introduce Differential Privacy, a general series of
mechanisms to cover both types of IoV applications (Section 2.3). Finally, we
identify our motivation and our goal in this work (Section 2.4).

###  2.1. The Concepts of Internet-of-Vehicles

Internet-of-Vehicles (IoV) is a large-scale connected network for efficient
cooperation’s among humans, vehicles, things, and environments. Such
cooperation consists of multiple interactions, which further provides
sufficient and diverse services for a wide range of in-vehicle participants
(e.g. city-wide or country-wide). As shown in Figure 1, the architecture of
IoV can be abstracted as follow. ➊ All agents, including humans, vehicles, and
things, connect with each other, within respective ranges of the particular
agent types. These broad and extensive connections enable multiple use cases,
such as traffic control (Chen and Chang, 2017), crash response (Davydov and
Bezzateev, 2020), convenience services (Sadiku et al., 2018) and etc.; ➋ IoV
also consists of many building blocks, with vehicles as the nodes, in
practice. Within a IoV, each node of this network is a vehicle, and the
vehicle is the primary unit of a complicated agent. All interactions from
other agents (e.g. humans, things, etc.) have to go through the interfaces
from vehicles, and then connect with others. It’s notable to mention that, to
manage a complicated architecture like IoV, centralized servers for groups of
nodes are essential in practice. Therefore, though IoV’s architecture achieves
tight associations among multiple agents (e.g., humans, vehicles, things and
environments), this design also incurs potential leakage of sensitive personal
statistics and substantially increases the risks of privacy disclosure.

Figure 1. The architecture of Internet-of-Vehicles: (1) Human: people in
vehicles and other people in the context of IoV, such as pedestrians; (2)
Vehicle: all vehicles that consume or provide services/applications of IoV;
(3) Thing: anything except human and vehicles, such as roads and remote
servers;

###  2.2. Classifying Application of IoV: Processing inside or outside the
Vehicles?

To better address the privacy concerns of IoV, we divide emerging applications
of IoV into the following two types: Processing-In-Vehicles or Processing-
Outside-Vehicles. We elaborate them in detail as follows. ➊ for Processing-In-
Vehicles, there are some applications that can be fully processed within a
vehicle. There are multiple examples in this scenario, such as the control of
in-vehicle devices (e.g. conditions, lights, etc.), in-vehicle sensors for
drivers (Cordero et al., 2020) and in-vehicle monitors of driving statistics
(Kim and Baek, 2020; Wang et al., 2021); and ➋ for Processing-Outside-
Vehicles, other applications may need assistance from centralized servers
within IoV, due to either the application requirements or computational
powers. There are many examples of applications, which require data
aggregations in centralized servers. For instance, Transportation Scheduling
demands all vehicles to update their coordinates and potential routines, to
perform scheduling decisions (Zhang et al., 2021a); also, there are many
examples for applications, which demand more computational power in
centralized servers (e.g., (Jin et al., 2021; Xiong et al., 2022; Zhang et
al., 2021b)). For instance, emerging trends of Deep Neural Networks demand
significant computational power for model training, which are usually
maintained in the centralized servers. Therefore, we consider our selected
privacy-preserving techniques for IoV shall be applicable for both types of
applications.

###  2.3. Differential Privacy

Differential Privacy (DP) is a series of general privacy-preserving
techniques, which are mathematically guaranteed for users’ privacy, regardless
of any auxiliary knowledge an adversary might have. The primary design concept
of this effective mechanism is, to narrow the possible statistical effects
from the increment of a single individual’s data. Formally speaking, this is
achieved by performing randomized functions subject to $\epsilon$-Differential
Privacy on a database, to obtain the desired query outcome. The following
definition gives out the mathematical description of a randomized function
$K_{f}$.

###### Definition 2.1.

(Randomized Function subject to $\epsilon$-Differential Privacy) We regard a
randomized function $K_{f}$ as satisfying $\epsilon$-Differential Privacy,
assume that $\forall$ dataset $D_{1}$ and $D_{2}$ with at most one element
difference, and $\forall S\subseteq Range(K_{f})$, the $K_{f}$ function obey
the below in-equation:

$Pr[K_{f}(D_{1})\in S]\leq exp(\epsilon)\times Pr[K_{f}(D_{2})\in S]$

Note that, in practice, the realization of DP (Laplace Mechanism) usually
depends on adding random noise, drawn from Laplace Distribution. The scale
parameter is set as $\Delta f/\epsilon$, for $f(D)$ (i.e. where $D$ is the
database, and $f$ is a query function): ➊ Three parameters $\epsilon$ should
be manually selected at the beginning, according to the privacy protection
degree the user anticipates. In other words, the smaller the $\epsilon$ is,
the more noises sensitive data are added; and ➋ The other parameter $\Delta f$
($L1-sensitivity$) relates to the query function $f$ to be used. After this
noise addition, the query $f(D)\,+\,Laplace\,noise$ can achieve the same
effects, as applying randomized function $K_{f}$ on $D$. In other words, the
conceptual $\epsilon$-Differential Privacy function $K_{f}$ is instantiated by
this approach. In this work, our focus lies on how to ensure our privacy
protection meets the $\epsilon$-Differential Privacy standard.

The reason why we select Differential Privacy as the core mechanism is that:
Differential Privacy can be deployed for both Processing-In-Vehicle and
Processing-Outside-Vehicles applications. We denote Processing-In-Vehicles
applications as data processing of sensitive data, and Processing-Outside-
Vehicles as data sharing of sensitive data. This features make Differential
Privacy stands out from other privacy-preserving mechanisms, which are
application-specific (e.g. Oblivious Inference (Gilad-Bachrach et al., 2016;
Chou et al., 2018), Authentication (Gabay et al., 2020; Aman et al., 2021;
Kilari et al., 2020) and etc.). Therefore, we believe DP is an ideal candidate
for IoV, and it’s worth the efforts to characterize its tradeoffs in detail.

###  2.4. Motivations and Our Goal

Though DP is a promising solution for privacy concerns of IoV in theory, it’s
still unclear how DP can introduce tradeoffs in the context of IoV. Obtaining
experimental results, by applying DP in IoV, is the first step towards
practical and effective solutions to privacy-preserving IoV. Our goal in this
work is to investigate representative mechanisms of DP for both processing and
sharing on sensitive data, and experimentally characterize the advantages and
disadvantages in the context of IoV. Our study consists of two parts, focusing
on data processing and data sharing on sensitive data respectively.

##  3\. Characterization Methodology

In this section, we describe our methodology to characterize differentially-
private techniques for IoV. We first recap the rationales of choosing
Differential Privacy as the core mechanism, based on the background of IoV
(Section 3.1). Then we cover the design of our characterization, which is
divided into two parts: data-processing and data-sharing (Section 3.2). Next,
we introduce the hardware, software and dataset used in our characterizations
in detail (Section 3.3). Finally, we describe our efforts to emulate IoV in
distributed machines (Section 3.4).

###  3.1. Rationales for Differential Privacy

The underlying reasons for choosing Differential Privacy as the core focus are
that: (1) DP is one of the most effective and prevalent modern privacy-
preserving techniques, which can be generalized for various applications for
processing sensitive data; and (2) DP supports dynamic parameter tuning and
several derivatives for context-specific purposes, which can also be
specialized for specific applications for sharing sensitive data.

Since our characterization aims to obtain lessons in a relatively broad range,
we believe the focus shall lie on a mechanism, which can be utilized in both
application scenarios. Therefore, we believe DP-enabled data processing and
sharing would become more important in IoV, and the characterization is an
essential step to stimulate more studies for privacy-preserving applications
in the era of IoV.

###  3.2. Study Design

To examine the tradeoffs of using Differential Privacy in the era of Internet-
of-Vehicles, we divide our study into two separate parts for comprehensive
characterizations. This is motivated by our classifications on IoV
applications, which can be divided into data-processing and data-sharing on
sensitive data. We describe our study design for separate parts in details as
follows.

The first part of our characterizations focuses on DP-enabled sensitive data
processing. We achieve so through three steps. First, we leverage existing
database systems to add random noises to the query operators via DP. In this
manner, querying driving physiological statistics (i.e. Heart Rate and Skin
Conductivity) can be considered as abstracted processing in IoV. Second, we
leverage four state-of-the-art DP-enabled queries (i.e., Fourier, Wavelet,
DataCube and Hierarchical), to perform queries from real-world query
workloads. To cover representative queries, we consider both All-Range Query
and One-Way Marginal Query. Third, we evaluate the service quality of these
queries, by using two distinctive metrics (i.e., Absolute Error and Relative
Error compared with queries over the raw data). In this way, we can obtain key
insights on how DP-enabled processing performs on sensitive data.

The second part of our characterizations emphasizes the DP-enabled sharing of
sensitive data. We achieve so via three steps. First, we leverage a state-of-
the-art application for in-vehicle drivers called Face2Multi-modal, a Deep-
Neural-Network-driven predictor for physiological and sensitive statistics.
Second, we identify key differences while training Face2Multi-modal using
sensitive data, between centralized training and decentralized training.
Third, we break down DP mechanisms into Global DP and Local DP, for
centralized and decentralized training respectively. Fourth, we evaluate the
service quality of both approaches, by using validation accuracy (i.e. how
accurate the predictor is), compared with predictors over the raw data. In
this way, we can retrieve key lessons on how DP-enabled sharing works on
sensitive data.

###  3.3. Hardware, Software and Dataset Supports

In this section, we cover details of the hardware, software and dataset
supports for our characterizations. We first describe our hardware and
software configurations. Then we elaborate the dataset for our studies in
details.

➊ Hardware Supports. Our characterizations and emulations of IoV are performed
on machines, whose detailed configurations are listed as follows. The CPU is
Intel(R) Core(TM) i5-7200U and the GPU is NVIDIA GeForce MX150. We assume this
configuration fits with a standard laptop/table-scale, which can be easily
equipped for modern and emerging (autonomous) vehicles.

➋ Software Supports. The implementations of user-level applications (i.e.
representative queries & Face2Multi-modal) are implemented in Python. We use
Python because it’s very expressive and powerful to be integrated with modern
database systems. In this manner, we can reduce as many variations of software
configurations as possible, to ensure the consistency of our two studies.

➌ Dataset Supports. We use BROOK dataset for our characterizations (Peng et
al., 2020). BROOK dataset has facial videos and multiple multi-modal/driving
status data of 34 drivers. Statistics of every participant are collected in
both manual mode and automated mode. Figure 2 shows 11 dimensions of data for
the current BROOK dataset, which includes: Facial Video, Vehicle Speed,
Vehicle Acceleration, Vehicle Coordinate, Distance of Vehicle Ahead, Steering
Wheel Coordinates, Throttle Status, Brake Status, Heart Rate, Skin
Conductance, and Eye Tracking.

Figure 2. The composition of the BROOK dataset.

###  3.4. The Emulation of Internet-of-Vehicles

We emulate IoV via the remote APIs of modern distributed database systems.
This ensures our characterizations not to mistake any procedure, by
overlooking the impacts of IoV. Since our focus lies on the effects on service
quality, by enabling DP on both processing and sharing of sensitive data, we
expect and validate our emulation to be sufficiently accurate, and there are
no side effects on the presented results in the following sections.

##  4\. Characterizing DP-enabled Processing of Sensitive Data

For DP-enabled processing, centralized servers perform queries to retrieve
sensitive data from vehicle nodes, and then perform the computation within
servers. In our characterizations, we select FOUR representative mechanisms
for DP-enabled processing, and design representative workloads according to
these mechanisms. Then we characterize the effects of these techniques and
analyze the results comprehensively, in terms of service quality.

In this section, we characterize the impacts of DP-enabled data processing, by
abstracting the interactions between vehicle nodes and centralized servers. We
first brief the representative workloads and queries, which we select from the
state-of-the-art approaches (Section 4.1). Next, we elaborate the setup for
examining the impacts of representative queries (Section 4.2). Then, we
describe how we construct various query structures and tune different
$\epsilon$ values for broad coverage of our characterizations (Section 4.3).
Finally, we describe our results by characterizing the impacts of DP-protected
data through the above two approaches, in terms of service quality (Section
4.4).

###  4.1. DP Mechanisms for Processing

In this section, we cover four state-of-the-art mechanisms for enabling DP on
queries, under $\epsilon$-Differential Privacy. We denote these mechanisms
using words reflecting their key ideas, which are Fourier, Wavelet, DataCube
and Hierarchical. We describe these mechanisms in details one-by-one.

➊ Fourier. The design purpose of Fourier is specifically for workloads having
all K-way marginals, for any given $K$ (Barak et al., 2007). This mechanism
transforms the cell counts, via the Fourier transformation, and obtains the
values of the marginals based on the Fourier parameters. If the workload is
not full rank, there are eliminations of the unnecessary queries from the
Fourier basis to reduce the sensitivity.

➋ Wavelet. The design goal of Wavelet is to support multi-dimensional range
workloads. This mechanism applies the Haar wavelet transformation to each
dimension (Xiao et al., 2010). Note that Wavelet is expected to be applied in
a hybrid manner if using $\epsilon$-differential privacy. Such a hybrid
algorithm depends on the number of distinct values in the selected dimension:
when the number is small, the identity strategy is applied to improve
Wavelet’s performance.

➌ DataCube The target applicability of DabaCube is to adaptably support
marginal workloads (Ding et al., 2011). To provide a deterministic comparison
point, the BMAX algorithm is applied in this case. This algorithm chooses a
subset of input marginals, to minimize the maximum errors of the answer to the
input workloads. DataCube is naturally adaptable with ($\epsilon$,
0)-differential privacy.

➍ Hierarchical The aim of Hierarchical (Hay et al., 2010) is to answer
workloads of range queries. The key idea is to use a binary tree structure of
queries: (1) the first query is the sum of all cells; and (2) the rest of the
queries recursively divide the first query into subsets. As the representative
comparison point, we apply binary hierarchical strategies (although higher
orders are possible). The strategy in (Hay et al., 2010) supports one-
dimensional range workloads, but is capable to be adapted to multiple
dimensions, similarly to Wavelet (Xiao et al., 2010).

###  4.2. Dataset Setup and Query Types

In this section, we aim to cover the concept definitions of All-Range Query
and One-Way Marginal Query. Due to the close relationship between these two
queries and the expression of our dataset, we will discuss the dataset set up
at the beginning and then concentrate on defining these two kinds of counting
queries.

➊ Dataset Setup To characterize the query errors from the above types of DP-
enabled query mechanisms, we rectify our dataset, to import them into our
emulation platform. More specifically, we pair arbitrary types of data
streams, in an evenly distributed manner, and iterate all possible cases with
average results reported. We use the combination of Heart Rates and Skin
Conductivity as a motivating example. Assume the table is of size $m\times n$,
and the constraint condition for cell $(i,j)$ could be written as $\rho_{ij}$.
Then, a $length-(m\times n)$ column vector
$\mathbf{x}=(x_{00},x_{01},...,x_{(m-1)(n-1)})^{T}$ where $x_{ij}$ meets the
condition $\rho_{ij}$ could be extracted from the table. In this case, it is
equal to $(12,17,0,0,2254,...,11)^{T}$, a column vector with length of 32.
With the existence of this data vector $\mathbf{x}$, it is convenient to
answer a linear query.

➋ All-Range Query All-Range Query, defined in this article, is closely
associated with the data vector. The dot product of a single $length-n$ row
vector, with (1) all positions being $1$ and (2) a $length-n$ column data
vector, is the expected result of the All-Range Query result. For a database
whose content is certain and unchanged, performing All-Range Query would
always generate the same results.

➌ One-Way Marginal Query One-Way Marginal query is a concept derived from the
K-Way Marginal query. We first provide a general explanation regarding the
K-Way Marginal query. A K-Way Marginal Query on a $n$-dimension database
expects a returned vector, whose each position corresponds to the counting
result of a query related to partial attributes (k attributes) only. Also, the
magnitude of all possible settings considering $K$ attributes is the length of
this vector. Similarly, the One-Way Marginal query is a special case of k-Way
Marginal query when $K=1$.

###  4.3. Metrics and Configurations of Workloads

➊ Absolute Error. The absolute error refers to the absolute difference between
the measured value and real value. In our experiment, we regard the query
outcomes generated by four DP-related query approaches as measured values. And
corresponding queries without DP processing directly performed on the database
will give rise to real values. For instance, given that an All-Range Query
utilizing Wavelet method obtains result $r^{\prime}$ when the dimension
setting is $4\times 8$, and, in the same condition, direct All-Range Query on
the database will get $r$, the absolute error, in this case, will be
$|r^{\prime}-r|$. Note that, when $r$ and $r^{\prime}$ are all vectors, the
absolute error becomes $\|r^{\prime}-r\|_{1}$.

➋ Relative Error. The relative error is defined as the ratio of the absolute
error to the real value. The formal mathematical expression for it is
$Relative\,Error=Absolute\,Error/Real\,Value$. Considering the afore-mentioned
example, the relative error in that situation should be $|(r^{\prime}-r)/r|$.

➌ Varied Configurations. In experiments for absolute error, the parameter
$\epsilon$ is previously fixed to $0.5$, and the dimension setting is the only
changeable variable, which rigorously follows the control variable method.
Concerning experiments related to the relative error, they are all conducted
on a $4\times 8$ database picked from BROOK with some study-worthy $\epsilon$
values (i.e. 0.1, 0.2, 0.5, 1 and 2.5) for the adjustment of four DP query
methods.

➍ Workloads for All-Range Query According to the definition of All-Range
Query, the desired counting result shall cover all possible constraint
conditions appearing in data vector, which, specifically speaking, for any
linear query in the workloads of All-Range Query, it is supposed to be a
vector with all positions be set as one, regardless of dataset dimensions.
➎ Workloads for One-Way Marginal Query As for one-way marginal query, we
rigorously examine the query results on various dimension settings (i.e. $32$,
$4\times 8$, $4\times 4\times 2$, $4\times 2\times 2\times 2$ and $2^{5}$).
Various One-way Marginal Queries regarding different data types such as HR and
SC could be realized through linear queries. By integrating all the linear
queries with different structural designs into one matrix, we will obtain the
one-way marginal query workload.

###  4.4. Characterizations Results

Figure 3. Relative Error of Four Query Methods

Figure 4. Absolute Error of Four Query Methods

In this section, we demonstrate the query results of the four representative
mechanisms under our settings, in terms of Relative Error and Absolute Error.
Figure 4 depicts the Relative Errors for the covered four mechanisms, for both
All-Range Query and One-Way Marginal Query. We make three key observations:
First, for both All-Range Query and One-Way Marginal Query, DataCube incurs
more errors than any other mechanisms in all scenarios, which suggests it may
not be an ideal candidate in practice. Second, Hierarchical is the best query
method for all the cases in All-Range Query, but Fourier performs the best for
every $\epsilon$ parameter setting in One-Way Marginal Query. Therefore, we
conclude that there are no one-size-fits-all mechanisms of DP for all
applications of IoV, in terms of Relative Error (i.e. Key Observation 1).
Third, all mechanisms gradually reduce their Relative Errors, when the
$\epsilon$ increases; moreover, all mechanisms achieve the best query results
at the maximum $\epsilon$ setting.

Figure 4 shows the results for Absolute Error of the covered four mechanisms.
Note that DataCube is incapable to evaluate Absolute Errors since this
approach is highly dataset-dependent. We make four key observations. First,
the query error of both Hierarchical and Wavelet has a positive correlation
with the growing granularity of query structures in both query types. Second,
the query errors of Fourier are more robust with the growing granularity of
query structures. Particularly, Fourier achieves the best results in complex
granularity of query structures for All-Range Query, and it also achieves the
best in all cases for One-Way Marginal Query. Third, for the query structure
$32$, $4\times 8$ in All-Range Query, Hierarchical obtains the best query
results rather than Fourier. Fourth, compared with Hierarchical, Wavelet can
achieve less Absolute Error in some simpler combinations for One-Way Marginal
Query. Therefore, we conclude that there are no one-size-fits-all mechanisms
of DP for all applications of IoV, in terms of Absolute Error as well (i.e.
Key Observation 2).

##  5\. Characterizing DP-enabled Sharing of Sensitive Data

In this section, we characterize the impact of DP-enabled data sharing, by
focusing on a novel in-vehicle application, Face2Multi-Modal (Huang et al.,
2020). We first brief the background of Face2Multi-Modal, and discuss the
potential privacy leakages in either centralized or decentralized manners
(Section 5.1). Next, we elaborate on two approaches to resolve privacy
leakages in centralized and decentralized manners, using DP (Section 5.2).
Then, we vary the configurations of GDP and LDP, to enable a broad coverage of
our characterizations (Section 5.3). Finally, we describe our results by
characterizing the impacts of DP-protected data through the above two
approaches, in terms of service quality (Section 5.4).

###  5.1. Face2Multi-modal: A Motivating Example for Privacy Concerns of In-
Vehicle Applications

Face2Multi-Modal provides an in-vehicle multi-modal data stream predictor, by
taking facial expressions as the only input (Huang et al., 2020). Face2Multi-
modal is motivated by the fact that: obtaining drivers’ physiological data, by
directly equipping bio-sensors, is expensive, impractical and user-unfriendly.
The key idea of Face2Multi-modal is to utilize the Convolutional Neural
Networks as the backbone for predicting driver statistics.

Since, for training, both input (i.e. facial expressions) and output (i.e.
drivers’ statistics like Heart Rate, Skin Conductance, etc.) are highly
sensitive information, the deployments of Face2Multi-modal can incur
significant concerns in terms of privacy. Moreover, the service quality of
Face2Multi-modal is highly related to the privacy-preserving techniques, and
the reasons are two-folded. First, the training of Face2Mutli-modal is
computationally expensive, and it’s ideal to perform such training on
centralized servers. and second, the service quality of Face2Multi-modal
relies on the model training on a high volume of raw data, and applying
privacy-preserving techniques can increase the difficulty of getting accurate
models.

In this characterization, we consider two types of model training for
Face2Multi-modal and enable DP for them using corresponding mechanisms. We
first consider that Face2Multi-modal is trained in a centralized manner. More
specifically, a training of Face2Multi-modal is performed based on aggregated
data received from different nodes and processed in the centralized servers.
Then we consider that Face2Multi-modal is trained in a decentralized manner,
where the training of Face2Multi-modal is performed solely in a vehicle node,
where no data are aggregated into the centralized servers.

###  5.2. DP-enabled Training of Face2Multi-modal: Two Approaches

To enable DP-protected training of Face2Multi-modal, we leverage Global
Differential Privacy (GDP) for centralized training, and Local Differential
Privacy (LDP) for decentralized training on sensitive data respectively.
Hereby, we provide sufficient details regarding these two approaches, via
elaborating key definitions and differences as follows.

➊ Global Differential Privacy (GDP). GDP manages data for a centralized
scenario. In GDP, the application has a central server as the aggregator,
which accepts data from vehicle nodes. In this way, the aggregator (i.e.
centralized servers in IoV) can have the access to all raw data, and therefore
all data are exposed to the aggregator. The idea of GDP is to allow the
aggregator to collect all sensitive data from vehicle nodes, and apply a DP
mechanism to add random noise on the whole aggregated dataset. In this way,
GDP can protect all sensitive data by adding the noise globally, thus reducing
the possibility of privacy leakage when the attacker obtains the same
privilege as the aggregator.

➋ Local Differential Privacy (LDP). LDP protects data in a decentralized
scenario. In contrast to GDP, LDP protects data on the client side. As edge
devices could collect extensive data about users, end-users might confront the
privacy leakage problems when an attacker cracks their devices and controls
the access to their sensitive data. LDP protects users’ data by adding random
noise locally to provide safety guarantees. The processed sensitive data would
then be used for data sharing either locally or among the Vehicle Network.

We use GDP and LDP for centralized and decentralized training respectively.
For centralized training, GDP protection works by 1) collecting all drivers’
statistics in centralized servers; 2) adding noises using DP on the gathered
data set; and 3) performing training for Face2Multi-modal. As for
decentralized training, LDP protection works by 1) adding noises to individual
drivers’ statistics in-vehicle nodes; and 2) performing training for
Face2Multi-modal.

###  5.3. Configurations of Our Characterizations

We vary the configurations during the training of Face2Multi-modal.
Specifically, we consider three aspects of our experiments. First, we vary the
range of $\epsilon$ (i.e. $\epsilon$ reflects the extent of noises on raw
data), when applying GDP and LDP. Then, we vary the objective types for DP-
enabled protection, including both images (i.e. input for Face2Multi-modal)
and textural information (i.e. output of Face2Multi-modal).

➊ GDP Parameters. Our emulation aggregates all data together, namely
$F=F_{1}+...+F_{N}$ , $S=S_{1}+...+S_{N}$, where $F$ and $S$ are fed into the
training of Face2Multi-Modal. During this procedure, we vary the value of
epsilon (i.e. $\epsilon$). Therefore, we can get different $F$ and/or $S$, as
the source data, to perform the training of Face2Multi-modal.

➋ LDP Parameters. Our emulation considers the whole procedure that is conduced
within each vehicle node under LDP protection. Therefore, each vehicle node
has its own $F$ and $S$, adds their own noises and trains their own models.
Similarly, we also vary the value of epsilon as we did in GDP parameters.

➌ Protected Data Types. Our characterizations focus on the data types,
suggested by Face2Multi-modal. We consider raw data for training, including
both input and output for training, can be protected by DP. We enable the
protection on reasonable combinations of images and textual information.

###  5.4. Characterization Results

In this section, we present the characterization results of GDP and LDP
protection approaches on Face2Multi-Modal. First, we visualize the
$\epsilon$-DP protection of image data in different settings of $\epsilon$.
Next, we showcase the validation accuracy of Face2Multi-modal, in terms of
textual information, for GDP and LDP approaches, with protecting drivers’
facial images only. Finally, we present the validation accuracy of Face2Multi-
Modal when both textual and images are protected under DP simultaneously.

Figure 6 presents the validation accuracy for GDP and LDP in various
$\epsilon$ settings. We make two key observations. First, LDP outperforms GDP
in the Face2Multi-Modal system by an average of 1.7X improvement of estimation
accuracy. This is because Deep Neural Network Models on each vehicle are
trained locally and customized for specific drivers. This suggests that LDP is
a better solution, compared with GDP, when deploying DP in the application-
specific manner (i.e. Key Observation 3).  Second, the validation accuracy of
both approaches keeps growing from the lowest $\epsilon$ value to the highest
value, since high $\epsilon$ value adds relatively low noises on the raw data.
Therefore, the validation accuracy is close to the validation accuracy, when
training on the raw data.

Figure 6 showcases the Validation Accuracy of GDP and LDP after introducing
$\epsilon$-Differential Privacy to protect textual information (i.e.
$\epsilon$=0.9) on the basis of various $\epsilon$ settings for image
protection. By comparing the results in Figure 6 and Figure 6, we can conclude
that both GDP and LDP protection on textual data can potentially destroy the
performance of the whole application (i.e. Key Observation 4), by reducing the
validation accuracy to one thirds of the previous one.

Figure 5. Validation Accuracy between GDP and LDP in different Epsilon
settings for facial images.

Figure 6. Validation Accuracy between GDP and LDP in different Epsilon
settings for textual information.

Figure 7 shows the visual effects of the same image under different $\epsilon$
settings. From visual inspection, the extent of noises, added into the image,
is in inverse proportion to the values of $\epsilon$. For ➊, the facial image
is ”over-protected”, which indicates that the random noises might destroy the
inherent biological facial features used for Deep-Neural-Network model
inference thus degrading the overall performance. While for ➑, it has a
relatively high $\epsilon$ value, which protects the image with minimum
effort. Therefore, to strike a balance between data integrity and privacy
preserving, a preferred setting of $\epsilon$ might be located in the range
between 0.01 and 1.2 (i.e. Key Observation 5).

Figure 7. Different $\epsilon$ settings and corresponding visual effects. For
each image, the $\epsilon$ values are ➊ $\epsilon$ =0.005; ➋ $\epsilon$ =0.01;
➌ $\epsilon$ =0.02; ➍ $\epsilon$ =0.05; ➎ $\epsilon$ =0.1; ➏ $\epsilon$ =1.

##  6\. A Summary of The Most Significant Observations & Our Takeaways

In this section, we summarize our FIVE significant observations, derived from
our 11 key findings through the rigorous characterizations, and discuss our
takeaways from them. We first describe our most significant observations in
detail (Section 6.1). Then we elaborate our takeaways from these observations
and suggest both challenges and opportunities for privacy-preserving IoV
(Section 6.2).

###  6.1. FIVE Most Significant Observations

We summarize our key observations into the following five aspects, derived
from two separate characterizations (Note that DP refers to Differential
Privacy, IoV refers to Internet-of-Vehicles, GDP refers to Global Differential
Privacy, LDP refers to Local Differential Privacy):

➊ There are no one-size-fits-all mechanisms of DP for all applications of IoV,
in terms of Relative Error.

➋ There are no one-size-fits-all mechanisms of DP for all applications of IoV,
in terms of Absolute Error;.

➌ To strike a balance between data integrity and privacy preserving, a
preferred setting of $\epsilon$ might be located in the range between 0.01 and
1.2.

➍ LDP is a better solution, compared with GDP, when deploying DP in the
application-specific manner.

➎ GDP and LDP protection on textual data can potentially destroy the
performance of the application.

We conjecture that these findings have covered all major design issues, when
applying DP in the context of IoV.

###  6.2. Takeaways of Our Characterizations

We discuss three major takeaways of our characterizations, based on the most
significant observations. First, since there are no one-size-fits-all
solutions for privacy-preserving IoV, by using DP. We believe there are a
large number of opportunities to explore privacy-preserving techniques for IoV
in practice. Second, for data processing on sensitive data, we already narrow
the large design scope into a restricted region, by examining the value of
$\epsilon$ in diverse settings. We believe this is very useful for follow-up
studies, to avoid redundant workloads of attempted settings. Third, for data
sharing on sensitive data, we already demonstrate that application-specific DP
protections can potentially destroy the service quality. Therefore, we believe
there are a huge amount of possibilities to co-design the application and
privacy-preserving mechanisms (e.g., DP-based protection for unbalanced batch
queries in IoV (Liu et al., 2022)), to ensure high service quality with
privacy protection at the same time.

##  7\. Conclusions

We present the most comprehensive study to characterize modern privacy-
preserving techniques for IoV. Our study provides 11 key findings for applying
DPtechniques in the context of IoVand we highlight FIVE most significant
observations from our detailed characterizations. We focus on Differential
Privacy (DP), a representative set of mathematically-guaranteed mechanisms for
both privacy-preserving processing and sharing on sensitive data. The purpose
of our study is to demystify the tradeoffs of leveraging DP techniques, with
service quality as the evaluation metrics. We first characterize four
representative privacy-preserving processing, enabled from advanced DP. Then
we perform a detailed study of an emerging in-vehicle, Deep-Neural-Network-
driven application, and study the strength and weakness of DP for diverse
types of data streams. We conclude that there is a large volume of challenges
and opportunities for future studies, by enabling privacy-preserving IoV with
low overheads for service quality.

## References

  * (1)
  * Aman et al. (2021) Muhammad Naveed Aman, Uzair Javaid, and Biplab Sikdar. 2021\.  A Privacy-Preserving and Scalable Authentication Protocol for the Internet of Vehicles.  _IEEE Internet Things J._ 8, 2 (2021), 1123–1139.  https://doi.org/10.1109/JIOT.2020.3010893
  * Barak et al. (2007) Boaz Barak, Kamalika Chaudhuri, Cynthia Dwork, Satyen Kale, Frank McSherry, and Kunal Talwar. 2007\.  Privacy, Accuracy, and Consistency Too: A Holistic Solution to Contingency Table Release. In _Proceedings of the Twenty-Sixth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems_ (Beijing, China) _(PODS ’07)_. Association for Computing Machinery, New York, NY, USA, 273–282.  https://doi.org/10.1145/1265530.1265569
  * Chen and Chang (2017) Lien-Wu Chen and Chia-Chen Chang. 2017.  Cooperative Traffic Control With Green Wave Coordination for Multiple Intersections Based on the Internet of Vehicles.  _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ 47, 7 (2017), 1321–1335.  https://doi.org/10.1109/TSMC.2016.2586500
  * Chou et al. (2018) Edward Chou, Josh Beal, Daniel Levy, Serena Yeung, Albert Haque, and Li Fei-Fei. 2018\.  Faster CryptoNets: Leveraging Sparsity for Real-World Encrypted Inference.  _CoRR_ abs/1811.09953 (2018).  arXiv:1811.09953 http://arxiv.org/abs/1811.09953
  * Cordero et al. (2020) Jorge Cordero, Jose Aguilar, Kristell Aguilar, Danilo Chávez, and Eduard Puerto. 2020\.  Recognition of the Driving Style in Vehicle Drivers.  _Sensors_ 20, 9 (2020).  https://doi.org/10.3390/s20092597
  * Dandala et al. (2017) Tej Tharang Dandala, Vallidevi Krishnamurthy, and Rajan Alwan. 2017.  Internet of Vehicles (IoV) for traffic management. In _2017 International Conference on Computer, Communication and Signal Processing (ICCCSP)_. 1–4.  https://doi.org/10.1109/ICCCSP.2017.7944096
  * Davydov and Bezzateev (2020) Vadim Davydov and Sergey Bezzateev. 2020.  Accident Detection in Internet of Vehicles using Blockchain Technology. In _2020 International Conference on Information Networking (ICOIN)_. 766–771.  https://doi.org/10.1109/ICOIN48656.2020.9016602
  * Ding et al. (2011) Bolin Ding, Marianne Winslett, Jiawei Han, and Zhenhui Li. 2011\.  Differentially private data cubes: optimizing noise sources and consistency. In _Proceedings of the 2011 ACM SIGMOD International Conference on Management of data_. 217–228.
  * Gabay et al. (2020) David Gabay, Kemal Akkaya, and Mumin Cebe. 2020\.  Privacy-Preserving Authentication Scheme for Connected Electric Vehicles Using Blockchain and Zero Knowledge Proofs.  _IEEE Trans. Veh. Technol._ 69, 6 (2020), 5760–5772.  https://doi.org/10.1109/TVT.2020.2977361
  * Gilad-Bachrach et al. (2016) Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin E. Lauter, Michael Naehrig, and John Wernsing. 2016.  CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. In _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016_ _(JMLR Workshop and Conference Proceedings)_ , Maria-Florina Balcan and Kilian Q. Weinberger (Eds.), Vol. 48. JMLR.org, 201–210.  http://proceedings.mlr.press/v48/gilad-bachrach16.html
  * Hay et al. (2010) Michael Hay, Vibhor Rastogi, Gerome Miklau, and Dan Suciu. 2010.  Boosting the Accuracy of Differentially Private Histograms through Consistency.  _Proc. VLDB Endow._ 3, 1–2 (Sept. 2010), 1021–1032.  https://doi.org/10.14778/1920841.1920970
  * Hua et al. (2017) Ling Hua, Hossein Anisi, Yee Por, and Muhammad Alam. 2017\.  Social Networking-based Cooperation Mechanisms in Vehicular Ad-Hoc Network- A Survey.  _Vehicular Communications_ 10 (11 2017).  https://doi.org/10.1016/j.vehcom.2017.11.001
  * Huang et al. (2020) Zhentao Huang, Rongze Li, Wangkai Jin, Zilin Song, Yu Zhang, Xiangjun Peng, and Xu Sun. 2020.  Face2Multi-modal: in-vehicle multi-modal predictors via facial expressions. In _12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_. 30–33.
  * Jin et al. (2021) Wangkai Jin, Xiaoxing Ming, Zilin Song, Zeyu Xiong, and Xiangjun Peng. 2021.  Towards Emulating Internet-of-Vehicles on a Single Machine. In _AutomotiveUI ’21: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Leeds, United Kingdom, September 9-14, 2021 - Adjunct Proceedings_. ACM, 112–114.  https://doi.org/10.1145/3473682.3480275
  * Khatoun et al. (2015) R. Khatoun, P. Gut, R. Doulami, L. Khoukhi, and A. Serhrouchni. 2015.  A reputation system for detection of black hole attack in vehicular networking. In _2015 International Conference on Cyber Security of Smart Cities, Industrial Control System and Communications (SSIC)_. 1–5.  https://doi.org/10.1109/SSIC.2015.7245328
  * Kilari et al. (2020) Vishnu Teja Kilari, Ruozhou Yu, Satyajayant Misra, and Guoliang Xue. 2020.  Robust Revocable Anonymous Authentication for Vehicle to Grid Communications.  _IEEE Trans. Intell. Transp. Syst._ 21, 11 (2020), 4845–4857.  https://doi.org/10.1109/TITS.2019.2948803
  * Kim and Baek (2020) Beomjun Kim and Yunju Baek. 2020.  Sensor-Based Extraction Approaches of In-Vehicle Information for Driver Behavior Analysis.  _Sensors_ 20, 18 (2020).  https://doi.org/10.3390/s20185197
  * Liu et al. (2022) Junyu Liu, Wangkai Jin, Zhenyong He, Xiaoxing Ming, Yicun Duan, Zeyu Xiong, and Xiangjun Peng. 2022.  HUT: Enabling High-UTility, Batched Queries under Differential Privacy Protection for Internet-of-Vehicles.  Technical Report-Feb-02 at User-Centric Computing Group, University of Nottingham Ningbo China.
  * Mehrish et al. (2017) Ambuj Mehrish, A.V. Subramanyam, and Mohan Kankanhalli. 2017\.  Multimedia signatures for vehicle forensics.  https://doi.org/10.1109/ICME.2017.8019488
  * Panichpapiboon and Pattara-atikom (2012) Sooksan Panichpapiboon and Wasan Pattara-atikom. 2012.  A Review of Information Dissemination Protocols for Vehicular Ad Hoc Networks.  _Communications Surveys & Tutorials, IEEE_ 14 (01 2012), 1 – 15.  https://doi.org/10.1109/SURV.2011.070711.00131
  * Peng et al. (2020) Xiangjun Peng, Zhentao Huang, and Xu Sun. 2020.  Building BROOK: A Multi-modal and Facial Video Database for Human-Vehicle Interaction Research.  (2020), 1–9.  arXiv:2005.08637 https://arxiv.org/abs/2005.08637
  * Sadiku et al. (2018) Matthew NO Sadiku, Mahamadou Tembely, and Sarhan M Musa. 2018\.  Internet of vehicles: An introduction.  _International Journal of Advanced Research in Computer Science and Software Engineering_ 8, 1 (2018), 11.
  * Sharma and Kaushik (2019) Surbhi Sharma and Baijnath Kaushik. 2019.  A survey on internet of vehicles: Applications, security issues & solutions.  _Vehicular Communications_ 20 (2019), 100182.  https://doi.org/10.1016/j.vehcom.2019.100182
  * Sze et al. (2017) Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel Emer. 2017\.  Efficient Processing of Deep Neural Networks: A Tutorial and Survey.  arXiv:cs.CV/1703.09039
  * Wang et al. (2021) Jiahao Wang, Zeyu Xiong, Yicun Duan, Junyu Liu, Zilin Song, and Xiangjun Peng. 2021\.  The Importance Distribution of Drivers’ Facial Expressions Varies over Time!. In _AutomotiveUI ’21: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Leeds, United Kingdom, September 9-14, 2021 - Adjunct Proceedings_. ACM, 148–151.  https://doi.org/10.1145/3473682.3480283
  * Xiao et al. (2010) Xiaokui Xiao, Guozhang Wang, and Johannes Gehrke. 2010\.  Differential privacy via wavelet transforms.  _IEEE Transactions on knowledge and data engineering_ 23, 8 (2010), 1200–1214.
  * Xiong et al. (2022) Zeyu Xiong, Jiahao Wang, Wangkai Jin, Junyu Liu, Yicun Duan, Zilin Song, and Xiangjun Peng. 2022.  Face2Statistics: User-Friendly, Low-Cost and Effective Alternative to In-Vehicle Sensors/Monitors for Drivers. In _International Conference on Human-Computer Interaction_.
  * Zhang et al. (2021b) Yu Zhang, Wangkai Jin, Zeyu Xiong, Zhihao Li, Yuyang Liu, and Xiangjun Peng. 2021b.  Demystifying Interactions Between Driving Behaviors and Styles Through Self-clustering Algorithms. In _International Conference on Human-Computer Interaction_ , Heidi Krömker (Ed.).  https://doi.org/10.1007/978-3-030-78358-7_23
  * Zhang et al. (2021a) Zheng Zhang, Junjun Huang, and Shouqi Cao. 2021a.  Marine Cold Chain Transportation Monitoring and Route Scheduling Optimization Based on IoV-BDS.  _IEEE Access_ 9 (2021), 20557–20574.  https://doi.org/10.1109/ACCESS.2021.3054388
  * Zhao et al. (2019) Zhong-Qiu Zhao, Peng Zheng, Shou-Tao Xu, and Xindong Wu. 2019\.  Object Detection With Deep Learning: A Review.  _IEEE Transactions on Neural Networks and Learning Systems_ 30, 11 (2019), 3212–3232.  https://doi.org/10.1109/TNNLS.2018.2876865
  * Zhou et al. (2020) Mohan Zhou, Yalong Bai, Wei Zhang, Tiejun Zhao, and Tao Mei. 2020.  Look-Into-Object: Self-Supervised Structure Modeling for Object Recognition. In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 11771–11780.  https://doi.org/10.1109/CVPR42600.2020.01179