# Privacy-Aware Human Mobility Prediction via Adversarial Networks

Yuting Zhan, Hamed Haddadi  Imperial College London UK and  Alex Kyllo, Afra
Mashhadi  University of Washington USA

(2018)

###### Abstract.

As various mobile devices and location-based services are increasingly
developed in different smart city scenarios and applications, many unexpected
privacy leakages have arisen due to geolocated data collection and sharing.
While these geolocated data could provide a rich understanding of human
mobility patterns and address various societal research questions, privacy
concerns for users’ sensitive information have limited their utilization. In
this paper, we design and implement a novel LSTM-based adversarial mechanism
with representation learning to attain a privacy-preserving feature
representation of the original geolocated data (i.e., mobility data) for a
sharing purpose. We quantify the utility-privacy trade-off of mobility
datasets in terms of trajectory reconstruction risk, user re-identification
risk, and mobility predictability. Our proposed architecture reports a Pareto
Frontier analysis that enables the user to assess this trade-off as a function
of Lagrangian loss weight parameters. The extensive comparison results on four
representative mobility datasets demonstrate the superiority of our proposed
architecture and the efficiency of the proposed privacy-preserving features
extractor. Our results show that by exploring Pareto optimal setting, we can
simultaneously increase both privacy (45%) and utility (32%).

mobility datasets, LSTM neural networks, mobility prediction, data privacy

††copyright: acmcopyright††journalyear: 2018††doi:
10.1145/1122445.1122456††conference: ; ; ††price: 15.00††isbn:
978-1-4503-XXXX-X/18/06††ccs: Security and privacy Privacy protections††ccs:
Human-centered computing Interactive systems and tools††ccs: Networks Network
performance analysis

##  1\. Introduction

Geolocation and mobility data collected by location-based services (LBS)
(Huang et al., 2018), can reveal human mobility patterns and address various
societal research questions (Kolodziej and Hjelm, 2017). For example, Call
Data Records (CDR) have been successfully used to provide real-time traffic
anomaly and event detection (Toch et al., 2019; Wang et al., 2020), and a
variety of mobility datasets have been used in shaping policies for urban
communities (Ferreira et al., 2020) and epidemic management in the public
health domain (Oliver et al., 2015, 2020). Human mobility prediction based on
users’ trajectories, a popular and emerging topic, supports a series of
important applications ranging from individual-level recommendation systems to
large-scale smart transportation.

While there is no doubt of the usefulness of predictive applications for
mobility data, privacy concerns regarding the collection and sharing of
individuals’ mobility traces have prevented the data from being utilized to
their full potential (Shokri et al., 2011; Beresford and Stajano, 2003; Krumm,
2009). A mobility privacy study conducted by De Montjoye et al (De Montjoye et
al., 2013) illustrates that four spatio-temporal points are enough to identify
95% of the individuals in a certain granularity. As human mobility traces are
highly unique, a mechanism capable of decreasing the user re-identification
risk can offer enhanced privacy protection in mobility data sharing.

In the past decade, the research community has extensively studied privacy of
geolocated data via various location privacy protection mechanisms (LPPM)
(Gedik and Liu, 2005, 2007). Some traditional privacy-preserving approaches
such as k-anonymity and geo-masking have shown to be insufficient to prevent
users from being re-identified (De Montjoye et al., 2013; Song et al., 2010;
Gonzalez et al., 2008; Malekzadeh et al., 2020). More recently, some related
works also try to apply machine-learning or deep-learning based approaches to
explore the effective LPPM. Rao et al proposed an LSTM-TrajGAN model to
generate privacy-preserving synthetic mobility datasets for data sharing and
publication (Rao et al., 2020). Feng et al investigated human mobility data
with privacy constraints via federated learning, achieving promising
prediction performance while preserving the personal data on the local devices
(Feng et al., 2020). Though these state-of-the-art models provide a reasonable
balance between utility and privacy, the effectiveness of the privacy
mechanism and utility metrics have not been fully investigated.

An appropriate and effective framework to allow researchers and practitioners
to easily assess the trade-off between utility and privacy of mobility
datasets at various granularity currently does not exist and can be highly
impactful for the research community.

(a)

(b)

Figure 1. (a) Schematic overview of the proposed privacy-preserving
adversarial architecture with representation learning, consisting of data
reconstruction risk unit, mobility prediction unit, and user re-identification
risk unit. (b) The baseline LSTM network for standalone classifiers.

proposed architecture

To this end, we posit an architecture for quantifying the utility-privacy
trade-off of mobility datasets in terms of data reconstruction leakage (i.e.,
Privacy I), user re-identification risk (i.e., Privacy II), and mobility
predictability (i.e., Utility). In order to do so, we explore a novel
mechanism to investigate these trade-offs and train a privacy-preserving
feature extractor $Enc_{L}$ based on representation learning and adversarial
learning. Inspired by PAN (Liu et al., 2019) (privacy adversarial network), we
employ adversarial learning to better balance the potential trade-off between
privacy and utility. In contrast to PAN, which focuses on the privacy of
images, our approach is designed for complex time-series data that exhibits
spatial-temporal characteristics. At the core of our architecture lies an LSTM
auto-encoder (AE) with three branches, corresponding to the three training
optimization objectives of the feature extractor $Enc_{L}$: i) to $maximize$
the loss associated with the reconstructed output by generative learning, ii)
to $minimize$ the prediction loss using the learned representation from the
$Enc_{L}$ by discriminative learning, and iii) to $maximize$ the percentage of
users who are re-identifiable through their trajectories by discriminative
learning. We use Lagrange multipliers to vary the weights that are given to
each of these objectives before combining them into a total loss, $L_{sum}$.
The output of this model is a Pareto-Frontier analysis that would guide the
user in investigating the trade-off between utility and privacy.

We report the analysis of our architecture by a thorough evaluation on four
real-world representative mobility datasets. A benchmark comparison is carried
out with the state-of-the-art algorithm based on Generative Adversarial
Network (GAN) (Goodfellow et al., 2014), namely LSTM-TrajGAN (Rao et al.,
2020). The results show that the synthetic dataset generated by TrajGAN is not
pareto-optimal in all four cases. That is, in the given spatial-temporal
granularity, it is possible to achieve a better privacy level for a dataset
with the same utility value and vice versa. The contributions of our work are
the following:

  * •

We posit an LSTM-based location privacy protection mechanism via
representation learning and adversarial learning to learn a privacy-preserving
feature extraction encoder, namely LSTM-PAE.

  * •

We provide extensive analysis on different mobility tasks and quantify the
privacy bound and utility bound of the target mobility dataset, along with a
trade-off analysis between these contrasting objectives.

  * •

We compare our model over four real-world mobility datasets with a state-of-
the-art GAN-based network that attempts to generate synthetic privacy-
preserving mobility data.

  * •

We offer the architecture as an open-source system to the researchers and
practitioners.

The rest of this paper is structured as follows: Section 2 describes our
proposed LSTM-PAE model. In Section 3, we describe the experimental settings
for our reported results and provide a brief overview of the datasets used to
evaluate our framework. Section 4 reports an in-depth analysis of our proposed
LSTM-PAE model. In Section 5, we demonstrate an evaluation of our framework
over four mobility datasets and report on the utility-privacy trade-off of our
proposed method in comparison with the state-of-the-art GAN based model.
Finally, we review the related work in Section 6 and conclude the paper with
future work directions in Section 7.

##  2\. Design of the Architecture

###  2.1. Problem Definitions

Before describing our proposed LSTM-PAE model in detail, we first give a brief
problem definition of the trade-off between mobility data utility and privacy
in terms of mobility prediction accuracy, user re-identification efficiency
and data reconstruction differences.

Data Utility: Mobility datasets are of great value for understanding human
behavior patterns, smart transportation, urban planning, public health issue,
pandemic management, and etc. Many of these applications rely on the next
location forecasting of individuals, which in the broader context can provide
an accurate portrayal of citizens’ mobility over time. Mobility prediction not
only can be analyzed to understand personalized mobility patterns, but can
also inform the allocation of public resources and community services. We
focus on the capability of _mobility prediction_ (_next location forecasting_)
in this paper, and leverage the accuracy of the prediction as an important
metric for quantifying the data utility. Hence, the definition of Utility is
concluded as followed:

Utility (U): the mobility predictability (i.e., prediction accuracy)

Privacy Protection: With more and more intelligent devices and sensors are
utilized to collect information about human activities, the trajectories also
expose increasing intimate details about users’ lives, from their social life
to their preferences. The capability of re-identification is important to
balance the risks and benefits of mobility data usage, for all data owners,
third parties, and researchers. We leverage the data reconstruction risk and
user re-identification risk as our privacy metrics to evaluate our proposed
privacy-aware architecture. Hence, the definition of Privacy is summarized as
followed:

Privacy I (PI): the differences between the reconstructed data $X^{\prime}$
and the original data $X$, that is, information loss in the reconstruction
process.

Privacy II (PII): the user re-identification inaccuracy, that is, the user de-
identification effectiveness.

###  2.2. Architecture Overview

Figure 1(b) presents the basic workflow of the proposed privacy-preserving
adversarial architecture with representation learning. We aim to train an
encoder $Enc_{L}$ to produce feature representations $f$ from the target
mobility data by jointly optimizing these extracted feature weights using the
combined losses of the data reconstruction risk unit, mobility prediction
unit, and user re-identification risk unit simultaneously, during adversarial
training. The encoder $Enc_{L}$ is trained to achieve a better trade-off
between predictability accuracy and user privacy budgets by extracting more
information about the mobility predictability but less about the user privacy.
The utility is measured via the mobility prediction task (i.e., prediction
accuracy), and the privacy leakage is quantified with two risks, listed as the
data reconstructor risk (i.e., information loss) and the user re-
identification risk (i.e., re-identification inaccuracy). Hence, the proposed
LSTM-PAE model is composed of three crucial units. We discuss the
functionality of each unit before we discuss the overall performance of the
model.

####  2.2.1. Architecture Composition Units

I. Mobility Prediction Unit:

i. Definition of Mobility Trajectory: the raw geolocated data or other
mobility data commonly contain three elements: user identifier u, timestamps
t, location identifiers l. Hence, each location records r could be denoted as
_$r_{i}$_ = [$u_{i}$, $t_{i}$, $l_{i}$], while each location sequence _S_ is a
set of ordered location records _$S_{n}$_ = {$r_{1}$, $r_{2}$, $r_{3}$,
$\cdots$, $r_{n}$}, namely mobility trajectory. Therefore, given the past
mobility trajectory _$S_{n}$_ = {$r_{1}$, $r_{2}$, $r_{3}$, $\cdots$,
$r_{n}$}, the mobility prediction task is to infer the most likely location _l
n+1_ at the next timestamp _t n+1_. The data fed into the encoder $Enc_{L}$
are a list of trajectories with specific sequence length _SL_ , that is
{$S_{sl}^{1}$, $S_{sl}^{2}$, $S_{sl}^{3}$, $\cdots$, $S_{sl}^{j}$}. For
instance, if the sequence length is 10, that indicates each trajectory
contains 10 history location records r, _$S_{10}$_ = {$r_{1}$, $r_{2}$,
$r_{3}$, $\cdots$, $r_{10}$}, and $SL=10$.

ii. Unit Design: the mobility prediction unit is regarded as the utility
discriminator $U_{D}$ in the proposed architecture, as shown in Figure 1(b).
This unit is composed of three parts, the input part with multi-modal
embedding of trajectory information, the sequential part with LSTM layers, and
an output part with the softmax activation function. As per the aforementioned
definition, the trajectories in this work are shown as location sequences _S_.
First, the location identifiers l and timestamps t are converted into one-hot
vectors. We then employ long short-term memory (LSTM) (Hochreiter and
Schmidhuber, 1997) layers to model the mobility patterns and sequential
transition relations in these mobility trajectories. As a prominent variant of
the recurrent neural network, LSTM networks exhibit brilliant performance on
modelling entire sequences of data, especially for learning long-term
dependencies via gradient descent (Zhan and Haddadi, 2019). Following the
sequential module, the softmax layer outputs the probability distribution of
the prediction results. This probability distribution is converted to the
top-n accuracy metrics to illustrate the unit performance.

II. User Re-identification Risk Unit:

i. Definition of User Re-identification: the privacy risk via user re-
identification risk arises due to the high uniqueness of human traces (De
Montjoye et al., 2013). We assume each trajectory _S_ is originally labeled
with a corresponding user identifier u, and the user re-identification risk
unit is to infer the user u to whom the target trajectory _$S_{n}$_ =
{$r_{1}$, $r_{2}$, $r_{3}$, $\cdots$, $r_{n}$} belongs. We thereby leverage
the user identifiers as the ground-truth values for the user identity classes.
However, this identity information is what we want to protect in the proposed
adversarial network, that is, the extracted features $f$ should convey as
little user identifiable information as possible, to decrease the user re-
identification accuracy.

ii. Unit Design: the user re-identification risk unit is regarded as the
privacy discriminator $P_{D}$ in the proposed architecture. the unit is
composed of three parts, the input part with one-hot embedding of user
identity, the sequential part with LSTM layers, and an output part with
softmax function. First, the user identity list is converted into one-hot
vectors. Similar to the mobility prediction unit, the user re-identification
risk unit also applies LSTM layers to better extract the spatial and temporal
characteristics of the context. A softmax function with cross-categorical
entropy loss function is applied to output a categorical probability
distribution of the user re-identification task. We then use the top-n
accuracy of this classifier as the metrics of user re-identification privacy
risk (i.e., Privacy II). The more accurately a classifier can re-identify the
user when given a trajectory, the higher the risk of disclosing private data.

III. Data Reconstruction Risk Unit: this unit is the encoder $Enc_{L}$ unit in
reverse. It is designed to evaluate the differences (i.e., Privacy I) between
the reconstructed data $X^{\prime}$ and the original input data $X$. A
malicious party is free to explore any machine learning model and reconstruct
the data if they have the shared extracted features $f$. We use a layer-to-
layer reverse architecture of our encoder $Enc_{L}$ to build a strong Data
Reconstruction Risk Unit. We leverage Euclidean and Manhattan distance as our
metrics to measure the differences between the $X$ and $X^{\prime}$.

Following the basic discussion of three units, we present the overall design
of the proposed LSTM-PAE model.

####  2.2.2. Overall Design

Dataset-City | Bounding Box | Record Counts | Number
---|---|---|---
Latitude | Longitude | Train | Test | User ID | POI | Weekday | Hour
MDC-Lausanne | 46.50 | 46.61 | 6.58 | 6.73 | 77393 | 19429 | 143 | 149 | 7 | 24
Privamov-Lyon | 45.70 | 45.81 | 4.77 | 4.90 | 62077 | 16859 | 58 | 129 | 7 | 24
GeoLife-Beijing | 39.74 | 40.07 | 116.23 | 116.56 | 95038 | 24578 | 145 | 960 | 7 | 24
FourSquare-NYC | 40.55 | 40.99 | -74.28 | -73.68 | 43493 | 11017 | 466 | 1712 | 7 | 24
Table 1. Overview of four mobility datasets.

Our proposed privacy-preserving adversarial feature encoder, the LSTM-PAE, is
based on representation learning and adversarial learning and aims to ease
data sharing privacy concerns. As shown in Figure 1(a), we design a multi-task
adversarial network to learn an LSTM-based encoder $Enc_{L}$, which can
generate the optimized feature representations $f=Enc_{L}(X)$ via lowering
privacy disclosure risk of user identification information and improving the
task accuracy (i.e., mobility predictability) concurrently. Two potential
malicious privacy leakages from the data reconstruction risk unit and the user
re-identification risk unit, are attempted to retrieve sensitive information
from the feature representations $f$.

Given mobility raw data $\mathcal{X}$ for Privacy I (e.g., data reconstruction
risk unit), the ground-truth label $z_{i}$ for Privacy II (e.g., user re-
identification risk unit), and the ground-truth label $y_{i}$ for utility
(e.g., mobility prediction), we train the LSTM encoder $Enc_{L}$ of this
multi-task network by adversarial learning to learn the representation
$\mathcal{F}=Enc_{L}(\mathcal{X})$. We design a specific loss function, namely
sum loss $\mathcal{L}_{sum}$, for this optimization process. Specifically,
when reconstructing the data $\mathcal{X^{\prime}}$, an LSTM decoder $Dec_{L}$
attempts to recreate the data based on the features $\mathcal{F}$, that is
$Dec_{L}:\mathcal{F}\rightarrow\mathcal{X^{\prime}}$. This data reconstruction
unit is trained by maximizing the reconstruction loss $\mathcal{L_{R}}$ while
minimizing the $\mathcal{L}_{sum}$. The mobility prediction unit, that is the
utility discriminator $U_{D}$, is trained to output a probability distribution
of the next location of interest, and this distribution has Y potential
classes. Discriminative training here is to maximize the prediction accuracy
by minimizing the utility loss $\mathcal{L_{U}}$, denoted as
$\min\mathcal{L_{U}}$. The user re-identification risk unit, that is the
privacy discriminator $P_{D}$, is trained to re-identify whom the target
trajectory belongs. Then in this privacy discriminator, the user re-
identification loss $\mathcal{L_{P}}$ is maximized, denoted as
$\max\mathcal{L_{P}}$.

In general, the encoder $Enc_{L}$ should satisfy high predictability (min
$\mathcal{L_{U}}$) and low user re-identification accuracy (max
$\mathcal{L_{P}}$) of the mobility data when maximizing the reconstruction
loss (max $\mathcal{L_{R}}$) in reverse engineering, where the training
objective can be written as (details is in Algorithm 1 attached in Appendix):

(1) $\displaystyle\text{min}\left(\mathcal{L}_{sum}\right)$
$\displaystyle=-\lambda_{1}\left(\max\mathcal{L_{R}}\right)+\lambda_{2}\left(\min\mathcal{L_{U}}\right)-\lambda_{3}\left(\max\mathcal{L_{P}}\right)$
$\displaystyle=-\lambda_{1}\|Dec_{L}(\mathcal{F})-\mathcal{X}\|^{2}+\lambda_{2}(-\sum_{i=1}^{\mathbf{Y}}y_{i}\text{log}(U_{D}(\mathcal{F})))$
$\displaystyle-\lambda_{3}(-\sum_{i=1}^{\mathbf{Z}}z_{i}\text{log}(P_{D}(\mathcal{F})))$

where $y_{i}$ is the ground-truth label for Utility, $z_{i}$ is the ground-
truth value for Privacy II; $\lambda_{1}$, $\lambda_{2}$ and $\lambda_{3}$ are
Lagrange multipliers (Beavis and Dobbs, 1990).

The overall training is to achieve privacy-utility trade-off by adversarial
learning on $\mathcal{L_{R}}$, $\mathcal{L_{U}}$, and $\mathcal{L_{P}}$
concurrently. The gradient of the loss (i.e., $\theta_{R}$, $\theta_{U}$,
$\theta_{P}$) back-propagates through the LSTM network to guide the training
of the encoder $Enc_{L}$. The encoder is updated with the sum loss function
$\mathcal{L}_{sum}$ until convergence. The Lagrange multipliers are utilized
to find the maxima or minima of a constrained problem. When two of them are
set to zero, the model is transformed to a specific evaluation tool for a
specific task. When three multipliers are utilized together, they control the
relative importance of each unit and guide the overall model to find the
maxima or minima given the specific trade-off choices.

##  3\. Experimental Setting

###  3.1. Datasets

Experiments are conducted on four representative mobility datasets: Mobile
Data Challenge Dataset (MDC) (Laurila et al., 2012), PRIVA’MOV (Mokhtar et
al., 2017),
FourSquare NYC (Yang et al., 2014), and GeoLife (Zheng et al., 2011). Once
imported into our architecture, each dataset was filtered and preprocessed
individually to derive their respective test and training sets illustrated in
Table 1.

MDC: The MDC dataset, recorded from 2009 to 2011, contains a large amount of
continuous mobility data for 184 volunteers with smartphones running a data
collection software, in the Lausanne/Geneva area. Each record of the gps-wlan
dataset represents a phone call or an observation of a WLAN access point
collected during the campaign (Laurila et al., 2012).

PRIVA’MOV: The PRIVA’MOV crowd-sensing campaign took place in the city of
Lyon, France from October 2014 to January 2016. Data collection was
contributed by roughly 100 participants including university students, staff,
and family members. The crowd-sensing application collected GPS, WiFi, GSM,
battery, and accelerometer sensor data. For the purpose of this project, we
only used the GPS traces from the dataset (Mokhtar et al., 2017).

GeoLife: The GeoLife dataset was collected by Microsoft Research Asia from 182
users in the four and a half year period from April 2007 to October 2011 and
contains 17,621 trajectories, mostly at a 5-second sampling rate (Zheng et
al., 2011).

FourSquare NYC: The Foursquare NYC dataset contains check-ins in NYC and Tokyo
collected during the approximately 10 months from 12 April 2012 to 16 February
2013, containing 227,428 check-ins from 1,083 subjects in New York City (Yang
et al., 2014).

(a) Utility discriminator $U_{D}$
Performance on MDC

(b) Privacy discriminator $P_{D}$ Performance on MDC

(c) Utility discriminator $U_{D}$
Performance on Priva’Mov

(d) Privacy discriminator $P_{D}$ Performance on Priva’Mov

Figure 2. Mobility prediction accuracy and user re-identification accuracy
change with the trajectory sequence length (SL) in our proposed mobility
prediction unit and user re-identification unit. The color bars indicate the
accuracy from top-1 to top-5, the black texts indicate the top-1 accuracy and
the purple texts indicate the top-5 accuracy. For instance, the top-1 mobility
prediction accuracy on MDC with SL = 2 is 0.473, and the top-5 one is 0.802.

The effect of the context length

###  3.2. Training

####  3.2.1. Training of LSTM-PAE

The main goal of the proposed adversarial network is to learn an efficient
feature representation based on the utility and privacy budgets, using all
users’ mobility histories. In most experiments in this work, the trajectory
sequences consist of 10 historical locations with timestamps (i.e., $SL=10$),
and the impact of the varying sequence lengths is discussed in Section 4.1.
After the pre-processing of the datasets, 70% of the records of each user are
segmented as the training dataset, and the remaining 30% as the testing
dataset. We utilize the mini-batch learning method to train the model until
the expected convergence. We take a gradient step to optimize the sum loss
$L_{sum}$ (i.e., Equation 1) in terms of $L_{R}$, $L_{U}$, and $L_{P}$
concurrently. Meanwhile, the sum loss $L_{sum}$ is also optimized by using the
Adam optimizer. All the experiments were performed with the Tesla V100 GPU; a
round of training would take 30 seconds on average and each experiment trains
for 1000 rounds.

####  3.2.2. Training of the Comparison Model

To provide a state of the art trained model for comparison, we re-implement
the LSTM-TrajGAN model described in (Rao et al., 2020) using the same hyper-
parameters, setting latent vector dimension to 100, using 100 LSTM units per
layer, a batch size of 256, utilizing the Adam optimizer with learning rate
0.001 and momentum 0.5, and training for 200 epochs (where one epoch is a pass
through the entire training set). We train LSTM-TrajGAN independently on the
training split of each benchmark mobility dataset, and then use it to generate
synthetic trajectories from the test set. Then we train the proposed LSTM-PAE
on the same training data and use it to generate a feature extraction from the
same test data. Finally, we evaluate the performance of the user re-
identification unit and mobility prediction unit on the real and synthetic
test sets generated by LSTM-TrajGAN, and compare the changes in accuracy to
assess the relative utility and privacy of the TrajGAN and PAE.

###  3.3. Metrics

We set Euclidean (Ball, 1960) and Manhattan distance (Black et al., 1998) as
our evaluation metrics for the data reconstruction unit to evaluate the
quality of the reconstructed data $X^{\prime}$ generated from extracted
features $f$. Euclidean distance gives the shortest or minimum distance
between two points, while Manhattan distance applies only if the points are
arranged in the form of a grid, and both definitions are feasible for the
problem we are working on. Note that these two distances have limited
capability in showing the quality of the reconstructed data $X^{\prime}$,
however, they intuitively capture the differences between the original data
$X$ and the reconstructed data $X^{\prime}$.

We leverage the top-n accuracy as our evaluation metric for both mobility
prediction and user re-identification risk units. The accuracy of the mobility
prediction unit is one of the most important factors in evaluating the utility
of the extracted feature representation f, where predictability of the f
increases as much as it can during the adversarial training. On the other
hand, the competing training objective is to decrease the accuracy of the user
re-identification unit to enhance the privacy of f. The top-n metric computes
the number of times where the correct label appears among the top $n$ labels
predicted. The top-n metric takes n predictions with higher probability into
consideration and it classifies the prediction as correct if one of them is a
true label. The top-1 to top-5 accuracies (Zhan and Haddadi, 2019) are
leveraged in our paper to discuss the performance of the proposed model.

##  4\. Experiments

In this section, we first discuss the impact of the varying sequence length
and varying Lagrange multipliers on the composition units before presenting
the overall performance of the proposed architecture in next section.

###  4.1. Impact of Varying Sequence Lengths

The performance of utility discriminator $U_{D}$ (i.e., mobility prediction
unit) and the privacy discriminator $P_{D}$ (i.e., user re-identification risk
unit) exert great impact on the overall performance of the proposed LSTM-PAE.
The trajectory length is the most important factor which could affect these
units’ performance. We use two representative datasets (i.e., MDC and
Priva’Mov) to present the impact of the varying sequence length on both
discriminators.

By changing the lengths of trajectory sequence $SL$ from 1 ($SL=1$) to 50
($SL=50$), we observe that the length of trajectory sequence has a high impact
on different tasks’ accuracy (i.e., mobility prediction accuracy for $U_{D}$
and user re-identification accuracy for $P_{D}$) of two different datasets,
while the impact in the MDC dataset is much higher than in the Priva’Mov
dataset, as shown in the Figure 2(d). Comparing the Figure 2(a) and Figure
2(c), there is a much sharper increase on the MDC dataset. More specifically,
when the sequence length is increased from 2 to 20, the top-1 mobility
prediction accuracy on MDC increases from 0.473 to 0.978 (i.e., +50.5%), while
accuracy on Priva’Mov increases from 0.918 to 0.959 (i.e., only +4.1%).
Similarly, more rapid growth appears in the user re-identification accuracy on
MDC, which is +68.0%, while the increase for Priva’Mov is only +30.8%. We
conclude that the mobility predictability and user re-identification accuracy
of a dataset might have a special link. The mobility predictability of
Priva’Mov is very high, almost higher than 90%, but the user re-identification
accuracy is always lower than 80%, which also means the uniqueness of
trajectories in this dataset is low. This low uniqueness suggests that the
users in this dataset might share similar daily routes, which is reasonable,
as we know these trajectories are collected from students at the same
university. For the MDC dataset, when $SL=10$, the user re-identification
accuracy is quite high, indicating that the locations are more sparse in this
dataset. However, the mobility predictability of this dataset is also high,
which also emphasizes that this sparseness does not affect the predictability.
These phenomena indicate that the deep training of mobility predictability and
user re-identification might share similar extracted features, while our
proposed architecture attempts to extract features more suitable for mobility
predictability but less suitable for user re-identification.

We note that the varying trajectory sequence length not only exerts impacts on
the model performance, but also has a great influence on the computation time.
For instance, the computation time at $SL=50$ costs six times as much as the
one at $SL=5$. The computation time also varies between datasets. Hence, an
appropriate choice of trajectory sequence length can avoid time-consuming
computation and to achieve expected task inference accuracy. In our work, we
place greater focus on the trajectory sequence lengths ranging from 5 to 10,
which exhibits great performance in both the $U_{D}$ and $P_{D}$ while also
keeps a low computation time.

(a) $\lambda_{1}=1-\lambda_{2},~{}\lambda_{3}=0$

(b) $\lambda_{3}=1-\lambda_{2},~{}\lambda_{1}=0$

Figure 3. The utility-privacy trade-offs can be tuned by varying Lagrange
multipliers. The primary y-axis (dashed line) represents utility, the
secondary y-axis (solid line) represents privacy. The x-axis represents the
value of the target Lagrange multiplier.

privacy and accuracy

###  4.2. Impact of Varying Lagrange Multipliers

As we discussed in Section 2.2, the sum loss function $L_{sum}$ is a linear
combination of $L_{R}$, $L_{U}$, and $L_{P}$ based on Lagrange multipliers. In
other words, the loss optimization in our proposed architecture LSTM-PAE could
be reframed as a Lagrangian optimization question. We evaluate the influence
of different combinations of Lagrangian multipliers $\lambda_{1}$,
$\lambda_{2}$, and $\lambda_{3}$ to the proposed adversarial network, as the
results shown in Figure 3(b).

In order to illustrate the adversarial effect between $L_{R}$ and $L_{U}$, as
shown in the Figure 3(a), we compare the overall model performance in terms of
the Utility and Privacy I by fixing the $\lambda_{3}=0$, and varying the other
two multipliers by subjecting to $\lambda_{1}=1-\lambda_{2}$. Figure 3(b)
represent the effect between $L_{P}$ and $L_{U}$ by setting the
$\lambda_{1}=0$, and compare the performance in terms of the Utility and
Privacy II. We could observe in both settings that the utility increases with
a larger $\lambda_{2}$, which means when the mobility prediction unit is given
more weight in the LSTM-PAE model, it would exert a positive impact on the
data utility. We conclude that the utility-privacy trade-offs could be tuned
by varying the Lagrange multipliers; the results in the Figure 3(b) also
verify the effectiveness of our adversarial architecture. Hence, in the next
section we provide an architecture evaluation to aid in achieving a
comprehensive and practical trade-off among Utility, Privacy I and Privacy II,
and use it to discuss the overall performance of the proposed LSTM-PAE. We
note that the balance of three units is far more complicated than the balance
of two. From the extensive experiment we conducted, initialing
$\lambda_{1}=0.1$, $\lambda_{2}=0.6$, $\lambda_{3}=0.3$ can guide the model
achieve the tradeoff most efficiently.

##  5\. Architecture Evaluation

In this section, we present the comparison results of the proposed
architecture LSTM-PAE and two baseline models under the same training setting.

Baseline Models

I. Standalone Model: It comprises three independent sub-models, namely data
reconstruction sub-model, mobility prediction sub-model, and user re-
identification sub-model. Each of the sub-models have a similar layer design
as the corresponding unit in the LSTM-PAE, however, the results of the three
sub-models are completely independent and have no effect on others.
Differently from the LSTM-PAE, which leverages adversarial learning to finally
attain an extracted feature representation $f$ that satisfies the utility
requirements and privacy budgets simultaneously, the standalone models only
are trained for optimal inference accuracy at the individual tasks.

II. LSTM-TrajGAN (Rao et al., 2020): It is an end-to-end deep learning model
to generate synthetic data which preserves essential spatial, temporal, and
thematic characteristics of the real trajectory data. Compared with other
common geomasking methods, TrajGAN can better prevent users from being re-
identified. While the mobility prediction ability of the synthetic data is not
in their utility metrics, the TrajGAN work claims to preserve essential
spatial and temporal characteristics of the original data, verified through
statistical analysis of the generated synthetic data distributions, which is
in a line with the mobility prediction based utility assessment in our work.
Hence, we train a mobility prediction model for each dataset and evaluate the
mobility predictability of synthetic data generated by the TrajGAN. In
contrast to the TrajGAN that aims to generate synthetic data, our proposed
LSTM-PAE is training an encoder $Enc_{L}$ that forces the extracted
representations f to convey maximal utility while minimizing private
information about user identities, via adversarial learning.

Datasets |  Models |  | Privacy I | Utility (% for decline) | Privacy II (% for gain)
---|---|---|---|---|---
| Euc(log) | Man(log) | top-1 | top-3 | top-5 | top-1 | top-3 | top-5
MDC | Standalone |  | 0.001 | 0.002 | 0.9347 | 0.9837 | 0.9922 | 0.9247 | 0.9819 | 0.9911
TrajGAN |  | 3.526 | 5.456 | -46.32% | -24.16% | -15.98% | +20.32% | +8.13% | +4.02%
Our Model | I | 2.294 | 4.542 | -54.56% | -34.74% | -25.10% | +69.80% | +50.44% | +39.95%
| II | 3.732 | 6.023 | -13.43% | -6.26% | -3.95% | +65.51% | +45.11% | +34.86%
Priva’Mov | Standalone |  | 1.281 | 2.554 | 0.9482 | 0.9878 | 0.9954 | 0.5643 | 0.8215 | 0.8765
TrajGAN |  | 3.704 | 5.779 | -6.60% | -1.89% | -0.93% | +14.17% | +14.35% | +8.88%
Our Model | I | 1.740 | 3.433 | -3.36% | -1.59% | -0.81% | +27.02% | +14.19% | +9.19%
| II | 4.164 | 5.803 | -10.81% | -6.83% | -4.91% | +35.29% | +14.97% | +10.05%
Geolife | Standalone |  | 1.903 | 3.804 | 0.4705 | 0.6842 | 0.7636 | 0.6572 | 0.8690 | 0.9294
TrajGAN |  | 4.581 | 6.680 | -62.31% | -50.45% | -43.72% | +66.73% | +47.89% | +37.22%
Our Model | I | 1.776 | 3.469 | -31.45% | -25.02% | -21.90% | +54.88% | +39.59% | +30.81%
| II | 4.616 | 6.928 | -21.13% | -18.78% | -17.11% | +55.49% | +40.40% | +32.34%
FourSquare | Standalone |  | 2.357 | 4.464 | 0.6468 | 0.8210 | 0.8823 | 0.8780 | 0.9735 | 0.9892
TrajGAN |  | 4.795 | 6.710 | -26.30% | -22.30% | -18.75% | +51.86% | +32.49% | +23.49%
Our Model | I | 2.418 | 4.533 | -51.05% | -41.45% | -35.20% | +53.47% | +35.26% | +25.86%
| II | 4.541 | 6.638 | -2.54% | -3.14% | -2.84% | +51.08% | +34.39% | +26.16%
Table 2. Performance comparison between our proposed models with standalone
model and the TrajGAN model. The Model I is our proposed architecture without
Lagrange multipliers, and the Model II is the one with multipliers
($\lambda_{1}$ = 0.1, $\lambda_{2}$ = 0.8, and $\lambda_{3}$ = 0.1). The
results shown in this table are all with trajectory sequence length 10 (i.e.,
SL = 10). The Privacy I intuitively shows the difference between the raw data
and reconstructed data; the Utility (%) represents the utility declines; and
the Privacy II(%) represents the privacy gains calculated via the user re-
identification inaccuracy rate.

###  5.1. Performance Comparison

We first compare our proposed models with the standalone model and the LSTM-
TrajGAN model on four representative mobility datasets, as details shown in
Table 2. The overall performance is evaluated in terms of the utility level
provided by the mobility prediction unit and the privacy threat provided by
two risk units The Model I is our proposed architecture but without applying
the Lagrange multipliers (i.e., where each losses are weighed equally), and
the Model II is the one with Lagrange multipliers (i.e., $\lambda_{1}=0.1,\
\lambda_{2}=0.8,\ \lambda_{3}=0.1$ for the results in Table 2). The results in
Table 2 are based on the input data with trajectory sequence length 10 (that
is $SL=10$). Because the standalone models are trained without the
consideration for the utility-privacy trade-offs, the results on the
standalone models can be leveraged to explain the best inference accuracy
(i.e., mobility prediction accuracy and user re-identification accuracy) that
each composition unit could achieve. Differently from the standalone model,
the TrajGAN and LSTM-PAE are both taking the utility-privacy trade-offs into
consideration and we compare their trade-offs with the standalone version.
Hence, in Table 2, these results are shown in utility decline and privacy
gain, both of which are in a percentage format. The similarity indexes are
leveraged to intuitively represent the difference between the original data
$X$ and reconstructed data $X^{\prime}$, where the larger value indicates
numerical differences between them.

Table 2 demonstrates that our proposed models, especially the one with
Lagrangian multipliers, outperforms the LSTM-TrajGAN model across various
datasets. For instance, when models are trained with the MDC dataset, our
Model II achieves the best privacy-utility trade-offs among different models,
as the utility decline is only 13.43% but with 65.51% privacy gain, while
46.32% utility decline and 20.32% privacy gain with the TrajGAN. The
similarity indexes also indicate the reconstructed data $X^{\prime}$ via Model
II has bigger differences than the one via the TrajGAN. Second, although the
utility decline of the TrajGAN on the Priva’Mov dataset is 4.21% higher than
our Model II, both two privacy metrics of the TrajGAN are worse than the Model
II. Our model has better overall trade-offs in utility requirements and
privacy budgets. The performance on Geolife and FourSquare are similar but
inverse, where the utility of our model is better than TrajGAN and with
slightly weaker privacy preservation. We leverage the composite metrics to
score the overall performance of different models, and demonstrates that our
model achieves better utility-privacy trade-offs. The comparisons between
Model I and Model II also illustrate the importance of the Lagrange
multipliers in not only providing flexibility to our proposed architecture
that enable its application in different scenarios, but also enhancing the
utility-privacy trade-offs in this special case.

(a) MDC

(b) PrivaḾov

(c) Geolife

(d) Foursquare

Figure 4. Pareto Frontier trade-off of Utility and Privacy on four datasets.
The hollow squares and diamonds present results of the proposed models. The
solid points present results of the TrajGAN. Blue color means SL = 5. Black
color means SL = 10.

Settings | MDC | PrivaḾov | Geolife | FourSquare
---|---|---|---|---
Euc | Man | Utility | Euc | Man | Utility | Euc | Man | Utility | Euc | Man | Utility
I | +3.814 | +6.122 | -0.565 | +1.800 | +1.230 | -0.160 | +1.451 | +0.845 | -0.208 | +1.045 | +0.580 | -0.608
II | +3.804 | +6.120 | -0.493 | +1.839 | +1.125 | -0.196 | +1.444 | +0.837 | -0.186 | +1.032 | +0.569 | -0.574
III | +3.746 | +6.049 | -0.166 | +1.999 | +1.277 | -0.097 | +1.444 | +0.834 | -0.146 | +0.988 | +0.535 | -0.406
IV | +3.732 | +6.023 | -0.125 | +2.251 | +1.272 | -0.102 | +1.426 | +0.821 | -0.099 | +0.926 | +0.487 | -0.016
V | +3.722 | +6.012 | -0.094 | +1.829 | +1.186 | -0.055 | +1.407 | +0.809 | -0.098 | +0.902 | +0.472 | +0.036
Table 3. Data reconstruction privacy leakage risk (PI) versus utility decline
(U) on four mobility datasets. We list five different settings of the Lagrange
multipliers to discuss the potential range of the utility-privacy tradeoffs.
Both ”Euc” and ”Man” are calculated by the log base 10 of the Euclidean and
Manhattan distances.

Figure 5. User re-identification privacy gain (PII) versus utility decline (U)
on four datasets. The orange area represents the utility decline while the
light-green area represents privacy gain. The dark-green area represents the
trade-offs between utility achievement and privacy budgets. The x-axis shows
five different settings of the model, and the y-axis shows the trade-offs.

###  5.2. Utility-Privacy Analysis

In this section, we first present the utility-privacy trade-off analysis
between TrajGAN and our proposed LSTM-PAE in terms of the mobility prediction
accuracy (i.e., U) and user de-identification efficiency (i.e., PII). We then
discuss two privacy risks (i.e., PI and PII) of our proposed framework among
four representative mobility datasets.

####  5.2.1. Trade-off Comparison

Figure 4 presents the trade-off comparisons of the four datasets in terms of
the U and PII under different Lagrangian settings, where the hollow squares
and hollow diamonds show the tradeoffs provided by the proposed LSTM-PAE in
$SL=5$ and $SL=10$, respectively. The solid points present the utility-privacy
trade-off provided by the TrajGAN under the same spatial granularity and same
trajectory sequence length. As can be seen from these results, in all four
cases the synthetic dataset generated by the TrajGAN is not pareto-optimal.
That is, in that given spatial-temporal granularity, the proposed architecture
is able to achieve a better privacy level for a dataset with the same utility
value. Compared with the TrajGAN, our proposed architecture improves utility
and privacy at the same time on four datasets. Especially for the performance
on the MDC dataset, the privacy improves 45.21% than the TrajGAN while the
utility also increases 32.89%. These results illustrate that our proposed
model achieves promising performance in training a privacy-sensitive encoder
$Enc_{L}$ for different datasets.

####  5.2.2. Privacy Leakage Risk Analysis

After evaluating the superior performance of our proposed framework, we
discuss the privacy leakage risks among four representative mobility datasets
in terms of data reconstruction loss (PI, ”Euc” and ”Man” in Table 3) and user
re-identification inaccuracy (PII, privacy gain in Figure 5). We use five
different combinations of Lagrangian multipliers to evaluate the comprehensive
performance of the proposed model, namely setting I, II, III, IV, and V in the
Table 3 and Figure 5.

Table 3 shows the impact of the data reconstruction privacy leakage risk (PI)
on the data utility decline (U). The ”Euc” and ”Man” in this table are
calculated by the Euclidean and Manhattan distances, shown as the logarithm to
the base 10 in the Table 3, which intuitively demonstrates the difference
between the reconstructed data $X^{\prime}$ and the original one $X$. The
results show that the impact of reconstructed data loss on the utility is
high, thereby emphasizing the importance of balancing the trade-off between
them. Indeed, more information loss implies less privacy leakage risk, which
also results in a larger utility decline.

(a) MDC

(b) PrivaḾov

(c) Geolife

(d) Foursquare

Figure 6. The effect of temporal granularity on the model performance of four
mobility datasets.

Figure 5 presents Utility and Privacy II (i.e., user re-identification risk)
trade-offs of the proposed system on the four datasets. The Zero line (i.e., y
= 0%) in each sub-figure is leveraged to indicate the original utility rate
(U) and privacy rate (PII) of the raw data. The blue line with square marker
is the privacy gain rate with top-1 accuracy and the blue line without marker
is the top-5 accuracy. The orange lines with and without triangle marker
present the utility decline rate with top-1 and top-5 accuracies,
respectively. Hence, the orange area represents the utility decline while the
light-green area represents the privacy gain when compared with original
results. The dark-green area represents the trade-offs between utility and
privacy budgets. The x-axis shows five different settings of the model, and
the y-axis shows the trade-offs (i.e., trade-offs = privacy gain \+ utility
decline).

In summary, these trade-offs are all positive in different model settings on
four different datasets. The performance on the Geolife data is the best,
while less than 20% utility decline but more than 50% privacy gains. The
performance on MDC and FourSquare also show the promising utility-privacy
trade-offs, especially for setting V on the FourSquare dataset, both the
utility and privacy increase. The uniqueness of human mobility trajectories is
high, and these trajectories are likely to be re-identified even with a few
location data points (De Montjoye et al., 2013). Our results emphasize that
the concern of user re-identification risk could be alleviated effectively
with our proposed model.

###  5.3. Discussion of Temporal Granularity

The timestamp is one of the basic components of the trajectory sequence, and
different choices on the temporal granularity affect the final performance of
any dataset. Figure 6 shows the impact of the varying temporal granularity on
the proposed architecture. We specifically present the top-1, top-5, and
top-10 accuracies for both utility and privacy dimensions, respectively. For
instance, when temporal granularity is 10-min, it indicates a location record
r is taken by every 10 minutes from the raw data. When using more coarser
temporal granularity, the quantity of interested location points decreases, so
as the difficulty of mobility prediction. However, the uniqueness of
trajectory decreases due to ignoring many of the unique locations from each
user, resulting in the lower privacy. To summarize on the Figure 6, the impact
of temporal granularity on the Priva’Mov is minimal. In terms of utility
(mobility prediction), Priva’Mov is the only dataset for which accuracy
decreases with increasing temporal granularity. This subtle decline emphasizes
the trajectory features only has a small change when varying granularity, in
line with the university students’ mobility.

##  6\. Related work

To contextualize our work, we briefly review the current state of the art in
machine learning-based privacy preservation techniques for mobility data.

Current location privacy protection studies focus on two research streams. One
is the differential privacy approach to grouping and mixing the trajectories
from different users so that the identification of individual trajectory data
is converted into a k-anonymity problem (Aktay et al., 2020; Xiao and Xiong,
2015; Andrés et al., 2013). The other stream focuses on synthetic data
generation (Rezaei et al., 2018; Huang et al., 2019; Choi et al., 2021; Ouyang
et al., 2018). Synthetic data generation methods have been extensively studied
in recent years as a way of tackling privacy concerns of location based
datasets. The majority of existing mobility synthesis schemes are mainly
categorized into two approaches: One is a more traditional, simulation-based
approach, while the other is a more recent, neural network based generative
modeling approach that utilizes recurrent autoencoders and generative
adversarial networks to produce realistic trajectories (Shin et al., 2020).
Simulation based approaches generate mobility traces by modeling overall user
behavior as a stochastic process, such as a Markov chain model of transition
probabilities between locations, and then drawing random walks, potentially
with additional stochastic noise added, as demonstrated in Xiao et al (Xiao et
al., 2017). These approaches require considerable feature engineering effort
and struggle to capture longer-range temporal and spatial dependencies in the
data (Luca et al., 2021) and are thus limited in their ability to preserve the
utility of the original datasets. In contrast, the generative neural network
approach synthesizes user mobility traces by learning, via gradient descent
back-propagation, then the optimal weights are utilized for decoding a high-
dimensional latent vector representation into sequences that closely resemble
the original data. Such traces can maintain important statistical properties
of the original data while taking advantage of noise introduced in the
reconstruction process, to improve data subject anonymity. Huang et al (Huang
et al., 2019) demonstrates the use of a variational autoencoder network to
reconstruct trajectory sequences, while Ouyang et al (Ouyang et al., 2018)
utilizes a convolutional GAN, but neither work directly makes a quantitative
assessment of the extent of privacy protection that their algorithms provide
(Huang et al., 2019; Ouyang et al., 2018). The LSTM-TrajGAN by Rao et al (Rao
et al., 2020) is a state of the art example of the generative trajectory
modeling approach, which quantifies its privacy protection by demonstrating a
significant decline in the performance of a second user ID classifier model on
the synthetic outputs compared to the original input trajectories. For these
reasons, we used it as both a baseline for comparison and for the design of
our proposed architecture.

Our proposed model takes the neural generative modeling approach, but differs
from existing methods in that we utilize a combined, multi-task neural network
to simultaneously reconstruct trajectories, predict next locations, and
reidentify users, from the same learned latent vector representation. We seek
an optimal trade-off between the three tasks’ individual losses by optimizing
a Lagrangian loss function with per-task weights, improving the
controllability of the relative utility and privacy of the outputs.

##  7\. Conclusion

In this paper, we presented a privacy-preserving architecture based on the
adversarial networks. Our model takes into account three different
optimization objectives and searches for the optimum trade-off for utility and
privacy of a given dataset. We reported an extensive analysis of our model
performances and the impact of its hyper-parameters using four real-world
mobility datasets. The Lagrange multipliers $\lambda_{1},\ \lambda_{2},$ and
$\lambda_{3}$ bring more flexibility to our framework that enable it to
satisfy different scenarios’ requirements according to the relative importance
of utility requirements and privacy budgets. We evaluated our framework on
four datasets and benchmarked our results against an LSTM-GAN approach. The
comparisons indicate the superiority of the proposed framework and the
efficiency of the proposed privacy-preserving feature extractor $Enc_{L}$.
Expanding this work, we will consider other utility functions for our model
such as community detection based on unsupervised clustering methods or deep
embedded clustering methods. In future work, we will leverage automated search
techniques, such as deep deterministic policy gradient algorithm, for
efficiency in searching for the optimal Lagrange multipliers.

## References

  * (1)
  * Aktay et al. (2020) Ahmet Aktay, Shailesh Bavadekar, Gwen Cossoul, John Davis, Damien Desfontaines, Alex Fabrikant, Evgeniy Gabrilovich, Krishna Gadepalli, Bryant Gipson, Miguel Guevara, et al. 2020\.  Google COVID-19 Community Mobility Reports: anonymization process description (version 1.1).  _arXiv preprint arXiv:2004.04145_ (2020).
  * Andrés et al. (2013) Miguel E Andrés, Nicolás E Bordenabe, Konstantinos Chatzikokolakis, and Catuscia Palamidessi. 2013\.  Geo-indistinguishability: Differential privacy for location-based systems. In _Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security_. 901–914.
  * Ball (1960) Walter William Rouse Ball. 1960\.  _A short account of the history of mathematics_.  Courier Corporation.
  * Beavis and Dobbs (1990) Brian Beavis and Ian Dobbs. 1990.  _Optimisation and stability theory for economic analysis_.  Cambridge university press.
  * Beresford and Stajano (2003) Alastair R Beresford and Frank Stajano. 2003.  Location privacy in pervasive computing.  _IEEE Pervasive computing_ 2, 1 (2003), 46–55.
  * Black et al. (1998) Paul E Black et al. 1998\.  Dictionary of algorithms and data structures.  (1998).
  * Choi et al. (2021) Seongjin Choi, Jiwon Kim, and Hwasoo Yeo. 2021.  TrajGAIL: Generating urban vehicle trajectories using generative adversarial imitation learning.  _Transportation Research Part C: Emerging Technologies_ 128 (2021), 103091\.
  * De Montjoye et al. (2013) Yves-Alexandre De Montjoye, César A Hidalgo, Michel Verleysen, and Vincent D Blondel. 2013.  Unique in the crowd: The privacy bounds of human mobility.  _Scientific reports_ 3 (2013), 1376.
  * Feng et al. (2020) Jie Feng, Can Rong, Funing Sun, Diansheng Guo, and Yong Li. 2020.  PMF: A privacy-preserving human mobility prediction framework via federated learning.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ 4, 1 (2020), 1–21.
  * Ferreira et al. (2020) Danielle L. Ferreira, Bruno A. A. Nunes, Carlos Alberto V. Campos, and Katia Obraczka. 2020\.  A Deep Learning Approach for Identifying User Communities Based on Geographical Preferences and Its Applications to Urban and Environmental Planning.  _ACM Transactions on Spatial Algorithms and Systems_ 6, 3 (May 2020), 1–24.  https://doi.org/10.1145/3380970
  * Gedik and Liu (2005) Bugra Gedik and Ling Liu. 2005.  Location privacy in mobile systems: A personalized anonymization model. In _25th IEEE International Conference on Distributed Computing Systems (ICDCS’05)_. IEEE, 620–629.
  * Gedik and Liu (2007) Bugra Gedik and Ling Liu. 2007.  Protecting location privacy with personalized k-anonymity: Architecture and algorithms.  _IEEE Transactions on Mobile Computing_ 7, 1 (2007), 1–18.
  * Gonzalez et al. (2008) Marta C Gonzalez, Cesar A Hidalgo, and Albert-Laszlo Barabasi. 2008\.  Understanding individual human mobility patterns.  _nature_ 453, 7196 (2008), 779–782.
  * Goodfellow et al. (2014) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014\.  Generative adversarial nets.  _Advances in neural information processing systems_ 27 (2014).
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ 9, 8 (1997), 1735–1780.
  * Huang et al. (2019) Dou Huang, Xuan Song, Zipei Fan, Renhe Jiang, Ryosuke Shibasaki, Yu Zhang, Haizhong Wang, and Yugo Kato. 2019\.  A variational autoencoder based generative model of urban human mobility. In _2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR)_. IEEE, 425–430.
  * Huang et al. (2018) Haosheng Huang, Georg Gartner, Jukka M Krisp, Martin Raubal, and Nico Van de Weghe. 2018.  Location based services: ongoing evolution and research agenda.  _Journal of Location Based Services_ 12, 2 (2018), 63–93.
  * Kolodziej and Hjelm (2017) Krzysztof W Kolodziej and Johan Hjelm. 2017.  _Local positioning systems: LBS applications and services_.  CRC press.
  * Krumm (2009) John Krumm. 2009\.  A survey of computational location privacy.  _Personal and Ubiquitous Computing_ 13, 6 (2009), 391–399.
  * Laurila et al. (2012) Juha K Laurila, Daniel Gatica-Perez, Imad Aad, Olivier Bornet, Trinh-Minh-Tri Do, Olivier Dousse, Julien Eberle, Markus Miettinen, et al. 2012\.  _The mobile data challenge: Big data for mobile computing research_.  Technical Report.
  * Liu et al. (2019) Sicong Liu, Junzhao Du, Anshumali Shrivastava, and Lin Zhong. 2019\.  Privacy adversarial network: representation learning for mobile data privacy.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ 3, 4 (2019), 1–18.
  * Luca et al. (2021) Massimiliano Luca, Gianni Barlacchi, Bruno Lepri, and Luca Pappalardo. 2021.  A Survey on Deep Learning for Human Mobility.  arXiv:2012.02825 [cs.LG]
  * Malekzadeh et al. (2020) Mohammad Malekzadeh, Richard G Clegg, Andrea Cavallaro, and Hamed Haddadi. 2020.  Privacy and utility preserving sensor-data transformations.  _Pervasive and Mobile Computing_ 63 (2020), 101132.
  * Mokhtar et al. (2017) Sonia Ben Mokhtar, Antoine Boutet, Louafi Bouzouina, Patrick Bonnel, Olivier Brette, Lionel Brunie, Mathieu Cunche, Stephane D’Alu, Vincent Primault, Patrice Raveneau, et al. 2017\.  PRIVA’MOV: Analysing Human Mobility Through Multi-Sensor Datasets. In _NetMob 2017_.
  * Oliver et al. (2020) Nuria Oliver, Bruno Lepri, Harald Sterly, Renaud Lambiotte, Sébastien Deletaille, Marco De Nadai, Emmanuel Letouzé, Albert Ali Salah, Richard Benjamins, Ciro Cattuto, et al. 2020\.  Mobile phone data for informing public health actions across the COVID-19 pandemic life cycle.
  * Oliver et al. (2015) Nuria Oliver, Aleksandar Matic, and Enrique Frias-Martinez. 2015\.  Mobile network data for public health: opportunities and challenges.  _Frontiers in public health_ 3 (2015), 189.
  * Ouyang et al. (2018) Kun Ouyang, Reza Shokri, David S. Rosenblum, and Wenzhuo Yang. 2018\.  A Non-Parametric Generative Model for Human Trajectories. In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_. International Joint Conferences on Artificial Intelligence Organization, 3812–3817.  https://doi.org/10.24963/ijcai.2018/530
  * Rao et al. (2020) Jinmeng Rao, Song Gao, Yuhao Kang, and Qunying Huang. 2020\.  Lstm-trajgan: A deep learning approach to trajectory privacy protection.  _arXiv preprint arXiv:2006.10521_ (2020).
  * Rezaei et al. (2018) Aria Rezaei, Chaowei Xiao, Jie Gao, and Bo Li. 2018\.  Protecting sensitive attributes via generative adversarial networks.  _arXiv preprint arXiv:1812.10193_ (2018).
  * Shin et al. (2020) Seungjae Shin, Hongseok Jeon, Chunglae Cho, Seunghyun Yoon, and Taeyeon Kim. 2020.  User Mobility Synthesis based on Generative Adversarial Networks: A Survey. In _2020 22nd International Conference on Advanced Communication Technology (ICACT)_. IEEE, 94–103.
  * Shokri et al. (2011) Reza Shokri, George Theodorakopoulos, Jean-Yves Le Boudec, and Jean-Pierre Hubaux. 2011.  Quantifying location privacy. In _2011 IEEE symposium on security and privacy_. IEEE, 247–262.
  * Song et al. (2010) Chaoming Song, Zehui Qu, Nicholas Blumm, and Albert-László Barabási. 2010\.  Limits of predictability in human mobility.  _Science_ 327, 5968 (2010), 1018–1021.
  * Toch et al. (2019) Eran Toch, Boaz Lerner, Eyal Ben-Zion, and Irad Ben-Gal. 2019\.  Analyzing large-scale human mobility data: a survey of machine learning methods and applications.  _Knowledge and Information Systems_ 58, 3 (2019), 501–523.
  * Wang et al. (2020) Senzhang Wang, Jiannong Cao, and Philip Yu. 2020.  Deep learning for spatio-temporal data mining: A survey.  _IEEE Transactions on Knowledge and Data Engineering_ (2020).
  * Xiao and Xiong (2015) Yonghui Xiao and Li Xiong. 2015.  Protecting locations with differential privacy under temporal correlations. In _Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security_. 1298–1309.
  * Xiao et al. (2017) Yonghui Xiao, Li Xiong, Si Zhang, and Yang Cao. 2017\.  LocLok: location cloaking with differential privacy via hidden markov model.  _Proceedings of the VLDB Endowment_ 10 (08 2017), 1901–1904.  https://doi.org/10.14778/3137765.3137804
  * Yang et al. (2014) Dingqi Yang, Daqing Zhang, Vincent W Zheng, and Zhiyong Yu. 2014.  Modeling user activity preference by leveraging user spatial temporal characteristics in LBSNs.  _IEEE Transactions on Systems, Man, and Cybernetics: Systems_ 45, 1 (2014), 129–142.
  * Zhan and Haddadi (2019) Yuting Zhan and Hamed Haddadi. 2019.  Towards automating smart homes: contextual and temporal dynamics of activity prediction. In _Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers_. 413–417.
  * Zheng et al. (2011) Yu Zheng, Hao Fu, Xing Xie, Wei-Ying Ma, and Quannan Li. 2011.  _Geolife GPS trajectory dataset - User Guide_ (geolife gps trajectories 1.1 ed.).  https://www.microsoft.com/en-us/research/publication/geolife-gps-trajectory-dataset-user-guide/ Geolife GPS trajectories 1.1.

APPENDIX

In general, the encoder $Enc_{L}$ should satisfy high predictability (min
$\mathcal{L_{U}}$) and low user re-identification accuracy (max
$\mathcal{L_{P}}$) of the mobility data when maximizing the reconstruction
loss (max $\mathcal{L_{R}}$) in the reversed engineering. The overall training
is to achieve privacy-utility trade-off by adversarial learning on
$\mathcal{L_{R}}$, $\mathcal{L_{U}}$, and $\mathcal{L_{P}}$ concurrently. The
gradient of the loss (i.e., $\theta_{R}$, $\theta_{U}$, $\theta_{P}$) back-
propagate through the LSTM network to guide the training of the encoder
$Enc_{L}$. The encoder is updated with the sum loss function
$\mathcal{L}_{sum}$ until convergence. Algorithm 1 summarizes our training
setting of the LSTM-PAE model.

1

Input : Mobility data X, real mobility prediction labels Y, real
identification recognition labels Z, Lagrangian multipliers: $\lambda_{1}$,
$\lambda_{2}$, $\lambda_{3}$

Output : Adversarial Encoder $Enc_{L}(X)$

2 Initialize model parameters

3 for  _n epochs_ do

4 for  _$k=1$ , $\cdots$$,K_{t}$_ do

5 1\. Sample a min-batch of mobility trajectories x, prediction labels y,
identification labels z

6 2\. Update $\theta_{R}$ with Adam optimizer on mini-batch loss
$L_{R}(\theta_{R})_{(x,\hat{x})}$

7 3\. Update $\theta_{U}$ with Adam optimizer on mini-batch loss
$L_{U}(\theta_{U})_{(y,\hat{y})}$

8 4\. Update $\theta_{P}$ with Adam optimizer on mini-batch loss
$L_{P}(\theta_{P})_{(z,\hat{z})}$

9

10 end for

11 Update with the gradient descent on
$L_{sum}(\theta_{R},\theta_{U},\theta_{P},\lambda_{1},\lambda_{2},\lambda_{3})$

12 end for

Algorithm 1 Training of the proposed adversarial network for representation
learning