11institutetext: AIT, Vienna, Austria 11email<EMAIL_ADDRESS>22institutetext: University of Vienna, Vienna, Austria

# MobilityDL: A Review of Deep Learning From Trajectory Data

Anita Graser 11 0000-0001-5361-2885 Anahid Jalali 11 0000-0002-8889-0735
Jasmin Lampert 11 0000-0002-0414-4525 Axel Weißenfeld 11 0000-0002-7246-2744
Krzysztof Janowicz 22

###### Abstract

Trajectory data combines the complexities of time series, spatial data, and
(sometimes irrational) movement behavior. As data availability and computing
power have increased, so has the popularity of deep learning from trajectory
data. This review paper provides the first comprehensive overview of deep
learning approaches for trajectory data. We have identified eight specific
mobility use cases which we analyze with regards to the deep learning models
and the training data used. Besides a comprehensive quantitative review of the
literature since 2018, the main contribution of our work is the data-centric
analysis of recent work in this field, placing it along the mobility data
continuum which ranges from detailed dense trajectories of individual movers
(quasi-continuous tracking data), to sparse trajectories (such as check-in
data), and aggregated trajectories (crowd information).

###### Keywords:

Deep learning Spatial data science Movement data Mobility Trajectories.

##  1 Introduction

Deep learning (DL) has become a popular approach for developing data-driven
prediction, classification, and anomaly detection solutions. Work on deep
learning from trajectory data is spread out over many domains, including but
not limited to computer science [3, 11, 14], geographic information science
[13, 17, 24], intelligent transportation systems [51], maritime sciences [8,
63], and ecology [34]. Consequently, the resulting body of work covers many
different use cases and trajectory dataset types. A data-centric way to
categorize trajectory datasets is according to their level of detail [19]
along the mobility data continuum ranging from detailed dense trajectories
(quasi-continuous tracking data of individual movement) to sparse trajectories
(such as check-in data of individuals), and finally, aggregated trajectories
(crowd-level information, typically aggregated to edges/nodes in a mobility
graph, to a grid, or to a set of points of interest)[2, 28, 67]. Even though
the level of detail of the underlying trajectory-based training data is an
important factor determining the potential capabilities as well as the scale
or resolution of derived machine learning models, cursory scanning of paper
titles and abstracts is usually insufficient to determine which type of
trajectory data was used to train the presented deep learning models. To gain
a better understanding of the state of the art, it is instead necessary to
review the data and methods sections in detail. While many papers start with
dense trajectories, most convert them into sparse trajectories [1, 8, 13, 24,
43, 62, 63] or even aggregate them to crowd-level [4, 55, 66]. Common
approaches to turning dense trajectories into sparse trajectories include:
converting them into a sequence of stop locations [1, 24] or a sequence of
traversed regions (grid cells) [13, 43], or converting them to trajectory
images [8, 62, 63].

The recent literature includes review papers for specific mobility use cases,
for example, in their review of location encoding methods for GeoAI [39], the
authors stress the analogy between NLP word-to-sentence relations and
location-to-trajectory relations. This analogy has led to Word2Vec-inspired
approaches encoding location into a location embedding using, for example,
Location2Vec [68], Place2Vec [61], or POI2Vec [15]. Another recent review [27]
is dedicated to deep learning for traffic flow prediction models, which are
primarily trained on aggregated trajectory data. Finally, Wang et al. [58]
provide a survey of deep learning for the wider field of spatio-temporal data
mining, including trajectories. However, to the best of our knowledge, there
is no review paper that provides a comprehensive overview of the different
neural network (NN) architectures used to learn from trajectory data,
specifically for mobility use cases.

The goal of this work, therefore, is to provide a first overview of the
current state of deep neural networks trained with trajectory data, structured
by:  1. Use case category (travel time, crowd flow, and location predictions;
location and trajectory classifications; as well as anomaly detection and
synthetic data generation),  2. Neural network architecture (including, for
example, CNN, RNN, LSTM, and GNN), and  3. Trajectory data granularity (dense,
sparse, aggregated) and representation.  For practical reasons, the in-depth
qualitative assessment part of this paper does not cover all relevant works
published in recent years exhaustively. However, we provide at least one paper
for each use case and DL combination that we have identified. We specifically
reviewed publications at recent events, including SIGSpatial
2022111https://sigspatial2022.sigspatial.org/accepted-papers/ [13, 24, 38, 43,
53, 55, 60, 65], Sussex-Huawei Locomotion (SHL) Challenge
2021222http://www.shl-dataset.org/activity-recognition-challenge-2021/ at the
ACM international joint conference on pervasive and ubiquitous computing
(UbiComp) [57], Traffic4cast challenge
2021333https://www.iarai.ac.at/traffic4cast/2021-competition/ at NeurIPS [36],
and Big Movement Data Analytics workshop BMDA
2021444https://www.datastories.org/bmda21/BMDA21Accepted.html at EDBT [4, 33,
54].

Even though we focus explicitly on deep learning, it is worth noting that deep
learning may not always be the best approach [21] to address a particular
challenge. For example, the SHL Challenge summary [57] reports that classic
machine learning (ML) models outperform deep learning models on their three
metrics: F1 score, train time, and test time. In Section 3 we therefore review
the main reasons stated by authors to motivate the use of deep learning as
well as the baselines and metrics used in evaluations.

This review does not attempt to compare the performance of different deep
learning approaches. Even though there are some commonly used open datasets
(such as the Porto taxi555https://www.kaggle.com/c/pkdd-15-predict-taxi-
service-trajectory-i/data, the T-Drive taxi666https://www.microsoft.com/en-
us/research/publication/t-drive-trajectory-data-sample/, the
GeoLife777https://www.microsoft.com/en-us/research/publication/geolife-gps-
trajectory-dataset-user-guide/, and the Gowalla check-in
datasets888http://snap.stanford.edu/data/loc-Gowalla.html) cross-paper
comparisons outside of dedicated data challenges (such as the DEBS Grand
Challenge 2018 [22], SHL Challenge 2021 [57]), and Traffic4cast challenge
2021${}^{\ref{Traffic4cast}}$) are notoriously difficult. For example,
“Despite the Porto dataset’s original use as a standardized benchmark for open
competition, design choices in subsequent work make cross-paper comparison
difficult. Firstly, different papers often augment the dataset with their
metadata not present in the original release, which may give some models an
advantage over others independent of architecture or training design.” [53]

The remainder of this paper is structured as follows: Section 2 presents the
recent deep learning research structured by the main use cases, neural network
architecture, and trajectory data granularity and representation. Section 3
summarizes and discusses the motivations provided for the use of DL over
classic ML and the benchmarks and metrics used to evaluate their performance.
Section 4 extends the time frame of our analysis to further analyze the trends
in this research area. Finally, Section 5 summarizes the findings and presents
our conclusions. Since the terminology used in different publications is not
necessarily consistent due to the large range of domains working on trajectory
data analysis, we, define the most important terms and abbreviations in a
glossary in the paper appendix.

##  2 Representing Trajectory Data For Deep Learning Use Cases

Trajectory datasets used in research and industry are highly heterogeneous
since they are affected by numerous technical and design choices. For example,
in Graser et al. [19], we distinguish 10 dimensions along which movement
dataset may vary:  1. Spatial resolution (fine or coarse / small or large
location errors),  2. Spatial dimensions (2D or 3D),  3. Temporal resolution
(sparse or frequent / quasi-continuous),  4. Sampling interval (regular or
irregular),  5. Representation (polylines or continuous curves),  6.
Constraints (network-constrained or unconstrained / open space),  7.
Collection models (Lagrangian or Eulerian / checkpoint-based),  8. Tracking
system (cooperative or uncooperative),  9. Privacy (personal or impersonal),
10. Data size (small or big datasets / streams).  It is therefore not hard to
imagine that there can be no one-size-fits-all approach to modelling
trajectory data for machine learning. Instead, it is necessary to pick the
right model for the use case and data available.

The literature on deep learning from trajectory data can be roughly divided
into eight use case categories, as shown in Table 1. The first three use
cases, consisting of: 1\. Trajectory prediction (& imputation), 2\. Arrival
time prediction mostly, and 3\. (Sub)trajectory classification mostly use
dense / quasi-continuous trajectory data, while the following use cases: 4\.
Anomaly detection, 5\. Next location (& destination) prediction, 6\. Synthetic
data generation, 7\. Location classification use cases mostly, and 8\. Traffic
volume / crowd flow prediction tend to use increasingly sparse / check-in data
or aggregated trajectories of multiple movers.

Table 1: Overview of trajectory data representations used to train neural networks using the following dataset categorization: trajectories of individual movers marked with $\Rightarrow$ for dense / quasi-continuous, $\to$ for sparse/check-in data or gridded data  and $\rightleftharpoons$ for aggregated trajectories of multiple movers. References marked with asterisk (*) mark review / summary papers Use case |  CNN |  (C)RNN |  LSTM |  GRU |  Trans
former |  GNN |  Other
---|---|---|---|---|---|---|---
Trajectory
prediction /
imputation [2.1] |  |  |  $\Rightarrow$ [41]
$\Rightarrow$ [5] |  $\Rightarrow$ [54]
$\to$ [13] |  $\to$ [6, 43] |  |
Arrival time
prediction [2.2] |  |  $\Rightarrow$ [56, 3] |  |  $\Rightarrow$ [3] |  |  $\rightleftharpoons$ [11] |
(Sub)trajectory
classification [2.3] |  $\to$ [8, 63] |  $\Rightarrow$ [57]* |  $\Rightarrow$ [57]* |  |  $\Rightarrow$ [57]* |  |  $\Rightarrow$ AdaNet: [57]*
Anomaly
detection [2.4] |  |  $\to$ [44]
$\rightleftharpoons$ [51] |  $\Rightarrow$ [33] |  |  |  |
Next location /
destination
prediction [2.5] |  $\to$ [16] |  $\to$ [14, 32] |  |  $\to$ [16] |  $\to$ [24] |  |  $\Rightarrow$ Hyper
network: [53]
$\to$ SAN: [30]
Synthetic data
generation [2.6] |  |  |  $\to$ [47] |  |  |  |  $\Rightarrow$ VAE-like: [65]
$\to$ GAN: [47]
$\rightleftharpoons$ MLP: [50]
Location
classification [2.7] |  $\to$ [62] |  |  |  |  |  $\rightleftharpoons$ [1] |  $\to$ Memory
network: [38]
Traffic volume
/ crowd flow
prediction [2.8] |  $\rightleftharpoons$ [58, 36, 27] |  $\rightleftharpoons$ [27] |  $\rightleftharpoons$ [4]
$\to$ [31] |  |  $\rightleftharpoons$ [60] |  $\to$ [17]
$\rightleftharpoons$ [31, 34] |  $\rightleftharpoons$ GAN: [66]
$\rightleftharpoons$ SAE: [27]

To further illustrate the wide variety of different trajectory data
representations encountered in these use cases, Figure 1 provides an overview
ordered by the identified trajectory representations from dense individual
trajectories at the top to crowd-level / aggregated movement trajectory data
at the bottom. In many instances, the borders between these categories are
fuzzy since they reside on a spectrum. While use cases such as Trajectory
prediction, Next location prediction and Traffic volume / crowd flow
prediction are clearly clustered in certain sections along this spectrum,
other use cases, such as Anomaly detection and Synthetic data generation
appear all over since, for example, anomalies can be analyzed both on the
level of individual trajectories as well as on the crowd level where analysts
may be interested in detecting unusual crowd patterns.

Figure 1: Overview of trajectory data representations used to train neural
networks

More details on the trajectory datasets and the data engineering steps applied
to the trajectory data before they are used as input to train the neural
networks are summarized in Tables 2-3. Some works (e.g. [17, 48]) use
additional data sources (such as the road network and POIs) in combination
with trajectory data to train their models. These additional data sources have
not been included in our review in favor of clarity and conciseness.

The following subsections describe each use case in more detail and introduce
the neural network designs used to address the use cases, as well as the
trajectory data representations used to train these neural networks.

###  2.1 Trajectory prediction / imputation

This use case category covers the prediction of trajectories in artificial
[6], urban [13], and maritime environments [41, 54, 5], as well as the
imputation of trajectories [43]. Depending on the context, trajectory
prediction is also called movement prediction [41, 54] or future location
prediction (FLP) [54] which can lead to confusions with the next location /
destination prediction use case. However, this use case differs from the next
location / destination prediction use case (which we will discuss later) in
that it aims to predict the future path of a mover, that is, along which path
the mover will get to its next location.

To predict trajectory paths with high spatial detail, it is necessary to learn
from high-resolution training data. Therefore, the training data for these
models consists of rather high-density trajectories of individual movers,
which may be resampled, generalized, or discretized, as shown in Figure 1. For
example, Capobianco et al. [5] resample each trajectory to a fixed sampling
interval of 15 minutes and create fixed-length trajectories with 12 positions,
equalling a fixed trajectory duration of three hours. Mehri et al. [41]
generalize AIS trajectories using context-aware piece-wise linear segmentation
before feeding them into an LSTM three vertices at a time. This enables their
model to perform short-term trajectory predictions with high spatial detail.
Tritsarolis et al. [54], on the other hand, represent trajectories by their
composition of differences in space $\Delta x$, $\Delta y$, and time $\Delta
t$ for the input of their RNN-based models to predict future positions $\Delta
t+1$. They even take the problem one step further and try to predict the co-
movement of multiple movers through evolving clusters.

An example of more long-term trajectory predictions is presented in Fan et al.
[13]. They discretize mobile phone GPS trajectories using the H3 hexagonal
grid 999https://github.com/uber/h3 and use the grid cell sequences to train
their GRU. The resulting model predicts cell sequences which are afterwards
used to search for similar high-resolution trajectories, which are returned as
the final trajectory prediction.

Transformers are another approach gaining traction. For example, Carroll et
al. [6] use synthetic discrete movement sequences in a minimalistic grid world
environment to train their transformers to predict trajectories. Another work
using transformers to impute trajectories is Musleh et al. [43]. They propose
TrajBERT, a model trained using H3-discretized (tokenized) GPS tracks. They
“down-sample the trajectories by dropping three-quarters of the points of each
trajectory and then run TrajBERT to fill the gaps by imputing the missing
points”.

The difficulty of a specific trajectory prediction task is influenced by the
type of movement (network-constrained or unconstrained), the complexity of the
movement patterns observed in the area of interest [20], as well as the
desired prediction horizon (how far into the future the prediction is
attempted) and the quality of the available training data. Additional
information, such as the vessel destination labels [5] can further improve the
prediction accuracy. The evaluation metrics for trajectory prediction tasks
usually include the RMSE, MAE, or MAPE of spatial and spatiotemporal distance
measures (see Table 5).

###  2.2 Arrival time prediction

This use case category covers the prediction of travel times or arrival times,
such as arrival time prediction in train networks [3] and street networks [11,
56]. This type of task is also known as Estimated Time of Arrival (ETA) task.
In classical intelligent transportation systems (ITS), ETA is commonly
computed using routing algorithms leveraging street network data with travel
time information. This travel time information may be historical averages for
a certain season, day of the week, and time of day, or the result of more
advanced ML/DL prediction models. Since future travel times often correlate
with historical travel times (for example, at a given time of day and day of
the week), recurrent mechanisms are commonly used to predict them [11, 56, 3].
For example, Derrow-Pinion et al. [11] train GNNs on aggregated trajectories
to provide travel time predictions in Google Maps. The GNN graph consists of
segment and supersegment-level embedding vectors. Nodes store street segment-
level data (average real-time and historical segment travel speeds and times,
segment length, and road class), while edges store supersegment-level data
(real-time supersegment travel times).

In contrast, Wang et al. [56] introduces the GEO-convolutional network layer
(GEO-Conv, also used by Buijse et la. [3]), which is trained on dense
trajectories rather than aggregated trajectories. The proposed GEO-Conv layer
takes dense trajectories as input and applies a non-linear mapping of each
trajectory point, followed by a GEO-Conv step with multiple kernels.

The difficulty of a specific arrival time prediction task is influenced by the
variability of the travel times in the area of interest, as well as the
desired prediction horizon and the quality of the available training data
(including sufficient data on relevant recurring patterns and events that may
influence travel times). The evaluation metrics for arrival time prediction
tasks usually include the RMSE, MAE and MAPE (see Table 5).

###  2.3 (Sub)trajectory classification

This use case category considers the classification of complete trajectories
[63] or sub-trajectories [8, 57]. Typical applications include the detection
of mover classes and movement types, such as ship maneuvers [8] or the
detection of transportation and locomotion modes of smartphone users [57].

An example of the classification of complete trajectories is the recognition
of ship types introduced by Yang et al. [63]. It relies on the same technique
as [8] for transforming the raw AIS trajectory data into colour-coded
trajectory images. The resulting images show characteristic trajectory
patterns, which can be used to identify the ship vessel type with a CNN
classifier.

By splitting trajectories into sub-trajectories, more fine-grained analyses
are possible. For example, Chen et al. [8] generate colour-coded trajectory
images from ship AIS data, where each pixel is assigned one of three colors
according to the movement type (static, normal, maneuvering). These trajectory
images are used to train a CNN-based ship maneuver classifier.

An example for human sub-trajectory classification is presented in the Sussex-
Huawei Locomotion-Transportation (SHL) Recognition Challenge [57] which aims
to identify different movement modes (i.e. still, walk, run, bike, car, bus,
train, and subway) from smartphone data. The SHL Challenge winner uses AdaNet,
a TensorFlow-based framework for learning NN models and ensembling models to
obtain even better models [10].

The difficulty of a specific (sub)trajectory classification task depends on
the the nature of the classes of interest (some classes may easy to
distinguish while others may have large overlaps with neighboring classes) as
well as the quality of the training dataset (in particular the class balance
which may be more or less skewed towards the popular classes and under-
represent rarer classes). Additional complexity is introduced if – in addition
to the classification – the splits between subtrajectories have to be
determined as well. The evaluation metrics for (sub)trajectory classification
tasks usually include F1 score, accuracy, precision and recall (see Table 5).

###  2.4 Anomaly detection

This use case category deals with identification and handling of unusual
observations and patterns in the data, often referred to as outliers or
anomalies. Both terms are often used interchangeably [7]. Outliers can be
defined as observations that “deviate so much from other observations as to
arouse suspicions that it was generated by a different mechanism” [23].
Similarly, anomalies are patterns that do not conform to an expected behavior
[7].

Types of movement anomalies include anomalous records (unusual spatiotemporal
or thematic attributes), anomalous (sub)trajectories, and anomalous events
(when “the trajectories of individual movers are unremarkable but their
combined spatio-temporal pattern is unusual” [19]). Since the definition of
anomalies is often context-dependent, ground truth labeled data is rare.
Therefore, anomaly detection approaches often resort to trying to identify
trajectories that deviate significantly from previously observed trajectories
based on some spatial, spatiotemporal, or other metrics. GeoTrackNet [44], for
example, is a model for maritime trajectory anomaly detection, which consists
of a probabilistic RNN-based (Recurrent Neural Network) representation of AIS
tracks and a contrario detection [12]. Anomalies detected by GeoTrackNet were
then evaluated by AIS experts. Similarly, Singh et al. [51] present an anomaly
detection system based on RNN regression models to detect anomalous
trajectories, on-off switching, and unusual turns. Again, a quantitative
accuracy analysis is not feasible due to the lack of ground truth data.

To address the issue of lacking ground truth, some researchers resort to using
synthetically generated anomalies [33]. For example, Liatsikou et al. [33]
developed an LSTM-based network for the automatic detection of movement
anomalies, such as the detection of synthetic anomalies in taxi trajectories.
The anomalies that can be detected are of limited length since the autoencoder
requires inputs of a certain fixed length, all trajectories are clipped to
nine points (and shorter ones discarded).

The difficulty of a specific anomaly detection task depends on the anomalies
of interest (and how much they deviate from regularly observed behavior) as
well as the availability of labelled training data. In settings where only
detected anomalies can be presented to experts for assessment, there is a lack
of dependable quantitative data on true and false negatives. The evaluation
metrics for anomaly detection tasks usually include accuracy, precision and
recall (see Table 5).

###  2.5 Next location / destination prediction

This use case category covers the prediction of the next locations or final
destinations of trips [14, 16, 24, 30, 32, 35, 53]. This usually boils down to
predicting the next location or final destination from a finite set of
potential locations. Besides GPS tracks, a commonly used data source in this
category are social media check-ins (e.g., from Foursquare). The task then
becomes to predict the next check-in location.

Attention mechanisms are particularly popular for next location prediction.
For example, Gao et al. [16] train VANext, a semi-supervised network
trajectory convolutional network, on check-in data. They convert each
individual user’s trajectory (check-in/POI sequence) into sequence embeddings
using a causal embedding method (similar to a high-order Markov process). The
resulting embeddings are the input for their GRU to learn the trajectory
patterns. They further apply attention to the embeddings for predicting the
user’s next POI. Feng et al. [14] tailor two attention mechanisms to generate
independent latent vectors from large and sparse trajectories. These
embeddings are then fed into their DeepMove GRUs and a historical attention
module. The learned attention weights can intuitively explain the prediction
based on the user’s history of movement behavior. Li et al. [30] introduce a
spatio-temporal self-attention network (STSAN). They generate trajectory
embeddings by concatenating the temporal (activity sequence), spatial
(distance matrix of locations), and location attentions (location sequence and
their categories). They feed these embeddings through a softmax layer and
predict the user’s next POI. They use a federated learning setting to tackle
the heterogeneity problem. Liao et al. [32] generate embeddings from location
sequences as well as graph embeddings from location-location and activity-
location graphs and train their MCARNN multi-task context-aware recurrent
neural network to solve both activity and location prediction tasks.

Other works use neural networks for dimensionality reduction and for creating
embeddings. Liu et al. [35] incorporate time and distance-specific transition
matrices as temporal and spatial embeddings generated by RNNs. Hong et al.
[24] reduce the dimensions of trajectories using a multilayered embedding
approach for transformers to predict next location and travel mode. Tenzer et
al. [53] generate two geospatial and temporal embeddings by  1. combining the
random picking and the nearest neighbor to create sequences of spatial
embeddings and  2. using a sinusoidal embedding to convert the timesteps to
temporal vectors.  They train a hyper network to learn to change its weights
in response to these embeddings.

The difficulty of a specific next location / destination prediction task
depends on many of the same factors discussed for trajectory prediction. The
evaluation metrics for next location / destination prediction tasks usually
include accuracy, precision and recall which pick up on the fact that
predicting the location from a finite set of potential locations is
reminiscent of a classification task. An alternative approach is to treat the
task like a regression task and measure, for example, the RMSE of the spatial
distance between the predicted and the true location (see Table 5).

###  2.6 Synthetic data generation

This category covers the generation of synthetic movement data, such as
synthetic trajectories [65, 47] and synthetic flows [50]. The creation of
synthetic data is of particular interest to either address data privacy
concerns or to deal with a lack of real data for certain regions. For example,
Rao et al. [47] address the privacy issue by developing an end-to-end deep
LSTM-TrajGAN model to generate privacy-preserving synthetic trajectory data
for data sharing and publication. Similarly, Zhang et al. [65] propose an end-
to-end trajectory generation model for generating synthetic trajectories using
VAE-like encoders and decoders where a prior generator based on variational
recurrent structure generates noise at time $t$ by considering the noise at
the previous time step.

Aggregated flow data, on the other hand is usually not privacy sensitive but
it may not be available in some regions or for certain time periods.
Therefore, Simini et al. [50] developed an MLP model (denoted Deep Gravity) to
generate mobility ﬂow probabilities. They evaluated Deep Gravity on mobility
ﬂows in England, Italy, and New York State and achieved a good performance
even for regions with no data available for training.

The difficulty of a synthetic data generation task, as well as the selection
of suitable evaluation metrics for synthetic data generation depend on the
specific details of the application (for example, whether it aims at
generating individual trajectories or aggregated flows) and method used (see
Table 6).

###  2.7 Location classification

This use case category covers the classification of locations, such as certain
areas or points of interest (POIs), using patterns derived from movement data.
The classification of regionally dominant movement patterns may be of interest
in and of itself [62] or it may help with the classification of POIs (e.g.
ports [1]) and trip destinations [38].

To detect regionally dominant movement patterns, Yang et al. [62] use
direction information and density maps to generate directional flow images.
They convert trajectories into images where each pixel contains the
directional flows. They use a CNN to classify the input image patterns and
detect the dominant regional movements.

An approach that makes more use of the temporal information contained in
trajectories is presented by Altan et al. [1]. They use a temporal GNN (TGNN)
to distinguish gateway ports from actual ports using AIS vessel movement data.
After extracting the ports (nodes) from the raw AIS messages using DBSCAN,
they extract trips between consecutive ports and build a graph for each time
step to generate the time-ordered daily graph sequence for the TGNN.

Lyu et al. [38] train a model to predict trip purposes based on destination
locations. Their model is trained using activity, origin, and destination
matrices derived from OD data. They use latent mode alignment to align the
geographic contextual latent with travel activities. This approach is called
the plug-in memory network.

The difficulty of a specific location classification task depends on the
nature of the location classes of interest and other factors previously
discussed for (sub)trajectory classification tasks. An additional factor is
whether the evaluation is performed using locations that fall into the same
geographic region as the training locations or whether the the evaluation
locations are from a different region (thus adding the challenge of model
transferability to different geographic contexts). The evaluation metrics for
location classification usually include precision and recall (see Table 6).

###  2.8 Traffic volume prediction

This use case category covers the prediction of traffic or crowd volumes and
flows, including, for example, predicting traffic volume on street segments
[4, 27], human activity at specific POIs [17] and metropolitan areas [31, 36],
or predicting animal movement dynamics [34].

Models for this use case are commonly trained with aggregated trajectory data.
For example, traffic movies are the training data provided for the
Traffic4Cast 2021 competition which challenged participants to predict traffic
under conditions of temporal domain shift (Covid-19 pandemic) and spatial
shift (transfer to entirely new cities). Lu [36] won this challenge using CNN
(U-Net) and multi-task learning. Their multi-task learning approach randomly
samples from all available cities and trains the U-Net model to jointly
predict the future traffic states for different cities. Wang et al. [55] also
use the traffic movie approach. They aggregate individual-level trajectories
into a grid with inflow referring to the total number of incoming traffic
entering this region from other regions during a given time interval and
outflow representing the total number of traffic leaving the region.
Similarly, Zhang et al. [66] create temporal grids of average traffic speed
and taxi inflow per cell.

Graph-based models represent another common approach for this use case. For
example, Li et al. [31] build a graph for their GCN by aggregating CDR data
and representing spatial statistical units as nodes and their relationship
(physical distance, physical movement, phone calls) as edges. Similarly,
Lippert et al. [34] build temporal graphs from bird migration data where nodes
represent radar locations, and edges represent the flows between the Voronoi
tessellation cells of the radar locations.

Other approaches include, for example, Buroni et al. [4] who provide a
tutorial using vehicle counts derived from GPS tracks to build and train a
Direct LSTM encoder-decoder model. The model is trained to predict counts of
vehicles per network edge per time step for the Belgian motorway network.
Similarly, Gao et al. [17] use their GPS tracks to count vehicles per POI per
time step (hourly) to train a GCN+GRU model that predicts these visit counts.
And Xue et al. [60] propose a translator called mobility prompting which
converts daily POI visit counts into natural language sentences so they can
use (and fine-tune) pre-trained NLP models such as Bert, RoBERTa, GPT-2, and
XLNet to predict these visit counts.

The difficulty of a specific traffic volume prediction task is influenced by
the variability of the traffic volumes in the area of interest, as well as the
desired prediction horizon and the quality of available training data. The
evaluation metrics for traffic volume or crowd flow prediction usually include
RMSE, MAE, and MAPE (see Table 6).

Table 2: Trajectory datasets and data engineering; public datasets are printed in bold Ref |  NN design |  Trajectory data |  Data engineering
---|---|---|---
Trajectory prediction / imputation
Mehri et al. (2021) [41] |  LSTM |  AIS data from NOAA101010https://coast.noaa.gov/htdata/CMSP/AISDataHandler/2017/index.html for the US East Coast, containing 58.5mio messages from 10.7k vessels over 2 months |  Trajectory is generalized using context-aware piecewise linear segmentation. The LSTM is trained on three vertices at a time
Tritsarolis et al. (2021) [54] |  RNN |  Ship AIS tracks in Piraeus111111https://zenodo.org/record/4498410 containing 138k records from 246 fishing vessels and GeoLife${}^{\ref{footnote-geolife}}$ |  Trajectories are resampled to regular 1 minute intervals and converted into sequences of differences in space $\Delta x$, $\Delta y$, and time $\Delta t$ to predict the vessel’s position in the next future step
Fan et al. (2022) [13] |  GRU |  Mobile phone GPS tracks in the Kanto area of Tokyo covering 220k users with a minimum reporting period of 5 minutes for 2 months |  Trajectories are discretized using the H3 hexagonal grid
Carroll et al. (2022) [6] |  Transformers |  Synthetic data in a minimalistic grid world environment |  Transformers are trained on trajectories as sequences of states, actions, and return-to-go tokens
Musleh et al. (2022) [43] |  Transformers |  GPS tracks in San Francisco from the GISCUP’17 dataset with 5M records |  Trajectories are discretized using the H3 hexagonal grid (tokenization) followed by the creation of spatial embeddings
Capobianco et al. (2023) [5] |  LSTM |  AIS data from the Danish Maritime Authority121212https://dma.dk/safety-at-sea/navigationalinformation/ais-data, 394 trajectories of tanker vessels belonging to two specific motion patterns of interest over one month |  Trajectories are resampled to a fixed sampling interval of 15 minutes and segmented into fixed-length trajectories with 12 positions
Arrival time prediction
Wang et al. (2018) [56] |  CRNN |  Taxi GPS tracks (FCD) in Chengdu and Beijing with a density of 2.6 to 5.5 GPS records per km |  Trajectories are fed into a GEO-Conv layer
Buijse et la. (2021) [3] |  CRNN+GRU |  Train GPS tracks in the Netherlands covering 350k train trips |  Trajectories are fed into a GEO-Conv layer
Derrow-Pinion et al. (2021) [11] |  GNN |  Google Maps road segments with travel times/speeds |  Road network graph edges are subdivided into shorter segments (modeled as GNN graph nodes) with associated aggregated travel time/speed information; segments are combined into supersegments (GNN graph edges)
(Sub)trajectory classification
Chen et al. (2020) [8] |  CNN |  Ship AIS tracks in Tianjin, China covering 23k trips |  Trajectories are converted into trajectory images with pixels coloured according to movement type (static, maneuvering, normal)
Yang et al. (2022) [63] |  CNN |  Ship AIS tracks in Northern America of the U.S. National Oceanic and Atmospheric Administration’s Office of Coastal Management with 259k records after pre-processing |  Trajectories are converted into trajectory images with the same method as described in [8].
Wang et al. (2021) [57] |  (C)RNN, LSTM, Transformers, AdaNet |  SHL dataset131313http://www.shl-dataset.org/activity-recognition-challenge-2021/: asynchronously sampled radio data of smartphones with up to 2,812 hours of labeled data, e.g. GPS reception and location, Wifi reception and GSM cell tower scans |  [SHL challenge summary paper] Mostly hand-crafted feature engineering as input, but also two teams using raw trajectory data
|  |  |
Table 3: Trajectory datasets and data engineering; public datasets are printed in bold – continued Ref |  NN design |  Trajectory data |  Data engineering
---|---|---|---
Anomaly detection
Liatsikou et al. (2021) [33] |  LSTM-based AE |  Porto taxi tracks${}^{\ref{footnote-porto}}$ (FCD) |  Trajectories are down-sampled to 60s and represented as a sequence of vectors (lat, lon) and clipped to the first nine points to fit the autoencoder requirements
Nguyen et al. (2022) [44] |  Probabilistic RNN |  Ship AIS tracks from a single receiver in Ushant, France |  Trajectories are down-sampled to 600s and converted to a four hot vectors (lat, lon, SOG, COG) with the resolutions of 0.01° for lat/lon, 1 knot for SOG, and 5° for COG.
Singh et al. (2022) [51] |  RNN regression model |  Ship AIS tracks in the Baltic sea region and near Bremerhaven, Germany for two months (including a comparison between satellite-based and coastal AIS) |  Trajectories are resampled and interpolated at the 60s and converted into a graph with nodes representing turning points for the vessel trajectories and edges representing the sea lanes traveled by vessels
Next location / destination prediction
Gao et al. (2019) [16] |  GRU & CNN |  Foursquare check-ins in New York and Singapore and Gowalla data in Houston and California with an average of 229k records of 3k users |  Location (¡id, lon, lat¿) sequences are converted into sequences embeddings using a causal embedding method
Feng et al. (2018) [14] |  Attention-GRU |  Foursquare check-ins141414https://sites.google.com/site/yangdingqi/home/foursquare-dataset?pli=1 with 294k records covering 15k users, other mobile application location records (search & check-ins) with 15mio records covering 5k users, and CDR with 491k records covering 1k users |  Location sequences are converted into embeddings using two independent attention mechanisms which are then fed into DeepMove’s GRUs and a historical attention module
Li et al. (2020) [30] |  SAN |  Foursquare check-ins, Tweets, Yelp and NYC data |  Location sequences are converted into embeddings
Liao et al. (2018) [32] |  RNN |  Foursquare check-ins in NYC and Tokyo with an average of 400k records of 1.7k users |  Location sequences and activity/location graphs are converted into embeddings for MCARNN
Liu et al. (2016) [35] |  RNN |  Gowalla check-ins${}^{\ref{footnote-gowalla}}$ and Global Terrorism Database (GTD) incidents151515http://www.start.umd.edu/gtd/ |  Location sequences are converted into time-specific and distance-specific transition matrices for ST-RNN
Hong et al. (2022) [24] |  Transformers |  Mobile phone GPS tracks in Switzerland from the Green Class (GC) study covering 139 participants for a year and from the yumuv study covering 498 participants for 3 months |  Location sequences are generated from GPS tracks by first filtering stay locations with a stay duration ¿25min and then spatially aggregating stays into locations. These location sequences are converted into location, time, day, and mode embeddings which are fed into the transformers
Tenzer et al. (2022) [53] |  Hyper network (LSTMs) |  Porto taxi tracks${}^{\ref{footnote-porto}}$ covering 1.7mio trips by 442 taxis in Porto, Portugal for 12 months |  Sequences of trajectory points are converted to sequences of spatial embeddings via a geospatial encoding mechanism
Table 4: Trajectory datasets and data engineering; public datasets are printed in bold – continued Ref |  NN design |  Trajectory data |  Data engineering
---|---|---|---
Synthetic data generation
Rao et al. (2020) [47] |  LSTM-GAN |  Foursquare check-ins161616https://github.com/bigdata-ufsc/petry-2020-marc/tree/master/data/foursquare_nyc in NYC covering 193 users with 3k trajectories and 67k records |  Trajectories are processed by a trajectory encoding model covering trajectory point encodings (location, temporal and categorical attributes) and trajectory padding (to ensure that all trajectories have the same length)
Zhang et al. (2022) [65] |  VAE-like deep generative models |  Porto taxi tracks${}^{\ref{footnote-porto}}$ (FCD); T-Drive${}^{\ref{footnote-tdrive}}$ data consisting of 10.3k taxis for one week; and Gowalla check-ins${}^{\ref{footnote-gowalla}}$ |  A coordinate encoding MLP converts two-dimensional points into a high-dimensional representation. Then, a Bidirectional LSTM is used to encode all representations with forward and backward information for a time step
Simini et al. (2021) [50] |  MLP |  England & Italy commuting flows, NY State flows including origin & destination geographic unit and estimated population flows between two geographic units |  Input to the model is of the origin & destination location as well as the distance between origin and destination. The output of the model is the probability to observe a trip between two locations.
Location classification
Yang et al. (2018) [62] |  CNN |  Synthetic data (manually drawn trajectories, rotated in a data augmentation step) |  Trajectories are converted to directional flow images (DFI) (resolution: 10×10)
Altan et al. (2022) [1] |  TGNN |  Ship AIS tracks in Halifax, Canada, covering 15 ports, 10 vessels, for 4 months (total: 513k AIS records) |  Trajectories are converted to temporal (daily) graphs of ports (nodes with associated port visit frequencies, waiting times, and speed statistics) and trips between consecutive ports (edges)
Lyu et al. (2022) [38] |  Memory network |  OD data from Tokyo Metro travel survey including travel mode, time, and purpose |  ODs are converted into a Space-Activity Matrix which is then decomposed into an activity, an origin, and a destination matrix
Traffic volume prediction
Lu (2021) [36] |  CNN (U-Net) |  Traffic movies for 10 cities in 2019+2020 with 8 dynamic channels encoding traffic speed and volume per direction and 9 static channels encoding the properties of the road maps |  The multi-task learning randomly samples from all available city traffic movies (resolution: 495×436)
Wang et al. (2022) [55] |  CNN |  Taxi GPS tracks (FCD) from TaxiBJ in Beijing for 17 months and bike GPS tracks from BikeNYC in New York City for 6 months |  Trajectories are converted to flow/traffic movies (32×32 for TaxiBJ & 16×8 for BikeNYC) with two dynamic channels encoding inflows and outflows
Li et al. (2021) [31] |  GCN+LSTM |  Sparse CDR trajectories in Senegal of 100,000 individuals for one year |  Trajectories are converted into a movement graph with edges representing the number of transitions from one cell phone tower node to the next
Buroni et al. (2021) [4] |  LSTM |  Lorry GPS tracks171717https://www.kaggle.com/datasets/giobbu/belgium-obu(FCD) in Belgium with 30s reporting interval, anonymous IDs (reset daily), timestamp, latitude, longitude, speed, and direction |  Trajectories are matched to street network to count vehicles per network edge per time step
Gao et al. (2022) [17] |  GCN+GRU |  Taxi GPS tracks (FCD) in Xi’an, China covering 7.7k taxis for 3 months with a sampling interval of 5–30 s |  Trajectories are used to count vehicles per POI (n=100) per time step (hourly)
Lippert et al. (2022) [34] |  Recurrent GNN |  Bird migration data in the form of simulated trajectories and measurements from the European weather radar network |  Trajectories are converted temporal graphs with to flows (edges) between Voronoi tessellation cells of radar locations (nodes).
Xue et al. (2022) [60] |  Transformers |  SafeGraph daily POI visit counts in NYC, Dallas and Miami |  Historical visitation data are translated into natural language sentences to fine-tune pre-trained NLP models
Zhang et al. (2020) [66] |  GAN |  Taxi GPS tracks (FCD) in Shenzhen, China for 6 months |  Trajectories are converted to traffic movies (resolution: 40×50, hourly) with average traffic speed and taxi inflow

##  3 Why Deep Learning?

This section attempts to summarize the reasons stated by authors to motivate
the use of deep learning (DL) instead of classical machine learning (ML) and
discusses the baselines, metrics, and implementations used to measure and
compare the performance of DL methods. Finally, this section ends with a
critical discussion of the findings.

###  3.1 DL Motivations

Novelty is a crucial factor determining what gets published and what does not.
Consequently, we encounter a lot of motivation statements stressing the
novelty of applying DL to certain trajectory-related tasks, along the lines
of, for example, “applying CNN for classiﬁcation of trajectory data is
relatively unexplored” [62] or “predicting the arrival times with the help of
deep learning models has been done in some recent works […] However, most of
these works aim to predict the arrival time for road vehicles […], and the
railroad industry lagged behind” [3].

Beyond the novelty factor, a key motivation for using DL rather than
traditional ML is that DL does not rely on hand-crafted features. Feature
engineering for mobility prediction and classification is challenging due to:
“1) the complex sequential transition regularities exhibited with time-
dependent and high-order nature; 2) the multi-level periodicity of human
mobility; and 3) the heterogeneity and sparsity of the collected trajectory
data” [14]. Hand-crafted features may therefore run into difficulties
capturing the full complex picture necessary for accurate predictions or
classifications [8, 38].

DL’s ability to take advantage of large volumes of data to learn complex non-
linear spatio-temporal relationships is therefore a key motivating factor.
This is particularly stressed by, for example, Derrow-Pinion et al. [11], who
argue that arrival time prediction “requires accounting for complex spatio-
temporal interactions (modelling both the topological properties of the road
network and anticipating events—such as rush hours—that may occur in the
future).” On a similar note, Liatsikou et al. [33] argue that DL models for
anomaly detection “can effectively address the challenges of feature
extraction, high dimensionality and non-linearity. Moreover, there is no need
to explicitly describe a normal pattern for a trajectory and to define the
type of the anomaly. […] the model is trained to reconstruct most of the
trajectories included in a dataset, while it fails on the more irregular ones,
which are classified as anomalies” And Buroni et al. [4] state that DL models
for road traffic forecasting “allowed to take advantage of the enormous volume
of mobility data to capture the complex non-linear space-temporal
relationships governing road traffic.” Moreover, DL may help to accurately
“perform multi-horizon road traffic predictions on large scale transportation
networks” and still “comply to real-time requirements” [37] (which may not be
achievable using established ITS or traffic simulation approaches).

Indeed, the black-box nature of DL may be advantageous in some use cases
dealing with the issue of location privacy. For example, Rao et al. [47] argue
that current approaches for privacy preservation “rely heavily on manually
designed procedures” and that “once the procedure is disclosed, one may have
the chance to recover the original trajectory data (e.g., using reverse
engineering).” Furthermore, for synthetic data generation, “the trade-off
between the effectiveness of trajectory privacy protection and the utility for
spatial and temporal analyses is still hard to control, and this issue has not
been fully discussed or evaluated. Besides, current studies mainly focus on
the spatial dimension of trajectory data whereas other semantics (e.g.,
temporal and thematic attributes) are rarely considered. In fact, these
characteristics have been proven to be crucial for trajectory user
identiﬁcation” [47].

The before-mentioned advantages notwithstanding, DL may not always be the best
approach [21]. Reasons include the black-box nature of many DL models [42] as
well as the large amounts of training data and computational resources needed
to train DL models [49]. For example, Jonietz et al. [26] point out that the
”lack of explainability can result in erosion of trust from users who may
question the results”, which in turn may limit the practical impact of DL
developments. The explainability of ML and DL models remains an open challenge
[42]. While classical ML models often rely on handcrafted features engineered
by domain experts, DL models automatically learn features from raw data. In
the absence of large, diverse, and representative datasets, DL models may not
always capture domain-specific nuances and may be prone to overfitting.
Additionally, “the rise of compute-intensive AI research can have significant,
negative environmental impacts” [29], depending on how and where the energy
for training DL models is generated, stored, and delivered [29, 49]. The
choice of DL models over traditional ML models should therefore be founded in
significantly better results than simpler classic ML baselines.

###  3.2 Baselines & Metrics

In order to evaluate the performance of new methods, it is essential for
authors to report benchmark datasets and models as well as the considered
metrics for evaluating their experiments. Since the machine learning tasks
vary according to the considered mobility use case, comparability can at best
be achieved within one use case. Therefore, Tables 5-6 summarize the different
traditional ML and DL methods that have been used as baselines as well as the
metrics for each use case.

The number of ML and DL baselines provided per publication varies widely, as
shown in Tables 5-6. While publications on Traffic volume / crowd flow
prediction and Next location / destination prediction tend to provide numerous
baselines for comparison, the number of baselines for Location classification,
Trajectory prediction, and Anomaly Detection are much lower. This may hint at
a general lack of suitable reference implementations.

Within most use cases, authors tend to adopt similar metrics: Classification
tasks, such as Location classification, (Sub)trajectory classification and –
to a degree – Next location / destination prediction strongly rely on F1
scores, accuracy and precision metrics. In addition to providing correct
destination predictions, another aspect is to provide the prediction as soon
as possible [22]. On the other hand, regression tasks, such Arrival time
prediction, Traffic volume / crowd flow prediction and Trajectory prediction
tend to rely on RMSE, MAE, and MAPE. Papers on Synthetic data generation,
however, do not feature prominent common metrics.

As Tables 5-6 show, some trajectory data-specific DL methods, such as VaNext,
DeepMove, and LSTM-TrajGAN, have been used repeatedly as baselines to evaluate
new methods. However, not all reviewed publications provide the source code of
the published models to facilitate reuse of the implementation. Table 7 lists
the publicly available implementations grouped by the used machine learning
library (PyTorch, TensorFlow, or Keras) and ordered by their popularity. It is
worth noting that most repositories are one-shot projects, that is, the
implementations are not actively worked on and maintained after publication of
the paper which likely negatively affects their re-usability.

In contrast, there is a growing number of trajectory analysis libraries which
offer essential functionality for machine learning from trajectory data, such
as Trackintel181818https://github.com/mie-lab/trackintel [40] (e.g., used by
[24]), MovingPandas191919http://movingpandas.org [18] (e.g., used by [41]) and
scikit-mobility202020https://scikit-mobility.github.io/scikit-mobility/ [46]
(e.g., used by [50]) as well as spatiotemporal deep learning frameworks, such
as TorchGeo212121https://torchgeo.readthedocs.io/en/stable/ [52] (which
focuses on raster data) and
GeoTorchAI222222https://github.com/wherobots/GeoTorchAI [9] (which includes a
bicycle flow prediction example using ST-ResNet [64]).

Table 5: ML and DL baselines used to compare proposed new methods, including papers that do not provide baseline comparisons (n/a) Ref |  ML baselines |  DL baselines |  Metrics
---|---|---|---
Trajectory prediction
Carroll et al. (2022) [6] |  Agent-based models |  (n/a) |  Model Validation Loss
Fan et al. (2022) [13] |  ARIMA, prophet |  (n/a) |  RMSE, MAE, MAPE, Cross Entropy
Mehri et al. (2021) [41] |  (n/a) |  ordinary LSTM |  RMSE, point-wise horizontal error
Musleh et al. (2022) [43] |  (n/a) |  (n/a) |  Mean and median shortest Euclidean distance
Tritsarolis et al. (2021) [54] |  (n/a) |  (n/a) |  For the co-movement prediction: spatial similarity, temporal similarity, Jaccard similarity
Capobianco et al. (2023) [5] |  (n/a) |  Previous version of their labeled architecture |  Average prediction error (APE), average displacement error (ADE)
Arrival time prediction
Derrow-Pinion et al. (2021) [11] |  Non-parametric models using
real-time & historical speeds |  DeepSets |  RMSE
Buijse et al. (2021) [3] |  (n/a) |  (n/a) |  RMSE, MAE
Wang et al. (2018) [56] |  Average speed, nearest neighbour, GBDT, Perceptron |  RNN |  RSME, MAE, MAPE
(Sub)trajectory classification
Chen et al. (2020) [8] |  KNN, SVM, DT |  (n/a) |  F1, Accuracy, AUC
Wang et al. (2021) [57] |  SHL Challenge summary paper comparing ML & DL submissions |  |  F1
Yang et al. (2022) [63] |  (n/a) |  (n/a) |  Accuracy, Precision, Recall, F1
Anomaly detection
Liatsikou et al. (2021) [33] |  Local Outlier Factor (LOF) |  Denoising autoencoder with FFNN |  Precision, Recall, F1
Nguyen et al. (2022) [44] |  (n/a) |  TREAD, LSTM, VRNN |  Qualitative manual expert evaluation
Singh et al. (2022) [51] |  Similarity model |  RNN |  Accuracy, false detection rate
Next location / destination prediction
Feng et al. (2018) [14] |  HMM |  RNN |  Accuracy
Gao et al. (2019) [16] |  HMM |  PRME, ST-RNN, Bi-LSTM, POI2Vec, DeepMove |  RMSE
Hong et al. (2022) [24] |  HMM |  LSTM, DeepMove, MobTcast, LSTM attn |  Accuracy, Precision, Recall, F1
Li et al. (2020) [30] |  (n/a) |  ST-RNN, MCARNN, DeepMove, VANext |  Negative Log-Likelihood, Accuracy, Average Percentage Rank
Liao et al. (2018) [32] |  Most Frequent, Context-
aware hybrid model |  ST-RNN, Semantic-Enriched-RM, Multi-Task GRU, SCARNN |  Accuracy, Negative Log-Likelihood, Mean Average Precision
Liu et al. (2016) [35] |  (n/a) |  RNN, ST-RNN |  F1, accuracy, precision, recall, AUC
Tenzer et al. (2022) [53] |  (n/a) |  RNN, Attention-based BiLSTM |  Mean Haversine Distance (MHD)
Gulisano et al. (2018) [22] |  DEBS 2018 Grand Challenge paper |  |  Average time span between a correct prediction and the arrival at the destination
Table 6: ML and DL baselines used to compare proposed new methods, including papers that do not provide baseline comparisons (n/a) – continued Ref |  ML baselines |  DL baselines |  Metrics
---|---|---|---
Synthetic data generation
Rao et al. (2020) [47] |  (n/a) |  LSTM-TrajGAN |  Accuracy, Precision, Recall, F1, Hausdorff Distance, Jaccard Index, temporal visit probability distribution
Simini et al. (2021) [50] |  Gravity Model (G), Nonlinear Gravity Model (NG) |  Multi-Feature Gravity Model (MFG) |  Common Part of Commuters (CPC), Relative improvement w.r.t. G
Zhang et al. (2022) [65] |  Rule-based models |  LSTM, IGMM-GAN, VAE |  Mean Distance Error (MDE), Maximum Mean Discrepancy (MMD), novelty score, Violation Score (VS)
Location classification
Altan et al. (2022) [1] |  (n/a) |  (n/a) |  Precision, recall, F-score
Lyu et al. (2022) [38] |  (n/a) |  (n/a) |  F1-scores (micro and macro)
Yang et al. (2018) [62] |  (n/a) |  (n/a) |  Precision
Traffic volume / crowd flow prediction
Buroni et al. (2021) [4] |  Seasonal Window Persistence Model (SW) |  LSTM (directed, iterated and with encoder-decoder approach) |  RMSE, MAE
Gao et al. (2022) [17] |  ARIMA, SVR |  GRU, DCRNN, TGNN |  RMSE, MAE, Accuracy
Li et al. (2021) [31] |  ARIMA, KNN, GBDT |  LSTM, AIP-PM, AIP-PGM, ASTGCN |  RMSE, MAE
Lippert et al. (2022) [34] |  Historical Average (HA),
GAM, GBT |  (n/a) |  RMSE
Wang et al. (2022) [55] |  Historical Average (HA) |  DeepST, ST-ResNet, ConvLSTM, DeepSTN, Graph WaveNet, DeepLGR |  RMSE, MAE
Zhang et al. (2020) [66] |  Spatial smoothing |  ConvLSTM, FC-SA, CNN-SA, FC-LSTM, CNN-LSTM, DyConv-LSTM |  RMSE, MAPE
Xue et al. (2022) [60] |  Linear Regression |  GRU, GRUAtt, Transformer |  RMSE, MAE
Table 7: Neural network implementations published together with their reviewed papers, ordered by ML library and stars. The stars column lists the number of Github stars, Kaggle upvotes, or Figshare/Zenodo downloads of the respective repository as of July 2023. |  Ref |  Model/System Name |  Code Repository |  Stars |  Last
updated
---|---|---|---|---|---
PyTorch
|  [14] |  DeepMove |  https://github.com/vonfeng/DeepMove |  131 |  08/19
|  [56] |  DeepTTE with
GEO-Conv layer |  https://github.com/UrbComp/DeepTTE |  127 |  06/20
|  [50] |  DeepGravity |  https://github.com/scikit-mobility/DeepGravity |  57 |  12/21
|  [66] |  Curb-GAN |  https://github.com/Curb-GAN/Curb-GAN |  25 |  02/20
|  [24] |  n/a |  https://github.com/mie-lab/location-mode-prediction |  9 |  07/23
|  [34] |  FluxRGNN |  https://github.com/FionaLippert/FluxRGNN
https://zenodo.org/record/6921595 |  3 |  07/22
|  [3] |  Deep Train Arrival
Time Estimator |  https://github.com/basbuijse/train-arrival-time-estimator |  2 |  07/21
|  [13] |  n/a |  https://github.com/fanzipei/crowd-context-prediction/tree/master |  0 |  02/21
|  [60] |  AuxMobLCast |  https://github.com/cruiseresearchgroup/AuxMobLCast |  0 |  11/22
|  [65] |  TrajGen |  https://github.com/tongjiyiming/TrajGen |  0 |  10/20
TensorFlow |  |
|  [31] |  n/a |  https://figshare.com/articles/dataset/Prediction_of_human_activity_intensity_using_the_interactions_in_physical_and_social_spaces_through_graph_convolutional_networks/11829306/1 |  122 |  03/21
|  [44] |  GeoTrackNet |  https://github.com/CIA-Oceanix/GeoTrackNet |  67 |  10/20
|  [51] |  Uncertainty EDL
Graph |  https://github.com/sansastra/uncertainty_edl_graph |  8 |  07/21
Keras |  |
|  [47] |  LSTM-TrajGAN |  https://github.com/GeoDS/LSTM-TrajGAN |  42 |  01/23
|  [8] |  CNN-SMMC |  https://github.com/rechardchen123/Ship_movement_classification_from_AIS |  19 |  04/20
|  [4] |  Tutorial on traffic forecasting with DL |  https://www.kaggle.com/code/giobbu/lstm-encoder-decoder-tensorflow |  9 |  09/21
|  [33] |  BMDA anomaly detection |  https://github.com/marialiatsikou/BMDA_anomaly_detection |  0 |  02/21

###  3.3 Discussion

The lack of common benchmarks methods and datasets makes it difficult to
compare methods and to systematically advance the research field. For example,
Wang et al. [57], remark that “to date, most research groups assess the
performance of their algorithms using their own datasets on their own
recognition tasks. These tasks often differ in the sensor modalities or in the
allowed recognition latency.” Additionally, there is a lack of benchmark
datasets for most use cases (as noted, e.g., regarding location classification
by Altan et al. [1]). For some use cases, researchers, therefore, resort to
synthetic data. For example, Liatsikou et al. (2021) create their own
synthetic trajectory anomalies to evaluate their model [33].

Another issue is the lack of agreed evaluation approaches and metrics. While
papers introducing new DL methods – quite naturally – claim superior
performance over existing ML/DL methods, data challenges, such as the SHL
Challenge on traffic flow prediction, paint a less clear picture, with regular
ML models often outperforming DL models on key metrics, such as F1 score,
train time, and test time. For example, Wang et al. [57] compare 7 ML SHL
Challenge submissions (including XGBoost, RF, DT and ensembles of classifiers)
and 8 DL submissions (including CNN, LSTM, CNN+RNN, Transformer, and AdaNet),
coming to the conclusion that “ML outperforms DL in global. ML has a similar
higher bound as DL, while achieving a much higher lower bound. The smaller
dynamic range verifies the better robustness of ML. While DL achieves the
highest F1 score (75.4%), it is only 1.1 percentage points higher than the
best ML approach (74.3%).” Furthermore, as would be expected “DL takes much
more time for training than ML, and also takes more time for testing.” [57]
Regarding DL’s independence from hand-crafted features, it is particularly
interesting that “Feature-based DL approaches achieve a much higher F1 score
(75.4%) than raw-data-based approaches (43.4%).” [57]

Since the majority of papers are not directly reproducible (either due to the
lack of data, code, or both) or prohibitively expensive to reproduce,
verification of stated claims remains a challenge. Typical pitfalls faced when
evaluating models for mobility use cases include [59]: 1) different datasets
employing often (undocumented) pre-processing pipelines 2) unnoticed over-
fitting due to spatial and or temporal auto-correlation and 3) the
accuracy/generality trade-off due to idealized conditions and lack of
variation in the data. In particular, Widhalm et al. (2018) [59] show that
evaluation with a random training/test split suggests a considerably higher
accuracy than with proper backtesting (reducing the F1 score from 96% to 84%
when accounting for auto-correlation and down to 54% when transferring the
model to different conditions). Therefore, they argue that results reported in
publications may not represent reliable indicators for the future performance
in real-world applications, where generality and robustness are essential.

##  4 Trends

While previous sections provided a detailed review of use cases and types of
deep neural networks for trajectory data, their focus on recent work is
insufficient to comprehensively understand longer-term trends in this field.
Therefore, in this section, we investigate these longer-term trends by through
the following research questions: How has the DL research on mobility
trajectory application domains evolved in recent years? Which DL architectures
are being utilized in research, and how have they changed over time?

To answer these questions, we performed a systematic literature review (SLR)
following the PRISMA method [45] for data collection and selection. PRISMA is
a widely used guideline for conducting SLRs. The search was carried out on
’Google Scholar’ and ’IEEE Xplore’ using the following keywords:

  * •

Google Scholar: "Deep Learning" AND "Trajectories" AND "Mobility" -"survey"

  * •

Google Scholar: "Machine Learning" AND "Trajectories" AND "Mobility"

  * •

IEEE Explor: "Deep Learning" AND "Trajectories" AND "Mobility"

and limited to the timeline between 2018 and 2023. The total number of
initially retrieved publications was 337. The following steps were performed
to select relevant papers: First, we discarded duplicate articles (8
publications) within and across databases. After that, we reviewed titles and
abstracts as well as other parts of the publication if necessary to manually
classify each publication regarding its use case and machine learning
technology. Publications that either do not address one of the previously
introduced eight use cases as listed in Tab. 1 (39 publications) or do not
apply neural networks are discarded (24 publications). In the next step, all
publications not published in a journal, at a conference, as a book chapter,
or as a preprint (14 publications) were excluded. We also considered
preprints, as many machine learning publications are initially released as
preprints before being presented in a journal or at a conference much later.
We hope that this will allow us to assess current trends more accurately. In
addition, review papers were excluded from further analysis (13 publications).
Finally, we included the papers analyzed in the previous sections (36
publications), which yielded a total of 275 relevant papers. The relevant
papers comprise 111 articles, 114 journals, 25 preprints, and 0 book chapters.

Figure 2 shows the trend analysis results for the eight previously introduced
use cases. By far, the most popular use case is ’Traffic volume / crowd flow
prediction’, responsible for approximately 33% of publications. The next most
common use cases are ’Trajectory prediction/ imputation’, followed by ’Next
location / destination prediction’ and ’(Sub)trajectory classification’. On
the other hand, the least common use cases are ’Anomaly detection’, ’Synthetic
data generation’, ’Arrival time prediction’, and ’Location classification’. It
is worth noting that the numbers for 2023 only represent publications up until
the time of writing (June 2023) and that publications can address more than
one use case. Please refer to the Section Appendix for the absolute number of
publications per use case.

Figure 2: Portions of publications by use case from 2018 to 2023. Total number
of publications per year is displayed at the top of each bar.

To effectively analyze the vast array of diverse neural network architectures
extracted from the relevant publications, we categorized and clustered them to
be able to discern prevailing trends and patterns. We applied the following
categorizations:

  1. 1.

RNN – Vanilla RNN, LSTM, or GRU

  2. 2.

GNN – Graphs and graph learning methods

  3. 3.

CNN – Convolutional networks

  4. 4.

AE – Autoencoders (Dense, LSTM, or CNNs)

  5. 5.

Attention – Attention mechanisms, including transformers

  6. 6.

GAN – Generative adversarial methods

  7. 7.

DRL – Deep reinforcement learning

  8. 8.

CRNN – Hybrid architectures combining CNN and RNN methods (e.g., LSTM or GRU)

  9. 9.

FNN – Fully connected dense networks

  10. 10.

Other DL – Any other deep learning architecture, such as Capsule Net, Gravity
Net, etc.

This categorization is more fine-grained than the summary in Table 1 to allow
for better differentiation between the less common NNs and their respective
trends.

Even more than the use case trends, this neural network design trends analysis
summarized in Figure 3 is dominated by a single class: RNNs, accounting for
approximately 39% of publications. The second most popular option for the
early years were CNNs, which decreased in popularity in 2020. CNNs have since
been overtaken by attention mechanisms and – potentially – GNNs (but the 2023
data here is still inconclusive). Autoencoders (AE) comprised approximately
18% of the publications in 2019 and spiked again in 2021, which seems to be
decreasing in popularity since. FNNs, CRNNs, DRL, GAN, and other DL methods
have remained niche applications. Note that publications can integrate
multiple neural network designs.

Figure 3: Portions of publications by NN design from 2018 to 2023. Total
number of publications per year is displayed at the top of each bar.

Bringing both neural network design and use case perspective together, we also
analyzed the techniques employed for each use case over the years, as shown in
Figure 4. This shows the prevalent use of RNNs for tasks such as traffic
volume prediction, trajectory prediction, and next location/final destination
prediction. Furthermore, we observed an emerging trend where GNNs are
increasingly being adopted as a viable solution for traffic volume and
trajectory prediction use cases. Similarly, CNNs have gained popularity as a
practical approach for traffic volume prediction. However, in trajectory
classification tasks, they serve as the second choice after RNNs.

Figure 4: Overview of the deep learning approaches per use cases from 2016 to
2023

##  5 Conclusion

In this work, we reviewed deep learning-based research focusing on trajectory
data in mobility use cases. In most cases, even if dense raw trajectory data
is used in the process, it is not ingested directly for training the neural
networks. Instead, data engineering steps are applied that convert
trajectories into more compact representations of individual trajectories
(sparse trajectories) or aggregations of multiple trajectories. This
aggregated trajectory data is commonly presented as time series of vectors,
graphs, or images (movies). Since these data engineering steps are a recurring
need in mobility data science, we expect further uptake of trajectory analysis
libraries, such as Trackintel, MovingPandas, and scikit-mobility since these
libraries implement many common trajectory generalization, aggregation, and
analysis methods and their focus on scientific software engineering aims at
reusability and long(er)-term availability of the software libraries. This is
an essential step towards more sustainable mobility data science, since most
of the implementations summarized in Table 7 have not been substantially
updated/maintained since being published which reduces their reusability since
the implementations will – sooner or later – become incompatible with newer
versions of the underlying data science libraries.

Future research should address the issues of model transferability, benchmark
availability, and model explainability. Current work rarely addresses the
issue of model transferability. Since most existing global ML models “cannot
perform well locally, or be transferred to study similar problems in other
regions”[25], transferability should be considered when evaluating or
comparing models. Additionally, developed models, even for the same
application and trajectory type, are difficult to evaluate (for example, due
to the lack of ground truth for anomaly detection) and to compare due to
different datasets and applied metrics. Therefore, more open benchmark
datasets are needed. Finally, to better understand the inner workings of
neural networks and to ensure the trustworthiness of model decisions,
explainability should play a more crucial role in model development.

## Declarations

### Competing interests

We, the authors, declare that we have no competing interests as defined by
Springer, or other interests that might be perceived to influence the results
and/or discussion reported in this paper.

### Authors’ contributions

All authors contributed to the study’s conception and design. All authors
contributed to the in-depth qualitative review of recent work. A.G. wrote the
original draft and compiled the tables and Figure 1. A.J and A.W. performed
the quantitative literature review and prepared Figures 2-6. All authors
participated in contributing to the text and the content of the manuscript,
including revisions and edits. All authors approve of the content of the
manuscript. All authors reviewed the manuscript.

### Funding

This work is mainly funded by the EU’s Horizon Europe research and innovation
program under Grant No. 101070279 MobiSpaces and No. 101093051 EMERALDS, and
the EU’s Horizon 2020 program under Grant No. 101021797 STARLIGHT.

### Availability of data and materials

The dataset and code to reproduce the quantitative literature review are
available at https://github.com/anahid1988/MobilityDL.

## References

  * [1] Altan, D., Etemad, M., Marijan, D., Kholodna, T.: Discovering Gateway Ports in Maritime Using Temporal Graph Neural Network Port Classification (Apr 2022), http://arxiv.org/abs/2204.11855, arXiv:2204.11855 [cs]
  * [2] Andrienko, G., Andrienko, N., Bak, P., Keim, D., Kisilevich, S., Wrobel, S.: A conceptual framework and taxonomy of techniques for analyzing movement. Journal of Visual Languages & Computing 22(3), 213–232 (2011). https://doi.org/https://doi.org/10.1016/j.jvlc.2011.02.003, https://www.sciencedirect.com/science/article/pii/S1045926X11000139
  * [3] Buijse, B.J., Reshadat, V., Enzing, O.W.: A Deep Learning-Based Approach for Train Arrival Time Prediction. In: Yin, H., Camacho, D., Tino, P., Allmendinger, R., Tallón-Ballesteros, A.J., Tang, K., Cho, S.B., Novais, P., Nascimento, S. (eds.) Intelligent Data Engineering and Automated Learning – IDEAL 2021, vol. 13113, pp. 213–222. Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-91608-4_22, https://link.springer.com/10.1007/978-3-030-91608-4_22, series Title: Lecture Notes in Computer Science
  * [4] Buroni, G., Bontempi, G., Determe, K.: A tutorial on network-wide multi-horizon traffic forecasting with deep learning. In: Workshop Proceedings of the EDBT/ICDT 2021 Joint Conference. Nicosia, Cyprus (2021), https://ceur-ws.org/Vol-2841/BMDA_6.pdf
  * [5] Capobianco, S., Forti, N., Millefiori, L.M., Braca, P., Willett, P.: Recurrent Encoder–Decoder Networks for Vessel Trajectory Prediction With Uncertainty Estimation. IEEE Transactions on Aerospace and Electronic Systems 59(3), 2554–2565 (Jun 2023). https://doi.org/10.1109/TAES.2022.3216823, https://ieeexplore.ieee.org/document/9946391/
  * [6] Carroll, M., Lin, J., Paradise, O., Georgescu, R., Sun, M., Bignell, D., Milani, S., Hofmann, K., Hausknecht, M., Dragan, A., Devlin, S.: Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers (Apr 2022), http://arxiv.org/abs/2204.13326, arXiv:2204.13326 [cs]
  * [7] Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM Computing Surveys 41(3), 1–58 (Jul 2009). https://doi.org/10.1145/1541880.1541882, http://portal.acm.org/citation.cfm?doid=1541880.1541882
  * [8] Chen, X., Liu, Y., Achuthan, K., Zhang, X.: A ship movement classification based on Automatic Identification System (AIS) data using Convolutional Neural Network. Ocean Engineering 218, 108182 (Dec 2020). https://doi.org/10.1016/j.oceaneng.2020.108182, https://linkinghub.elsevier.com/retrieve/pii/S0029801820311124
  * [9] Chowdhury, K., Sarwat, M.: GeoTorch: a spatiotemporal deep learning framework. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–4. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3561036, https://dl.acm.org/doi/10.1145/3557915.3561036
  * [10] Cortes, C., Gonzalvo, X., Kuznetsov, V., Mohri, M., Yang, S.: AdaNet: Adaptive Structural Learning of Artificial Neural Networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 874–883. PMLR (Aug 2017), https://proceedings.mlr.press/v70/cortes17a.html
  * [11] Derrow-Pinion, A., She, J., Wong, D., Lange, O., Hester, T., Perez, L., Nunkesser, M., Lee, S., Guo, X., Wiltshire, B., Battaglia, P.W., Gupta, V., Li, A., Xu, Z., Sanchez-Gonzalez, A., Li, Y., Velickovic, P.: ETA Prediction with Graph Neural Networks in Google Maps. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management. pp. 3767–3776. ACM, Virtual Event Queensland Australia (Oct 2021). https://doi.org/10.1145/3459637.3481916, https://dl.acm.org/doi/10.1145/3459637.3481916
  * [12] Desolneux, A., Moisan, L., Morel, J.M.: From gestalt theory to image analysis: a probabilistic approach, vol. 34. Springer Science & Business Media (2007)
  * [13] Fan, Z., Yang, X., Yuan, W., Jiang, R., Chen, Q., Song, X., Shibasaki, R.: Online trajectory prediction for metropolitan scale mobility digital twin. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–12. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3561040, https://dl.acm.org/doi/10.1145/3557915.3561040
  * [14] Feng, J., Li, Y., Zhang, C., Sun, F., Meng, F., Guo, A., Jin, D.: DeepMove: Predicting Human Mobility with Attentional Recurrent Networks. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW ’18. pp. 1459–1468. ACM Press, Lyon, France (2018). https://doi.org/10.1145/3178876.3186058, http://dl.acm.org/citation.cfm?doid=3178876.3186058
  * [15] Feng, S., Cong, G., An, B., Chee, Y.M.: POI2Vec: geographical latent representation for predicting future visitors. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. pp. 102–108 (2017)
  * [16] Gao, Q., Zhou, F., Trajcevski, G., Zhang, K., Zhong, T., Zhang, F.: Predicting Human Mobility via Variational Attention. In: The World Wide Web Conference on - WWW ’19. pp. 2750–2756. ACM Press, San Francisco, CA, USA (2019). https://doi.org/10.1145/3308558.3313610, http://dl.acm.org/citation.cfm?doid=3308558.3313610
  * [17] Gao, Y., Chiang, Y., Zhang, X., Zhang, M.: Traffic volume prediction for scenic spots based on multi‐source and heterogeneous data. Transactions in GIS 26(6), 2415–2439 (Sep 2022). https://doi.org/10.1111/tgis.12975, https://onlinelibrary.wiley.com/doi/10.1111/tgis.12975
  * [18] Graser, A.: MovingPandas: Efficient Structures for Movement Data in Python. GI_Forum 1, 54–68 (2019). https://doi.org/10.1553/giscience2019_01_s54, https://hw.oeaw.ac.at?arp=0x003aba2b
  * [19] Graser, A., Dragaschnig, M., Koller, H.: Exploratory Analysis of Massive Movement Data. In: Werner, M., Chiang, Y.Y. (eds.) Handbook of Big Geospatial Data, pp. 285–319. Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-55462-0_12, https://link.springer.com/10.1007/978-3-030-55462-0_12
  * [20] Graser, A., Schmidt, J., Dragaschnig, M., Widhalm, P.: Data-driven Trajectory Prediction and Spatial Variability of Prediction Performance in Maritime Location Based Services. In: 15th International Conference on Location-Based Services. pp. 129–134. Vienna, Austria (2019). https://doi.org/10.34726/lbs2019.23
  * [21] Grinsztajn, L., Oyallon, E., Varoquaux, G.: Why do tree-based models still outperform deep learning on tabular data? (Jul 2022), http://arxiv.org/abs/2207.08815, arXiv:2207.08815 [cs, stat]
  * [22] Gulisano, V., Jerzak, Z., Smirnov, P., Strohbach, M., Ziekow, H., Zissis, D.: The DEBS 2018 Grand Challenge. In: Proceedings of the 12th ACM International Conference on Distributed and Event-based Systems. pp. 191–194. ACM, Hamilton New Zealand (Jun 2018). https://doi.org/10.1145/3210284.3220510, https://dl.acm.org/doi/10.1145/3210284.3220510
  * [23] Hawkins, D.M.: Identification of outliers, vol. 11. Springer (1980)
  * [24] Hong, Y., Martin, H., Raubal, M.: How do you go where?: improving next location prediction by learning travel mode information using transformers. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–10. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3560996, https://dl.acm.org/doi/10.1145/3557915.3560996
  * [25] Janowicz, K., Zhu, R., Verstegen, J., McKenzie, G., Martins, B., Cai, L.: Six GIScience Ideas That Must Die. AGILE: GIScience Series 3, 1–8 (Jun 2022). https://doi.org/10.5194/agile-giss-3-7-2022, https://agile-giss.copernicus.org/articles/3/7/2022/
  * [26] Jonietz, D., Sester, M., Stewart, K., Winter, S., Tomko, M., Xin, Y.: Urban mobility analytics: Report from dagstuhl seminar 22162. Dagstuhl Reports 12(4), 26–53 (2022)
  * [27] Kashyap, A.A., Raviraj, S., Devarakonda, A., Nayak K, S.R., K V, S., Bhat, S.J.: Traffic flow prediction models – A review of deep learning techniques. Cogent Engineering 9(1), 2010510 (Dec 2022). https://doi.org/10.1080/23311916.2021.2010510, https://www.tandfonline.com/doi/full/10.1080/23311916.2021.2010510
  * [28] Konzack, M.P.: Trajectory analysis: Bridging algorithms and visualization. Ph.D. thesis, Technische Universiteit Eindhoven, Eindhoven (2018)
  * [29] Lacoste, A., Luccioni, A., Schmidt, V., Dandres, T.: Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700 (2019)
  * [30] Li, A., Wang, S., Li, W., Liu, S., Zhang, S.: Predicting Human Mobility with Federated Learning. In: Proceedings of the 28th International Conference on Advances in Geographic Information Systems. pp. 441–444. ACM, Seattle WA USA (Nov 2020). https://doi.org/10.1145/3397536.3422270, https://dl.acm.org/doi/10.1145/3397536.3422270
  * [31] Li, M., Gao, S., Lu, F., Liu, K., Zhang, H., Tu, W.: Prediction of human activity intensity using the interactions in physical and social spaces through graph convolutional networks. International Journal of Geographical Information Science 35(12), 2489–2516 (Dec 2021). https://doi.org/10.1080/13658816.2021.1912347, https://www.tandfonline.com/doi/full/10.1080/13658816.2021.1912347
  * [32] Liao, D., Liu, W., Zhong, Y., Li, J., Wang, G.: Predicting activity and location with multi-task context aware recurrent neural network. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. pp. 3435–3441 (2018)
  * [33] Liatsikou, M., Papadopoulos, S., Apostolidis, L., Kompatsiaris, I.: A Denoising Hybrid Model for Anomaly Detection in Trajectory Sequences. In: Workshop Proceedings of the EDBT/ICDT 2021 Joint Conference. Nicosia, Cyprus (2021), https://ceur-ws.org/Vol-2841/BMDA_4.pdf
  * [34] Lippert, F., Kranstauber, B., Forré, P.D., van Loon, E.E.: Learning to predict spatiotemporal movement dynamics from weather radar networks. Methods in Ecology and Evolution pp. 2041–210X.14007 (Oct 2022). https://doi.org/10.1111/2041-210X.14007, https://onlinelibrary.wiley.com/doi/10.1111/2041-210X.14007
  * [35] Liu, Q., Wu, S., Wang, L., Tan, T.: Predicting the next location: A recurrent model with spatial and temporal contexts. In: Thirtieth AAAI conference on artificial intelligence (2016)
  * [36] Lu, Y.: Learning to Transfer for Traffic Forecasting via Multi-task Learning (2021). https://doi.org/10.48550/ARXIV.2111.15542, https://arxiv.org/abs/2111.15542, publisher: arXiv Version Number: 1
  * [37] Lv, Y., Duan, Y., Kang, W., Li, Z., Wang, F.Y.: Traffic Flow Prediction With Big Data: A Deep Learning Approach. IEEE Transactions on Intelligent Transportation Systems pp. 1–9 (2014). https://doi.org/10.1109/TITS.2014.2345663, https://ieeexplore.ieee.org/document/6894591
  * [38] Lyu, S., Han, T., Nishiyama, Y., Sezaki, K., Kusakabe, T.: A plug-in memory network for trip purpose classification. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–12. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3560969, https://dl.acm.org/doi/10.1145/3557915.3560969
  * [39] Mai, G., Janowicz, K., Hu, Y., Gao, S., Yan, B., Zhu, R., Cai, L., Lao, N.: A review of location encoding for GeoAI: methods and applications. International Journal of Geographical Information Science 36(4), 639–673 (Apr 2022). https://doi.org/10.1080/13658816.2021.2004602, https://www.tandfonline.com/doi/full/10.1080/13658816.2021.2004602
  * [40] Martin, H., Hong, Y., Wiedemann, N., Bucher, D., Raubal, M.: Trackintel: An open-source Python library for human mobility analysis. Computers, Environment and Urban Systems 101, 101938 (2023). https://doi.org/10.1016/j.compenvurbsys.2023.101938
  * [41] Mehri, S., Alesheikh, A.A., Basiri, A.: A Contextual Hybrid Model for Vessel Movement Prediction. IEEE Access 9, 45600–45613 (2021). https://doi.org/10.1109/ACCESS.2021.3066463, https://ieeexplore.ieee.org/document/9380635/
  * [42] Mokbel, M., Sakr, M., Xiong, L., Züfle, A., Almeida, J., Aref, W., Andrienko, G., Andrienko, N., Cao, Y., Chawla, S., et al.: Towards mobility data science (vision paper). arXiv preprint arXiv:2307.05717 (2023)
  * [43] Musleh, M.: Towards a unified deep model for trajectory analysis. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–2. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3565529, https://dl.acm.org/doi/10.1145/3557915.3565529
  * [44] Nguyen, D., Vadaine, R., Hajduch, G., Garello, R., Fablet, R.: GeoTrackNet —A Maritime Anomaly Detector Using Probabilistic Neural Network Representation of AIS Tracks and A Contrario Detection. IEEE Transactions on Intelligent Transportation Systems 23(6), 5655–5667 (Jun 2022). https://doi.org/10.1109/TITS.2021.3055614, https://ieeexplore.ieee.org/document/9353410/
  * [45] Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., et al.: The prisma 2020 statement: an updated guideline for reporting systematic reviews. International journal of surgery 88, 105906 (2021)
  * [46] Pappalardo, L., Simini, F., Barlacchi, G., Pellungrini, R.: scikit-mobility: a Python library for the analysis, generation and risk assessment of mobility data. arXiv:1907.07062 [physics.soc-ph] (2021), https://arxiv.org/abs/1907.07062
  * [47] Rao, J., Gao, S., Kang, Y., Huang, Q.: LSTM-TrajGAN: A Deep Learning Approach to Trajectory Privacy Protection. In: 11th International Conference on Geographic Information Science (GIScience 2021)-Part I. pp. 12:1–12:17. Leibniz International Proceedings in Informatics, Schloss Dagstuhl-Leibniz-Zentrum für Informatik, Poznan/online (Jun 2020), http://arxiv.org/abs/2006.10521, arXiv:2006.10521 [cs]
  * [48] Rao, J., Gao, S., Li, M., Huang, Q.: A privacy‐preserving framework for location recommendation using decentralized collaborative machine learning. Transactions in GIS 25(3), 1153–1175 (Jun 2021). https://doi.org/10.1111/tgis.12769, https://onlinelibrary.wiley.com/doi/10.1111/tgis.12769
  * [49] Shi, M., Currier, K., Liu, Z., Janowicz, K., Wiedemann, N., Verstegen, J., McKenzie, G., Graser, A., Zhu, R., Mai, G.: Thinking Geographically about AI Sustainability. AGILE: GIScience Series 4, 1–7 (Jun 2023). https://doi.org/10.5194/agile-giss-4-42-2023, https://agile-giss.copernicus.org/articles/4/42/2023/
  * [50] Simini, F., Barlacchi, G., Luca, M., Pappalardo, L.: A Deep Gravity model for mobility flows generation. Nature Communications 12(1), 6576 (Nov 2021). https://doi.org/10.1038/s41467-021-26752-4, https://www.nature.com/articles/s41467-021-26752-4
  * [51] Singh, S.K., Fowdur, J.S., Gawlikowski, J., Medina, D.: Leveraging Graph and Deep Learning Uncertainties to Detect Anomalous Maritime Trajectories. IEEE Transactions on Intelligent Transportation Systems 23(12), 23488–23502 (Dec 2022). https://doi.org/10.1109/TITS.2022.3190834, https://ieeexplore.ieee.org/document/9839418/
  * [52] Stewart, A.J., Robinson, C., Corley, I.A., Ortiz, A., Ferres, J.M.L., Banerjee, A.: TorchGeo: deep learning with geospatial data. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–12. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3560953, https://dl.acm.org/doi/10.1145/3557915.3560953
  * [53] Tenzer, M., Rasheed, Z., Shafique, K., Vasconcelos, N.: Meta-learning over time for destination prediction tasks. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–10. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3560980, https://dl.acm.org/doi/10.1145/3557915.3560980
  * [54] Tritsarolis, A., Chondrodima, E., Tampakis, P., Pikrakis, A.: Online Co-movement Pattern Prediction in Mobility Data. In: Workshop Proceedings of the EDBT/ICDT 2021 Joint Conference. Nicosia, Cyprus (2021), https://ceur-ws.org/Vol-2841/BMDA_15.pdf
  * [55] Wang, C., Liang, Y., Tan, G.: Periodic residual learning for crowd flow forecasting. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–10. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3560947, https://dl.acm.org/doi/10.1145/3557915.3560947
  * [56] Wang, D., Zhang, J., Cao, W., Li, J., Zheng, Y.: When Will You Arrive? Estimating Travel Time Based on Deep Neural Networks. Proceedings of the AAAI Conference on Artificial Intelligence 32(1) (Apr 2018). https://doi.org/10.1609/aaai.v32i1.11877, https://ojs.aaai.org/index.php/AAAI/article/view/11877
  * [57] Wang, L., Ciliberto, M., Gjoreski, H., Lago, P., Murao, K., Okita, T., Roggen, D.: Locomotion and Transportation Mode Recognition from GPS and Radio Signals: Summary of SHL Challenge 2021. In: Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. pp. 412–422. ACM, Virtual USA (Sep 2021). https://doi.org/10.1145/3460418.3479373, https://dl.acm.org/doi/10.1145/3460418.3479373
  * [58] Wang, S., Cao, J., Yu, P.S.: Deep Learning for Spatio-Temporal Data Mining: A Survey. IEEE Transactions on Knowledge and Data Engineering 34(8), 3681–3700 (Aug 2022). https://doi.org/10.1109/TKDE.2020.3025580, https://ieeexplore.ieee.org/document/9204396/
  * [59] Widhalm, P., Leodolter, M., Brändle, N.: Top in the Lab, Flop in the Field?: Evaluation of a Sensor-based Travel Activity Classifier with the SHL Dataset. In: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. pp. 1479–1487. ACM, Singapore Singapore (Oct 2018). https://doi.org/10.1145/3267305.3267514, https://dl.acm.org/doi/10.1145/3267305.3267514
  * [60] Xue, H., Voutharoja, B.P., Salim, F.D.: Leveraging language foundation models for human mobility forecasting. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–9. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3561026, https://dl.acm.org/doi/10.1145/3557915.3561026
  * [61] Yan, B., Janowicz, K., Mai, G., Gao, S.: From ITDL to Place2Vec: Reasoning About Place Type Similarity and Relatedness by Learning Embeddings From Augmented Spatial Contexts. In: Proceedings of the 25th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. pp. 1–10. ACM, Redondo Beach CA USA (Nov 2017). https://doi.org/10.1145/3139958.3140054, https://dl.acm.org/doi/10.1145/3139958.3140054
  * [62] Yang, C., Gidofavi, G.: Classification of regional dominant movement patterns in trajectories with a convolutional neural network. In: Martin Raubal, Shaowen Wang, Mengyu Guo, David Jonietz, Peter Kiefer (eds.) Spatial Big Data and Machine Learning in GIScience. Melbourne, Australia (Aug 2018)
  * [63] Yang, T., Wang, X., Liu, Z.: Ship Type Recognition Based on Ship Navigating Trajectory and Convolutional Neural Network. Journal of Marine Science and Engineering 10(1), 84 (Jan 2022). https://doi.org/10.3390/jmse10010084, https://www.mdpi.com/2077-1312/10/1/84
  * [64] Zhang, J., Zheng, Y., Qi, D.: Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction. Proceedings of the AAAI Conference on Artificial Intelligence 31(1) (Feb 2017). https://doi.org/10.1609/aaai.v31i1.10735, https://ojs.aaai.org/index.php/AAAI/article/view/10735
  * [65] Zhang, L., Zhao, L., Pfoser, D.: Factorized deep generative models for end-to-end trajectory generation with spatiotemporal validity constraints. In: Proceedings of the 30th International Conference on Advances in Geographic Information Systems. pp. 1–12. ACM, Seattle Washington (Nov 2022). https://doi.org/10.1145/3557915.3560994, https://dl.acm.org/doi/10.1145/3557915.3560994
  * [66] Zhang, Y., Li, Y., Zhou, X., Kong, X., Luo, J.: Curb-GAN: Conditional Urban Traffic Estimation through Spatio-Temporal Generative Adversarial Networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 842–852. ACM, Virtual Event CA USA (Aug 2020). https://doi.org/10.1145/3394486.3403127, https://dl.acm.org/doi/10.1145/3394486.3403127
  * [67] Zheng, Y.: Trajectory Data Mining: An Overview. ACM Transactions on Intelligent Systems and Technology 6(3), 1–41 (May 2015). https://doi.org/10.1145/2743025, http://dl.acm.org/citation.cfm?doid=2764959.2743025
  * [68] Zhu, M., Chen, W., Xia, J., Ma, Y., Zhang, Y., Luo, Y., Huang, Z., Liu, L.: Location2vec: A Situation-Aware Representation for Visual Exploration of Urban Locations. IEEE Transactions on Intelligent Transportation Systems 20(10), 3981–3990 (Oct 2019). https://doi.org/10.1109/TITS.2019.2901117, https://ieeexplore.ieee.org/document/8664648/

## Appendix

### Absolute numbers of literature review results

Figure 5: Publication counts by use case from 2018 to 2023. Figure 6:
Publication counts by NN design from 2018 to 2023.

### Glossary

  * •

AE – Autoencoder

  * •

AIS – Automatic Identification System

  * •

APE – Average prediction error

  * •

ADE – Average displacement error

  * •

AUC – Area under the ROC curve

  * •

CDR – Call Detail Records

  * •

CNN – Convolutional Neural Network

  * •

COG – Course over ground

  * •

DNN – Deep Neural Network

  * •

DL – Deep Learning

  * •

DT – Decision Tree

  * •

FNN –- Fully connected dense networks

  * •

FFNN – Feed-Forward Neural Network

  * •

FCD – Floating Car Data

  * •

GAN – Generative Adversarial Network

  * •

GBDT – Gradient-boosted Decision Tree

  * •

GeoAI – Geospatial Artificial Intelligence

  * •

GIS – Geographic Information Science

  * •

GNN – Graph Neural Network

  * •

GPS – Global Positioning System, often used synonymously for all GNSS (incl.
Galileo, GLONASS, and Beidou)

  * •

HMM – Hidden Markov Models

  * •

KNN – K-Nearest Neighbours

  * •

LSTM – Long Short-Term Memory

  * •

MAE – Mean absolute error

  * •

MAPE – Mean absolute percentage error

  * •

ML – Machine Learning

  * •

MLP – Multilayer Perceptron

  * •

NLP – Natural Language Processing

  * •

OBU – On-board Unit

  * •

OD – Origin-Destination

  * •

POI – Point of Interest

  * •

PRME – Personalized Ranking Metric Embedding

  * •

RF – Random Forest

  * •

RNN – Recurrent Neural Network

  * •

SAE – Stacked Autoencoder

  * •

SAN – Self-Attention Network

  * •

SOG – Speed over ground

  * •

SVM – Support Vector Machine

  * •

VAE – Variational AutoEncoder

  * •

XGBoost – Extreme Gradient Boost