XX XX XXXXX 1234567 TAES.2024.Doi Number

Senior Member, IEEE

This work is supported by the Korea Agency for Infrastructure Technology
Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and
Transport (Grant 22DATM-C163373-02). (Corresponding author: Han-Lim Choi)

Authors’ addresses: Thaweerath Phisannupawong, Joshua J. Damanik, and Han-Lim
Choi are with the Department of Aerospace Engineering, Korea Advanced
Institute of Science and Technology, Daejeon, 34141, Republic of Korea, Email:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
The aircraft trajectory classification datasets used in this paper are
available at huggingface.co/datasets/petchthwr/ATFMTraj.
The source code is publicly available at github.com/petchthwr/ATSCC.

# Aircraft Trajectory Segmentation-based Contrastive Coding: A Framework for
Self-supervised Trajectory Representation

THAWEERATH PHISANNUPAWONG  JOSHUA J. DAMANIK  HAN-LIM CHOI  Korea Advanced
Institute of Science and Technology
Daejeon, 34141, Republic of Korea

(2024)

###### Abstract

Air traffic trajectory recognition has gained significant interest within the
air traffic management community, particularly for fundamental tasks such as
classification and clustering. This paper introduces Aircraft Trajectory
Segmentation-based Contrastive Coding (ATSCC), a novel self-supervised time
series representation learning framework designed to capture semantic
information in air traffic trajectory data. The framework leverages the
segmentable characteristic of trajectories and ensures consistency within the
self-assigned segments. Intensive experiments were conducted on datasets from
three different airports, totaling four datasets, comparing the learned
representation’s performance of downstream classification and clustering with
other state-of-the-art representation learning techniques. The results show
that ATSCC outperforms these methods by aligning with the labels defined by
aeronautical procedures. ATSCC is adaptable to various airport configurations
and scalable to incomplete trajectories. This research has expanded upon
existing capabilities, achieving these improvements independently without
predefined inputs such as airport configurations, maneuvering procedures, or
labeled data.

Air Traffic Management, Contrastive Learning, Representation Learning, Time
Series, Trajectory Clustering

##  1 INTRODUCTION

Air transportation plays a crucial role in the world economy; as economic
growth increases, so does air traffic. Operational concepts have been
developed to accommodate this continuous growth in air traffic, for the
current air traffic management (ATM) system and procedures will not be able to
maintain the level of safety and efficiency in handling the increasing
traffic, especially for airports with dense traffic flow and complex
maneuvering patterns [1]. Therefore, airspace modernization has become a
challenging aim for the ATM society and the regulatory authorities. The ATM
concept is expected to shift from a clearance-based to a trajectory-based
operation (TBO), which allows more flexible clearance and efficiency in
utilizing airspace for high-density traffic. Although the trajectory is human-
interpretable, managing higher density, more flexible traffic can be
overwhelming for human operators; therefore, intelligence management and
recognition systems have been developed to assist the air traffic controller
in characterizing flight trajectories, estimating the airport capacity, and
decision-making.

Nowadays, a vast amount of aircraft trajectory data, including Automatic
Dependent Surveillance-Broadcast (ADS-B) recording, is publicly available.
This availability holds significant potential for trajectory recognition
research. Classification and clustering are frequently explored in literature
as fundamental recognition tasks. For instance, classification assists air
traffic controllers in organizing trajectories for monitoring and capacity
estimation. Clustering has continuously attracted increased attention from the
community, mainly because trajectory datasets are usually found unlabeled.
However, the trajectories as time series are typically rich in information,
complex, and highly dimensional. Many algorithms designed for these downstream
tasks are vulnerable to the high dimensionality of data and usually
necessitate feature extraction. Transforming them into a more generalizable
data representation is known to be an effective method for enhancing
downstream recognition task performance.

Self-supervised contrastive representation learning has shown effectiveness in
enhancing recognition tasks in many applications, namely vision, natural
language processing, and time series. Although this approach has been
extensively applied to real-world time series data, including medical signals,
electromagnetics, sound waves, and more, the exploration of moving object
trajectories still offers considerable opportunity to be explored. This paper
introduces Aircraft Trajectory Segmentation-based Contrastive Coding (ATSCC),
a self-supervised contrastive representation learning framework designed for
aircraft trajectory data motivated by the operational contextualization of
ATM. An assumption has been made that the states within a trajectory segment
share identical contexts. Consequently, ATSCC put together the representation
within the same segment while distancing them from others, resulting in a
representation that reflects the comprehension of patterns. The results show
that our method outperforms existing approaches in self-supervised time series
representation learning for trajectory classification and clustering tasks and
resolves various utilization issues. This paper’s contributions can be listed
as follows:

  * •

To our knowledge, we proposed, for the first time, a novel self-supervised
contrastive time series representation learning framework for multivariate
aerospace trajectory.

  * •

We propose an effective concept of similarity within trajectory data using
segmentation from the iterative Ramer-Douglas-Peucker (RDP) algorithm. We have
integrated the capabilities of a causal language model to generate a
representation where each timestep collects all preceding information, making
it suitable for real-time monitoring and tracing incomplete and variable-
length trajectories.

  * •

We introduced labeled trajectory datasets essential for assessing
classification accuracy and evaluating clustering results’ fidelity to the
labels, an often overlooked aspect in prior works.

  * •

ATSCC has proven exceptionally suitable for aircraft trajectory data,
outperforming state-of-the-art baselines. The model has overcome limitations
associated with locational interpretation by allowing analysis with the
semantic representation.

##  2 RELATED WORKS

###  2.1 Time Series Representation Learning

Several studies have proposed models that generate time series representation
vectors by leveraging the similarity among samples. Random Warping Series
(RWS) [2] demonstrated a kernel method based on Dynamic Time Warping (DTW),
capable of generating a time series representation using random features
approximation. SPIRAL [3] proposed a method that converts a set of time series
into vectors by solving non-convex and nom-deterministic polynomial-time hard
optimization, preserving the pairwise similarity of instances.

Autoencoders are the most widely applied representation learning for the
aerospace domain. An encoder compresses the data sample into a representation
vector, which is then used by the decoder for reconstruction. The model is
trained with a reconstruction loss formulated using arbitrary differentiable
distance functions and can be constructed with various architectures such as
multi-layer perceptrons (MLP) [4, 5], convolutional neural networks (CNN) [6,
7], recurrent neural networks (RNN) [8], long-short term memory (LSTM) [9,
10], dilated causal convolutional networks [11], as implemented by [12, 13],
or transformers [14] as in [15].

Contrastive learning optimizes the encoded representation in the embedding
space, ensuring the positive pairs or groups are similar in the embedding
space while being distinct from the negative ones. The main idea of
contrastive representation learning is to develop an effective definition of
positive and negative samples, architecture, and training strategy. T-loss
[16] employs a triplet loss for training, where a positive sample is obtained
from a sub-series within the random reference area of an instance, and the
negatives are all other sub-series. TNC [17] leverages the local smoothness of
the time series to model a positive temporal neighborhood, while the negatives
are sub-series that are temporally distant. TNC maximizes similarity
likelihood within the neighborhood while minimizing that of the negatives. TS-
TCC [18] maximizes the agreement between two augmented views of an instance
while setting them apart from other samples. TS2Vec [19] employs random
cropping and binomial masking for data augmentation. The encoder is trained by
applying the hierarchical contrastive loss on the embedding of two cropped
time series, both temporally and instance-wise, at every temporal max pooling
level. InfoTS [20] incorporates meta-learning techniques to identify the most
suitable time series augmentation method. The encoder is trained using InfoNCE
loss, contrasting temporally and instance-wise, while the meta-learner learns
by balancing fidelity and variety loss, similar to [21]. Contrastive learning
allows us to define semantic similarities and rules for elements in data,
aligning data representation more closely with contextual understanding.

###  2.2 Trajectory Classification

One of the fundamental challenges in trajectory recognition is runway
classification [22]. The study demonstrated the classification of landing
trajectories using a labeled dataset, comparing various classifiers. However,
recent works on traffic patterns are relatively fewer than those on
clustering, as public trajectory data are often unlabeled. Efforts for data
labeling have been demonstrated in several works, where clustering results are
analyzed and used as labels to train classifiers such as the Random Forest
algorithm [23, 24, 25], MLP [26], and LSTM [27]. Directly enforcing a
classification loss such as cross-entropy tends to result in representations
that are close to one-hot vectors. Moreover, these works emphasize the
necessity of labeled datasets.

###  2.3 Trajectory Clustering

In the simplest way, trajectories can be resampled to a fixed length and then
clustered using Euclidean distance, as demonstrated in [23, 24, 25, 28], or
using weighted Euclidean distance, as shown in [29]. For variable-length
datasets, clustering can be performed using a matrix of time series pairwise
distances. Examples include DTW [30] combined with agglomerative clustering
[31] as in [27, 32], route similarity [33] with progressive clustering [34]
using OPTICS [35] as demonstrated in [36], and Symmetrized Segment-Path
Distance (SSPD) [37] with HDBSCAN [38] on simplified trajectories using the
RDP algorithm [39, 40] as shown in [28]. Aircraft flight paths can also be a
feature for clustering. For example, DTW combined with HDBSCAN [38] was used
on track angle sequences [41], while in [42], track sequences were modeled
using Von Mises distributions and clustered with K-medoids [43] using
Bhattacharyya distance [44].

The partitioning-based method can be traced back to TRACLUS [45], which
identifies points where states change rapidly for partitioning, uses DBSCAN
[46] with line segment distance to cluster line segments, and then combines
these segments to form clusters. Similarly, the methods in [47, 48] identify
turning points based on heading changes and group those points using K-means
[49] and DBSCAN. These approaches then form the clusters by aggregating the
sequences of waypoints from a dependency tree. These methods often rely on
spatial interpretations and often result in a large number of clusters. The
hidden Markov model in [50] converts trajectories into string sequences of
turning actions, which are then clustered using K-medoids with the edit
distance.

Clustering on raw samples can be inefficient with high-dimensional samples.
Many works have demonstrated clustering on the dimensionality-reduced samples,
such as PCA [47, 51, 7], and t-SNE [6, 52]; however, these methods are
typically used for visualization. Moreover, they focus on preserving the
dataset’s characteristics rather than those of data instances. For
representation-based methods, autoencoders are commonly used, and the trained
encoder’s output serves as the trajectory representation; for example, the MLP
with Gaussian Mixture Model (GMM) [53] in [4] or with K-Means in [5], and the
CNN for visual representation with GMM as in [7]. Deep clustering is typically
an extension of the representation-based methods, often utilizing either a
pretrained feature extractor or learning the representation concurrently with
the training process. A popular framework for trajectory deep clustering is
Deep Embedded Clustering (DEC) [54], implemented with various architecture
choices such as CNN in [6], bidirectional LSTM in [9], and time and attribute
LSTM (TA-LSTM), an LSTM with a time regulator gate for irregular time
intervals, as in [10]. For both clustering on representations and deep
clustering, having a high-fidelity representation can enhance performance by
effectively converging into distinct classes.

##  3 METHODOLOGY

Figure 1: Diagram of the Proposed Framework for Aircraft Trajectory
Segmentation-based Contrastive Coding (ATSCC): Includes Segment ID Assignments
Using the Iterative RDP Algorithm, Geometric Feature Extraction, Training
Pipeline with a Causal Transformer Encoder, ID Remapping, and Training with
Soft Nearest Neighbor Loss.

###  3.1 Problem Definition

For $N$ instances of time series in a variable length trajectory dataset, a
multivariate time series, denoted as
$\mathcal{X}_{i}=\\{x_{i,1},x_{i,2},x_{i,3},\dots,x_{i,T_{i}}\\}$, has a
dimension of $T_{i}\times F$, where $T_{i}$ is the sequence length of $i^{th}$
sample and $F$ is the number of features which varies based on the selected
geometric features. In detail, $x_{i,t}$ initially consists of position
$\\{x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z}\\}$ in the East-North-Up (ENU)
coordinates. In this paper, it can be extended up to 9 features, incorporating
directional vectors $\\{x_{i,t}^{u_{x}},x_{i,t}^{u_{y}},x_{i,t}^{u_{z}}\\}$
and polar components
$\\{x_{i,t}^{r},x_{i,t}^{\sin\theta},x_{i,t}^{\cos\theta}\\}$. It should be
noted that the sequence length $T_{i}$ mentioned is the explicit time, and
$T_{i}$ varies among instances as we deliberately avoided interpolating the
trajectory data to a uniform length. The encoder $f_{w}(\mathcal{X}_{i})$,
with learnable parameter $w$, is trained to best describe $\mathcal{X}_{i}$
with the representation
$\mathcal{Z}_{i}=\\{z_{i,1},z_{i,2},z_{i,3},\dots,z_{i,T_{i}}\\}$, that has a
dimension of $T_{i}\times K$. Each $z_{i,t}\in\mathbb{R}^{K}$ simmarizes
$\mathcal{X}^{\prime}_{i,t}=\\{x_{i,k}:k\leq t\\}$ where
$\mathcal{X}^{\prime}_{i,t}\subseteq\mathcal{X}_{i}$. The training of the
encoder $f_{w}(\mathcal{X}_{i})$ involves the assignment of the local segment
ID denoted as
$\Upsilon_{i}=\\{\upsilon_{i,1},\upsilon_{i,2},\upsilon_{i,3},\dots,\upsilon_{i,T_{i}}\\}$
on each $\mathcal{X}_{i}$ as the notation for the loss function.

###  3.2 Segment ID Assignments

Many air traffic control (ATC) tasks exhibit segmenting behavior. For example,
aircraft vectoring instructions remain unchanged until the aircraft reaches a
designated turning position or is given a new instruction. In waypoint
navigation, a sequence of waypoints instructs the aircraft operator to comply,
forming the trajectory. In TBO, planned trajectories are divided by detailed
checkpoints. Inspired by these tasks, we infer the semantic context within
segments partitioned by significant positions.

The ATSCC framework (Fig 1), inspired by these ATC tasks, utilizes the
geometric properties of trajectories instead of data augmentation, which has
been debated for its infeasibility across diverse datasets due to their unique
temporal dynamics [20]. Specifically, trajectories can be segmented into
significant points [47, 48], which are assumed to be close enough to the
instructed waypoints, thus forming segments within the trajectory. Using this
characteristic, the ATSCC framework aims to group together the representations
at timestamps within a segment while distancing them from those at timestamps
outside that segment and from those within other instances.

The RDP algorithm [39, 40] reduces points in a curve while preserving its
essential shape and geometric integrity, resulting in a simplified curve with
timestamps of significant points that segment the curve. We used an open-
source Python library for the iterative RDP algorithm [55], as described in
Algorithm 1. It starts by marking the beginning and end points of the curve as
significant. Then, for each segment between significant points, the algorithm
identifies points that exceed a perpendicular distance threshold, $\epsilon$,
from the line segment and marks these points as significant. This process
repeats until no further points exceed the threshold. The perpendicular
distance from a point $x_{i,k}$ to a line segment defined by points $x_{i,s}$
and $x_{i,e}$, is denoted as $d(x_{i,k},x_{i,s},x_{i,e})$ and can be
calculated as follows:

$d(x_{i,k},x_{i,s},x_{i,e})=\begin{cases}\|x_{i,k}-x_{i,s}\|&\text{if
}x_{i,s}=x_{i,e},\\\
\frac{\left|\|(x_{i,e}-x_{i,s})\times(x_{i,s}-x_{i,k})\|\right|}{\|x_{i,e}-x_{i,s}\|}&\text{otherwise}.\end{cases}$
(1)

Here, $s$, $k$, and $e$ denote the indices of the start, considering, and end
points of the trajectory segment, respectively. $x_{i,s}$, $x_{i,k}$ and
$x_{i,e}$ refer to thier corresponding points within $\mathcal{X}_{i}$. Note
that for this algorithm, $x_{i,t}=\\{x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z}\\}$.
The iterative RDP algorithm produces the mask
$M_{i}=\\{m_{i,1},m_{i,2},m_{i,3},\dots,m_{i,T}\\}$, where each
$m_{i,t}\in\\{0,1\\}$. An instance’s segment IDs
$\Upsilon_{i}=\\{\upsilon_{i,1},\upsilon_{i,2},\upsilon_{i,3},\ldots,\upsilon_{i,T}\\}$
is obtained from the cumulative sum by each segment ID
$\upsilon_{i,t}=\sum_{k=1}^{t}m_{i,k}$ and $\upsilon_{i,T}=\upsilon_{i,T-1}$
where $\upsilon_{i,t}\in\mathbb{Z}$ is marked corresponded to the temporal
indices, $t$ on $x_{i,t}$.

The process of segment ID assignment can be described as a function where
$\Upsilon_{i}=RDP(\mathcal{X}_{i},\epsilon)$, indicating that $\Upsilon_{i}$
is the result of performing segment ID assignment via the RDP algorithm with
an allowable perpendicular line segment distance of $\epsilon$ to the
trajectory $\mathcal{X}_{i}$. $\epsilon$ serves as a measure of the allowable
error between $\mathcal{X}_{i}$ and its simplified version. Thus, changing in
$\epsilon$ changes how the operational context boundary is defined, so
$\epsilon$ is taken as a hyperparameter of the ATSCC. All $\Upsilon_{i}$ for
$N$ instances of $\mathcal{X}_{i}$ are precomputed for computational
efficiency. They are padded with NaN to $T_{\text{max}}=\max_{i\in 1\dots
N}(T_{i})$, in the same manner as the unequal length trajectory dataset, and
stored within the PyTorch Dataloader module for later use in the training
pipeline.

Algorithm 1 Local Label Marking via Iterative Ramer-Douglas-Peucker Algorithm

1:procedure RDP($\mathcal{X}_{i}$, $\epsilon$)

2: // Step 1: Iterative RDP Algorithm [55]

3: $\mathcal{S}\leftarrow\\{(1,T_{i})\\}$

4: $M_{i}\leftarrow\\{1:k=1,\dots,T_{i}\\}$

5: $n_{\mathcal{S}}=|\mathcal{S}|$

6: while $n_{\mathcal{S}}\not=0$ do

7: $(s,e)\leftarrow\mathcal{S}[n_{\mathcal{S}}]$

8: $\mathcal{S}\leftarrow\mathcal{S}-\\{(s,e)\\}$

9: $d_{max}\leftarrow 0.0$

10: $t\leftarrow s$

11: for $k\leftarrow t+1$ to  $t_{e}-1$ do

12: if $m_{i,k}$ then

13: $d\leftarrow d(x_{i,k},x_{i,s},x_{i,e})$

14: if $d>d_{max}$ then

15: $t\leftarrow k$

16: $d_{max}\leftarrow d$

17: end if

18: end if

19: end for

20: if $d_{max}>\epsilon$ then

21: $\mathcal{S}\leftarrow\mathcal{S}+\\{(s,t),(t,e)\\}$

22: else

23: for $k\leftarrow s+1$ to  $e-1$ do

24: $m_{i,k}\leftarrow 0$

25: end for

26: end if

27: $n_{\mathcal{S}}=|\mathcal{S}|$

28: end while

29: // Step 2: Local segment ID assignment

30: $\Upsilon_{i}\leftarrow\\{\\}$

31: for $t$ in $T$ do

32: if $t\not=T$ then

33: $\upsilon_{i,t}=\sum_{k=1}^{t}m_{i,k}$

34: else

35: $\upsilon_{i,t}=\sum_{k=1}^{t-1}m_{i,k}$

36: end if

37: $\Upsilon_{i}\leftarrow\Upsilon_{i}+\\{\upsilon_{i,t}\\}$

38: end for

39: return $\Upsilon_{i}$

40:end procedure

###  3.3 Geometric Feature Extraction

The trajectory data describe the sequence of aircraft’s positions in ENU
coordinates as
$\mathcal{X}_{i}\in\\{x_{i,1},x_{i,2},x_{i,3},\dots,x_{i,T}\\}$; each
$x_{i,t}$ initially consists of $x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z}$. To
extend the underlying information in trajectory states, this feature
extraction calculates additional geometric features prior to the training
stage. Since contrastive representation learning can handle features of
different scales, these features enhance information without typical scaling
issues found in feature-based methods or autoencoders. Empirically, this
inclusion improves performance, as discussed in the ablation study.

Aircraft path angle is a useful feature for representing trajectory shape [5,
41, 42]. Moreover, most ATM instructions involve path angles, such as
headings, glide paths, or directing aircraft between waypoints and specific
altitudes. In other words, ATC uses not only positions but also the direction
of the aircraft. This paper uses the directional unit vector to represent
aircraft paths, avoiding trigonometric singularities of angles. The aircraft’s
path vector at each timestamp is given by

$\\{x_{i,t}^{u_{x}},x_{i,t}^{u_{y}},x_{i,t}^{u_{z}}\\}=\frac{x_{i,t+1}-x_{i,t}}{\left\|x_{i,t+1}-x_{i,t}\right\|},$
(2)

where $\left\|x_{i,t+1}-x_{i,t}\right\|$ denotes the Euclidean norm of the
vector difference, and $x_{i,t}=\\{x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z}\\}$.
For the final time step $T_{i}$, it is assumed that without further updates,
the aircraft maintains the last computed path, such that

$\\{x_{i,T_{i}}^{u_{x}},x_{i,T_{i}}^{u_{y}},x_{i,T_{i}}^{u_{z}}\\}=\\{x_{i,T_{i}-1}^{u_{x}},x_{i,T_{i}-1}^{u_{y}},x_{i,T_{i}-1}^{u_{z}}\\}.$
(3)

Interpreting positions in polar coordinates centered at reference waypoints is
crucial for navigational aids such as VOR/DME (Very High-Frequency
Omnidirectional Range with Distance Measuring Equipment) stations. The
sensitivity of these states increases as the aircraft maneuvers near the
reference point, in this case, the airport. This behavior establishes rapid
changes in features, providing ease of recognition. The polar components
feature at each timestamp can be calculated with,

$\\{x_{i,t}^{r},x_{i,t}^{\sin\theta},x_{i,t}^{\cos\theta}\\}=\left\\{\begin{matrix}\sqrt{\left(x_{i,t}^{x}\right)^{2}+\left(x_{i,t}^{y}\right)^{2}}\\\
\sin\left(\arctan 2(x_{i,t}^{y},x_{i,t}^{x})\right)\\\ \cos\left(\arctan
2(x_{i,t}^{y},x_{i,t}^{x})\right)\end{matrix}\right\\}^{T}.$ (4)

We use the sine and cosine of the angle from the $\arctan 2$ function to avoid
singularities and discontinuities at 0 and 360 degrees. This method ensures a
smooth transition across quadrant boundaries. The features are combined to
form
$x_{i,t}=\\{x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z},x_{i,t}^{u_{x}},x_{i,t}^{u_{y}},x_{i,t}^{u_{z}},x_{i,t}^{r},x_{i,t}^{\sin\theta},x_{i,t}^{\cos\theta}\\}$.
These are then stored in the PyTorch DataLoader module along with the segment
IDs $\Upsilon_{i}$.

###  3.4 Encoder Architecture

The encoder $f_{w}$ consists of a random masking module, a Causal Transformer
backbone, and input/output projection layers, as shown in Fig 1. The encoder
first applies binomial timestamp masking to the inputs during training,
similar to [19], but our encoder employs masking before the input projection
to mimic missing states caused by irregular transmissions, providing
regularization like dropout. This random mask is later merged with the source
padding mask and causal mask for attention to ignore the masked tokens.

The input linear projection expands each state $x_{i,t}$ from dimension $K$ to
the token dimension $E$ of the encoder backbone. The architecture of the
Causal Transformer backbone, illustrated in Fig 2, is similar to the designs
of GPT-2 [56] and GPT-3 [57]. For each batch of inputs
$\mathcal{X}=\\{\mathcal{X}_{1},\mathcal{X}_{2},\mathcal{X}_{3},\dots,\mathcal{X}_{B}\\}$,
the encoder handle unequal-length sequences by applying key padding masks on
inputs, outputs, and attention matrices. The causal attention mechanism
restricts attention only to previous tokens, ensuring that the output for any
position reflects the set encoding of all preceding tokens. Thus, the model’s
output at any given time, $z_{i,t}$, is the summary of
$\mathcal{X}^{\prime}_{i,t}=\\{x_{i,k}:k\leq t\\}$. Each encoder state, having
a dimension of $E$, is then linearly projected into $z_{i,t}$ with a
representation size of $K$.

We follow the NoPos approach [58], which shows that the transformer causal
language model (CLM) can perceive positional information without explicit
positional encoding. Thus, when applied to time series data, the CLM
effectively captures their sequential behavior. We apply L2 normalization
after each linear projection layer as suggested in [59] that it can improve
performance by making the embeddings linearly separable. Additionally, L2
normalization on tokens before encoding slightly enhances the learned
representation’s effectiveness for downstream tasks.

Figure 2: Architecture of Causal Transformer Encoder

###  3.5 Training Pipeline

As the ATSCC framework defines contextual similarity using the segment IDs
assigned by the RDP algorithm, this contrastive representation learning
becomes simple yet effective. First, a batch time series data,
$\mathcal{X}=\\{\mathcal{X}_{1},\mathcal{X}_{2},\mathcal{X}_{3},\dots,\mathcal{X}_{B}\\}$,
is encoded through the encoder $f_{w}$, resulting in a batch of representation
$\mathcal{Z}=\\{\mathcal{Z}_{1},\mathcal{Z}_{2},\mathcal{Z}_{3},\dots,\mathcal{Z}_{B}\\}$.
As $z_{i,t}$ represents $\mathcal{X}^{\prime}_{i,t}=\\{x_{i,k}:k\leq t\\}$,
the contrasting applied to each element of an encoded batch $\mathcal{Z}$ is
equivalent to both temporal and instance-wise contrasting of a set the
incomplete instances $\\{\mathcal{X}^{\prime}_{i,t}\\}$ for
$t\in\\{1,2,3,\dots,T_{i}\\}$ and $i\in\\{1,2,3,\dots,B\\}$. The contrasting
on an encoded batch $\mathcal{Z}$ is dictated by the coresponding batch of
segment IDs,
$\Upsilon=\\{\Upsilon_{1},\Upsilon_{2},\Upsilon_{3},\dots,\Upsilon_{B}\\}$.
The framework define positive samples as $z_{i,t}$ within the same segment
that shares the same $\upsilon_{i,t}$, while treating other $z_{i,t}$ in
different segments and $z_{j,t}$ from other instances
$j\in\\{1,2,3,\dots,B\\}$ with $j\neq i$ as negative samples. The procedure of
loss calculation is described in Algorithm 2.

Algorithm 2 Calculation of Soft Nearest Neighbor Loss

1:procedure SNNL($\mathcal{Z},\Upsilon,\tau$)

2: $\mathcal{Z}_{f}\leftarrow\\{\\}$

3: $\Upsilon_{f}\leftarrow\\{\\}$

4: $\upsilon_{\text{current}}\leftarrow 1$

5: for $i\leftarrow 1\text{ to }B$ do

6: // Flatten $\mathcal{Z}$

7:
$K\leftarrow\\{t|\upsilon_{i,t}\not=\text{NaN},\upsilon_{i,t}\in\Upsilon_{i}\\}$

8: $\mathcal{Z}_{f}\leftarrow\mathcal{Z}_{f}+\\{z_{i,t}|t\in
K,z_{i,t}\in\mathcal{Z}_{i}\\}$

9: // Segment ID remapping

10: $g_{m}:\\{\\}\rightarrow\\{\\}$

11: $\Upsilon_{i,\text{new}}\leftarrow\\{\\}$

12: $\Upsilon_{i}\leftarrow\\{\upsilon_{i,t}|t\in
K,\upsilon_{i,t}\in\Upsilon_{i}\\}$

13: for $\upsilon_{i,t}$ in $\Upsilon_{i}$ do

14: if $g_{m}(\upsilon_{i,t})\text{ does not exist}$ then

15: $g_{m}(\upsilon_{i,t})\leftarrow\upsilon_{\text{current}}$

16: $\upsilon_{\text{current}}\leftarrow\upsilon_{\text{current}}+1$

17: end if

18:
$\Upsilon_{i,\text{new}}\leftarrow\Upsilon_{i,\text{new}}+\\{g_{m}(\upsilon_{i,t})\\}$

19: end for

20: $\Upsilon_{f}\leftarrow\Upsilon_{f}+\Upsilon_{i,\text{new}}$

21: end for

22: // Calculate soft-nearest neighbor loss

23: $\text{Loss}=\mathcal{L}_{ssn}(\mathcal{Z}_{f},\Upsilon_{f},\tau)$;
Equation (6)

24: return Loss

25:end procedure

The batch of embedding $\mathcal{Z}$ is flattened into 2-dimensional
$\mathcal{Z}_{f}$ having the number of features of $F$ and the length of
$\sum_{i=1}^{B}T_{i}$, for $T_{i}$ is not equal across instances. During the
ID assigning, each sequence of segment IDs $\Upsilon_{i}$ is marked locally,
starting from 1. The ID remapping on the batch of segment IDs $\Upsilon$ is
performed using the updatable mapping function $g_{m}$. This procedure ensures
that when local segment IDs are flattened, the uniqueness of IDs across
segments is guaranteed, preventing segments from different samples from being
mistakenly identified as positives. The length of the flattened segment IDs,
$\Upsilon_{i}$ is $\sum_{i=1}^{B}T_{i}$, where each ID $\upsilon_{i,t}$
matches the corresponding representation vector $z_{i,t}$. Given the presence
of multiple positive samples, the ATSCC framework employs a modified soft-
nearest neighbor loss [60], which utilizes scaled vector multiplication
instead of distances, as each $z_{i,t}$ has been L2 normalized. The rearranged
soft-nearest neighbor loss is given by equation (5).

$\mathcal{L}_{snn}(\mathcal{Z}_{f},\Upsilon_{f},\tau)=\\\
-\mathop{{}\mathbb{E}}_{\begin{subarray}{c}z\sim\mathcal{Z}_{f}\\\
\upsilon\sim\Upsilon_{f}\\\
i\sim|\mathcal{Z}_{f}|\end{subarray}}\left(\log\sum_{\begin{subarray}{c}j=1\\\
j\neq i\\\
\upsilon_{i}=\upsilon_{j}\end{subarray}}^{|\mathcal{Z}_{f}|}e^{\left(\frac{z_{i}^{T}z_{j}}{\tau}\right)}-\log\sum_{\begin{subarray}{c}k=1\\\
k\neq
i\end{subarray}}^{|\mathcal{Z}_{f}|}e^{\left(\frac{z_{i}^{T}z_{k}}{\tau}\right)}\right).$
(5)

A single model parameter update step in ATSCC processes a large number of
$z_{i,t}$ in one batch; for example, a batch size of 16 trajectories with an
average length of 500 steps results in 8,000 representation vectors in
$\mathcal{Z}_{f}$. We assume that the inclusion of positive contrasting terms
in the sum of negative terms has a minimal effect. Moreover, omitting positive
contrasts in the negative terms, as in equation (6), leads to empirically
better performance.

$\mathcal{L}_{snn}(\mathcal{Z}_{f},\Upsilon_{f},\tau)=\\\
-\mathop{{}\mathbb{E}}_{\begin{subarray}{c}z\sim\mathcal{Z}_{f}\\\
\upsilon\sim\Upsilon_{f}\\\
i\sim|\mathcal{Z}_{f}|\end{subarray}}\left(\log\sum_{\begin{subarray}{c}j=1\\\
j\neq i\\\
\upsilon_{i}=\upsilon_{j}\end{subarray}}^{|\mathcal{Z}_{f}|}e^{\left(\frac{z_{i}^{T}z_{j}}{\tau}\right)}-\log\sum_{\begin{subarray}{c}k=1\\\
k\neq i\\\
\upsilon_{i}\neq\upsilon_{k}\end{subarray}}^{|\mathcal{Z}_{f}|}e^{\left(\frac{z_{i}^{T}z_{k}}{\tau}\right)}\right)$
(6)

The loss function maximizes agreement between the representation vectors in
the assigned segments while explicitly differentiating them from those in
other segments and other instances in a batch. This modified form ensures that
enforcing the soft-nearest neighbor loss supports our fundamental motivation;
that is, the collective trajectory states within the same operational context
area should result in a similar representation clearly distinct from those
outside.

Figure 3: Visualization of Trajectory Datasets and Histogram of Class
Distribution (by order): Incheon Airport Arrival and Departure Trajectories,
Stockholm Arlanda Airport Arrival Trajectories, and Zurich Airport Arrival
Trajectories, with Corresponding Histograms of Class Distribution.

##  4 RESULTS AND DISCUSSION

In this section, we first elaborate on the dataset preparation used for
training and evaluation. We outline the experiment setting for evaluating the
learned trajectory representation on fundamental air traffic management
downstream tasks, including classification and clustering. We compare the
performance of our learned representation with state-of-the-art representation
learning methods and autoencoders to demonstrate ATSCC’s suitability for air
traffic data. The feasibility of model components was analyzed in the ablation
study. Additionally, we provide a detailed visual explanation of the learned
representation. We further demonstrate the utilization of ATSCC for trajectory
categorization, addressing a common problem in the field.

###  4.1 Datasets

####  4.1.1 Dataset Sources

This study used trajectory data from three different airports, totaling four
datasets. We aim to demonstrate the broad applicability of the ATSCC framework
across various airport configurations. This paper uses data from the authors
and other researchers to ensure the framework’s applicability, reliability of
datasets, and experiment validity. The data sources are:

  * •

Incheon International Airport (Denoted as ICAO code: RKSI): The ADS-B data
were sourced from the Opensky database [61]. The data for flights from 2018 to
2023 were queried using the flight identification numbers from the Airportal
website, [62]. We downsampled the trajectory data from the southbound and
southeastbound flights to improve balance. The datasets for arrivals and
departures are denoted as RKSIa and RKSId, respectively.

  * •

Stockholm Arlanda Airport (Denoted as ICAO code: ESSA): The data is from the
Swedish Civil Air Traffic Control (SCAT) dataset [63], which includes
surveillance data, weather, flight plans, and airspace data. We focused on
surveillance data for positional states and flight plans to assist our manual
labeling. We only use arrival data for this airport, as the departure data was
found incomplete.

  * •

Zurich Airport (Denoted as ICAO code: LSZH): The data was sourced from the
[64], where the focus was on analyzing Zurich Airport flight arrivals.

####  4.1.2 Dataset Preprocessing

The dataset preprocessing involves data cleaning, transformation, resampling,
smoothing, and scaling. We first omitted the preprocessing for incomplete
trajectories. Then, we extracted positional states from the surveillance data,
including latitude, longitude, and barometric altitude. Barometric altitude
was chosen for its lower noise levels and greater reliability compared to GPS
vertical accuracy [65] and because it is commonly used for traffic control.
The positional features were transformed into Cartesian vectors on the ENU
(East-North-Up) coordinate centered at the airport. Then, trajectories were
bounded by a radius, $r_{\text{max}}$, determined by the farthest waypoints in
the area chart of the Aeronautical Information Publication (AIP) with 120 km
for Incheon airport, 100 km for Stockholm Arlanda airport, and 40 nautical
miles for Zurich airport [64]. The trajectories were resampled to 1-second
intervals without interpolating into a fixed number of timestamps. Then, the
outlier states were removed, and the Savizky-Golay filter [66] was applied to
smooth the trajectories. Each trajectory was scaled to the range $[-1,1]$ by
dividing by $r_{max}$, such that
$\mathcal{X}_{i}\leftarrow\mathcal{X}_{i}/r_{max}$. A preprocessed instance
$\mathcal{X}_{i}$ has the states
$x_{i,t}=\\{x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z}\\}$. The trajectories are
padded with NaN to $T_{\text{max}}=\max_{i\in 1\dots N}(T_{i})$. The training
and testing sets were split with a 50% proportion, as is common with many
datasets; moreover, a significant amount of test data is needed to evaluate
clusterability.

####  4.1.3 Dataset Labeling

The lack of labeled datasets restricts the advancement in developing
classification models, which require training with true labels as done by
[22]. The studies cited in [23, 24, 25, 27, 26] utilized clustering to
generate labels; however, this method is commonly practiced in the artificial
intelligence field as pseudo-labeling. Moreover, many clustering studies omit
the labels in their evaluations, focusing on the proximity of instances within
clusters [4, 5, 9, 52, 10]; nevertheless, the primary concern remains the
fidelity of class assignments assessed using the true label. For these
reasons, it is essential to have data labeled by humans, as in [67, 68], to
align with humans’ semantic understanding of data.

The datasets were manually labeled using information published in the AIPs.
Labeling arrival trajectories at Incheon Airport and Zurich Airport followed
the same procedures. For arrival corridors labeling, performing K-means on the
set of all $\\{x^{x}_{i,1},x^{y}_{i,1}\\}$ for $i\in 1\dots N$ provided a good
initialization of segments before manual adjustments. The runways were labeled
by manually drawing linear classification lines on
$\\{x^{x}_{i,T_{i}},x^{y}_{i,T_{i}}\\}$ for $i\in 1\dots N$, to classify the
touchdown points. A similar manual classification on initial approach fixes
(IAFs) was performed to classify the approach procedures. As a result, the
classes are defined as the combination of runways, IAFs, and Standard Terminal
Arrival Routes (STARs). For the Incheon departure dataset, labeling departure
corridors mirrored that of arrivals but was implemented using the set
$\\{x^{x}_{i,T_{i}},x^{y}_{i,T_{i}}\\}$. In the AIP, the pairs of dependent
runways comply with the same Standard Instrument Departure (SID); we labeled
each $\\{x^{x}_{i,1},x^{y}_{i,1}\\}$ as dependent runway pairs, using the same
manual classification procedure. The labels for departure data denote the
SIDs. The labeling of the trajectory data at Stockholm Arlanda airport was
straightforward as the surveillance data were given with the flight plan data
[63]. The STARs and the situated runways are recorded; therefore, after
minimal corrections and labeling with the initial approach fixes, the classes
are given by combinations of runways, IAFs, and STARs.

Each set of class descriptions was then converted to integers denoted as
$Y=\\{y_{1},y_{2},y_{3},\dots y_{N}\\}$ for $y_{i}\in\mathbb{Z}$. It is
important to emphasize that these instance-level labels will not be used in
representation learning via the self-supervised ATSCC framework or any
baseline methods discussed in this paper; they are only for evaluation
purposes. The trajectory datasets visualization with class distribution is
illustrated in Fig 3.

###  4.2 Experiments

####  4.2.1 Hyperparameters and Reproduction Remarks

Our model employs the CLM configuration from [58], featuring 12 layers with a
model dimension of 768, feed-forward dimensions of 3072 using Gaussian Error
Linear Unit (GELU) activation, 12 attention heads with a 0.35 drop rate, and a
binomial random masking probability set at 0.2. Following [16, 19], the
dimension of the representation $z_{i,t}$ is set at 320. The batch size, $B$,
is set at 16 for training with the AdamW optimizer, using a learning rate of
$1\times 10^{-5}$ and a weight decay $\lambda$ for regularization set at
$1\times 10^{-5}$. For unscaled ATM trajectory data, we recommend using the
RDP threshold as $\epsilon\cdot r_{max}$ for the maximum control radius
$r_{max}$. However, as our data has been scaled to $[-1,1]$, we omitted the
notation of $r_{max}$. We performed a grid search based on performance
evaluation over the RDP threshold, $\epsilon\in\\{0.0001,0.001,0.01,0.1\\}$
and the temperature in loss function
$\tau\in\\{0.01,0.05,0.1,0.5,1.0,5.0,10.0\\}$. For ATSCC, the learned
representation $z_{i,t}$ represents all its previous states; thus, the last
timestamp’s representation $z_{i,T_{i}}$ is extracted as the instance-level
representation for evaluation. To reduce the computational burden, the
trajectory dataset was downsampled every 5 seconds without standardization. To
ensure the consistency of our results, we trained and evaluated the model with
5 different seeds. For all training and evaluation, this paper used Python
3.11.8 and PyTorch 2.0.1 with CUDA Toolkit 11.8, running on an Nvidia GeForce
RTX 4090 GPU.

####  4.2.2 Reproduction of Baselines

This paper discusses the ATM trajectory recognition tasks at the instance
level, including classification and clustering. In the same manner, as the
training and evaluation of ATSCC, we downsampled the trajectories every 5
seconds and used identical geometric features across all baselines with
$x_{i,t}=\\{x_{i,t}^{x},x_{i,t}^{y},x_{i,t}^{z},x_{i,t}^{u_{x}},x_{i,t}^{u_{y}},x_{i,t}^{u_{z}},x_{i,t}^{r},x_{i,t}^{\sin\theta},x_{i,t}^{\cos\theta}\\}$.
Following [16, 19], the dimension of the representation vector is set to 320.
We referred to the original code provided by the authors of the baseline
papers to reproduce the results. The reproduction details are described as
follows:

  * •

SPIRAL [3]: As an example of a non-neural network optimization-based method,
we have reproduced SPIRAL using the DTW. We kept all parameters as default
except for the representation size.

  * •

TCN-AE: We use the Temporal Convolutional Network architecture from [16] to
construct this autoencoder. Following [13, 12], we employ the TCN as both the
encoder and decoder with maxpooling and upsampling in the middle.

  * •

TF-AE: We built a transformer autoencoder following the architecture in [14].
The encoding involves feature extraction via a class token, while the decoding
uses positional encoding as a time query on the latent vector for
reconstruction, similar to [15].

  * •

T-Loss [16] encourages the consistency of random subsequences using triplet
loss. We used the default hyperparameters for the UCR and UEA datasets.

  * •

TNC [17] leverages local smoothness of time series and uses a sliding window
to encode the data into shorter sequences, which we use to represent the
instances, as aggregation to a single vector was not demonstrated. We used the
waveform CNN encoder and HAR configuration. The size of the sub-series
representation was set to 64, with a window size of $\text{int}(T_{max}/20)$
and a sliding gap of 5.

  * •

TS-TCC [18] maximize the agreement between two augmented samples. We referred
to the hyperparameters for the HAR Dataset. Following [19], we exhibit the
instance-level representation using Maxpooling on the base convolutional
network.

  * •

TS2Vec [19] maximize contextual consistency of time series using the
hierarchical contrastive loss. We used the reproduction details the author
demonstrated on the UCR and UEA datasets.

  * •

InfoTS [20] implements meta-learning technique in finding the best
augmentation for representation learning. We reproduced it with the same
hyperparameters used for the UCR and UEA datasets.

Table 1: Performance of Self-Supervised Representation Learning Baselines: Evaluation of Learned Representations Across All Trajectory Datasets, Measured by Accuracy, NMI, and ARI Scores | RKSIa |  |  | RKSId |  |  | ESSA |  |  | LSZH |  |
---|---|---|---|---|---|---|---|---|---|---|---|---
Baselines | ACC | NMI | ARI | ACC | NMI | ARI | ACC | NMI | ARI | ACC | NMI | ARI
Non-neural network |  |  |  |  |  |  |  |  |  |  |  |
• SPIRAL [3] | 0.8344 | 0.6211 | 0.3102 | 0.9946 | 0.8732 | 0.7191 | 0.8503 | 0.7406 | 0.4352 | 0.9173 | 0.7920 | 0.6234
Autoencoders |  |  |  |  |  |  |  |  |  |  |  |
• TCN-AE | 0.9753 | 0.6682 | 0.3593 | 0.9982 | 0.8706 | 0.6514 | 0.9942 | 0.7527 | 0.4420 | 0.9915 | 0.8060 | 0.6566
• TF-AE | 0.9181 | 0.6420 | 0.2881 | 0.9971 | 0.8545 | 0.6229 | 0.9563 | 0.7489 | 0.4394 | 0.9674 | 0.7994 | 0.6159
Contrastive Learning |  |  |  |  |  |  |  |  |  |  |  |
• T-Loss [16] | 0.9735 | 0.6660 | 0.3238 | 0.9984 | 0.8484 | 0.5969 | 0.9967 | 0.7552 | 0.4553 | 0.9916 | 0.8301 | 0.6349
• TNC [17] | 0.9728 | 0.6982 | 0.4222 | 0.9980 | 0.8784 | 0.6771 | 0.9904 | 0.7581 | 0.4837 | 0.9897 | 0.8983 | 0.7908
• TS-TCC [18] | 0.9847 | 0.7207 | 0.4304 | 0.9980 | 0.8831 | 0.6834 | 0.9989 | 0.7656 | 0.4670 | 0.9950 | 0.8667 | 0.7248
• TS2Vec [19] | 0.9786 | 0.6890 | 0.3412 | 0.9984 | 0.8660 | 0.6691 | 0.9981 | 0.7572 | 0.4463 | 0.9933 | 0.8373 | 0.6381
• InfoTS [20] | 0.9764 | 0.6668 | 0.3703 | 0.9986 | 0.8758 | 0.6621 | 0.9986 | 0.7623 | 0.4608 | 0.9930 | 0.8604 | 0.6851
• ATSCC | 0.9946 | 0.8723 | 0.8195 | 0.9987 | 0.9129 | 0.7723 | 0.9990 | 0.8640 | 0.6517 | 0.9977 | 0.9480 | 0.9037

((a)) SPIRAL

((b)) TCN-AE

((c)) TF-AE

((d)) T-Loss

((e)) TNC

((f)) TS-TCC

((g)) TS2Vec

((h)) InfoTS

((i)) ATSCC

Figure 4: t-SNE Visualization of Learned Representations from the Arrival
Trajectory Dataset at Incheon International Airport (RKSIa) Across All
Representation Learning Baselines

####  4.2.3 Trajectory Classification

An effective representation should have fidelity to class labels and enable
linear separability. These properties can be evaluated using classification
benchmarks, as instance-level labels $Y$ were manually labeled according to
AIPs. The evaluation follows the same standard classification protocol as
demonstrated in [16, 19]. This protocol involves training a Support Vector
Machine (SVM) classifier with a Radial Basis Function (RBF) kernel on the
learned representation of the training data. For TNC, we fitted an SVM using
the Global Alignment Kernel (GAK) [69]. We then assess the SVM’s accuracy
using the representations derived from the testing data. We have integrated
the classification challenges in [22, 23, 24, 27, 26], classifying both
runways or approach patterns.

According to the accuracy results in Table 1, we observed a significant
improvement in the accuracy of the arrival dataset for Incheon Airport, which
features a complex configuration with four parallel runways. The experiment on
departure trajectory data showed slightly better accuracy, as departure
trajectories are relatively straightforward compared to arrivals. At Stockholm
Arlanda Airport, which features two parallel runways, the results follow the
same trend as Incheon arrivals, resulting in the highest accuracy on this
larger dataset. Lastly, Zurich Airport has the most distinguishable pattern,
yet we still observed significantly better performance compared to the other
baselines.

The ATSCC model outperforms all baselines in accuracy, demonstrating that its
learned representation is class-preserving and easily separable in the
encoding space. This leads to significant improvements in the classifier’s
ability to identify runways and terminal maneuvering procedures compared to
state-of-the-art baselines, highlighting ATSCC’s suitability for air traffic
trajectory data. Fig 4 further illustrates the separability of the learned
representation, with classes being well separated.

####  4.2.4 Trajectory Clustering

We evaluate the uniformity of the learned representation in the encoding space
to assess whether a simple clustering algorithm, when employed on the
representation, can effectively differentiate data instances and ensure
correct clustering without relying on true labels. We conduct the clustering
experiment similarly to [17] by implementing K-Means on the instance-level
representation of trajectories using Euclidean distance, and setting the
number of clusters equal to the number of unique test labels; however, for
TNC, DTW distance was used.

Due to the absence of instance labels in recent ATM clustering works (e.g.,
[6, 29, 28, 50, 51]), clusters are typically analyzed visually. Besides,
metrics such as the Silhouette score, Calinski-Harabasz Index, and Davies-
Bouldin Index (DBI) were used (e.g., [4, 5, 9, 10, 52]). However, the fidelity
and accuracy of cluster assignments are more crucial than the proximity of
representations within clusters. Therefore, following recent clustering
literature [54, 70, 71, 72], we use the Normalized Mutual Information (NMI)
score and Adjusted Rand Index (ARI) to evaluate the similarity of clustering
results with the true labels. This evaluation method provides a sensible
performance assessment by testing whether the learned representation enhances
clustering to recognize unique maneuvering procedures and correctly assign
labels to the samples.

The NMI and ARI results tabulated in Table 1 show that both scores are
significantly enhanced across arrival datasets, from those with complex
arrival trajectories, such as Incheon and Stockholm Arlanda with their
parallel runways, to those with less ambiguous configurations, like Zurich.
Although the other baselines perform well with departure data, our approach
still achieves superior scores. These results validate that the learned
representation aligns well with the operational semantics. For example,
trajectories with landing patterns that are geometrically similar become
distinguishable in the encoding space. The superior scores reflect the
framework’s capability to generate the representations with exceptional
uniformity, enabling K-Means to establish high-fidelity clusters. The
experiment further confirms that the ATSCC framework is a more suitable
representation learning method for air traffic management trajectory data.
Moreover, besides separability, uniformity can also be observed in Fig 4, as
the same classes align well in distinct groups.

We further conducted a clustering experiment using a greater number of
clusters than the number of classes because trajectory clustering is typically
used for trajectory categorization, which requires more clusters than
procedures, according to [5, 27]. This experiment aimed to assess the ability
of the representation to enable clustering to generate high-fidelity
subclusters, as ideally, these clusters should be subclasses of the
procedures. To maximize fidelity, it is important to maximize mutual
information [20, 21]. Therefore, we evaluate the mutual information score (MI)
on clusters generated by K-means, with the number of clusters ranging from the
number of existing classes to 100 clusters.

Fig 5 shows that our learned representation outperforms other methods for
complex airports like Incheon and Stockholm Arlanda. However, for simpler
cases such as Zurich and the departure trajectory of Incheon airports, ATSCC
demonstrates slightly better performance due to minimal runway ambiguity. The
MI scores converge to higher values, highlighting the importance of
representation learning over the choice of clustering method and parameters.
The results demonstrate that increasing the number of clusters to separate
ambiguous groups is ineffective without an effective data representation.
Along with classification and clustering benchmarks, this further highlights
that the separability and uniformity of clusters lead to higher fidelity and
accuracy in downstream tasks.

Figure 5: Comparison of MI Score Plot Evaluation on Learned Representation
Using K-Means Clustering: Number of Clusters Ranges from the Number of Classes
Up to 100, Incremented in Steps of 5

###  4.3 Analysis

####  4.3.1 Ablation Study

In this section, we evaluate the effectiveness of the framework’s components
by evaluating the average performance on four trajectory datasets upon
modifications, as tabulated in Table 2. We begin by analyzing the geometric
feature extraction by excluding certain features and examining the impact on
the results. Polar positions have a greater impact on performance than path
vector sequences, as they change rapidly near the airport, resulting in
distinct tokens. Conversely, adding path sequences as the only additional
feature reduces scores due to potential confusion from parallel landing paths.
However, we found that including both polar positions and path sequence
significantly enhances overall performance.

We have observed that applying random binomial masking on the input and
attention masks generates significantly effective representation because the
random masking enhances regularization and generalization, performing
similarly to dropout. Additionally, it provides more variation of data,
similar to data augmentation.

L2 normalization, which constrains the representation within a hypersphere,
can enhance downstream task performances by promoting linear separation in the
encoding space. Therefore, we observed that eliminating L2 normalization after
both projection layers resulted in significant performance loss, similar to L2
normalizing only on tokens. On the other hand, L2 normalization on the
representation significantly enhanced the scores, consistent with [59];
however, slight additional improvement was observed when adding token
normalization.

Given that the ATSCC encoder does not incorporate explicit positional
encoding, we compared our results with those of commonly used positional
encodings. We referenced [14] for sinusoidal positional encoding and [73] for
initializing the learnable positional encoding. We observed that the encoder
following the NoPos [58] outperforms those with explicit positional encoding,
which can alter the information in tokens. For self-supervised learning, it is
important to utilize underlying data information, rather than relying solely
on expected output as in supervised learning. Introducing distortions through
positional encoding can fundamentally impact training.

We verify the suitability of the casual transformer by implementing different
encoder backbones. The ATSCC performs binomial masking on both the input and
the attention matrices. However, different architectures perform this random
masking differently. The dilated causal convolution network encoder
architecture and its random masking technique are referenced in [19]. Later,
we replace the backbone of this CNN model with a 3-layer LSTM model having a
hidden layer dimension of 512. Both encoders lead to a significant decrease in
overall performance, as CNN can become locally focused, and LSTM loses
information when encoding long sequences.

The soft nearest neighbor loss [60] was rearranged and modified into Equation
(6). We have empirically proven that omitting the positive contrasting terms
in the sum with negatives results in better performance in our framework and
experiment setting.

Table 2: Ablation Results on Four Trajectory Datasets Configurations | ACC | NMI | ARI
---|---|---|---
Full Configuration | 0.9977 | 0.8993 | 0.7868
Geometric Feature Extraction |  |  |
• without aircraft path vectors | 0.9971 | 0.8751 | 0.7537
• without polar positions | 0.9966 | 0.7883 | 0.5624
• catesian positions only | 0.9974 | 0.8080 | 0.6588
Random Masking |  |  |
• without random masking | 0.9962 | 0.8044 | 0.5818
L2 Normalization |  |  |
• without token L2 norm | 0.9977 | 0.8941 | 0.7691
• without representation L2 norm | 0.9952 | 0.7902 | 0.5784
• without both L2 norm | 0.9961 | 0.8090 | 0.6089
Positional Encoding |  |  |
• Sinusoidal | 0.9723 | 0.1863 | 0.0712
• Learnable | 0.9892 | 0.7751 | 0.6400
Backbone Architecture |  |  |
• LSTM | 0.6195 | 0.5359 | 0.2606
• Dilated Causal Convolution | 0.9456 | 0.4903 | 0.2752
soft nearest neighbor loss |  |  |
• $\mathcal{L}_{snn}$: Equation (5) | 0.9973 | 0.8718 | 0.7361

((a)) Trajectory of Class 8: Arriving via STAR REBIT2, Landing on Runway 15L
via IAF MUNAN

((b)) Trajectory of Class 28: Arriving via STAR OLMEN2H, Landing on Runway 15L
via IAF MUNAN

Figure 6: Visualization of Embedded Trajectories, 2D Top-View Trajectory Plot
and Geometric Features Figure 7: Visualization of Clustering Results on Four
Trajectory Datasets: Illustrating Representative Trajectories Using the Mean
of Cluster Members

####  4.3.2 Visual Explanation of Embedding

For ease of visual explanation, we have taken two trajectories from the
Incheon arrival dataset and their learned representations at all timestamps,
which were trained with the output dimension set at 32. Both flights landed on
the same runway but arrived via different STARs. At each timestamp, trajectory
states in different scales and units are collectively converted into scale-
invariant representation vectors. According to Fig 6, the representation
$z_{i,t}$ changes over time as causal attention collects information $x_{i,t}$
in $\mathcal{X}^{\prime}_{i,t}\subseteq\mathcal{X}_{i}$ from current and
previous states at each timestamp $t\in\\{1,\dots,t\\}$. A clear difference in
$z_{i,t}$ of the two trajectories is noticeable at earlier timestamps due to
different STARs. Later, $z_{i,t}$ of two trajectories diverges, despite
landing on the same runway, because the information of the STARs was carried
along while encoding. The ATSCC does not forcibly discretize the states, nor
does it use the notation of Euclidean vectors or waypoints. Instead, the
values $z_{i,t}$ represent scale-invariant semantic embeddings that preserve
temporal transitions in continuous encoding space. This enables their
applicability to machine learning models, which can be flexibly adapted for
other tasks.

####  4.3.3 Analysis for Trajectory Categorization

Many previous studies have focused on air traffic categorization; therefore,
this section shows the feasibility of our model in addressing this problem.
The trajectory can be analyzed by varying the number of clusters to 100 or
more, as suggested by [7, 27]. Fig. 5 shows that increasing the number of
clusters maximizes the MI score; thus, we performed a brief analysis using 100
clusters as an example of clustering results for a categorization task.

Since the trajectory data undergoes semantic analysis using the learned
representation, trajectories with spatially close but operationally distinct
segments, such as parallel landing paths, can be differentiated, as
demonstrated by the results on Incheon and Stockholm Arlanda Airports (Fig.
7). Unlike other studies that address complex airports with parallel runways
(e.g., [4, 5, 23, 24, 27, 33, 52]), ATSCC effectively enables separation of
ambiguous classes without requiring airport information, resulting in high-
fidelity subclasses of the procedures. Similar to [7], the visualization shows
the identifications of non-standard patterns, notably at Incheon Airport, and
trajectories with holding patterns are clearly grouped at both Stockholm
Arlanda and Zurich Airport. This confirms that the ATSCC is also applicable
for characterizing trajectories and outlier detection.

##  5 CONCLUSION

This paper has proposed a representation learning framework for multivariate
air traffic trajectory data. ATSCC trains a causal transformer encoder by
assigning segment IDs via the RDP algorithm for training with the soft nearest
neighbor loss. We proposed fidelity benchmarking using manually labeled
trajectory data, guided by AIPs, to evaluate the representation. The
classification and clustering results show that ATSCC is the most suitable
framework for ATM trajectory data. Our learned representation enables the
separation of ambiguous paths that are semantically different, non-standard
patterns, and holding points. We have addressed the limitations of several
prior works, for ATSCC does not rely on computationally intensive distance
metrics, nor does it require the airport information. Patterns are recognized
through the training, showing adaptability to various airport configurations.
The recommendations for future works include extending the model’s
applicability to other trajectory or time series data or adapting the model to
tasks such as trajectory prediction, imputation, or anomaly detection.
Moreover, ATSCC can potentially train a pre-trained trajectory model for
various machine-learning tasks. Additionally, incorporating weather or
operational data could extend the model’s utility across diverse scenarios.

## References

  * [1] G. Enea and M. Porretta, “A comparison of 4d-trajectory operations envisioned for ’nextgen’ and ’sesar’, some preliminary findings,” in _Proceedings of the 28th Congress of the International Council of the Aeronautical Sciences_ , Sep. 2012, pp. 23–28.
  * [2] “Random warping series: A random features method for time-series embedding,” 2018, pp. 793–802, 21st International Conference on Artificial Intelligence and Statistics, AISTATS 2018.
  * [3] Q. Lei, J. Yi, R. Vaculin, L. Wu, and I. S. Dhillon, “Similarity preserving representation learning for time series clustering,” in _Proceedings of the 28th International Joint Conference on Artificial Intelligence_ , ser. IJCAI’19. AAAI Press, 2019, p. 2845–2851.
  * [4] W. Zeng, Z. Xu, Z. Cai, X. Chu, and X. Lu, “Aircraft trajectory clustering in terminal airspace based on deep autoencoder and gaussian mixture model,” _Aerospace_ , vol. 8, no. 9, 2021.
  * [5] X. Chu, X. Tan, and W. Zeng, “A clustering ensemble method of aircraft trajectory based on the similarity matrix,” _Aerospace_ , vol. 9, no. 5, 2022.
  * [6] X. Olive, L. Basora, B. Viry, and R. Alligier, “Deep trajectory clustering with autoencoders,” in _ICRAT 2020, 9th International Conference for Research in Air Transportation_ , 2020.
  * [7] Y. Liu, K. K. H. Ng, N. Chu, K. K. Hon, and X. Zhang, “Spatiotemporal image-based flight trajectory clustering model with deep convolutional autoencoder network,” _Journal of Aerospace Information Systems_ , vol. 20, no. 9, p. 575–587, Sep. 2023.
  * [8] P. Malhotra, V. TV, L. Vig, P. Agarwal, and G. Shroff, “TimeNet: Pre-trained deep recurrent neural network for time series classification,” _arXiv preprint arXiv:1706.08838_ , 2017.
  * [9] G. Liu, Y. Fan, J. Zhang, P. Wen, Z. Lyu, and X. Yuan, “Deep flight track clustering based on spatial–temporal distance and denoising auto-encoding,” _Expert Systems with Applications_ , vol. 198, p. 116733, Jul. 2022.
  * [10] Y. Fan, J. Liu, H. Ye, and Z. Lyu, “TA-LSTM: A time and attribute aware lstm for deep flight track clustering,” _IEEE Transactions on Aerospace and Electronic Systems_ , p. 1–14, 2023.
  * [11] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “WaveNet: A generative model for raw audio,” _arXiv preprint arXiv:1609.03499_ , 2016.
  * [12] R. Mo, Y. Pei, N. V. Venkatarayalu, P. Nathaniel Joseph, A. B. Premkumar, S. Sun, and S. K. K. Foo, “Unsupervised TCN-AE-Based Outlier Detection for Time Series With Seasonality and Trend for Cellular Networks,” _IEEE Transactions on Wireless Communications_ , vol. 22, no. 5, pp. 3114–3127, 2023.
  * [13] M. Thill, W. Konen, H. Wang, and T. Bäck, “Temporal convolutional autoencoder for unsupervised anomaly detection in time series,” _Applied Soft Computing_ , vol. 112, p. 107751, 2021.
  * [14] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in Neural Information Processing Systems_ , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017.
  * [15] G. Tevet, B. Gordon, A. Hertz, A. H. Bermano, and D. Cohen-Or, “Motionclip: Exposing human motion generation to clip space,” in _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII_. Springer, 2022, pp. 358–374.
  * [16] J.-Y. Franceschi, A. Dieuleveut, and M. Jaggi, “Unsupervised scalable representation learning for multivariate time series,” in _Advances in Neural Information Processing Systems_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32. Curran Associates, Inc., 2019.
  * [17] S. Tonekaboni, D. Eytan, and A. Goldenberg, “Unsupervised representation learning for time series with temporal neighborhood coding,” in _International Conference on Learning Representations_ , sep 2020.
  * [18] E. Eldele, M. Ragab, Z. Chen, M. Wu, C. K. Kwoh, X. Li, and C. Guan, “Time-series representation learning via temporal and contextual contrasting,” in _Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21_ , 2021, pp. 2352–2359.
  * [19] Z. Yue, Y. Wang, J. Duan, T. Yang, C. Huang, Y. Tong, and B. Xu, “TS2vec: Towards universal representation of time series,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 36, no. 8, 2022, pp. 8980–8987.
  * [20] D. Luo, W. Cheng, Y. Wang, D. Xu, J. Ni, W. Yu, X. Zhang, Y. Liu, Y. Chen, H. Chen _et al._ , “Time series contrastive learning with information-aware augmentations,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 37, no. 4, 2023, pp. 4534–4542.
  * [21] Y. Tian, C. Sun, B. Poole, D. Krishnan, C. Schmid, and P. Isola, “What makes for good views for contrastive learning?” _Advances in neural information processing systems_ , vol. 33, pp. 6827–6839, 2020.
  * [22] C. S. Bosson and T. Nikoleris, “Supervised learning applied to air traffic trajectory classification,” in _2018 AIAA Information Systems-AIAA Infotech @ Aerospace_. American Institute of Aeronautics and Astronautics, Jan. 2018.
  * [23] M. C. R. Murça, R. DeLaura, R. J. Hansman, R. Jordan, T. Reynolds, and H. Balakrishnan, “Trajectory clustering and classification for characterization of air traffic flows,” in _16th AIAA Aviation Technology, Integration, and Operations Conference_. American Institute of Aeronautics and Astronautics, Jun. 2016.
  * [24] M. C. R. Murça, R. J. Hansman, L. Li, and P. Ren, “Flight trajectory data analytics for characterization of air traffic flows: A comparative analysis of terminal area operations between new york, hong kong and sao paulo,” _Transportation Research Part C: Emerging Technologies_ , vol. 97, p. 324–347, Dec. 2018.
  * [25] S. Madar, T. G. Puranik, and D. N. Mavris, “Application of trajectory clustering for aircraft conflict detection,” in _2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC)_. IEEE, Oct. 2021.
  * [26] N. Jiménez-Campfens, A. Colomer, J. Núñez, J. M. Mogollón, A. L. Rodríguez, and V. Naranjo, “Deep learning in aeronautics: Air traffic trajectory classification based on weather reports,” in _Intelligent Data Engineering and Automated Learning – IDEAL 2020: 21st International Conference, Guimaraes, Portugal, November 4–6, 2020, Proceedings, Part II_. Springer-Verlag, 2020, p. 148–155.
  * [27] C. Deng, H.-C. Choi, H. Park, and I. Hwang, “Trajectory pattern identification and classification for real-time air traffic applications in area navigation terminal airspace,” _Transportation Research Part C: Emerging Technologies_ , vol. 142, p. 103765, 2022.
  * [28] L. Basora, J. Morio, and C. Mailhot, “A trajectory clustering framework to analyse air traffic flows,” in _Proceedings of the 7th SESAR Innovation Days_ , Nov. 2017.
  * [29] S. J. Corrado, T. G. Puranik, O. J. Pinon, and D. N. Mavris, “Trajectory clustering within the terminal airspace utilizing a weighted distance function,” _Proceedings_ , vol. 59, no. 1, 2020.
  * [30] H. Sakoe and S. Chiba, “Dynamic programming algorithm optimization for spoken word recognition,” _IEEE Transactions on Acoustics, Speech, and Signal Processing_ , vol. 26, no. 1, pp. 43–49, 1978.
  * [31] D. Müllner, “Modern hierarchical, agglomerative clustering algorithms,” _arXiv preprint arXiv:1109.2378_ , 2011.
  * [32] F. Rehm, “Clustering of flight tracks,” in _AIAA Infotech@Aerospace 2010_. American Institute of Aeronautics and Astronautics, Apr. 2010.
  * [33] G. Andrienko, N. Andrienko, P. Bak, D. Keim, and S. Wrobel, _Visual Analytics of Movement_. Springer Berlin Heidelberg, 2013.
  * [34] S. Rinzivillo, D. Pedreschi, M. Nanni, F. Giannotti, N. Andrienko, and G. Andrienko, “Visually driven analysis of movement data by progressive clustering,” _Information Visualization_ , vol. 7, no. 3–4, p. 225–239, Sep. 2008.
  * [35] M. Ankerst, M. M. Breunig, H.-P. Kriegel, and J. Sander, “Optics: ordering points to identify the clustering structure,” ser. SIGMOD ’99. Association for Computing Machinery, 1999, p. 49–60.
  * [36] G. Andrienko, N. Andrienko, G. Fuchs, and J. M. C. Garcia, “Clustering trajectories by relevant parts for air traffic analysis,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 24, no. 1, pp. 34–44, 2018.
  * [37] P. C. Besse, B. Guillouet, J.-M. Loubes, and F. Royer, “Review and perspective for distance-based clustering of vehicle trajectories,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 17, no. 11, pp. 3306–3317, 2016.
  * [38] R. J. Campello, D. Moulavi, and J. Sander, “Density-based clustering based on hierarchical density estimates,” in _Pacific-Asia conference on knowledge discovery and data mining_. Springer, 2013, pp. 160–172.
  * [39] U. Ramer, “An iterative procedure for the polygonal approximation of plane curves,” _Computer Graphics and Image Processing_ , vol. 1, no. 3, p. 244–256, Nov. 1972.
  * [40] D. H. DOUGLAS and T. K. PEUCKER, “Algorithms for the reduction of the number of points required to represent a digitized line or its caricature,” _Cartographica: The International Journal for Geographic Information and Geovisualization_ , vol. 10, no. 2, p. 112–122, Dec. 1973.
  * [41] G. Xuhao, Z. Junfeng, and P. Zihan, “Trajectory clustering for arrival aircraft via new trajectory representation,” _Journal of Systems Engineering and Electronics_ , vol. 32, no. 2, pp. 473–486, 2021.
  * [42] A. Mcfadyen, M. O’Flynn, T. Martin, and D. Campbell, “Aircraft trajectory clustering techniques using circular statistics,” in _2016 IEEE Aerospace Conference_ , 2016, pp. 1–10.
  * [43] L. Rdusseeun and P. Kaufman, “Clustering by means of medoids,” in _Proceedings of the statistical data analysis based on the L1 norm conference, neuchatel, switzerland_ , vol. 31, 1987.
  * [44] A. Bhattacharyya, “On a measure of divergence between two multinomial populations,” _Sankhyā: The Indian Journal of Statistics (1933-1960)_ , vol. 7, no. 4, pp. 401–406, 1946.
  * [45] J.-G. Lee, J. Han, and K.-Y. Whang, “Trajectory clustering: a partition-and-group framework,” in _Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data_ , ser. SIGMOD ’07. Association for Computing Machinery, 2007, p. 593–604.
  * [46] M. Ester, H.-P. Kriegel, J. Sander, X. Xu _et al._ , “A density-based algorithm for discovering clusters in large spatial databases with noise,” in _kdd_ , vol. 96, no. 34, 1996, pp. 226–231.
  * [47] M. Gariel, A. N. Srivastava, and E. Feron, “Trajectory clustering and an application to airspace monitoring,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 12, no. 4, pp. 1511–1524, 2011.
  * [48] X. Olive and J. Morio, “Trajectory clustering of air traffic flows around airports,” _Aerospace Science and Technology_ , vol. 84, pp. 776–781, 2019.
  * [49] J. MacQueen _et al._ , “Some methods for classification and analysis of multivariate observations,” in _Proceedings of the fifth Berkeley symposium on mathematical statistics and probability_ , vol. 1, no. 14. Oakland, CA, USA, 1967, pp. 281–297.
  * [50] S. Chakrabarti and A. E. Vela, “Modeling and characterizing aircraft trajectories near airports using extracted control actions,” _Journal of Aerospace Information Systems_ , vol. 20, no. 2, pp. 81–101, 2023.
  * [51] W. J. Eerland and S. Box, “Trajectory clustering, modeling and selection with the focus on airspace protection,” in _AIAA Infotech @ Aerospace_. American Institute of Aeronautics and Astronautics, Jan. 2016.
  * [52] J. J. Damanik, C. Shin, and H.-L. Choi, “Spatial trajectory clustering of incheon airport flights,” in _Proceedings of the Korean Society for Aeronautical & Space Sciences 2022 Fall Conference_. Korean Society for Aeronautical & Space Sciences (KSAS), Nov. 2022, pp. 284–285.
  * [53] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the em algorithm,” _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , vol. 39, no. 1, p. 1–22, Sep. 1977.
  * [54] J. Xie, R. Girshick, and A. Farhadi, “Unsupervised deep embedding for clustering analysis,” in _International conference on machine learning_. PMLR, 2016, pp. 478–487.
  * [55] H. Fabian, “rdp,” https://github.com/fhirschmann/rdp/, 2016.
  * [56] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019.
  * [57] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in _Advances in Neural Information Processing Systems_ , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 1877–1901.
  * [58] A. Haviv, O. Ram, O. Press, P. Izsak, and O. Levy, “Transformer language models without positional encodings still learn positional information,” in _Findings of the Association for Computational Linguistics: EMNLP 2022_ , Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Association for Computational Linguistics, Dec. 2022, pp. 1382–1390.
  * [59] T. Wang and P. Isola, “Understanding contrastive representation learning through alignment and uniformity on the hypersphere,” in _International conference on machine learning_. PMLR, 2020, pp. 9929–9939.
  * [60] N. Frosst, N. Papernot, and G. Hinton, “Analyzing and improving representations with the soft nearest neighbor loss,” in _International conference on machine learning_. PMLR, 2019, pp. 2012–2020.
  * [61] M. Schäfer, M. Strohmeier, V. Lenders, I. Martinovic, and M. Wilhelm, “Bringing up opensky: A large-scale ads-b sensor network for research,” in _Proceedings of the 13th IEEE/ACM International Symposium on Information Processing in Sensor Networks_ , April 2014, pp. 83–94.
  * [62] Ministry of Land, Infrastructure and Transport, South Korea, “Airportal,” Electronic resource, 2024, accessed: April 7, 2023. [Online]. Available: https://www.airportal.go.kr/
  * [63] J. Nilsson and J. Unger, “Swedish civil air traffic control dataset,” _Data in Brief_ , vol. 48, p. 109240, Jun. 2023.
  * [64] X. Olive and L. Basora, “Detection and identification of significant events in historical aircraft trajectory data,” _Transportation Research Part C: Emerging Technologies_ , vol. 119, p. 102737, Oct. 2020.
  * [65] E. Axell, P. Johansson, M. Alexandersson, and J. Rantakokko, “Estimation of the position error in gps receivers,” Swedish Defence Research Agency (FOI), FOI-R–3840–SE E36044, February 2014, approved by Christian Jönsson.
  * [66] A. Savitzky and M. J. E. Golay, “Smoothing and differentiation of data by simplified least squares procedures.” _Analytical Chemistry_ , vol. 36, no. 8, p. 1627–1639, Jul. 1964.
  * [67] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.
  * [68] T. Lin, M. Maire, S. J. Belongie, L. D. Bourdev, R. B. Girshick, J. Hays, P. Perona, D. Ramanan, P. Doll’a r, and C. L. Zitnick, “Microsoft COCO: common objects in context,” _CoRR_ , vol. abs/1405.0312, 2014. [Online]. Available: http://arxiv.org/abs/1405.0312
  * [69] M. Cuturi, “Fast global alignment kernels,” in _Proceedings of the 28th international conference on machine learning (ICML-11)_ , 2011, pp. 929–936.
  * [70] M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , September 2018.
  * [71] J. R. Regatti, A. A. Deshmukh, E. Manavoglu, and U. Dogan, “Consensus clustering with unsupervised representation learning,” in _2021 International Joint Conference on Neural Networks (IJCNN)_ , 2021, pp. 1–9.
  * [72] G. Vardakas, I. Papakostas, and A. Likas, “Deep clustering using the soft silhouette score: Towards compact and well-separated clusters,” _arXiv preprint arXiv:2402.00608_ , 2024.
  * [73] G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff, “A transformer-based framework for multivariate time series representation learning,” in _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_, ser. KDD ’21. Association for Computing Machinery, 2021, p. 2114–2124.

[]Thaweerath Phisannupawong received a B.Eng. degree in Aeronautical
Engineering and Commercial Pilot from King Mongkut’s Institute of Technology
Ladkrabang (KMITL), Bangkok, Thailand, in 2021. Since 2022, he has been
pursuing an M.S. degree in Aerospace Engineering at the Korea Advanced
Institute of Science and Technology (KAIST) in Daejeon, South Korea. His
research focuses on aerospace data applications and representation learning.

[]Joshua J. Damanik received the B.S. degree in engineering physics from
Institut Teknologi Bandung, Indonesia, in 2018, and the M.S. degree in
aerospace engineering from the Korea Advanced Institute of Science and
Technology (KAIST), Daejeon, South Korea, in 2021, where he is currently
pursuing the Ph.D. degree in aerospace engineering. His current research
interests include robotics estimation and control, and data mining.

[]Han-Lim Choi (Senior Member, IEEE) received the B.S. and M.S. degrees in
aerospace engineering from the Korea Advanced Institute of Science and
Technology (KAIST), Daejeon, South Korea, in 2000 and 2002, respectively, and
the Ph.D. degree in aeronautics and astronautics from the Massachusetts
Institute of Technology (MIT), Cambridge, MA, USA, in 2009. Then, he studied
at MIT as a Postdoctoral Associate until he joined KAIST, in 2010. He is
currently a Professor of aerospace engineering at KAIST. His research
interests include estimation and control for sensor networks and decision
making for multi-agent systems. He was a recipient of the Automatic
Applications Prize, in 2011 (together with Dr. Jonathan P. How).