$\displaystyle=-\frac{k_{\perp}^{2}}{4\Omega_{a}}\sum_{p=0}^{\infty}\sum_{j=0}^{\infty}\sum_{g=0}^{l+2k}\sum_{h=0}^{k+\lfloor
l/2\rfloor}\sum_{r=0}^{p+2j}\sum_{s=0}^{j+\lfloor
p/2\rfloor}\sum_{s_{1}=0}^{s}\sum_{h_{1}=0}^{h}L^{r}_{ss_{1}}L^{g}_{hh_{1}}\frac{\left(T^{-1}\right)^{gh}_{lk}}{\sqrt{2^{l}l!}}\frac{\left(T^{-1}\right)^{rs}_{pj}}{\sqrt{2^{p}p!}}N_{a}^{pj}$
$\displaystyle\times\frac{2}{\sqrt{\pi}}\int d\xi\int
ds_{a}e^{-s_{a}^{2}}s_{a}^{r+g+4+2s_{1}+2h_{1}}P_{r}(\xi)P_{g}(\xi)$
$\displaystyle\times\left[\nu_{ab}^{D}(v)\left(\xi^{2}+1\right)+\nu_{ab}^{\parallel}(v)(1-\xi^{2})\right].$
(156)

Finally, using the Legendre orthogonality relation and the identity given by
respectively,

$\displaystyle\int d\xi P_{g}(\xi)P_{r}(\xi)=\frac{2\delta_{g}^{r}}{2g+1},$
(157) $\displaystyle\int d\xi P_{g}(\xi)P_{r}(\xi)\xi^{2}=d_{g}^{r},$ (158)

with $d_{g}^{r}$ defined in Eq. 103, and introducing the speed integrated
frequencies, $\overline{\nu}_{ab}^{\parallel
k}=4\int_{0}^{\infty}ds_{a}s_{a}^{2k}\nu_{ab}^{\parallel}(v)e^{-s_{a}^{2}}/\sqrt{\pi}$
and $\overline{\nu}_{ab}^{Dk}=4\int
ds_{a}e^{-s^{2}}s_{a}^{2k}\nu_{ab}^{D}(v)/\sqrt{\pi}$ with closed formulas
found in Eq. 104, the Hermite-Laguerre expansion of $R_{ab}^{\perp}$, i.e.
$R_{ab}^{\perp lk}$ is derived and is given in Eq. 100.

## References

  * Abel et al. (2008) Abel, I. G., Barnes, M., Cowley, S. C., Dorland, W. & Schekochihin, A. A. 2008 Linearized model Fokker-Planck collision operators for gyrokinetic simulations. i. theory. Physics of Plasmas 15 (12), 122509.
  * Barnes et al. (2009) Barnes, M., Abel, I. G., Dorland, W., Ernst, D. R., Hammett, G. W., Ricci, P., Rogers, B. N., Schekochihin, A. A. & Tatsuno, T. 2009 Linearized model Fokker–Planck collision operators for gyrokinetic simulations. ii. numerical implementation and tests. Physics of Plasmas 16 (7), 072107\.
  * Belli & Candy (2011) Belli, E. A. & Candy, J. 2011 Full linearized Fokker–Planck collisions in neoclassical transport simulations. Plasma physics and controlled fusion 54 (1), 015015.
  * Belli & Candy (2017) Belli, E. A. & Candy, J. 2017 Implications of advanced collision operators for gyrokinetic simulation. Plasma Physics and Controlled Fusion 59 (4), 045005.
  * Boyd (2001) Boyd, J. P. 2001 Chebyshev and Fourier spectral methods. Courier Corporation.
  * Brizard & Mishchenko (2009) Brizard, A. J. & Mishchenko, A. 2009 Guiding-center recursive Vlasov and Lie-transform methods in plasma physics. Journal of Plasma Physics 75 (3), 675.
  * Candy & Waltz (2003) Candy, J. & Waltz, R. E. 2003 Anomalous transport scaling in the diii-d tokamak matched by supercomputer simulation. Physical review letters 91 (4), 045001.
  * Cary (1981) Cary, J. R. 1981 Lie transform perturbation theory for Hamiltonian systems. Physics Reports 79 (2), 129.
  * Chang et al. (2017) Chang, C. S., Ku, S, Tynan, G. R., Hager, R., Churchill, R. M., Cziegler, I., Greenwald, M., Hubbard, AE & Hughes, J. W. 2017 Fast low-to-high confinement mode bifurcation dynamics in a tokamak edge plasma gyrokinetic simulation. Physical Review Letters 118 (17), 175001.
  * Chen et al. (1991) Chen, L., Briguglio, S. & Romanelli, F. 1991 The long-wavelength limit of the ion-temperature gradient mode in tokamak plasmas. Physics of Fluids B: Plasma Physics 3 (3), 611–614.
  * Crandall et al. (2020) Crandall, P., Jarema, D., Doerk, H., Pan, Q., Merlo, G., Görler, T., Navarro, A. Bañón, Told, D., Maurer, M. & Jenko, F. 2020 Multi-species collisions for delta-f gyrokinetic simulations: Implementation and verification with gene. Computer Physics Communications 255, 107360.
  * Dimits & Cohen (1994) Dimits, A. M. & Cohen, B. I. 1994 Collision operators for partially linearized particle simulation codes. Phys. Rev. E 49, 709.
  * Dimits et al. (1996) Dimits, A. M., Williams, T. J., Byers, J. A. & Cohen, B. I. 1996 Scalings of ion-temperature-gradient-driven anomalous transport in tokamaks. Physical review letters 77 (1), 71\.
  * Donnel et al. (2019) Donnel, P., Garbet, X., Sarazin, Y., Grandgirard, V., Asahi, Y., Bouzat, N., Caschera, E, Dif-Pradalier, G., Ehrlacher, C., Ghendrih, P. & others 2019 A multi-species collisional operator for full-f global gyrokinetics codes: Numerical aspects and verification with the Gysela code. Computer Physics Communications 234, 1.
  * Dorf et al. (2014) Dorf, M. A., Cohen, R. H., Dorr, M, Hittinger, J. & Rognlien, T. D. 2014 Progress with the COGENT edge kinetic code: Implementing the Fokker-Planck collision operator. Contributions to Plasma Physics 54 (4-6), 517.
  * Dougherty (1964) Dougherty, J. P. 1964 Model Fokker-Planck equation for a plasma and its solution. The Physics of Fluids 7 (11), 1788\.
  * Dudson et al. (2009) Dudson, B. D., Umansky, M. V., Xu, X. Q., Snyder, P. B. & Wilson, H. R. 2009 Bout++: A framework for parallel plasma fluid simulations. Computer Physics Communications 180 (9), 1467.
  * Ernst et al. (2004) Ernst, D. R., Bonoli, P. T., Catto, P. J., Dorland, W., Fiore, CL, Granetz, RS, Greenwald, M, Hubbard, AE, Porkolab, M, Redi, MH & others 2004 Role of trapped electron mode turbulence in internal transport barrier control in the alcator c-mod tokamak. Physics of Plasmas 11 (5), 2637.
  * Francisquez et al. (2020) Francisquez, M., Bernard, T. N., Mandell, N. R., Hammett, G. W. & Hakim, A. 2020 Conservative discontinuous Galerkin scheme of a gyro-averaged dougherty collision operator. Nuclear Fusion 60 (9), 096021.
  * Frei et al. (2020) Frei, B. J., Jorge, R. & Ricci, P. 2020 A gyrokinetic model for the plasma periphery of tokamak devices. Journal of Plasma Physics 86 (2), 905860205.
  * Garbet et al. (2004) Garbet, X., Mantica, P., Angioni, C, Asp, E, Baranov, Y, Bourdelle, C, Budny, R, Crisanti, F, Cordey, G, Garzotti, L & others 2004 Physics of transport in tokamaks. Plasma Physics and Controlled Fusion 46 (12B), B557.
  * Giacomin et al. (2020) Giacomin, M., Stenger, L. N. & Ricci, P. 2020 Turbulence and flows in the plasma boundary of snowflake magnetic configurations. Nuclear Fusion 60 (2), 024001.
  * Görler et al. (2011) Görler, Tobias, Lapillonne, Xavier, Brunner, Stephan, Dannert, Tilman, Jenko, Frank, Aghdam, Sohrab Khosh, Marcus, Patrick, McMillan, Ben F, Merz, Florian, Sauter, Olivier & others 2011 Flux-and gradient-driven global gyrokinetic simulation of tokamak turbulence. Physics of Plasmas 18 (5), 056103.
  * Gradshteyn & Ryzhik (2014) Gradshteyn, I. S. & Ryzhik, I. M. 2014 Table of integrals, series, and products. Academic press.
  * Hager et al. (2016) Hager, R., Yoon, E. S., Ku, S., D’Azevedo, Eduardo F., Worley, Patrick H. & Chang, C.-S. 2016 A fully non-linear multi-species Fokker-Planck-landau collision operator for simulation of fusion plasma. Journal of Computational Physics 315, 644.
  * Hahm & Tang (1989) Hahm, T. S. & Tang, W. M. 1989 Properties of ion temperature gradient drift instabilities in h-mode plasmas. Physics of Fluids B: Plasma Physics 1 (6), 1185–1192.
  * Hallatschek & Zeiler (2000) Hallatschek, K. & Zeiler, A. 2000 Nonlocal simulation of the transition from ballooning to ion temperature gradient mode turbulence in the tokamak edge. Physics of Plasmas 7 (6), 2554\.
  * Halpern et al. (2016) Halpern, F., Ricci, P., Jolliet, S., Loizu, J., Morales, J., Mosetto, A., Musil, F., Riva, F., Tran, T.-H. & Wersal, C. 2016 The gbs code for tokamak scrape-off layer simulations. Journal of Computational Physics 315, 388\.
  * Hammett et al. (1993) Hammett, G. W., Beer, M. A., Dorland, W., Cowley, S. C. & Smith, S. A. 1993 Developments in the gyrofluid approach to tokamak turbulence simulations. Plasma physics and controlled fusion 35 (8), 973.
  * Hasegawa et al. (1979) Hasegawa, A., Maclennan, C. G & Kodama, Y. 1979 Nonlinear behavior and turbulence spectra of drift waves and rossby waves. The Physics of Fluids 22 (11), 2122.
  * Hazeltine & Meiss (2003) Hazeltine, R. D. & Meiss, J. D. 2003 Plasma confinement. Courier Corporation.
  * Helander & Sigmar (2002) Helander, P. & Sigmar, D. J. 2002 Collisional transport in magnetized plasmas cambridge university press.
  * Held et al. (2015) Held, E. D., Kruger, S. E., Ji, J.-Y., Belli., E. A. & Lyons, B. C. 2015 Verification of continuum drift kinetic equation solvers in nimrod. Physics of Plasmas 22 (3), 032511\.
  * Hinton & Rosenbluth (1999) Hinton, F. L. & Rosenbluth, M. N. 1999 Dynamics of axisymmetric (E × B) and poloidal flows in tokamaks. Plasma Physics and Controlled Fusion 41 (3A).
  * Hirshman & Sigmar (1976) Hirshman, S. P. & Sigmar, D. J. 1976 Approximate Fokker-Planck collision operator for transport theory applications. Physics of Fluids 19 (10), 1532.
  * Holtkamp (2009) Holtkamp, N. 2009 The status of the iter design. Fusion Engineering and Design 84 (2-6), 98.
  * Idomura et al. (2008) Idomura, Y., Ida, M., Kano, T., Aiba, N. & Tokuda, S. 2008 Conservative global gyrokinetic toroidal full-f five-dimensional vlasov simulation. Computer Physics Communications 179 (6), 391.
  * Idomura et al. (2003) Idomura, Y., Tokuda, S. & Kishimoto, Y. 2003 Global gyrokinetic simulation of ion temperature gradient driven turbulence in plasmas using a canonical maxwellian distribution. Nuclear Fusion 43 (4), 234.
  * Jenko et al. (2000) Jenko, F., Dorland, W., Kotschenreuther, M. & Rogers, B. N. 2000 Electron temperature gradient driven turbulence. Physics of plasmas 7 (5), 1904.
  * Ji & Held (2006) Ji, J.-Y. & Held, E. D. 2006 Exact linearized Coulomb collision operator in the moment expansion. Physics of plasmas 13 (10), 102103.
  * Jorge et al. (2019) Jorge, R., Frei, B. J. & Ricci, P. 2019 Nonlinear gyrokinetic Coulomb collision operator. Journal of Plasma Physics 85 (6), 905850604.
  * Jorge et al. (2017) Jorge, R., Ricci, P. & Loureiro, N. F. 2017 A drift-kinetic analytical model for scrape-off layer plasma dynamics at arbitrary collisionality. Journal of Plasma Physics 83 (6).
  * Jorge et al. (2018) Jorge, R., Ricci, P. & Loureiro, N. F. 2018 Theory of the drift-wave instability at arbitrary collisionality. Physical review letters 121 (16), 165001.
  * Kolesnikov et al. (2010) Kolesnikov, R. A., Wang, W. X., Hinton, F. L., Rewoldt, G. & Tang, W. M. 2010 Drift-kinetic simulation of neoclassical transport with impurities in tokamaks. Physics of Plasmas 17 (2), 022506.
  * Landreman & Ernst (2012) Landreman, M. & Ernst, D. R. 2012 Local and global fokker–planck neoclassical calculations showing flow and bootstrap current modification in a pedestal. Plasma Physics and controlled fusion 54 (11), 115006.
  * Landreman & Ernst (2013) Landreman, M. & Ernst, D. R. 2013 New velocity-space discretization for continuum kinetic calculations and fokker–planck collisions. Journal of Computational Physics 243, 130.
  * Li & Ernst (2011) Li, B. & Ernst, D. R. 2011 Gyrokinetic Fokker-Planck collision operator. Physical Review Letters 106 (19).
  * Madsen (2013) Madsen, J. 2013 Full-f gyrofluid model. Physics of Plasmas 20 (7), 072301.
  * Mandell et al. (2018) Mandell, N. R., Dorland, W. & Landreman, M. 2018 Laguerre–Hermite pseudo-spectral velocity formulation of gyrokinetics. Journal of Plasma Physics 84 (1).
  * Merlo et al. (2016) Merlo, G., Sauter, O., Brunner, S., Burckel, A., Camenen, Y., Casson, F. J., Dorland, W., Fable, E., Görler, T., Jenko, F. & others 2016 Linear multispecies gyrokinetic flux tube benchmarks in shaped tokamak plasmas. Physics of plasmas 23 (3), 032104.
  * Mosetto et al. (2015) Mosetto, A., Halpern, F. D., Jolliet, S., Loizu, J. & Ricci, P. 2015 Finite ion temperature effects on scrape-off layer turbulence. Physics of Plasmas 22 (1), 012308.
  * Nakata et al. (2015) Nakata, M., Nunami, M., Watanabe, T.-H. & Sugama, H. 2015 Improved collision operator for plasma kinetic simulations with multi-species ions and electrons. Computer Physics Communications 197, 61–72.
  * Nunami et al. (2015) Nunami, M., Nakata, M., Watanabe, T.-H. & Sugama, H. 2015 Development of linearized collision operator for multiple ion species in gyrokinetic flux-tube simulations. Plasma and Fusion Research 10, 1403058–.
  * Pan & Ernst (2019) Pan, Qingjiang & Ernst, Darin R 2019 Gyrokinetic landau collision operator in conservative form. Physical Review E 99 (2), 023201.
  * Pan et al. (2020) Pan, Q., Ernst, D. R. & Crandall, P. 2020 First implementation of gyrokinetic exact linearized Landau collision operator and comparison with models. Physics of Plasmas 27 (4), 042307\.
  * Pan et al. (2021) Pan, Q., Ernst, D. R. & Hatch., D. R 2021 Importance of gyrokinetic exact fokker-planck collisions in fusion plasma turbulence. Physical Review E 103 (5), L051202.
  * Paruta et al. (2018) Paruta, P., Ricci, P., Riva, F., Wersal, C., Beadle, C. & Frei, B. 2018 Simulation of plasma turbulence in the periphery of diverted tokamak by using the gbs code. Physics of Plasmas 25 (11), 112301.
  * Romanelli (1989) Romanelli, F. 1989 Ion temperature-gradient-driven modes and anomalous ion transport in tokamaks. Physics of Fluids B: Plasma Physics 1 (5), 1018–1025.
  * Romanelli & Briguglio (1990) Romanelli, F. & Briguglio, S. 1990 Toroidal semicollisional microinstabilities and anomalous electron and ion transport. Physics of Fluids B: Plasma Physics 2 (4), 754–763.
  * Rosenbluth et al. (1972) Rosenbluth, M. N., Hazeltine, R. D. & Hinton, F. L. 1972 Plasma transport in toroidal confinement systems. The Physics of Fluids 15 (1), 116–140.
  * Rosenbluth & Hinton (1998) Rosenbluth, M. N. & Hinton, F. L. 1998 Poloidal flow driven by ion-temperature-gradient turbulence in tokamaks. Physical Review Letters 80 (4), 724–727.
  * Rosenbluth et al. (1957) Rosenbluth, M. N., MacDonald, W. M. & Judd, D. L. 1957 Fokker-Planck equation for an inverse-square force. Physical Review 107 (1), 1.
  * Shimada et al. (2007) Shimada, M., Campbell, D. J., Mukhovatov, V., Fujiwara, M., Kirneva, N., Lackner, K., Nagami, M., Pustovitov, V. D., Uckan, N., Wesley, J. & others 2007 Progress in the iter physics basis-chapter 1: overview and summary. Nuclear Fusion 47, S1.
  * Snider (2017) Snider, R. F. 2017 Irreducible Cartesian Tensors. De Gruyter Studies in Mathematical Physics 1\. De Gruyter.
  * Stegmeir et al. (2019) Stegmeir, A., Ross, A .and Body, T, Francisquez, M., Zholobenko, W., Coster, D., Maj, O., Manz, P., Jenko, F., Rogers, B. N. & others 2019 Global turbulence simulations of the tokamak edge region with grillix. Physics of Plasmas 26 (5), 052517.
  * Sugama et al. (2019) Sugama, H., Matsuoka, S., Satake, S., Nunami, M. & Watanabe, T.-H. 2019 Improved linearized model collision operator for the highly collisional regime. Physics of Plasmas 26 (10), 102108.
  * Sugama et al. (2009) Sugama, H., Watanabe, T.-H. & Nunami, M. 2009 Linearized model collision operators for multiple ion species plasmas and gyrokinetic entropy balance equations. Physics of Plasmas 16 (11), 112503.
  * Tamain et al. (2016) Tamain, P., Bufferand, H., Ciraolo, G., Colin, C., Galassi, D., Ghendrih, P., Schwander, F. & Serre, E. 2016 The tokam3x code for edge turbulence fluid simulations of tokamak plasmas in versatile magnetic geometries. Journal of Computational Physics 321, 606.
  * Villard et al. (2013) Villard, L., Angelino, P., Bottino, A., Brunner, S., Jolliet, S., McMillan, B. F., Tran, T.-M. & Vernay, T. 2013 Global gyrokinetic ion temperature gradient turbulence simulations of ITER. Plasma Physics and Controlled Fusion 55 (7), 074017.
  * Wong (1998) Wong, M. W. 1998 The Weyl Transform. Springer.
  * Xiao et al. (2007) Xiao, Y., Catto, P. J. & Molvig, K. 2007 Collisional damping for ion temperature gradient mode driven zonal flow. Physics of Plasmas 14 (3).
  * Xu & Rosenbluth (1991) Xu, X. Q. & Rosenbluth, M. N. 1991 Numerical simulation of ion-temperature-gradient-driven modes. Physics of Fluids B: Plasma Physics 3 (3), 627.
  * Zeiler et al. (1997) Zeiler, a., Drake, J. F. & Rogers, B. 1997 Nonlinear reduced Braginskii equations with ion thermal dynamics in toroidal plasma. Physics of Plasmas 4 (6), 2134.
  * Zhu et al. (2018) Zhu, B., Francisquez, M. & Rogers, B. N. 2018 GDB: A global 3D two-fluid model of plasma turbulence and transport in the tokamak edge. Computer Physics Communications 232, 46\.


# PeR-ViS: Person Retrieval in Video Surveillance using Semantic Description

Parshwa Shah
School of Engineering and Applied Science, Ahmedabad University
<EMAIL_ADDRESS>
Arpit Garg, Vandit Gajjar
School of Computer Science, The University of Adelaide
{arpit.garg<EMAIL_ADDRESS>
###### Abstract

A person is usually characterized by descriptors like age, gender, height,
cloth type, pattern, color, etc. Such descriptors are known as attributes
and/or soft-biometrics. They link the semantic gap between a person’s
description and retrieval in video surveillance. Retrieving a specific person
with the query of semantic description has an important application in video
surveillance. Using computer vision to fully automate the person retrieval
task has been gathering interest within the research community. However, the
Current, trend mainly focuses on retrieving persons with image-based queries,
which have major limitations for practical usage. Instead of using an image
query, in this paper, we study the problem of person retrieval in video
surveillance with a semantic description. To solve this problem, we develop a
deep learning-based cascade filtering approach (PeR-ViS), which uses Mask
R-CNN [14] (person detection and instance segmentation) and DenseNet-161 [16]
(soft-biometric classification). On the standard person retrieval dataset of
SoftBioSearch [6], we achieve 0.566 Average IoU and 0.792 %w $IoU>0.4$,
surpassing the current state-of-the-art by a large margin. We hope our simple,
reproducible, and effective approach will help ease future research in the
domain of person retrieval in video surveillance. The source code and
pretrained weights available at https://parshwa1999.github.io/PeR-ViS/.

##  1 Introduction

Recently, pedestrian attribute recognition such as age, gender, height, cloth
color, and type, etc. has obtained increasing attention due to its promising
outcomes in applications such as person re-identification, attribute-based
person search, and person retrieval in video surveillance. Nowadays
metropolitan cities are equipped with thousands of surveillance cameras, which
stores a gigantic amount of surveillance data every second. To retrieve a
specific person manually from large-scale videos possibly takes months to
complete. Using computer vision techniques to fully automate the above task
has been gathering interest within the research community. The current trend
mainly solves this task based on image queries, which have major limitations
and might not be suitable for practical usage. Therefore, we studied the
problem of person retrieval with semantic descriptions to face these
limitations. Figure 1 illustrates the example of the person retrieval using a
semantic description.

Figure 1: Person retrieval using a semantic description. Given the semantic
description of a person, our approach PeR-ViS uses each description in a
filtering mode to retrieve the correct person.

Person retrieval with image-based queries is known as Person Search and/or
Person Re-identification in computer vision [38, 20, 33]. Given image query to
network, it finds the similarity between the query and that surveillance
footage. The most identical person than retrieved from the footage according
to similarity score. However, this problem requires at least one image as a
query for the network, which has a major limitation in practice. In many
cases, such as a lost person, there might be only a description provided of
the person(s) appearance available. Facing the limitation of image-based
person retrieval, we propose to use a semantic description for person
retrieval. It does not require a person(s) image to be applied to the network.
The semantic description can also accurately describe the information of the
person(s) appearance.

To use this semantic description for person retrieval, we propose PeR-ViS - a
deep learning-based cascade filtering approach for person retrieval in video
surveillance. The PeR-ViS takes a semantic description and a surveillance
frame as an input and outputs the correct retrieved person with a bounding
box. It uses Mask R-CNN [14] for precise detection and instance segmentation
of every person in the surveillance frame. It uses Height, Torso Module (Color
$>>$ Type $>>$ Pattern), Leg Module (Color $>>$ Pattern) and Gender as cascade
filter. These descriptors are chosen due to their distinguishable capability.
For example, the height descriptor is view and distance invariant, while
predicting color is also invariant to angle and direction [27]. The height
filter is designed using camera calibration parameters, while all other
filters are based on convolutional neural networks: DenseNet-161 [16]. The
complete approach is illustrated in Figure 2, and we will talk about more in
Section 3.

In summary, the main contributions of this paper are four-fold.

  * •

We study the problem of person retrieval with the semantic description in
video surveillance, which often arises in real-life scenarios, but remains
wide open in the research community.

  * •

Mask R-CNN is used by us because of following advantages

    * –

Unnecessary background clutter is removed.

    * –

As accurate segmented boundary provides precise head and feet points. Better
estimation of real-world height can be made.

    * –

It helps to extract accurate patch for Torso and Leg module attribute
classification.

  * •

The estimated height can also be used to distinguish between the standing and
sitting position of the person. This ability narrows down search space for the
person of interest in the standing position.

  * •

We propose a new person retrieval approach (PeR-ViS) which uses cascade
filters of person(s) descriptors to narrow down the search space of detected
people to leave only the target. The approach surpasses the current state-of-
the-art on the SoftBioSearch dataset, achieving 0.566 Average IoU and 0.792 %w
IoU $>$ 0.4.

The rest of this paper is as follows. Section 2 describes work-related to
person search and retrieval in video surveillance. Our approach to PeR-ViS and
its modules is briefly mentioned in Section 3. The experiment, the
implementation details, and the results are described and shown in Section 4.
Section 5 discusses the experimental results and ablation studies. Finally,
Section 6 focuses on the possible future work and concludes the paper.

##  2 Related work

Person Search: A person search is a recently introduced problem, where an
image query is applied to the network and the same/similar person can be
found. Li et al. [34] have proposed a person search task that aims to find a
similar person(s) in the photo-gallery without bounding box annotation. The
respective data is similar to that in the person re-identification. The major
difference is that the bounding-box is unavailable in the person search task.
Moreover, it can also be seen as a task to combine person detection and person
re-identification. There are some other works, which try to search a person
with other modalities of data, such as attribute-based [28, 8], and natural
language-based [18], which are more similar to the problem we aim to tackle in
this task.

Person re-identification: Person search is indeed an extension of the Person
re-identification task [36, 10], which objective is to match a person(s) image
from various cameras within a short span. The problem has drawn much attention
in the computer vision research community since the last decade. Several
datasets have been [11, 15, 17, 19, 30] proposed to tackle the task of person
re-identification. Early person re-identification methods focus on manually
designing distinguishable features [13, 31], and learning distance metrics
[19, 21, 24]. With the growth of deep learning in recent years, many
researchers have proposed several deep learning-based solutions to solve this
task. Li et al. [19] and Ahmed et al. [2] created CNN models for person re-
identification. Both the CNNs uses a pair of cropped person(s) images as input
and utilizes a binary verification loss function to train the different
parameters. Ding et al. [7] and Cheng et al. [3] utilized triplet instances
for training CNNs to minimize the feature distance between the similar person
and maximize the distance between different people. Instead of utilizing the
triplet loss function, Xiao et al. [32] proposed to learn features by
categorizing identities.

Figure 2: Overview of the proposed approach - PeR-ViS. We first apply Mask
R-CNN to the input image for a person(s) accurate detection and instance
segmentation. The detected person(s) then fed into the linear filter, which
ultimately narrows down the search space and leaves only the target at the
end. The filtering sequence follows the given order: Height; Torso cloth
color, type, pattern; Leg cloth color, pattern; and Gender. (Best viewed in
color and magnification.)

Person Retrieval: Before the growth of deep learning in the field of computer
vision, hand-crafted methods had developed algorithms to learn part or local
features. In the work of Gray et al. [12], they have partitioned person(s)
into horizontal stripes to extract color and texture features. Similar
partition has also been used by many other works [20, 25, 23]. We also have
adopted the strategy for extracting torso and leg patches accurately. Some
other works utilize a more refined strategy. In the work of Gheissari et al.
[10], authors have divided person(s) into different triangular parts for
feature extraction. Cheng et al. [4] used a genteel structure to parse the
person(s) into different semantic parts. Das et al. [5] have applied
histograms on the head, torso, and leg portion to extract the spatial
information. The state-of-the-art on most person retrieval datasets is
currently maintained by deep learning algorithms [37]. There are other
methods, which are more or less similar to our work for person retrieval,
however, the strategy of cascade filtering that aims to narrow down the
detected people to leave only the target is our major contribution.

##  3 Proposed approach - PeR-ViS

This section introduces a deep learning-based cascade filtering approach for
person retrieval in video surveillance (PeR-ViS). Figure 2 illustrates the
complete flow diagram of PeR-ViS. Each video surveillance frame is given to
state-of-the-art Mask R-CNN [14] in order to detect and instantiate segment(s)
of person(s). For person(s) head and feet, points are extracted for all
detected and segmented person(s). For height estimation, using the camera
calibration technique, it is calculated based on the height given in the
semantic description. In this complete approach, height acts as a primary
filter to narrow down the search space of person(s) in the frame. In the case
of multiple matches, where more than one person matches the height
description, additional filtering is performed using torso color, type,
pattern; leg color, pattern, and gender. Instance segmentation helps us in
obtaining background free extraction of the patch from the torso and leg. The
number of identities is further narrow down by comparing the semantic
description with the extracted patches. The preciseness of the final output is
further improved by exploiting gender classification. The following sections
describe the process of filter modules for person retrieval.

Height estimation: Person height is view-invariant, which helps to distinguish
between standing and sitting position of the person. Tsai camera calibration
approach [29] is used to estimate detected person(s) height by matching
bounding box coordinates to real-world coordinates. The dataset of
SoftBioSearch [6] provided 6 calibrated cameras for the calculation of real-
world coordinates. Detected person(s) head and feet points are computed from
the instance segmentation, which can be seen in the height filter (Figure 1).
Steps for Computation of height estimation are as follows:

  * •

From the given camera calibration parameters Intrinsic parameters matrix
($I_{m}$), rotation matrix ($R_{m}$) and a translation vector ($t_{v}$) are
computed.

  * •

The respective transformation matrix is computed as $T=I_{m}[R_{m}|t_{v}]$.

  * •

By using radial distortion parameters, head and feet points are undistorted.

  * •

Using inverse transformation of $T$ global coordinate for feet is set to $F=0$
and global coordinates $X$, $Y$ are derived.

  * •

By using $X,Y$ coordinate to compute the F coordinate of the head which also
characterizes height.

Calculated height assists in narrowing down the search space within the test
surveillance frame based on the semantic description (e.g. Average height
(150-170 cm)). After that test surveillance frame now only contains the
person(s) which matches the height’s semantic description. During training
time, annotated head and feet points are used to calculate the height from all
the input surveillance frames of the video sequence. The average height
($H_{avg}$) is computed over all the surveillance frames in a given video
sequence. Over the similar training video sequence, we noticed that the
average height estimated from automated head and feet point is greater than
was $H_{avg}$. This dissimilarity holds the wrong computation, therefore; it
is subtracted from $H_{avg}$ during the testing time to equalize the error.

Figure 3: Extraction of torso and leg region from detected person(s) body.

Torso and Leg Module Filter: Mask R-CNN generates a person(s) label with a
confidence score and instance segmentation. Using the golden ratio for height,
the torso and leg regions are extracted. From the detected bounding box and
instance segmentation, the upper 20-50% region is classified as the torso,
while the 50-70% represents the leg portion of the person(s). The extraction
of these regions is shown in Figure 3. Instance segmentation is utilized to
get the torso and leg portion without the background to refine the cloth
color, pattern, and type classification. Patch for prediction is extracted
from a region marked by points ‘a’, ‘b’, ‘c’, and ‘d’ (ref. Figure 1 \- Torso
and Leg module filter). The extracted patches are used to fine-tune DenseNet -
161 [16], for predicting the confidence score. The SoftBioSearch dataset [6]
annotation contains 13 Torso primary and secondary color, 4 torso cloth type,
8 torso cloth pattern, 13 Leg cloth primary color and secondary color, and 8
leg cloth patterns for each identity. In the case of different similar
matches, the approach will refine the results by using each given modality.
This feature of cascade filtering helps to narrow down the search space.
Furthermore, to get and/or verify the result of the last filter (Leg cloth
pattern), we use the gender filter. We noticed that in some of the cases
gender filter helps to improve the performance when the surveillance frame is
crowded.

Gender Classification: By using height and other semantic descriptions in a
cascade filtering mode, the proposed approach retrieves the person for most
cases. But when multiple matches come after the last filter, the approach uses
Gender as the final filter for retrieving or verifying the result. Full-body
images of male and female categories were used in fine-tuning DenseNet.

##  4 Experiments

This section discusses the details about the overview of the dataset,
performance metric for evaluation, and implementation details. The PeR-ViS
uses Mask R-CNN, which uses the pre-trained weights of Microsoft COCO [22]
dataset for detection and instance segmentation. The Mask R-CNN model uses the
ResNet-101 FPN as the backbone architecture, which has achieved Average
Precision (AP) of 35.7 on the COCO test-dev set.

###  4.1 Dataset overview

Our work uses the SoftBioSearch database [6], which uses 6 stationary
calibrated cameras and consists of 110 unconstrained training video segments.
Each of these 6 cameras is annotated with Tsai’s camera calibration technique
[29]. Every video sequence in the training set is labeled with a set of
semantic descriptions to describe the person(s) identity. In addition, with
the provided description, nine key-points were also annotated in the case to
use human key-point estimation and exploits with the person retrieval task.
More precise details on the camera calibration and human body key-point can be
found in [6].

The test set consists of further 41 person identities taken from 4 of the 6
cameras (Here camera numbers 1 and 6 are not used) used in collecting the
training set. Compilation and annotation of the test set follow the same
instructions of the train set to ensure parity, with at least the first 30
surveillance frames of each video sequence reserves to allow the person(s)
identity to fully enter in the camera view.

For more details on the performance evaluation, the test person identities
were separated into very easy, easy, medium, and hard.

The given labels are defined below:

  * •

Very Easy: randomly populated scenario, no complex factors, and target person
identity clearly visible.

  * •

Easy: Scenario consists of one or more people, but the person(s) identity is
clearly distinguishable.

  * •

Medium: From the following factors one of the factors will be present: Similar
type of the identity present in the scenario, Very heavy occlusion with the
target, a crowded scene.

  * •

Hard: Two or more of the above factors present in the scenario. In the test
set out of 41 person identities, 6 are labeled as very easy, 13 as easy, 12 as
the medium, and 10 as hard.

###  4.2 Performance metric

As described in SoftBioSearch dataset, the metrics use the Intersection Over
Union ($IoU$) given by Eq. 1. An $IoU_{avg}$ is calculated per video sequence,
and video sequences results are averaged over all video sequences to obtain a
final accuracy measure. $IoU=\frac{D\cap GT}{D\cup\ GT}$ Where D is the
obtained bounding-box from the approach and GT is the Ground Truth bounding
box.

###  4.3 Implementation details

The fine-tuning is accomplished on a workstation with an Intel Xeon core
processor and accelerated by NVIDIA TitanX 12 GB GPU. All experiments run in
Tensorflow 1.8 [1].

Data Augmentation: In order to achieve generalization of training data for
improved performance and robustness data augmentation is used. E.g. In the
training set after removing the surveillance frames with partial occlusion,
the final set contains 8577 images from 110 subjects. Thus training DenseNet
with only 8577 images may result in over-fitting, which is avoided using a
data augmentation scheme. Each training frame is horizontally and vertically
flipped, rotated with 10 angles $1^{o}$,$2^{o}$, $3^{o}$, $4^{o}$, $5^{o}$,
$-1^{o}$, $-2^{o}$, $-3^{o}$, $-4^{o}$, $-5^{o}$ and brightness increased with
a gamma factor of 1.5.

DenseNet training for Cloth Color, type, pattern; and gender: Cloth Color,
Type, Pattern, and gender models are fine-tuned using DenseNet - 161 which is
pre-trained on the ImageNet dataset. The SoftBioSearch dataset consists of
1704 patches divided into 13 torso and leg primary and secondary colors; 8
torso and leg patterns; and 4 torso type. Extra patches for training these
attributes are extracted using 4 human key-points provided in annotations
(Left-right shoulder and left-right waist). In order to deal with light
changes, these patches are augmented by increasing the brightness with a gamma
factor of 1.5. Thereafter, almost 17000 patches are generated and further
divided into 80-20% train and validation set.

All the networks for the torso and leg module are trained using mini-batch
stochastic gradient descent (SGD). Due to the computation cost, the fine-
tuning strategy is the same for all the descriptors. The networks were trained
for 20 epochs with a learning rate is set to 0.001, dropout set to 0.35, and
effective mini-batch size is set to 32. Table 1 shows the validation accuracy
of different descriptors.

For the gender classification, the initial data augmentation generated 105980
images for training gender descriptor, which is almost 13 times larger than
the original training set (8577). 20% of total images were used for
validation. For the gender fine-tuning, the network was trained for 30 epochs
with a learning rate of 0.01, dropout rate of 0.25, and effective batch size
is set to 64.

Descriptor | Validation Accuracy
---|---
Torso Color | 81.29%
Torso Type | 79.14%
Torso Pattern | 76.5%
Leg Color | 71.52%
Leg Pattern | 72.5%
Gender | 77.79%
Table 1: Accuracy results on the validation set of different semantic
descriptor based on the above hyper-parameter setting.

##  5 Experimental results

This section covers the qualitative and quantitative experimental results
derived from a test set of 41 person identities. Overall results are shown in
Table 2, including a comparison with the baseline algorithm and current state-
of-the-art methods. Form this we can see that our approach - PeR-ViS
outperformed the current state-of-the-art by a large margin.

Approach | Average IoU | %w $IoU>0.4$
---|---|---
Baseline [6] | 0.290 | 0.669
Galiyawala et al. [9] | 0.363 | 0.522
Schumann et al. [26] | 0.503 | 0.759
Yaguchi et al. [35] | 0.418 | -
Yaguchi et al. [35] | 0.462 | -
Yaguchi et al. [35] | 0.511 | 0.669
Ours | 0.566 | 0.792
Table 2: Overall IoU of different methods on the test set.

Figure 4: Performance broken down. (a) by sequence difficulty and compared
with Average $IoU$. (b) by sequence difficulty and compared with %w $IoU>0.4$.
(from top to bottom - a, b)

Considering the algorithms used, [35], [9], and [26] uses deep learning method
that deploys a CNNs for detecting person(s) identity in the surveillance
frame. The baseline of [6] uses an avatar, which is a non-deep learning
approach, that is constructed from the input semantic query to drive a
gradient descent search. It is noticeable that both [9] and [6] uses very few
descriptors, while [35] and [26] uses the full set of available descriptors,
likely improving the performance.

Figure 5: Per Sequence Performance for test set.

To further analyze performance, we break the results by difficulty level
(Figure 4) described in Section 4.1 and into individual video sequences
(Figure 5). From Figure 4 it is clear performance decrease with an increase in
difficulty. The “Very Easy” video sequences mainly contain only a single
moving person identity, and as seen in the figure all other methods perform
well on these video sequences. As the difficulty level increases, [9] suffers
a very much decrease in the performance compared to others. A very slight
difference can be spotted in the performance of [35] and [26]. Both achieve
very similar IoU’s, but as compared with our approach their algorithms fail to
perform well in the hard video sequences. Thus indeed our approach performs
very well on medium and hard sequences and breaks the current state-of-the-art
by a margin of more than 5%. Figure 5 shows the performance of PeR -ViS on
each video sequence. It is noticeable that performance varies across each
video sequence and several sequences pose a challenge for retrieval. From
Figure 4 as compared with [26], the authors use a tracking approach, which
helps to reduce the error that may impact non-tracking approaches such as ours
and others. From this, it is noticeable that ours and [35] have the
inclination to either detect a person’s identity very precisely or very badly.

Surprisingly, two of the video sequences (20 and 39) contain very less crowd,
but very complicated by having a very similar person identity. The other two
video sequences consist high amount of crowd, and we see almost all the
systems are having a decrease in performance. To further showcase the person
retrieval results, Figure 6 and Figure 7 show some of the examples of True
positive and False negative cases based on semantic descriptions.

(a) Person retrieved using only height.

(b) Person retrieved using height and torso module filter.

(c) Person retrieved using height, torso, leg and gender module filter.

Figure 6: True positive cases of person retrieval with semantic description.
(Best viewed in color and magnification.) Figure 7: False negative cases of
person retrieval.(a) incorrect color classification with multiple person(s),
(b) multiple person(s) with occlusion, (c) multiple person(s) with same torso
color and height class and (d) person detection fails. The supplementary
material contains more true positive results. (from left to right – a, b, c,
d) (Best viewed in color and magnification.)

Figure 6 shows true positive cases of person retrieval using semantic
descriptions. In Figure 6, images from left to right indicate the input test
frame, the output of the height module filter, the output of Torso and Leg
module filter and gender module filter respectively. Figure 7 shows the
results when the approach fails to retrieve the person correctly. It also
indicates that the dataset is created in challenging conditions. In Figure 7,
the approach fails due to following results: (a) Multiple persons with the
same torso color yields incorrect color classification, (b) Multiple person(s)
with occlusion, (c) Same height class appears when multiple persons comes in
the surveillance frame and (d) Mask R-CNN fails for person detection.

###  5.1 Ablation experiments

We run some ablations to analyze our PeR-ViS approach

Choice of classification network: To test the influence of a classification
network on the proposed approach, we have formed five video sequences from
test set consists of Easy, Medium, and Hard category. We have used AlexNet,
VGG-16, and ResNet-152 for ablation experiment and average IoU is used for
evaluation. All the networks perform well on the easy video sequences. Next on
the medium sequences, where background clutter and less occlusion are present,
AlexNet and VGG-16 perform poorly. Here ResNet-152 shows equivalent
performance to DenseNet-161. The true performance of DenseNet-161 is noticed
in the hard category, where it achieves excellent performance. The Table 3
shows the result of IoU using different networks.

| Very Easy | Easy | Easy | Medium | Hard
---|---|---|---|---|---
AlexNet | 0.567 | 0.534 | 0.523 | 0.325 | 0.183
VGG-16 | 0.641 | 0.621 | 0.615 | 0.336 | 0.237
ResNet-152 | 0.742 | 0.712 | 0.64 | 0.492 | 0.36
DenseNet-161 | 0.762 | 0.733 | 0.642 | 0.582 | 0.461

Table 3: This table shows the IoU resulton 5 video sequences (Sequence Number
4, 13, 21, 23, and 28) using different network architecture in our approach.

More descriptors: In our work, we have used Height; Torso and Leg module; and
Gender descriptors. The choice of these descriptors is purely based on
perceptive ability. We have tried to add more descriptors such as Age, Shoe
color, Luggage, and Human body’s build but none of the modality was capable to
improve the performance reported in our work.

##  6 Discussion and Conclusion

The proposed approach - PeR-ViS retrieves the person in video surveillance
based on the semantic description of Height; Torso Cloth color, type, and
pattern; Leg Cloth color and pattern; and Gender. The major benefit from this
filtering sequence is that Height, and Torso Color, Type, and Pattern are
easily differentiable compare to other descriptors, where the heavy crowd is
present, leg patch won’t easily extract. Thus the choice of this filtering
sequence is most important in our work. Also, instance segmentation allows
precise height estimation and accurate color patch extraction from the torso
and leg. Thus, our algorithm achieves an average IoU of 0.566 and %w $IoU>0.4$
of 0.792, surpassing the current state-of-the-art by a large margin. We have
also provided the code snippet up-to Torso type filter (Height $>>$ Torso
Cloth Color $>>$ Torso Cloth Type and Gender for verification). The possible
future work will now focus on how to improve the results by incorporating
human pose estimation and other descriptors and investigate the architecture
for generalization in real-world scenarios. Furthermore, the tracking approach
is also useful when the person is retrieved with Height and/or Torso module
filter, which will essentially lower the computation usage.

## Acknowledgements

We would like to thank anonymous reviewers for providing us their valuable
feedback on our paper. We would like to express our deep gratitude to Dr.
Mehul S. Raval, Dr. Hiren Galiyawala and Mr. Kenil Shah for providing useful
comments and discussion. We would also like to thank Ms. Ayesha Gurani, Mr.
Viraj Mavani and Mr. Yash Khandhediya for their help with manuscript

## References

  * [1] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al.  Tensorflow: A system for large-scale machine learning.  In 12th $\\{$USENIX$\\}$ symposium on operating systems design and implementation ($\\{$OSDI$\\}$ 16), pages 265–283, 2016.
  * [2] Ejaz Ahmed, Michael Jones, and Tim K Marks.  An improved deep learning architecture for person re-identification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3908–3916, 2015.
  * [3] De Cheng, Yihong Gong, Sanping Zhou, Jinjun Wang, and Nanning Zheng.  Person re-identification by multi-channel parts-based cnn with improved triplet loss function.  In Proceedings of the iEEE conference on computer vision and pattern recognition, pages 1335–1344, 2016.
  * [4] Dong Seon Cheng, Marco Cristani, Michele Stoppa, Loris Bazzani, and Vittorio Murino.  Custom pictorial structures for re-identification.  In Bmvc, volume 1, page 6, 2011.
  * [5] Abir Das, Anirban Chakraborty, and Amit K Roy-Chowdhury.  Consistent re-identification in a camera network.  In European conference on computer vision, pages 330–345. Springer, 2014.
  * [6] Simon Denman, Michael Halstead, Clinton Fookes, and Sridha Sridharan.  Searching for people using semantic soft biometric descriptions.  Pattern Recognition Letters, 68:306–315, 2015.
  * [7] Shengyong Ding, Liang Lin, Guangrun Wang, and Hongyang Chao.  Deep feature learning with relative distance comparison for person re-identification.  Pattern Recognition, 48(10):2993–3003, 2015.
  * [8] Rogerio Feris, Russel Bobbitt, Lisa Brown, and Sharath Pankanti.  Attribute-based people search: Lessons learnt from a practical surveillance system.  In Proceedings of International Conference on Multimedia Retrieval, pages 153–160, 2014.
  * [9] Hiren Galiyawala, Kenil Shah, Vandit Gajjar, and Mehul S Raval.  Person retrieval in surveillance video using height, color and gender.  In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–6. IEEE, 2018.
  * [10] Niloofar Gheissari, Thomas B Sebastian, and Richard Hartley.  Person reidentification using spatiotemporal appearance.  In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1528–1535. IEEE, 2006.
  * [11] Mengran Gou, Srikrishna Karanam, Wenqian Liu, Octavia Camps, and Richard J Radke.  Dukemtmc4reid: A large-scale multi-camera person re-identification dataset.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 10–19, 2017.
  * [12] Douglas Gray and Hai Tao.  Viewpoint invariant pedestrian recognition with an ensemble of localized features.  In European conference on computer vision, pages 262–275. Springer, 2008.
  * [13] Omar Hamdoun, Fabien Moutarde, Bogdan Stanciulescu, and Bruno Steux.  Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences.  In 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras, pages 1–6. IEEE, 2008.
  * [14] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
  * [15] Martin Hirzer, Csaba Beleznai, Peter M Roth, and Horst Bischof.  Person re-identification by descriptive and discriminative classification.  In Scandinavian conference on Image analysis, pages 91–102. Springer, 2011.
  * [16] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [17] S Karanam, M Gou, Z Wu, A Rates-Borras, O Camps, and RJ Radke.  A systematic evaluation and benchmark for person re-identification: features, metrics, and datasets.
  * [18] Shuang Li, Tong Xiao, Hongsheng Li, Bolei Zhou, Dayu Yue, and Xiaogang Wang.  Person search with natural language description.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1970–1979, 2017.
  * [19] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang.  Deepreid: Deep filter pairing neural network for person re-identification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 152–159, 2014.
  * [20] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li.  Person re-identification by local maximal occurrence representation and metric learning.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2197–2206, 2015.
  * [21] Shengcai Liao and Stan Z Li.  Efficient psd constrained asymmetric metric learning for person re-identification.  In Proceedings of the IEEE International Conference on Computer Vision, pages 3685–3693, 2015.
  * [22] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In European conference on computer vision, pages 740–755. Springer, 2014.
  * [23] Andy J Ma, Pong C Yuen, and Jiawei Li.  Domain transfer support vector ranking for person re-identification without target camera label information.  In Proceedings of the IEEE international conference on computer vision, pages 3567–3574, 2013.
  * [24] Sakrapee Paisitkriangkrai, Chunhua Shen, and Anton Van Den Hengel.  Learning to rank in person re-identification with metric ensembles.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1846–1855, 2015.
  * [25] Bryan James Prosser, Wei-Shi Zheng, Shaogang Gong, Tao Xiang, and Q Mary.  Person re-identification by support vector ranking.  In BMVC, volume 2, page 6, 2010.
  * [26] Arne Schumann, Andreas Specker, and Jürgen Beyerer.  Attribute-based person retrieval and search in video sequences.  In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–6. IEEE, 2018.
  * [27] Priyansh Shah, Mehul S Raval, Shvetal Pandya, Sanjay Chaudhary, Anand Laddha, and Hiren Galiyawala.  Description based person identification: use of clothes color and type.  In National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics, pages 457–469. Springer, 2017.
  * [28] Chi Su, Shiliang Zhang, Junliang Xing, Wen Gao, and Qi Tian.  Deep attributes driven multi-camera person re-identification.  In European conference on computer vision, pages 475–491. Springer, 2016.
  * [29] Roger Tsai.  A versatile camera calibration technique for high-accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses.  IEEE Journal on Robotics and Automation, 3(4):323–344, 1987.
  * [30] Taiqing Wang, Shaogang Gong, Xiatian Zhu, and Shengjin Wang.  Person re-identification by discriminative selection in video ranking.  IEEE transactions on pattern analysis and machine intelligence, 38(12):2501–2514, 2016.
  * [31] Xiaogang Wang, Gianfranco Doretto, Thomas Sebastian, Jens Rittscher, and Peter Tu.  Shape and appearance context modeling.  In 2007 ieee 11th international conference on computer vision, pages 1–8. Ieee, 2007.
  * [32] Tong Xiao, Hongsheng Li, Wanli Ouyang, and Xiaogang Wang.  Learning deep feature representations with domain guided dropout for person re-identification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1249–1258, 2016.
  * [33] Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, and Xiaogang Wang.  End-to-end deep learning for person search.  arXiv preprint arXiv:1604.01850, 2(2), 2016.
  * [34] Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, and Xiaogang Wang.  Joint detection and identification feature learning for person search.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3415–3424, 2017.
  * [35] Takuya Yaguchi and Mark S Nixon.  Transfer learning based approach for semantic person retrieval.  In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1–6. IEEE, 2018.
  * [36] Wojciech Zajdel, Zoran Zivkovic, and Ben JA Krose.  Keeping track of humans: Have i seen this person before?  In Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pages 2081–2086. IEEE, 2005.
  * [37] Liang Zheng, Yi Yang, and Alexander G Hauptmann.  Person re-identification: Past, present and future.  arXiv preprint arXiv:1610.02984, 2016.
  * [38] Wei-Shi Zheng, Shaogang Gong, and Tao Xiang.  Person re-identification by probabilistic relative distance comparison.  In CVPR 2011, pages 649–656. IEEE, 2011.


# A Simplified Framework for Air Route Clustering Based on ADS-B Data

Quan Duong
ICT Department
John von Neumann Institute
Ho Chi Minh, Vietnam
<EMAIL_ADDRESS>
Tan Tran
ICT Department
John von Neumann Institute
Ho Chi Minh, Vietnam
<EMAIL_ADDRESS>
Duc-Thinh Pham
Air Traffic Management Research Institute
School of Mechanical and Aerospace Engineering
Nanyang Technological University
Singapore, Singapore
<EMAIL_ADDRESS>
An Mai
ICT Department
John von Neumann Institute
Ho Chi Minh, Vietnam
<EMAIL_ADDRESS>
###### Abstract

The volume of flight traffic gets increasing over the time, which makes the
strategic traffic flow management become one of the challenging problems since
it requires a lot of computational resources to model entire traffic data. On
the other hand, Automatic Dependent Surveillance - Broadcast (ADS-B)
technology has been considered as a promising data technology to provide both
flight crews and ground control staff the necessary information safely and
efficiently about the position and velocity of the airplanes in a specific
area. In the attempt to tackle this problem, we presented in this paper a
simplified framework that can support to detect the typical air routes between
airports based on ADS-B data. Specifically, the flight traffic will be
classified into major groups based on similarity measures, which helps to
reduce the number of flight paths between airports. As a matter of fact, our
framework can be taken into account to reduce practically the computational
cost for air flow optimization and evaluate the operational performance.
Finally, in order to illustrate the potential applications of our proposed
framework, an experiment was performed using ADS-B traffic flight data of
three different pairs of airports. The detected typical routes between each
couple of airports show promising results by virtue of combining two indices
for measuring the clustering performance and incorporating human judgment into
the visual inspection.

###### Index Terms:

trajectory clustering, unsupervised learning, ads-b, cluster validity index,
visual inspection

††publicationid: pubid:  Proceedings of The 13th International Conference on
Computing and Communication Technology (RIVF 2019), Da Nang, Vietnam.
Copyright 2019 by the author(s)  ††publicationid: pubid: Proceedings of the
$13^{th}$ International Conference on Computing and Communication Technology
(RIVF 2019), Danang, Vietnam. © 2019 by the author(s)

##  I Introduction

Together with the continuously growing need of traveling by aviation, one of
the recent most challenging issue arise in air traffic management is flight
arrivals delayed in almost all airports in the world. Some of the causes of
flight delays or cancellation are maintenance problems related to the
aircraft, fueling, inclement weather, etc., and more importantly airline
glitches is one of the the top cause of flight delays. In the attempt to
tackle partly the airline glitches issues, people tends to address to a more
specific problem of route optimization since it’s seen that a non-optimal
flight plan route may cause the airline glitches and indirectly cause the
flight delay. From this aspect, we propose a simplified framework which
leverages the advantage of machine learning to cluster the air route based on
trajectory clustering and ADS-B data to support the optimization for the route
schedule in air traffic management. Herein, ADS-B data is a kind of data
generated by a new surveillance technology, in which the position of an
aircraft is determined via satellite navigation and also periodically
broadcasted, enabling it to be tracked in real time. ADS-B requires no
external input since it depends only on the signal from the navigation system
of the aircraft, and hence is more and more promising to provide significant
and simplified operational enhancements to military and civilian applications.
However, to our best knowledge, there are very few studies until now on
leveraging this kind of data in advanced studies from machine learning/AI
perspectives for air traffic management. Therefore, there may be a big chance
to open up many interesting research on this field of application. From the
machine learning perspectives, recently people realizes that there are many
rooms for development in trajectory clustering problems due to rapid
improvements in satellites and tracking facilities which make it possible to
collect a large amount of trajectory data of moving objects (for e.g.,
hurricane track data, animal movement data, flight data, etc.). To study the
trajectory clustering of moving objects, Gaffney and the team[1] have proposed
a mixture regression model-based trajectory clustering algorithm, in which
cluster memberships are determined by using EM algorithm. In [2], the authors
have presented a framework to classify vehicle motion trajectories based on
hierarchical clustering; including two steps: trajectories preprocessing and
resampling at the first step, then carrying out the trajectories spectral
clustering based on similar spatial patterns at the second step. To provide a
holistic understanding and deep insight into this interesting topic, a
comprehensive survey of the development of trajectory clustering was proposed
in [3]. From this, the authors group the existing trajectory clustering
methods into three categories: unsupervised, supervised and semi-supervised
algorithms based on machine learning perspectives. They also show the
appearance of different trajectory data in many modern intelligent systems for
surveillance security, abnormal behavior detection, crowd behavior analysis,
and traffic control, which has attracted growing attention.

Regarding to the data sources, Radar Track data was often used in various
works from the literature (see in [4], [5], [6], and [7]). The other work from
Bombelli and the team [8] used the historical data containing the set of
Future ATM Concept Evaluation Tool (FACET) Track (TRX) file. And from Yulin
Lui [9] work, they used the data of the flights’ information from FAA Traffic
Flow Management System and Aviation System Performance Metric. However, as we
know there are recently three publications ([10], [11], and [12]) in the
Twelfth USA/Europe Air Traffic Management Research and Development Seminar
(ATM2017) shared their studies on ADS-B data. From the work of Junzi and the
team [11], they leveraged the large amount of trajectory data for extracting
different aircraft performance parameters and they generated different data
mining models corresponding to these parameters. While Dhief and the team [10]
makes use of the ADS-B based systems and can prove their effectiveness in
oceanic area. The United States have plans for mandating ADS-B Out by 2020 for
all airplanes, both air transport and general aviation, and FAA expects to
improve the operational performance by using ADS-B based systems. These kinds
of system are increasing around the world and applying ADS-B data is an
essential requirement for all aircraft operating on the European, Canadian,
and Australian in NAS since 2015 [10]. Besides, there are several
investigations about the quality of ADS-B data such as [13], [14], [15], [12].
Busyairah and the team [14] proposed a framework for evaluating ADS-B data in
the London Terminal Maneuvering Area, the result showed that 66.7% of aircraft
meet the requirement in term of accuracy, integrity, latency, availability,
and update rate. Especially, Rekkas and the team [15] and Barsheshat [13]
showed similar results, in which their evaluation results considering ADS-B
performance are very positive. Barsheshat [13] also stated that implementing
an ADS-B system provides many benefits, including the reduction for the need
of maintaining and/or upgrading radar infrastructure.

For the behind techniques, an extensive comparative study of cluster validity
indices has been carried out by Arbelaitz and the team [16]. They experimented
with 30 different cluster validity indices with the goal is to choose the best
index for each individual application. From their work, the authors also
recommend to employ several indices at the same time to receive the robust
results. Regarding to trajectory clustering evaluation, most of the works
([5], [7],[9], and [17]) preferred to use Silhouette score alone for choosing
the best number of clusters. Another combination from the work of Bombelli and
the team [8] leveraged three different indices which is average Silhouette
index, Davies-Boulding, and Dun index. And the other work from Maraya and the
team [4] combined Silhouette score and Davies-Bouldin (DB) score.

Inspired by the above results, we propose to leverage the ideas from these
analysis in a combination with a simplified framework, in order to adapt for
the case of ADS-B data in air route clustering problem. The paper is organized
as follows: In Section I, we discussed about the motivation and previous works
related to this paper. Then the Section II presents our proposed framework, in
which the considered ADS-B data and appropriate techniques involving in the
step of data preprocessing will be covered respectively in Section II-A and
II-B. The experimental design and corresponding results taking into account
real ADS-B data will be placed in Section III. Finally, we come to a
conclusion and future works in Section IV.

##  II A PROPOSED FRAMEWORK

Figure 1: The detail flow of our proposed framework for air route clustering
based on ADS-B data

.

TABLE I: The Benchmark Methodologies Step | Methods
---|---
Trajectory data | Radar track
Distance between curve | Fréchet distance
Clustering | DBSCAN
Decide Number of Cluster | Silhouette Score
TABLE II: Trajectories Flow Clustering Description Step | Description
---|---
1.1 Traffic Data Extraction |  Received raw traffic data in JSON format
1.2 Data Standardization |  Group traffic data to identified flights and convert to CSV format
2.1 Flight Traffic Filtering |  Filter out the unwanted airports, remove flights that land or depart outside of terminal, and drop the duplicated data points
2.2 Flight Traffic Interpolation |  Convert individual flight’s tracks into vectors of equal length. Apply the cubic spline [18] to interpolate the spaced data points
2.3 Dissimilarity Matrix Construction |  Measure the dissimilarities between flight paths with the same number of features by computing Fréchet distance
3.1 Parameters Generation |  The clustering algorithm replied on two input parameters are $\varepsilon$ (a distance threshold) and MinPts (a minimum number of points). 100 values of $\varepsilon$ were generated and MinPts was retrieved by observations
3.2 Flight Path Clustering |  Apply a density-based clustering algorithm called DBSCAN with defined MinPts and $\varepsilon$
3.3 Quality of Clusters Measurement |  Silhouette and Davies-Bouldin indices were combined for measuring the clustering performance
3.4 Log files Storage |  Record all the performances and were produced for each parameter
4.1 Clustering Result |  Visualize the detected clusters along with original and interpolated trajectories
4.2 Visual Analysis |  Analyze the clustering outcome with the recommended ranking from step 3.3, 3.4, and incorporate with human judgment
4.3 Decision Making |  Make the decision by virtue of domain expert

For the main technical part of this paper, we perform the trajectory
clustering based on DBSCAN [19] approach and ADS-B data, then combine two
different evaluation indices (Silhouette and DB) for the clustering outputs,
which were used to support the visual inspection step. The detail flow of our
framework are shown in Figure 1, in which the flow composes four different
phases including Data Extraction, Data Processing, Clustering, and Visual
Inspection. Additionally, the detail description of involved techniques for
each step in the framework can be found in the Table II. Finally, for a
comparison, all experiments are carried out by using the same data, on
specific dates, and for particular pairs of origin and destination airports.
For a benchmark, we re-implement the route clustering framework from Adria
[17], which is summarized the used methodologies in the Table I, considering
the same assumption for both of their system and ours.

###  II-A Data Description

(a) Original Trajectories

(b) Interpolated Trajectories

Figure 2: The Visualization of Trajectories in Data Processing Step performed
by our framework. Figures 2(a) is done by using original trajectories up to
2057 spaced points. Figures 2(b) showed the interpolated trajectories with
only 100 spaced points

Figure 3: The result from benchmark method discussed in [17], adapted to ADS-B
data. Here, each color represents for each cluster while the grey color is
always used to represent for detected outlier trajectories

(a) Best air route clustering result based on single Silhouette index

(b) Best air route clustering result based on Silhouette-DB index

(c) Best air route clustering result based on Visual Inspection step

Figure 4: Our framework’s result for the detected clusters from different
scenarios, in case of Sydney-Suvarnabhumi airports

All flights’ trajectories data related to this paper are crawled from
FlightAware111https://flightaware.com, the company operates a website and
mobile application which offers free flight tracking services for both private
and commercial air traffic. FlightAware produces FlightFeeder which is known
as a network of ADS-B receiver. The message consisting of the ADS-B data is
received from airplane transponders directly via a small antenna. Then the
ADS-B data is extracted from the received messages and uploaded to
FlightAware’s server over any available internet connection. For carrying out
the analysis, we are going to employ only the flight trajectory data from
three different Origin and Destination pairs of airport, which are shown
clearly in the Table III. The chosen flights for this analysis are collected
for entire January 2017.

Here, the whole process of data generation involves of satellites,
transmitters, and receivers. The airplane determines its own position and
velocity by communicating with satellites via GNSS (global navigation
satellite system) or GPS receivers. Then it broadcasts the its position and
velocity via ADS-B Out to other airplanes and ground ADS-B receivers.
Ultimately, the receivers around the world send this information to
FlightAware’s servers over a real-time connection.

###  II-B Data Processing

After receiving raw traffic data in JSON format, firstly the data
standardization steps were employed by grouping traffic flights and converting
them to CSV format, that can be consumed for other tasks such as EDA
(exploratory data analysis), data processing, and data clustering. Secondly,
in order to make sure the clustering algorithm consuming qualified data, the
flight traffic filtering step was executed, including filter out the unwanted
pairs of origin and destination terminals, remove flights that land or depart
at the coordinates outside of terminals, and drop the duplicated data points.
The Figure 2(a) show visually the original trajectories after filtering step.
Finally, by virtue of flight traffic interpolation, each individual flight’s
trajectories will be converted into vector of equal length. We then apply the
cubic spline interpolation [18] to the spaced data points with the assumption
that the spatial and temporal dimensions are used here in the phase. The
Figure 2(b) then visualize the interpolated trajectories, which are able to
represent the entire flight’s path with a pre-defined number of spaced points.

##  III Experimental Results

This section intents to show the major results obtained from reimplementing
the benchmark method from Aria [17] in case of our adaptation for ADS-B data,
and incorporating our framework’s application with human intervention in the
visual inspection step. Specifically, a density-based clustering algorithm
called DBSCAN is taken into account in the Flight Path Clustering step. The
DBSCAN algorithm receives two primary parameters (MinPts and $\epsilon$) to
cluster the trajectory data, where the MinPts is the minimum number of data
points belong to a cluster determined by the actual observation, and
$\epsilon$ is a distance threshold. In this experimental setting, we generated
100 values for $\epsilon$ combined with the real observed MinPts value for
running the DBSCAN algorithm. The best parameters is determined by using two
evaluation stages, quantitative metrics including Silhouette and DB (known as
cluster validity indices), and a qualitative validation in visual inspection
phase.

TABLE III: Origin Destination Pair Summary Codename | Airports | Cities
---|---|---
YBBN-WSSS | Brisbane-Changi | Brisbane-Singapore
YSSY-VTBS | Sydney-Suvarnabhumi | Sydney-Bangkok
NZCH-WSSS | Christchurch-Changi | Christchurch-Singapore

In Figure 2, we make a visual comparison of the flight paths based on original
trajectories and interpolated trajectories from three different pairs of
airports (see Table III), in which you can see that only using 100 spaced
points we could describe the whole flight paths between two airports. Next,
the Figure 3 shows the final clusters of flight paths for three different
pairs of airports, in which the highest Silhouette score is selected as best
result. Here, each color represents for each cluster while the grey color is
always used to represent for detected outlier trajectories in this analysis.
In our framework, to obtain more robust results we combine two different
indices for measuring the performance of clustering algorithm. In particular,
the Silhouette [20] and Davies-Bouldin (DB) [21] indices are employed
altogether to select the best clustering result, in the event of these two
indices all agree in choosing the most appropriate route clusters that
supports strongly for decision making. On the other hand, when the Silhouette
index and combined Silhouette-DB index show different rankings (see in Figure
4(a) and Figure 4(b)), the visual inspection step will be activated to help
the user makes the final decision on air route clustering (see Figure 4(c)).

(a) Aggregated Routes in case of Christchurch-Changi airports

(b) Aggregated Routes in case of Brisbane-Changi airports

(c) Aggregated Routes in case of Sydney-Suvarnabhumi airports

Figure 5: Aggregated Routes as a final output of our framework

Last but not least, after clustering the routes into particular clusters, we
then employ a simple route matching technique to aggregate all flights’ paths
in each cluster into unique route that represents for the air route of each
pair of airports (see the Figure 5). This visualization, once employed in real
application, will act as a visual suggestion for Airline Route Planning. In
summary, a statistical report of the most important information from our
framework are put in the Table IV, taking into account the experiments in
three pairs of airports.

TABLE IV: The Summary of Proposed Framework Pair of Airports | NZCH-WSSS | YBBN-WSSS | YSSY-VTBS
---|---|---|---
No. Flight Path | 43 | 152 | 94
No. Original Trajectories | 65102 | 169545 | 135586
No. Interpolated Trajectories | 4300 | 15200 | 9400
MinPts | 2 | 30 | 3
Epsilon | 0.523 | 1.583 | 0.603
Noise Percentage (%) | 30.00 | 2.98 | 18.87
No. Clusters via Quantitative Metric | 4 | 2 | 2
No. Clusters via Visual Inspection | 4 | 2 | 3

##  IV Conclusions

Based on the potentials and advantages of ADS-B data ,we propose in this paper
a simplified and workable framework for air trajectories clustering which is
one of the essential parts in the air management process. To our best
knowledge, there is a lack of in-depth analytical study from machine learning
based approach, with emphasis on this promising data technology. The
experimental results, carried out using the data of three different pairs of
airports, can show the effectiveness of our proposed framework, as well as the
clustering results based on the combination of two different metrics. Finally,
from using the interpolation considered in the framework, it is able to reduce
significantly the complexity of data processing, which is very important in
real application. The whole framework’s source code is made available in
github222https://github.com/quandb/atc. In this direction for the future
research, we are going to consider the framework’s extension to generate
predictive capabilities in measuring the operation performance in air traffic
management. Furthermore, we also aim to provide guidance and auto-generate
inputs for real time decision support system.

## References

  * [1] S. Gaffney and P. Smyth, “Trajectory clustering with mixtures of regression models,” in _Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , ser. KDD ’99. New York, NY, USA: ACM, 1999, pp. 63–72. [Online]. Available: http://doi.acm.org/10.1145/312129.312198
  * [2] Z. Fu, W. Hu, and T. Tan, “Similarity based vehicle trajectory clustering and anomaly detection,” in _IEEE International Conference on Image Processing 2005_ , vol. 2, Sep. 2005, pp. II–602.
  * [3] J. Bian, D. Tian, Y. Tang, and D. Tao, “A survey on trajectory clustering analysis,” _arXiv preprint arXiv:1802.06971_ , 2018.
  * [4] M. Conde Rocha Murca, R. DeLaura, R. J. Hansman, R. Jordan, T. Reynolds, and H. Balakrishnan, ser. AIAA AVIATION Forum. American Institute of Aeronautics and Astronautics, Jun 2016, ch. Trajectory Clustering and Classification for Characterization of Air Traffic Flows, 0. [Online]. Available: https://doi.org/10.2514/6.2016-3760
  * [5] A. Eckstein, “Automated flight track taxonomy for measuring benefits from performance based navigation,” in _2009 Integrated Communications, Navigation and Surveillance Conference_ , May 2009, pp. 1–12.
  * [6] W. J. Eerland and S. Box, ser. AIAA SciTech Forum. American Institute of Aeronautics and Astronautics, Jan 2016, ch. Trajectory Clustering, Modeling and Selection with the focus on Airspace Protection, 0. [Online]. Available: https://doi.org/10.2514/6.2016-1411
  * [7] M. Gariel, A. N. Srivastava, and E. Feron, “Trajectory clustering and an application to airspace monitoring,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 12, no. 4, pp. 1511–1524, Dec 2011.
  * [8] A. Bombelli, A. Segarra Torne, E. Trumbauer, and K. D. Mease, _Automated Route Clustering for Air Traffic Modeling_ , ser. AIAA SciTech Forum. American Institute of Aeronautics and Astronautics, Jan 2017, 0. [Online]. Available: https://doi.org/10.2514/6.2017-1318
  * [9] Y. Liu, M. Hansen, D. J. Lovell, C. Chuang, M. O. Ball, and J. Gulding, “Causal analysis of en route flight inefficiency-the us experience,” in _Twelfth USA/Europe Air Traffic Management Research and Development Seminar_ , vol. 570, 2017.
  * [10] I. Dhief, N. E. Dougui, D. Delahaye, and N. Hamdi, “Strategic planning of North Atlantic Oceanic air traffic based on a new wind-optimal route structure,” in _ATM Seminar, 12th USA/Europe Air Traffic Management R &D Seminar_, Seattle, United States, Jun. 2017. [Online]. Available: https://hal-enac.archives-ouvertes.fr/hal-01592231
  * [11] J. H. Junzi Sun, Joost Ellerbroek, “Modeling Aircraft Performance Parameters with Open ADS-B Data,” in _ATM Seminar, 12th USA/Europe Air Traffic Management R &D Seminar_, Seattle, United States, Jun. 2017.
  * [12] T. Verbraak, J. Ellerbroek, J. Sun, and J. Hoekstra, “Large-scale ads-b data and signal quality analysis,” in _Proceedings of the 12th USA/Europe Air Traffic Management Research and Development Seminar_ , 2017.
  * [13] A. A. Barsheshat, “Implementation of ads-b systems — benefits and considerations,” in _Proceedings of ESAV’11_ , Sept 2011, pp. 197–201.
  * [14] B. S. Ali, A. Majumdar, W. Y. Ochieng, and W. Schuster, “Ads-b: the case for london terminal manoeuvring area (ltma),” 2013.
  * [15] C. Rekkas and M. Rees, “Towards ads-b implementation in europe,” in _2008 Tyrrhenian International Workshop on Digital Communications - Enhanced Surveillance of Aircraft and Vehicles_ , Sept 2008, pp. 1–4.
  * [16] O. Arbelaitz, I. Gurrutxaga, J. Muguerza, J. M. Pérez, and I. Perona, “An extensive comparative study of cluster validity indices,” _Pattern Recognition_ , vol. 46, no. 1, pp. 243 – 256, 2013. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S003132031200338X
  * [17] A. S. Torne, “Route clustering for strategic planning in air traffic management,” Master dissertation, University of California, Irvine, Open Access Publications from the University of California, 2015. [Online]. Available: https://escholarship.org/uc/item/0jb7748c
  * [18] P. Dierckx, “A fast algorithm for smoothing data on a rectangular grid while using spline functions,” _SIAM Journal on Numerical Analysis_ , vol. 19, no. 6, pp. 1286–1304, 1982. [Online]. Available: http://www.jstor.org/stable/2157211
  * [19] M. Ester, H.-P. Kriegel, J. Sander, and X. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise,” in _Proc. 2nd Int. Conf. on Knowledge Discovery and Data Mining_ , ser. KDD’96. AAAI Press, 1996, pp. 226–231. [Online]. Available: http://dl.acm.org/citation.cfm?id=3001460.3001507
  * [20] P. J. Rousseeuw, “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis,” _Journal of Computational and Applied Mathematics_ , vol. 20, 1987. [Online]. Available: http://gen.lib.rus.ec/scimag/index.php?s=10.1016/0377-0427(87)90125-7
  * [21] D. W. Davies, David L.; Bouldin, “A cluster separation measure,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. PAMI-1, 1979\. [Online]. Available: http://gen.lib.rus.ec/scimag/index.php?s=10.1109/tpami.1979.4766909