11institutetext: Laboratoire d’astrophysique de Bordeaux, Univ. Bordeaux,
CNRS, B18N, allée Geoffroy Saint-Hilaire, 33615 Pessac, France. 11email:
<EMAIL_ADDRESS>22institutetext: Department of Astronomy, Graduate
School of Science, The University of Tokyo, Tokyo, Japan 33institutetext:
National Astronomical Observatory of Japan, Tokyo, Japan 44institutetext:
Centro de Astrobiología (CSIC-INTA), Depto. de Astrofísica, ESAC Campus,
Camino Bajo del Castillo s/n, 28692, Villanueva de la Cañada, Madrid, Spain
55institutetext: Grantecan S. A., Centro de Astrofísica de La Palma, Cuesta de
San José, 38712 Breña Baja, La Palma, Spain 66institutetext: University of
Vienna, Department of Astrophysics, Türkenschanzstraße 17, 1180 Wien, Austria
77institutetext: Institute of Astronomy, Faculty of Science, the University of
Tokyo, Osawa 2-21-1, Miaka, Tokyo 181-0015, Japan 88institutetext: Kiso
Observatory, Institute of Astronomy, Faculty of Science, the University of
Tokyo, Mitake 10762-30, Kiso-machi, Kiso-gun, Nagano 397-0101, Japan
99institutetext: Canada - France - Hawaii Telescope Corporation, 65-1238
Mamalahoa Highway, Kamuela, HI 96743, USA 1010institutetext: Depto. de
Inteligencia Artificial , UNED, Juan del Rosal, 16, 28040 Madrid, Spain
1111institutetext: Depto. Estadística e Investigación Operativa. Universidad
de Cádiz, Avda. República Saharaui s/n, 11510 Puerto Real, Cádiz, Spain
1212institutetext: AIM, CEA, CNRS, Université Paris-Saclay, Université de
Paris, F-91191 Gif-sur-Yvette, France 1313institutetext: Núcleo de Astrofísica
Teórica, Universidade Cidade de São Paulo, R. Galvão Bueno 868, Liberdade,
01506-000, São Paulo, SP, Brazil 1414institutetext: Steward Observatory,
University of Arizona, 933 North Cherry Avenue, Rm. N204 Tucson, AZ
85721-0065, USA

# Infrared Spectroscopy of free-floating planet candidates in Upper Scorpius
and Ophiuchus

H. Bouy 11 M. Tamura 2233 D. Barrado 44 K. Motohara 33 N. Castro Rodríguez 55
N. Miret-Roig 1166 M. Konishi 22 S. Koyama 77 H. Takahashi 88 N. Huélamo 44 E.
Bertin 99 J. Olivares 1010 L. M. Sarro 1010 A. Berihuete 1111 J.-C. Cuillandre
1212 P.A.B. Galli 1313 Y. Yoshii 771414 T. Miyata 77

(Received ; accepted )

###### Abstract

Context. A rich population of low-mass brown dwarfs and isolated planetary
mass objects has been reported recently in the Upper Scorpius and Ophiuchus
star forming complex.

Aims. We investigate the membership, nature and properties of 17 of these
isolated planetary mass candidates using low-resolution near-infrared spectra.

Methods. We investigate the membership by looking for evidences of youth using
four diagnostics: the slope of the continuum between the J and Ks band, the
Hcont and TLI-g gravity sensitive indices, and by comparing the spectra to
young and field (old) M and L-dwarf standards.

Results. All the targets but one are confirmed as young ultracool objects,
with spectral types between L0 and L6 and masses in the range 0.004–0.013 M⊙
according to evolutionary models. The status of the last target is unclear at
this point.

Conclusions. Only one possible contaminant has been identified among the 17
targets, suggesting that the contamination level of the original sample must
be low ($\lesssim$6%).

###### Key Words.:

Stars: brown dwarfs; Planets and satellites: formation

##  1 Introduction

Free-floating planets are planetary-mass objects that do not orbit a star, but
roam the galaxy isolated. Apart from micro-lensing detections, only a few tens
of directly imaged free-floating planet candidates are known to date (for
example among others Tamura et al., 1998; Lucas & Roche, 2000; Zapatero Osorio
et al., 2000; Luhman et al., 2005; Peña Ramírez et al., 2012; Suárez et al.,
2019; Lodieu et al., 2018; Luhman & Hapich, 2020, and references therein) and
only a small fraction have been confirmed so far. Because of the degeneracy in
the mass-luminosity relationship for these ultracool objects, it is indeed
impossible to distinguish a low mass brown dwarf from a planetary mass object
when the age and distance are unknown. This deadlock can be overcome by
studying free-floating planets members of young associations where the age and
distance are precisely known. In the context of the COSMIC-DANCE111project-
dance.com survey Miret-Roig et al. (2022) obtained deep optical and near-
infrared photometry and measured accurate proper motions 5 mag beyond Gaia’s
limit in the nearby Upper Scorpius OB association (USco) and $\rho$-Ophiuchus
(Oph) molecular clouds. They identified over 3 500 members, including between
70 and 170 planetary mass objects, depending on the age assumed. This large
number of planetary mass object candidates in a young association has
important implications for the theories and models of star, brown dwarf and
planet formation. In order to confirm this important discovery, we performed
follow-up spectroscopic observations of 18 free-floating planet candidates to
confirm their nature and membership to the association and validate Miret-Roig
et al. (2022) analysis. In the following, we describe the observations and the
processing of the data obtained at the Grantecan and Subaru telescopes. We
discuss the membership to the association by looking for spectral features
characteristic of young ultracool objects. Finally, we estimate the spectral
types of the objects and derive effective temperature and mass estimates, as
well as a contamination rate in Miret-Roig et al. (2022) sample.

##  2 Observations

###  2.1 Targets

We selected 18 targets within Miret-Roig et al. (2022) sample, after
discarding objects already observed spectroscopically in the literature (e.g.
Lodieu et al., 2018; Luhman & Esplin, 2022). A total of 18 objects were
randomly selected in the range between $17.3<J<19.2$ mag, corresponding to
masses between $7\lesssim$M$\lesssim 10$MJup and effective temperatures
between $1\,500\lesssim M\lesssim 1\,900$ K according to Baraffe et al. (2002)
evolutionary models for an age of 5 Myr and a distance of 145 pc. One brighter
target (DANCe J16064553-2121595 = 3355, $J=16.19$ mag) was added during the
course of the observations as clouds were degrading the sensitivity and
preventing us to observe our original targets. Figure 1 shows a $(M_{J},J-Ks)$
color-magnitude diagram of the sample and Fig. 2 shows the location of the
targets in Upper Scorpius and Ophiuchus. Two sources (3213 and 3214) are
separated by only 120″.

Figure 1: $(M_{J},J-Ks)$ colour-magnitude diagram of our targets (red stars),
with low-gravity (blue squares) and field gravity (grey dots) ultracool dwarfs
from the literature (Burgasser, 2014, and references therein). An arrow
represents a $A_{\rm V}=2$ mag extinction vector. Figure 2: Position of our
targets in Upper Scorpius and Ophiuchus. Background photograph credit: Mario
Cogo (galaxlux.com) Table 1: Targets observed

Name |  IDaa$a$From Miret-Roig et al. (2022) | RA | Dec |  $A_{v}$ max. | i | z | y | J | H | Ks | Exp. Time
---|---|---|---|---|---|---|---|---|---|---|---
|  | (J2000) | (J2000) | mag | mag | mag | mag | mag | mag | mag |  NDIT$\times$DIT s
SWIMS
DANCe J16135217-2443562 | 3213 | 16:13:52.17 | -24:43:56.2 | 0.82 |  23.76$\pm$0.13 |  22.03$\pm$0.05 |  21.52$\pm$0.06 |  18.84$\pm$0.06 |  17.70$\pm$0.05 |  16.70$\pm$0.05 |  40$\times$300
DANCe J16134589-2442310 | 3214 | 16:13:45.89 | -24:42:31.0 | 0.82 |  23.72$\pm$0.20 |  22.05$\pm$0.06 |  21.37$\pm$0.05 |  19.17$\pm$0.08 |  17.73$\pm$0.05 |  16.81$\pm$0.05 |  36$\times$300
DANCe J16081299-2304316 | 3345 | 16:08:12.99 | -23:04:31.6 | 0.52 |  23.69$\pm$0.08 |  22.01$\pm$0.05 |  22.15$\pm$0.08 |  18.94$\pm$0.05 |  17.80$\pm$0.05 |  16.82$\pm$0.05 |  24$\times$300
DANCe J16064553-2121595 | 3355 | 16:06:45.53 | -21:21:59.5 | 0.49 |  21.03$\pm$0.05 |  19.11$\pm$0.05 |  18.40$\pm$0.05 |  16.19$\pm$0.05 |  15.31$\pm$0.05 |  14.55$\pm$0.05 |  8$\times$300
DANCe J16091010-1930376 | 3378 | 16:09:10.10 | -19:30:37.6 | 0.60 |  24.09$\pm$0.11 |  22.19$\pm$0.05 |  21.58$\pm$0.05 |  19.03$\pm$0.09 |  17.81$\pm$0.05 |  16.74$\pm$0.05 |  22$\times$300
DANCe J15454024-2422072 | 3404 | 15:45:40.24 | -24:22:07.2 | 0.74 |  23.41$\pm$0.10 |  21.99$\pm$0.07 | $\cdots$ |  18.55$\pm$0.06 |  17.55$\pm$0.05 |  16.67$\pm$0.05 |  22$\times$300
EMIR
DANCe J16221885-2440029 | 3293 | 16:22:18.85 | -24:40:02.9 | 2.99 |  22.20$\pm$0.05 |  20.42$\pm$0.05 |  19.70$\pm$0.05 |  17.33$\pm$0.05 |  16.25$\pm$0.05 |  15.48$\pm$0.05 |  16$\times$200
DANCe J16064467-2033428 | 3314 | 16:06:44.67 | -20:33:42.8 | 0.77 |  22.29$\pm$0.05 |  20.45$\pm$0.05 |  19.85$\pm$0.05 |  17.56$\pm$0.05 |  16.50$\pm$0.05 |  15.72$\pm$0.05 |  28$\times$200
DANCe J16140756-2211524 | 3231 | 16:14:07.56 | -22:11:52.4 | 0.63 |  23.02$\pm$0.05 |  21.09$\pm$0.05 |  20.78$\pm$0.05 |  18.28$\pm$0.05 |  17.28$\pm$0.05 |  16.44$\pm$0.05 |  24$\times$200
DANCe J16255679-2113354 | 3091 | 16:25:56.79 | -21:13:35.4 | 1.81 |  23.53$\pm$0.11 |  21.76$\pm$0.05 |  21.19$\pm$0.06 |  18.39$\pm$0.08 |  17.33$\pm$0.05 |  16.36$\pm$0.05 |  28$\times$200
DANCe J16123953-2614510 | 3210 | 16:12:39.53 | -26:14:51.0 | 0.88 |  23.05$\pm$0.14 | $\cdots$ |  20.24$\pm$0.05 |  18.43$\pm$0.06 |  17.31$\pm$0.05 |  16.34$\pm$0.05 |  24$\times$200
DANCe J16053908-2403330 | 3244 | 16:05:39.08 | -24:03:33.0 | 0.36 |  23.14$\pm$0.07 |  21.40$\pm$0.05 |  20.87$\pm$0.07 |  18.47$\pm$0.05 |  17.33$\pm$0.05 |  16.33$\pm$0.05 |  32$\times$200
DANCe J16020000-2057342 | 3324 | 16:02:00.00 | -20:57:34.2 | 0.66 |  23.15$\pm$0.05 |  21.66$\pm$0.05 |  21.11$\pm$0.05 |  18.54$\pm$0.05 |  17.50$\pm$0.05 |  16.35$\pm$0.05 |  32$\times$200
DANCe J16393029-2454135 | 3421 | 16:39:30.29 | -24:54:13.5 | 0.69 |  23.36$\pm$0.07 |  21.69$\pm$0.06 |  20.82$\pm$0.05 |  18.55$\pm$0.05 |  17.44$\pm$0.05 |  16.59$\pm$0.05 |  32$\times$200
DANCe J16041234-2127472 | 3326 | 16:04:12.34 | -21:27:47.2 | 0.52 |  23.69$\pm$0.07 |  21.91$\pm$0.05 |  21.53$\pm$0.05 |  18.70$\pm$0.05 |  17.43$\pm$0.05 |  16.57$\pm$0.05 |  32$\times$200
DANCe J16254583-2347372 | 3144 | 16:25:45.83 | -23:47:37.2 | 3.21 |  23.64$\pm$0.14 |  21.93$\pm$0.06 |  21.42$\pm$0.05 |  18.74$\pm$0.05 |  17.49$\pm$0.05 |  16.46$\pm$0.05 |  46$\times$200
DANCe J15481655-2307430 | 3200 | 15:48:16.55 | -23:07:43.0 | 0.77 |  23.65$\pm$0.12 | $\cdots$ | $\cdots$ |  18.76$\pm$0.09 |  17.50$\pm$0.05 |  16.56$\pm$0.05 |  32$\times$200
DANCe J15582895-2530319 | 3299 | 15:58:28.95 | -25:30:31.9 | 0.36 |  23.70$\pm$0.12 |  22.01$\pm$0.05 |  21.41$\pm$0.06 |  18.76$\pm$0.06 |  17.40$\pm$0.05 |  16.41$\pm$0.05 |  32$\times$200

222$A_{v}$ max. corresponds to the integrated line-of-sight interstellar
reddening to 250 pc in the direction of the source as computed by Green et al.
(2019). The near-infrared JHKs photometry comes in the 2MASS system.

###  2.2 SWIMS at Subaru

A total of six objects were observed with the _Simultaneous-color Wide-field
Infrared Multi-object Spectrograph_ (SWIMS, Motohara et al., 2014, 2016;
Konishi et al., 2018, 2020) mounted on the Subaru Telescope in May 2021
(Program S21A-047, PI: M. Tamura). SWIMS was used in long-slit mode with its
simultaneous _zJ_ ($700<R<1200$) and _HK_ (600¡R¡1000) grisms. The 300 s
individual exposures were acquired following a standard ABBA procedure to
efficiently remove the sky emission. The seeing was generally very good
(between 0$\aas@@fstack{\prime\prime}$3 and 0$\aas@@fstack{\prime\prime}$6)
but clouds were hindering the observations at times. A slit of
0$\aas@@fstack{\prime\prime}$5 or 0$\aas@@fstack{\prime\prime}$8 was used
depending on the seeing. Table 1 gives the list of targets observed with SWIMS
and the corresponding number of exposures and individual exposure times. Three
B stars of the Upper Scorpius associations were observed (HIP81145, HIP82133
and HIP78702) to be used as telluric standards.

The raw data were processed following standard procedures for infrared
spectroscopy using a combination of custom made Python code using the
_astropy_ and _specutils_ libraries (Price-Whelan et al., 2018; Earl et al.,
2022) and IRAF/PyRAF’s _apall_ package for the spectra extraction and telluric
correction. The closest-in-time B stars spectrum was used to remove the
telluric contamination in each of our targets spectra. Because the B stars
belong to Upper Scorpius as well, the typical airmass differences with the
target were less than 0.2$\sim$0.3.

###  2.3 EMIR at GTC

A total of 13 objects were observed with _Espectrógrafo Multiobjeto Infra-
Rojo_ (EMIR, Garzón et al., 2014) mounted on the Grantecan telescope (Program
GTC2-21A, PI: D. Barrado) in May 2021. EMIR was used in long-slit mode with
its _HK_ grism and a slit of 1$\aas@@fstack{\prime\prime}$2 chosen to match
the ambient seeing during the observations, and leading to an effective
resolution of $R\sim 500$. The weather was mostly clear during the
observations. Table 1 gives the list of targets observed with EMIR and the
corresponding number of exposures and exposure times. The 200 s individual
exposures were acquired following a standard ABBA procedure to efficiently
remove the sky emission.

The data were reduced using _RedEmIR_ , a new GTC pipeline written in Python.
_RedEmIR_ eliminates the contribution of the sky background in the near
infrared using the consecutive A-B pairs. The sky-subtracted images are
subsequently flat-fielded, calibrated in wavelength and average combined to
obtain the final spectrum. The telluric correction is achieved using a
customized version of Xtellcor (Vacca et al., 2003) adjusted to the
atmospheric conditions of the La Palma observatory (Ramos Almeida et al.,
2009). The spectra are then divided by the spectrum of an A0 star spectrum
obtained after or before the targets to remove telluric contamination.

##  3 Evidences of youth

Young ultra-cool dwarfs such as the ones targeted in the present study have
not yet contracted into their final configurations and their gravity is
significantly lower than their older field counterparts. At the resolution of
our spectroscopic observations, gravity will affect mostly three features:

  * •

$J$ to $Ks$ continuum slope: a $J-Ks$ color redder than for field brown dwarfs
has been systematically reported for young brown dwarfs (e.g. Kirkpatrick et
al., 2008; Delorme et al., 2017a) as well as some young planets and brown
dwarf companions to stars (e.g. Barman et al., 2011; Delorme et al., 2017b,
and Fig. 1). The lower gravity leads to more clouds in the upper layers of the
atmosphere. These reduce the amount of emergent flux at shorter wavelengths
and lead to fainter absolute $J$-band magnitudes and redder $J-Ks$ colors.
Additionally, the lower density associated to a lower gravity results in
reduced collision-induced $H_{2}$ and FeH absorption which in turn leads to
less suppression of the K-band flux and therefore a redder $J-Ks$ color
(Mohanty et al., 2007; Faherty et al., 2013).

  * •

triangular H-band continuum: only part of the $H$-band flux is affected by
this reduced collision-induced FeH and $H_{2}$ absorption, producing an
$H$-band continuum with a characteristic triangular shape (Lucas et al.,
2001), which can be quantified and measured using the Hcont index proposed by
Allers & Liu (2013). It is insensitive to reddening but the presence of dust
in the upper layer of the atmosphere can mimic the effect of low-gravity on
the H-band shape. For this reason, Allers & Liu (2013) recommends to
complement the Hcont index with other diagnostics to test the youth of
ultracool objects.

  * •

gravity-sensitive absorption lines: collision induced pressure broadening
depends on both the temperature and gravity (through the density) in the
ultracool dwarf atmosphere. For a given effective temperature, an older
ultracool dwarf with a higher gravity will therefore have more prominent
absorption lines than a younger (lower gravity) counterpart (Martin et al.,
1996; Gorlova et al., 2003; Allers & Liu, 2013). At the resolution of our
observations, the 1.244 and 1.253 $\mu$m K I lines are the most gravity
sensitive lines detectable in the SWIMS spectra. Unfortunately, the relatively
low signal-to-noise ratio of our SWIMS spectra in the J-band results in large
uncertainties and inconclusive values of the KIJ index of Allers & Liu (2013).

  * •

TLI-g gravity sensitive index: taking advantage of the growing number of near-
infrared spectra available in the literature and in various databases,
Almendros-Abad et al. (2022) recently used machine learning techniques to
define a new gravity sensitive index. Their TLI-g index is designed to
separate young objects from older field objects with a performance superior to
other indices from the literature. It seems in particular less sensitive to
the presence of dust in the upper layer of the atmosphere, but is however
sensitive to extinction.

In the following, we therefore discuss the $J-Ks$ color, Hcont and TLI-g
indices of our targets and compare their spectra to young and field M and
L-dwarf standards.

###  3.1 $J-Ks$ colours

Figure 1 shows that all our targets have $J-Ks$ colors redder than older field
counterparts from the literature and similar to known young low-gravity
ultracool dwarfs. Both multiplicity and extinction can shift objects in this
diagram and mimic the effect of youth. The presence of an unresolved companion
can indeed shift the position mostly vertically in a colour-magnitude diagram
and by at most 0.75 mag333For an equal mass binary the individual fluxes are
half the combined flux and the individual magnitudes are
$2.5\log_{10}(2)=0.75$ mag fainter.. While we cannot rule out the presence of
unresolved companions, Figure 1 shows that our targets remain redder than most
field L-dwarfs even when adding 0.75 mag to their luminosity. Any unresolved
pair would therefore be made of individual components redder than the older
field sequence. Extinction is unlikely to have shifted the objects given that
the cumulative line-of-sight reddening towards our objects is low in most
cases (see Table 1) and still places them on the redder low-gravity sequence
even in the worst cases of 3293 and 3144. While it is not a conclusive proof,
the very red $J-Ks$ color certainly adds to the list of evidences indicating
youth and hence membership to USco or Oph.

###  3.2 Comparison with young and old M and L standards

To further assess the youth of our targets and at the same time derive their
spectral type, we perform an empirical comparison with spectra of young and
old ultracool objects from the literature. The comparison is made using _The
SpeX Prism Library Analysis Toolkit_ (SPLAT, Burgasser & Splat Development
Team, 2017). A number of spectral libraries of young ultracool dwarfs have
been presented in the literature, and we chose to use the very-low gravity
spectral standards included in SPLAT (see Table 5 and Burgasser & Splat
Development Team, 2017). The field-gravity (older) standards chosen for the
comparison are presented in Table 6.

The degeneracy between extinction and spectral type can affect and compromise
the comparison (see e.g. Luhman et al., 2017). To partially lift this
degeneracy and explore the effect of extinction on the results of our
analysis, we perform the comparison after dereddening our spectra by the
cumulative extinction in the line-of-sight of each target up to 250 pc
reported in the 3D extinction map of Green et al. (2019). Assuming that all
our targets belong to USco, this should represent a worst case scenario in
terms of reddening and provide an estimate of the lower limit on the spectral
type. USco is indeed located at approximately 145 pc and Oph at 125 pc,
although they possibly extend up to $\sim$200 pc (Damiani et al., 2019). The
value of 250 pc was therefore chosen to be conservative and be sure to include
the entire depth of the clouds associated to these two regions.

_SPLAT_ was first used to scale the instrumental fluxes to physical units
using the target H-band photometry reported in Miret-Roig et al. (2022) as
reference. The closest match in each of the two libraries of standards is
found using a standard $\chi^{2}$-minimization. Figures 9 to 11 show the
results, with the original spectra on the left and the dereddened spectra on
the right. As expected the spectral types obtained using the high-gravity
(older) field standards are systematically later than those obtained with the
low-gravity (young) standards.

Figures 9 to 11 show that 3210, 3200, 3314, 3091, 3326, 3214, 3355, 3144,
3299, 3345 are clearly better matched by a young spectral standard than by an
old spectral standard independently of the reddening.

Among the rest of the sample we can see that:

  * •

3378 and 3213 have K-band fluxes significantly higher than the best-match
field standards which is clearly pointing towards a young age.

  * •

the best matches for 3244, 3421, 3231 and 3404 are obtained with a young
standard but the $\chi^{2}$ difference is only marginal and inconclusive.

  * •

3293 has a marginally better fit with an old standard but the $\chi^{2}$
difference is only marginal as well.

  * •

3324 displays a very peculiar spectral energy distribution, that cannot be
matched by any of our old or young standards. While we cannot rule out that
the peculiar continuum emission is related to variability, we note that 3324
was observed during degraded ambient conditions and the discrepancy is
probably due to telluric clouds absorption, as suggested by the discrepancy
between the $H-K_{s}$=1.15 mag photometry reported by Miret-Roig et al. (2022)
and the synthetic $H-K_{s}$=0.8 mag computed from the spectral data. This
spectrum is therefore considered as dubious and discarded for the rest of the
analysis.

###  3.3 Sharp H-band continuum

As mentioned earlier the shape of the H-band continuum varies from a typical
triangular shape at young ages to a flatter continuum at more advanced ages as
gravity increases. The Hcont index defined by Allers & Liu (2013) is commonly
used to quantify the sharpness of the H-band continuum and look for evidence
of youth.

We measured the Hcont index in all our spectra, as well as in the 891 spectra
from the SPEX Ultracool dwarfs library with a good quality (QUALITY_FLAG=OK
and MEDIAN_SNR$\geq$50) and a resolution R$\geq$120\. The uncertainty was
estimated by simply propagating the standard errors of the means used in the
Hcont index formula. We derived near-infrared spectral types for each spectrum
from the SPEX Ultracool dwarfs library based on the Kirkpatrick et al. (2010)
L dwarf classification scheme and a gravity classification between very-low
(VL-G), intermediate (INT-G) and field (FLD-G) gravity using Allers & Liu
(2013) classification scheme. Figure 3 shows the results in the form of a
violin graph, using the spectral types that will be presented in Section 4 for
our targets. The index measurement, spectral type and gravity classification
for the 891 spectra are available in electronic form in Table 7.

Within the relatively large error bars, a number of our targets seem to have
an Hcont favoring low or intermediate gravity and hence a young age. These
include 3210, 3200, 3314, 3091, 3326, 3214, 3355, 3144, 3299, 3345. Two
objects have Hcont values more consistent with high-gravity older objects:
3404 and 3293, although the large uncertainties make them also consistent with
intermediate gravity objects. Objects 3244, 3421, 3231 have Hcont values
consistent with intermediate or low gravity, while 3378 and 3213 have
inconclusive Hcont indices compatible within the uncertainties either with
high or intermediate gravity objects.

Figure 3: Hcont gravity index from Allers & Liu (2013) for our targets (red
dots) over-plotted over a violin graph of the distributions for ultracool
dwarfs with very-low gravity (red), intermediate gravity (blue) and field-
gravity (cyan) from the SPEX library of ultracool dwarfs. Our targets are
shifted randomly horizontally for clarity. Figure 4: TLI-g gravity index from
Almendros-Abad et al. (2022) for our targets (red dots) over-plotted over a
violin graph of the distributions for ultracool dwarfs with very-low gravity
(red), intermediate gravity (blue) and field-gravity (cyan) from the SPEX
library of ultracool dwarfs. Our targets are shifted randomly horizontally for
clarity.

###  3.4 TLI-g index

The TLI-g index was invented recently by Almendros-Abad et al. (2022) using
machine learning techniques to specifically distinguish low and field gravity
ultracool objects. We measured the TLI-g index in all our spectra, as well as
in the 891 spectra from the SPEX library mentioned in the previous section
(see Table 7). Figure 4 shows the results in the form of a violin graph.

Within the error bars we can see that all our targets have a TLI-g index
favoring low or intermediate gravity and hence a young age except in the case
of:

  * •

3144 with a TLI-g index favoring a field gravity and hence an older age

  * •

3378 and 3404 have such large uncertainties that the TLI-g index is
inconclusive

Although Almendros-Abad et al. (2022) defined the TLI-g index using a sample
of M0–L3 ultracool dwarfs, we can see in Fig. 4 that it seems to work equally
well on the L4 and L6 dwarfs of our sample.

###  3.5 Final youth status

Table 2 gives a summary of the 4 youth diagnostics as well as the final status
for each target. Among the 17 targets, 9 have the four diagnostics indicating
a young age, and are therefore firmly confirmed as young ultracool dwarfs
members of the USco and Oph associations. Another 5 have inconclusive Hcont
indices within the large uncertainties and/or inconclusive comparison with
standards but have all other diagnostics indicating low-gravity, and are
therefore confirmed as young ultracool dwarfs with a high level of confidence
as well.

The remaining cases are discussed here:

  * •

3144 is classified as young from the $J-Ks$ color, the comparison with
spectral standards, and the Hcont index, but as old from the TLI-g index.
Extinction can affect the TLI-g measurement, but would move the object down in
the diagram and make it look younger. Instead we find that 3144 has a higher
TLI-g value. Its value is still consistent with intermediate gravity object
within the uncertainties and given that all the other diagnostics are favoring
a young age we classify 3144 as a young ultracool dwarf as well.

  * •

3404 is classified as young using 2 diagnostics and is only classified as
possibly old using the Hcont index while the TLI-g index is inconclusive
because of the large uncertainties. Given that the large uncertainties on the
Hcont index make it fully compatible with intermediate and low gravity objects
as well, we classify 3404 as young as well.

  * •

3293 has a marginally better fit with an old standard and has a Hcont index
clearly favoring a high-gravity older object. But the $J-Ks$ color and TLI-g
index are favoring a young object. The integrated line-of-sight extinction at
250 pc is relatively large (AV=2.99 mag) and its $J-Ks$ color and TLI-g index
might therefore be affected by reddening. With the current data it is
difficult to draw any conclusion on 3293 youth.

In conclusion, 16 of the 17 targets have multiple evidences of youth and one
(3293) is inconclusive. In total we therefore firmly confirm the youth of 16
candidates out of 17 as young L-dwarfs members of the USco and Oph
associations, or 94%.

Table 2: Diagnostics of youth  ID | $J-Ks$ | Comparison | Hcont | TLI-g | Young?
---|---|---|---|---|---
|  | with standard |  |  |
3091 | Young | Young | Young | Young | Young
3144 | Young | Young | Young | Old | Young
3200 | Young | Young | Young | Young | Young
3210 | Young | Young | Young | Young | Young
3213 | Young | Young | ? | Young | Young
3214 | Young | Young | Young | Young | Young
3231 | Young | Young? | Int? | Young | Young
3244 | Young | Young? | Int? | Young | Young
3293 | Young | Old? | Old? | Young | ?
3299 | Young | Young | Young | Young | Young
3314 | Young | Young | Young | Young | Young
3326 | Young | Young | Young | Young | Young
3345 | Young | Young | Young | Young | Young
3355 | Young | Young | Young | Young | Young
3378 | Young | Young | ? | ? | Young
3404 | Young | Young? | Old? | ? | Young
3421 | Young | Young? | Int? | Young | Young

##  4 Spectral types, effective temperature and masses

We now use Figures 9 to 11 to estimate the spectral types of the 16 targets
with evidences of youth identified in the previous section. With the question
of their youth settled, we are left to decide whether extinction affects the
spectrum of the targets. By using the integrated line-of-sight extinction
towards each target until 250 pc, we can partially break the degeneracy
between spectral type and reddening and check the worst-case scenario in which
the target eventually lies at the far edge of the association.

Figures 9 to 11 show that extinction does not affect the result of the
comparison for 3091, 3200, 3231, 3244, 3299, 3324, 3326, 3421, 3345. All
indeed have fairly small integrated extinction (see Table 1).

The match is significantly better without extinction in the cases of 3213
(L2), 3314 (L0), 3210 (L0), 3144 (L4), specially in the K-band.

Finally, the match is equally good with or without dereddening for 3404
(L3–L4), 3378 (L2–L4), 3355 (L1–L3), 3214 (L2–L4). We note that the difference
is always smaller than 2 subclasses, and we adopt the corresponding ranges as
final spectral type.

These results show that reddening does not affect much our target, in
agreement with the integrated line-of-sight extinction at 250 pc, and confirm
that they are most likely not reddened background contaminants.

These spectral types are translated into effective temperatures using the
empirical relationship for young L-dwarfs reported in Faherty et al. (2016),
which in turn are translated into masses using the Saumon & Marley (2008)
models for 3, 6 and 10 Myr. Given their location in the Scorpius OB2 complex,
most of our targets are expected to belong to Upper Scorpius and have ages in
the range 6–10 Myr. Three of them (3421, 3293, 3144, see Fig. 2) are
nevertheless located on top of the $\rho-$Ophiuchus molecular clouds and
probably belong to the young (1$\sim$3Myr) association. Uncertainties on the
spectral types (1 to 2 subclasses) translate into uncertainties of the order
of 250 K for the effective temperatures and 0.002 M⊙ for the masses at a given
age. Table 3 gives the results and shows that all the candidates seem to have
masses in the planetary domain, the least massive having a mass of only
0.004–0.006 M⊙ depending on the age. These objects will cool steadily over
time, dissolve in the galactic field population and become field T and
Y-dwarfs within the next 50$\sim$100 Myr, as illustrated in Figure 5.

Table 3: Adopted spectral types and estimated effective temperatures and masses for the confirmed members.  ID | SpT |  Teff | Mass 3Myr | Mass 6Myr | Mass 10Myr
---|---|---|---|---|---
|  | (K) |  (M⊙) |  (M⊙) |  (M⊙)
3213 | L2 | 1880 | 0.007 | 0.010 | 0.012
3214 | L2–L4 | 1560–1880 | 0.005–0.007 | 0.007–0.010 | 0.009–0.012
3345 | L6 | 1220 | 0.003 | 0.004 | 0.006
3355 | L1–L3 | 1740–1970 | 0.006–0.008 | 0.008–0.011 | 0.011–0.013
3378 | L2–L4 | 1560–1880 | 0.005–0.007 | 0.007–0.010 | 0.009–0.012
3404 | L3–L4 | 1560–1740 | 0.005–0.006 | 0.007–0.008 | 0.009–0.011
3314 | L0 | 2060 | 0.009 | 0.012 | 0.014
3231 | L1 | 1970 | 0.008 | 0.011 | 0.013
3091 | L0 | 2060 | 0.009 | 0.012 | 0.014
3210 | L0 | 2060 | 0.009 | 0.012 | 0.014
3244 | L1 | 1970 | 0.008 | 0.011 | 0.013
3421 | L1 | 1970 | 0.008 | 0.011 | 0.013
3326 | L2 | 1880 | 0.007 | 0.010 | 0.012
3144 | L4 | 1560 | 0.005 | 0.007 | 0.009
3200 | L0 | 2060 | 0.009 | 0.012 | 0.014
3299 | L4 | 1560 | 0.005 | 0.007 | 0.009
Figure 5: Effective temperature vs age between 1 Myr and 15 Gyr according to
the Marley et al. (2021) evolutionary models. The targets are represented as
well assuming an age between 1 and 10 Myr.

##  5 Objects of interest

In the following we discuss a couple of objects of interest: the coolest of
our targets and a ultra-wide pair of planetary mass objects.

###  5.1 The coolest target: DANCe J16081299-2304316 (3345)

In this section we compare the spectrum of the coolest of our targets, the L6
DANCe J16081299-2304316 (3345), with spectra of exoplanets and free-floating
planets with similar ages and spectral types from the literature.

The H-band is in general well matched by all these young exoplanet and free-
floating planet spectra, but the overall slope of DANCe J16081299-2304316 is
shallower than that of all these objects and the higher J-band flux must be
due to the slightly earlier spectral type. The 2.3$\mu$m CO overtone is well
matched in most free-floating planet spectra but is not as pronounced in the
exoplanet spectra. On the other hand the drop observed in DANCe
J16081299-2304316 at wavelength greater than 2.3$\mu$m is observed in HR8799c
spectrum only, making it a good free-floating analog of this directly imaged
young gas-giant planet. Overall, the spectrum of DANCe J16081299-2304316
appears most similar to that of HR8799c, and both objects have near-infrared
spectral types of L6.

Figure 6: Smoothed spectrum of DANCe J16081299-2304316 (red) and the planetary
mass companion 2MASS J12073346-3932539 b and HR8799c from Greenbaum et al.
(2018).  Figure 7: Smoothed spectrum of DANCe J16081299-2304316 (red) compared
to the TW Hydra L7$\gamma$ free-floating planets WISEA J114724.10-204021.3
(Schneider et al., 2016) and 2MASS J11193254-1137466AB (Kellogg et al., 2016),
the young L7pec free-floating planet WISE J174102.78-464225.5 (Schneider et
al., 2014), and the $\beta$-Pic L7$\gamma$ free-floating planet PSO
J318.5338-22.8603 (Liu et al., 2013)

###  5.2 DANCe J16135217-2443562 and DANCe J16134589-2442310: an ultra-wide
pair?

DANCe J16135217-2443562 (3213) and DANCe J16134589-2442310 (3214) form a wide
visual pair with a separation on 120″corresponding to a projected separation
of $\sim$17 400 A.U. at a distance of 145pc, as shown in Fig. 8. Such a wide
separation for such low mass objects suggest that it is probably a coincidence
rather than a bound physical pair. But on the other hand the very low spatial
density of free-floating planets reported in Miret-Roig et al. (2022) (between
0.4 and 1 FFP per square degree) suggests that such a coincidence is highly
unlikely, and calls for follow-up observations of this intriguing pair.
Improved proper motions measurements, as well as parallaxes and radial
velocities would help understand and eventually confirm their common origin.
Such a pair could indeed also be a remnant of an extreme case of ultra-wide
multiple system such as the ones reported in Taurus (Joncour et al., 2017) or
the result of a simultaneous dynamical ejection of two planets in a planetary
system.

Figure 8: Three color image ($r,Y,Ks$ as blue, green, red) of the field around
DANCe J16135217-2443562 and DANCe J16134589-2442310. Both are indicated by a
square.

##  6 Conclusions

We have obtained near-infrared spectra of 18 ultracool candidate members of
Upper Scorpius and Ophiuchus discovered by Miret-Roig et al. (2022) using
SWIMS at the Subaru telescope and EMIR at the Grantecan telescope. One of the
spectra was affected by the poor ambient conditions (clouds) and we discard it
in the analysis.

The spectra allow us to confirm the low gravity, and hence youth, using 4
diagnostics: i) the shape of their H-band continuum measured by the Hcont
index; ii) their $J-Ks$ color redder than field counterparts; iii) by
comparison with near-infrared spectra of young L-dwarf standards; iv) using
the TLI-g gravity sensitive index. Among the 17 targets, 16 have multiple
evidences of youth and one (3293) is inconclusive. In total we therefore
firmly confirm the youth of 16 candidates out of 17 as young L-dwarfs members
of the USco or Ophiuchus association, corresponding to a contamination rate of
only $\lesssim$6% and indicating that the methodology devised by Bouy et al.
(2013) and Sarro et al. (2014) and used by Miret-Roig et al. (2022) is very
reliable.

The spectral types of the targets are estimated by comparison with young
L-dwarf standards, and range between L0 and L6. Using Faherty et al. (2016)
empirical relationship for young L-dwarfs we transform these spectral types
into effective temperatures, and found that the objects have temperatures in
the range between 1 220 and 2 060 K, corresponding to masses in the range
0.004–0.013 M⊙ according to the models of Saumon & Marley (2008) for ages
between 3 and 10 Myr, consistent with Miret-Roig et al. (2022) estimate based
on the photometry only. Interestingly even the brightest target (DANCe
J16064553-2121595 = 3355) is an early L-dwarf, suggesting that many objects
fainter than $M_{\rm J}\gtrsim 10.5$ mag must indeed have masses in the
planetary mass domain.

###### Acknowledgements.

We thank A. Burgasser for his help with _SPLAT_. We are grateful to M.
Bonnefoy, M. Liu, K. Luhman and B. Bowler for sharing spectra of young
L-dwarfs and planets. This research has received funding from the European
Research Council (ERC) under the European Union’s Horizon 2020 research and
innovation programme (grant agreement No 682903, P.I. H. Bouy), and from the
French State in the framework of the ”Investments for the future” Program,
IdEx Bordeaux, reference ANR-10-IDEX-03-02. P.A.B. Galli acknowledges
financial support from São Paulo Research Foundation (FAPESP) under grants
2020/12518-8 and 2021/11778-9. DB and NH have been partially funded by the
Spanish State Research Agency (AEI) Project No. PID2019-107061GB-C61 and
MDM-2017-0737 Unidad de Excelencia María de Maeztu \- Centro de Astrobiología
(CSIC-INTA). Based on observations made with the Gran Telescopio Canarias
(GTC), installed in the Spanish Observatorio del Roque de los Muchachos of the
Instituto de Astrofísica de Canarias, in the island of La Palma. This work is
partly based on data obtained with the instrument EMIR, built by a Consortium
led by the Instituto de Astrofísica de Canarias. EMIR was funded by GRANTECAN
and the National Plan of Astronomy and Astrophysics of the Spanish Government.
Based in part on data collected at Subaru Telescope which is operated by the
National Astronomical Observatory of Japan and obtained from the SMOKA, which
is operated by the Astronomy Data Center, National Astronomical Observatory of
Japan. This research has made use of the VizieR catalogue access tool, CDS,
Strasbourg, France. The original description of the VizieR service was
published in A&AS 143, 23. This work made use of GNU Parallel (Tange, 2011),
astropy (Astropy Collaboration et al., 2013; Price-Whelan et al., 2018),
Topcat (Taylor, 2005), specutils (Earl et al., 2022), matplotlib (Hunter,
2007), Plotly (Plotly Technologies Inc., 2015), Numpy (Harris et al., 2020),
aplpy (Robitaille & Bressert, 2012; Robitaille, 2019).

Figure 9: Comparison of our targets spectra (black) with very-low gravity
standards (red) and field-gravity standards (blue). The left panels show the
original spectrum, and the right panels show the spectrum derredened by the
cumulative line-of-sight extinction indicated in the plot title and in Table 1
Figure 10: Same as Fig. 9 Figure 11: Same as Fig. 9

## References

  * Allers & Liu (2013) Allers, K. N. & Liu, M. C. 2013, ApJ, 772, 79
  * Almendros-Abad et al. (2022) Almendros-Abad, V., Mužić, K., Moitinho, A., Krone-Martins, A., & Kubiak, K. 2022, A&A, 657, A129
  * Astropy Collaboration et al. (2013) Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013, A&A, 558, A33
  * Baraffe et al. (2002) Baraffe, I., Chabrier, G., Allard, F., & Hauschildt, P. H. 2002, A&A, 382, 563
  * Bardalez Gagliuffi et al. (2014) Bardalez Gagliuffi, D. C., Burgasser, A. J., Gelino, C. R., et al. 2014, ApJ, 794, 143
  * Barman et al. (2011) Barman, T. S., Macintosh, B., Konopacky, Q. M., & Marois, C. 2011, ApJ, 735, L39
  * Bouy et al. (2013) Bouy, H., Bertin, E., Moraux, E., et al. 2013, A&A, 554, A101
  * Burgasser (2007) Burgasser, A. J. 2007, ApJ, 659, 655
  * Burgasser (2014) Burgasser, A. J. 2014, in Astronomical Society of India Conference Series, Vol. 11, Astronomical Society of India Conference Series, 7–16
  * Burgasser et al. (2008) Burgasser, A. J., Liu, M. C., Ireland, M. J., Cruz, K. L., & Dupuy, T. J. 2008, ApJ, 681, 579
  * Burgasser et al. (2004) Burgasser, A. J., McElwain, M. W., Kirkpatrick, J. D., et al. 2004, AJ, 127, 2856
  * Burgasser & Splat Development Team (2017) Burgasser, A. J. & Splat Development Team. 2017, in Astronomical Society of India Conference Series, Vol. 14, Astronomical Society of India Conference Series, 7–12
  * Cruz et al. (2009) Cruz, K. L., Kirkpatrick, J. D., & Burgasser, A. J. 2009, AJ, 137, 3345
  * Damiani et al. (2019) Damiani, F., Prisinzano, L., Pillitteri, I., Micela, G., & Sciortino, S. 2019, A&A, 623, A112
  * Delorme et al. (2017a) Delorme, P., Dupuy, T., Gagné, J., et al. 2017a, A&A, 602, A82
  * Delorme et al. (2017b) Delorme, P., Schmidt, T., Bonnefoy, M., et al. 2017b, A&A, 608, A79
  * Earl et al. (2022) Earl, N., Tollerud, E., Jones, C., et al. 2022, astropy/specutils: V1.7.0
  * Faherty et al. (2013) Faherty, J. K., Rice, E. L., Cruz, K. L., Mamajek, E. E., & Núñez, A. 2013, AJ, 145, 2
  * Faherty et al. (2016) Faherty, J. K., Riedel, A. R., Cruz, K. L., et al. 2016, ApJS, 225, 10
  * Garzón et al. (2014) Garzón, F., Castro-Rodríguez, N., Insausti, M., et al. 2014, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 9147, Ground-based and Airborne Instrumentation for Astronomy V, ed. S. K. Ramsay, I. S. McLean, & H. Takami, 91470U
  * Gizis (2002) Gizis, J. E. 2002, ApJ, 575, 484
  * Gorlova et al. (2003) Gorlova, N. I., Meyer, M. R., Rieke, G. H., & Liebert, J. 2003, ApJ, 593, 1074
  * Green et al. (2019) Green, G. M., Schlafly, E., Zucker, C., Speagle, J. S., & Finkbeiner, D. 2019, ApJ, 887, 93
  * Greenbaum et al. (2018) Greenbaum, A. Z., Pueyo, L., Ruffio, J.-B., et al. 2018, AJ, 155, 226
  * Harris et al. (2020) Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature, 585, 357
  * Hunter (2007) Hunter, J. D. 2007, Computing in Science & Engineering, 9, 90
  * Joncour et al. (2017) Joncour, I., Duchêne, G., & Moraux, E. 2017, A&A, 599, A14
  * Kellogg et al. (2016) Kellogg, K., Metchev, S., Gagné, J., & Faherty, J. 2016, ApJ, 821, L15
  * Kirkpatrick et al. (2008) Kirkpatrick, J. D., Cruz, K. L., Barman, T. S., et al. 2008, ApJ, 689, 1295
  * Kirkpatrick et al. (2010) Kirkpatrick, J. D., Looper, D. L., Burgasser, A. J., et al. 2010, ApJS, 190, 100
  * Konishi et al. (2020) Konishi, M., Motohara, K., Takahashi, H., et al. 2020, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 11447, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, 114475H
  * Konishi et al. (2018) Konishi, M., Motohara, K., Takahashi, H., et al. 2018, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 10702, Ground-based and Airborne Instrumentation for Astronomy VII, ed. C. J. Evans, L. Simard, & H. Takami, 1070226
  * Liu et al. (2013) Liu, M. C., Magnier, E. A., Deacon, N. R., et al. 2013, ApJ, 777, L20
  * Lodieu et al. (2018) Lodieu, N., Zapatero Osorio, M. R., Béjar, V. J. S., & Peña Ramírez, K. 2018, MNRAS, 473, 2020
  * Lucas & Roche (2000) Lucas, P. W. & Roche, P. F. 2000, MNRAS, 314, 858
  * Lucas et al. (2001) Lucas, P. W., Roche, P. F., Allard, F., & Hauschildt, P. H. 2001, MNRAS, 326, 695
  * Luhman et al. (2005) Luhman, K. L., Adame, L., D’Alessio, P., et al. 2005, ApJ, 635, L93
  * Luhman & Esplin (2022) Luhman, K. L. & Esplin, T. L. 2022, AJ, 163, 26
  * Luhman & Hapich (2020) Luhman, K. L. & Hapich, C. J. 2020, AJ, 160, 57
  * Luhman et al. (2017) Luhman, K. L., Mamajek, E. E., Shukla, S. J., & Loutrel, N. P. 2017, AJ, 153, 46
  * Marley et al. (2021) Marley, M. S., Saumon, D., Visscher, C., et al. 2021, ApJ, 920, 85
  * Martin et al. (1996) Martin, E. L., Rebolo, R., & Zapatero-Osorio, M. R. 1996, ApJ, 469, 706
  * Miret-Roig et al. (2022) Miret-Roig, N., Bouy, H., Raymond, S. N., et al. 2022, Nature Astronomy, 6, 89
  * Mohanty et al. (2007) Mohanty, S., Jayawardhana, R., Huélamo, N., & Mamajek, E. 2007, ApJ, 657, 1064
  * Motohara et al. (2016) Motohara, K., Konishi, M., Takahashi, H., et al. 2016, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 9908, Ground-based and Airborne Instrumentation for Astronomy VI, ed. C. J. Evans, L. Simard, & H. Takami, 99083U
  * Motohara et al. (2014) Motohara, K., Konishi, M., Takahashi, H., et al. 2014, in Ground-based and Airborne Instrumentation for Astronomy V, ed. S. K. Ramsay, I. S. McLean, & H. Takami, Vol. 9147, International Society for Optics and Photonics (SPIE), 2070 – 2081
  * Peña Ramírez et al. (2012) Peña Ramírez, K., Béjar, V. J. S., Zapatero Osorio, M. R., Petr-Gotzens, M. G., & Martín, E. L. 2012, ApJ, 754, 30
  * Plotly Technologies Inc. (2015) Plotly Technologies Inc. 2015, Collaborative data science
  * Price-Whelan et al. (2018) Price-Whelan, A. M., Sipőcz, B. M., Günther, H. M., et al. 2018, AJ, 156, 123
  * Ramos Almeida et al. (2009) Ramos Almeida, C., Pérez García, A. M., & Acosta-Pulido, J. A. 2009, ApJ, 694, 1379
  * Reid et al. (2006) Reid, I. N., Lewitus, E., Allen, P. R., Cruz, K. L., & Burgasser, A. J. 2006, AJ, 132, 891
  * Robitaille (2019) Robitaille, T. 2019, APLpy v2.0: The Astronomical Plotting Library in Python
  * Robitaille & Bressert (2012) Robitaille, T. & Bressert, E. 2012, APLpy: Astronomical Plotting Library in Python, Astrophysics Source Code Library
  * Sarro et al. (2014) Sarro, L. M., Bouy, H., Berihuete, A., et al. 2014, A&A, 563, A45
  * Saumon & Marley (2008) Saumon, D. & Marley, M. S. 2008, ApJ, 689, 1327
  * Schneider et al. (2014) Schneider, A. C., Cushing, M. C., Kirkpatrick, J. D., et al. 2014, AJ, 147, 34
  * Schneider et al. (2016) Schneider, A. C., Windsor, J., Cushing, M. C., Kirkpatrick, J. D., & Wright, E. L. 2016, ApJ, 822, L1
  * Suárez et al. (2019) Suárez, G., Downes, J. J., Román-Zúñiga, C., et al. 2019, MNRAS, 486, 1718
  * Tamura et al. (1998) Tamura, M., Itoh, Y., Oasa, Y., & Nakajima, T. 1998, Science, 282, 1095
  * Tange (2011) Tange, O. 2011, ;login: The USENIX Magazine, 36, 42
  * Taylor (2005) Taylor, M. B. 2005, in Astronomical Society of the Pacific Conference Series, Vol. 347, Astronomical Data Analysis Software and Systems XIV, ed. P. Shopbell, M. Britton, & R. Ebert, 29
  * Vacca et al. (2003) Vacca, W. D., Cushing, M. C., & Rayner, J. T. 2003, PASP, 115, 389
  * Zapatero Osorio et al. (2000) Zapatero Osorio, M. R., Béjar, V. J. S., Martín, E. L., et al. 2000, Science, 290, 103

##  Appendix A Tables

Table 4: Gravity sensitive indices  Object | Hcont | TLI-g
---|---|---
3091 | 0.97$\pm$0.05 | 0.84$\pm$0.05
3144 | 0.99$\pm$0.07 | 1.03$\pm$0.07
3200 | 1.01$\pm$0.03 | 0.95$\pm$0.04
3210 | 0.99$\pm$0.03 | 0.97$\pm$0.03
3213 | 0.92$\pm$0.05 | 0.85$\pm$0.05
3214 | 0.96$\pm$0.09 | 0.97$\pm$0.07
3231 | 0.93$\pm$0.03 | 0.92$\pm$0.03
3244 | 0.94$\pm$0.03 | 0.90$\pm$0.02
3293 | 0.93$\pm$0.03 | 0.95$\pm$0.03
3299 | 0.94$\pm$0.03 | 0.91$\pm$0.03
3314 | 0.99$\pm$0.02 | 0.93$\pm$0.02
3324 | 0.96$\pm$0.02 | 0.98$\pm$0.02
3326 | 0.99$\pm$0.04 | 0.92$\pm$0.04
3345 | 0.99$\pm$0.05 | 0.86$\pm$0.05
3355 | 0.96$\pm$0.04 | 0.90$\pm$0.05
3378 | 0.91$\pm$0.12 | 0.93$\pm$0.14
3404 | 0.84$\pm$0.10 | 0.96$\pm$0.15
3421 | 0.93$\pm$0.03 | 0.94$\pm$0.03
Table 5: SPEX library of very-low gravity ultracool standards

Name | RA | Dec | SpT | Ref.
---|---|---|---|---
2MASSW J1207334-393254A | 181.88959 | -39.548443 | M8 | (1)
TWA 26 | 174.96304 | -31.989279 | M9 | (1)
2MASS J01415823-4633574 | 25.492626 | -46.565945 | L0$\gamma$ | (2)
2MASS J05184616-2756457 | 79.692337 | -27.946028 | L1$\gamma$ | (3)
2MASS J05361998-1920396 | 84.083244 | -19.344334 | L2$\gamma$ | (3)
2MASSW J2208136+292121 | 332.05679 | 29.355972 | L3$\gamma$ | (2)
2MASS J05012406-0010452 | 75.35025 | -0.17922223 | L4$\gamma$ | (2)
2MASSW J2244316+204343 | 341.13196 | 20.728695 | L7$\gamma$ | (4)

(1) Gizis (2002) ; (2) Cruz et al. (2009) ; (3) Bardalez Gagliuffi et al.
(2014) ; (4) Kirkpatrick et al. (2008)

Table 6: Library of field ultracool standards

Name | RA (J2000) | Dec (J2000) | SpT | Ref.
---|---|---|---|---
VB 10 | 289.2400917 | 5.1506056 | M8 | (2)
LP 944-20 | 54.89675 | -35.4289139 | L0 | (1)
GJ 1048B | 38.9997083 | -23.5223611 | L1 | (1)
2MASSI J2057540-025230 | 314.4753917 | -2.8750722 | L2 | (2)
2MASSW J1506544+132106 | 226.7267125 | 13.3516889 | L3 | (3)
2MASSW J0036159+182110 | 9.0674 | 18.3529083 | L4 | (1)
SDSSp J144600.60+002452.0 | 221.5025833 | 0.4144444 | L5 | (4)
2MASSI J1010148-040649 | 152.5616958 | -4.1138694 | L6 | (5)
2MASSI J1526140+204341 | 231.5585417 | 20.7281833 | L7 | (2)

(1) Burgasser et al. (2008) ; (2) Burgasser et al. (2004) ; (3) Burgasser
(2007) ; (4) Bardalez Gagliuffi et al. (2014) ; (5) Reid et al. (2006)

Table 7: Gravity sensitive indices measured in 891 spectra of the SPEX library  Name | RA (J2000) | Dec (J2000) | SpT | SPEX_CLASS | Hcont | e_Hcont | TLI-g | e_TLI-g
---|---|---|---|---|---|---|---|---
2MASS J0000286-124515 | 0.11945834 | -12.75425 | M9.0 | FLD-G | 0.947 | 0.003 | 1.015 | 0.01
2MASS J00013044+1010146 | 0.37683335 | 10.170723 | M6.0 | VL-G | 0.988 | 0.006 | 1.009 | 0.023
LHS 102B | 1.1451666 | -40.734943 | L4.0 | FLD-G | 0.855 | 0.002 | 1.011 | 0.008
LEHPM 1-162 | 1.4486667 | -21.954889 | M8.0 | FLD-G | 0.963 | 0.002 | 0.994 | 0.009
2MASSI J0006205-172051 | 1.5854167 | -17.347389 | L2.0 | FLD-G | 0.897 | 0.006 | 0.998 | 0.006
SDSS J000632.60+140606.4 | 1.6358334 | 14.101778 | L1.0 | FLD-G | 0.931 | 0.009 | 0.989 | 0.017
SDSS J000649.16-085246.3 | 1.7048334 | -8.879528 | M8.0 | FLD-G | 0.921 | 0.004 | 1.035 | 0.008
2MASS J0007078-245804 | 1.7827917 | -24.967834 | M8.0 | INT-G | 0.987 | 0.003 | 1.008 | 0.017
2MASS J00085931+2911521 | 2.2471251 | 29.197805 | M8.0 | INT-G | 0.967 | 0.002 | 1.013 | 0.007
2MASS J00100009-2031122 | 2.500375 | -20.520056 | M8.0 | FLD-G | 0.933 | 0.003 | 1.049 | 0.003
2MASSI J0013578-223520 | 3.4907918 | -22.58889 | L5.0 | FLD-G | 0.859 | 0.01 | 0.992 | 0.01
WISE J001450.14-083823.1 | 3.7089167 | -8.6397219 | M7.0 | FLD-G | 0.945 | 0.008 | 1.042 | 0.016
2MASS J00145575-4844171 | 3.7322917 | -48.738083 | L3.0 | FLD-G | 0.883 | 0.006 | 1.022 | 0.013
2MASSW J0015447+351603 | 3.9364998 | 35.267387 | L2.0 | INT-G | 0.9 | 0.003 | 1.025 | 0.006
SDSS J001608.44-004302.3 | 4.0351253 | -0.71722221 | L3.0 | INT-G | 0.898 | 0.008 | 0.961 | 0.023
2MASS J00163761+3448368 | 4.1567087 | 34.810223 | M8.0 | FLD-G | 0.939 | 0.004 | 1.016 | 0.013


Handuo Zhang, Karunasekera Hasith, and Han Wang 11institutetext: School of
Electrical & Electronic Engineering,
Nanyang Technological University, Singapore,
11email:  {hzhang032<EMAIL_ADDRESS><EMAIL_ADDRESS>
# GMC: Grid Based Motion Clustering in Dynamic Environment

Handuo Zhang  Karunasekera Hasith  Han Wang

###### Abstract

Conventional SLAM algorithms takes a strong assumption of scene
motionlessness, which limits the application in real environments. This paper
tries to tackle the challenging visual SLAM issue of moving objects in dynamic
environments. We present GMC, grid-based motion clustering approach, a
lightweight dynamic object filtering method that is free from high-power and
expensive processors. GMC encapsulates motion consistency as the statistical
likelihood of detected key points within a certain region. Using this method
can we provide real-time and robust correspondence algorithm that can
differentiate dynamic objects with static backgrounds.

We evaluate our system in public TUM dataset. To compare with the state-of-
the-art methods, our system can provide more accurate results by detecting
dynamic objects.

###### keywords:

visual SLAM, motion coherence, dynamic envrionment

##  1 Introduction

Localization has always been a core and challenging task in mobile robotics,
autonomous driving, autonomous UAVs and other navigation applications. With
the development of all kinds of smart algorithms together with more powerful
sensors and processors, now the translation error of SLAM has even been
reduced to around $0.8\%\sim 1.2\%$ according to KITTI odometry and SLAM
evaluation benchmark [11]111KITTI Benchmark:
http://www.cvlibs.net/datasets/kitti/eval_odometry.php with monocular, stereo
and RGBD cameras. Some significant visual odometry methods like SVO [8, 9],
ORB-SLAM [17], DSO-SLAM [7] and S-PTAM [18], etc are listed in the rankings.
They are mostly inspired by the groundbreaking key-frame based monocular SLAM
framework PTAM [13] and become popular partially because cameras are
relatively cheap and handy to set up, and they passively acquire texture-rich
cues from the environment, which makes it applicable in real-world
environments.

However, most of current visual SLAM system are still sensitive to highly
dynamic scenarios due to the strong assumption that the surroundings are
static. Currently unexpected change of surroundings will probably corrupt the
quality of the state estimation process and even lead to system corruption.
For example, the presence of dynamics in the environments like walking
pedestrians and moving vehicles, might give rise to misleading data
association for vision based SLAM systems. Although there have been some
researches to reduce the sensitivity to dynamic objects on laser based SLAM
[28, 21] based on Iterative Closest Point (ICP) method, the problem is still
not well studied on vision-based SLAM.

In recent years with the rapid development of deep learning, some researchers
turn to high-level understanding of the surroundings [20, 16, 19]. This can
help remove the moving objects and indirectly solve the problem. However, the
property of deep neural network makes the hardware requirements really high,
also real-time performance can hardly be reached on mobile robot platforms.

In this paper, we focus on reducing the impact of dynamic objects without the
need of neural network, which can be employed on lightweight mobile platform.
The main contributions of our work are:

  * •

We proposed a dynamic feature point filtering method called grid-based motion
clustering (GMC). It can significantly reduce the impact of dynamic object on
pose estimation.

  * •

The GMC method also can used to reject outliers during feature matching based
on motion consistency within certain neighborhood.

  * •

We build a complete SLAM system that adopts GMC method and the performance of
the system is validated on TUM dataset.

  * •

the proposed approach can be easily integrated into other existing SLAM
frameworks.

The rest of this paper is organized as follows. Related work is presented in
Sec. (2). Sec. (3) discusses the proposed algorithm, and based on that we
discuss how to integrate into a SLAM system. Finally in Sec. (4) we evaluate
the performance of our proposed framework in the popular KITTI benchmark and
in real applications, validating our performance.

##  2 Related Work

The basic assumption of most SLAM methods is that the landmarks observed are
static, which limits the application of visual SLAM. Some recent researches
start researching active moving objects using dense SLAM by incorporating
optical flow technique. Optical flow is generated with movement of pixels, so
theoretically static background and moving targets can be distinguished by
that [1, 26]. However, this kind of dense sampling requires a large amount of
calculation. Also as the association cannot be guaranteed, these methods are
not robust, and RANSAC might fail easily, especially when the object size is
big. Some learning based methods are utilized to minimize the reprojection
error or pixel intensity with the association issue one of the values to be
solve using EM [6] or CRF [27], but the results cannot be achieved in real-
time.

Semantic segmentation has gained significant improved over past year due to
deep learning based methods. Segnet [2] and Mask R-CNN [12] are two recent
popular semantic segmentation methods. Semantic segmentation provides the
information for which object (e.g. human, car, road, building and etc.) each
pixel in the image belongs to. In SLAM problem features are tracked across
frames to localize the camera. However, these features suffer from
illumination variance and vie-point variations making them loose track in few
frames. Furthermore, most SLAM approaches assume that the most parts of the
image represent the static environment, which is not the ideal assumption for
a reliable road scenes. Semantic segmentation provides the high level scene
understanding which could facilitate rectifying the errors introduced in the
above two scenarios.

Simply associating semantic information with multiple fixed points VO can be
improved [15] because of the long term association of features across frames.
Other than point features, object level information can also be used in
localizing ego position, coupled with inertial and point features into a
single optimization framework [6]. In extreme viewing variations due to
occlusion, large illumination changes based on the time of the day and viewing
angle changes can cause great challenges to localization problem. [23]
proposes a method to handle this problem using a generative model to learn
descriptors combining geometric and semantic information of the scene.
Additionally semantic information can be used to classify static and dynamic
objects and thus features belonging to static and dynamic classes. DS-SLAM
[29], DynaSLAM [4] and SIVO [10] uses semantic information to remove the
outliers in consistently moving features in dynamic objects such as humans.
DynaSLAM [4] later uses semantic info to inpaint the occluded static scenes.
In contrast to just using semantic information to remove dynamic objects, it
can be used to calculate both ego motion and motions of each object using
stereo visuals [14]. Use of semantic information is useful to keep only the
static permanent information (e.g. features on buildings, road and etc.) when
generating the map [29] while making use of static temporary information (e.g.
features on parked vehicles) and dynamic information (e.g. features on dynamic
vehicles and persons) in local frames to improve the localization.

##  3 Grid based Motion Clustering Approach

In some researches, CRF is utilized to do clustering considering the distance
between pairwise point distance and neighboring constraints [27]. This type of
approach derives from the idea of image segmentation and cannot be achieved in
real-time. In this section the GMC is addressed for dynamic object clustering
even without counting every pixel. The strategy of accumulating keypoints and
classification based distance measurement makes the process fast. Meanwhile,
the Quadtree [22] aided adjustment process makes the algorithm robust.

This approach utilizes the basic property of motion coherence constraints,
i.e. neighboring pixels share similar motion. While in localization task,
exhaustively comparing each pixel is time expensive. So we take discriminative
features considering motion coherence as a statistics likelihood within a
region. This property will be proved in section (3.2).

In the following, we will extract the feature points and explicitly use the
initial guess of rotational $R$ and translational $T$ parameter of the motion
vector to convert the triangulated new frame keypoints into 3D locations in
global coordinate, so as to further calculate the distance measurements which
will be discussed in sec. (3.1).

###  3.1 Distance Measurement

The normal metrics of 3D points is the Euclidean distance between them in the
reference frame $\mathcal{F}_{re}$ and the associated frame
$\mathcal{F}_{ma}$.

$e_{i}^{j}=\|re_{i}-(T_{j}+R_{j}ma_{i})\|^{2}$ (1)

where $j$ represents the frame index, $i$ is the feature point index, thus
$R_{j}$ and $T_{j}$ are initial estimated rotation and translation parameters.

For an image with $M$ keypoints which are all static, the theoretical
assumption is that their 3D displacement of keypoint locations are zero. While
for dynamic objects, displacement does not infer full motion information, i.e.
does not differentiate any motion directions, like illustrated in Fig. (1).
Unlike 2D reprojection error, the 3D Euclidean error triangulated from 2D
feature point pairs are impacted by the distance. So on top of equation (1) we
incorporate the covariance of reconstructed 3D scene feature points [3].
According to multiview geometry, we can conclude that the covariance of 3D
reconstructed points have relationship with 2D image point covariance
matrices, the 3D locations, and the projection matrices.

$\displaystyle
cov=\left(\begin{array}[]{cc}\mathbf{A}^{\intercal}\left(\mathbf{B}\left(\begin{array}[]{cc}\mathbf{C}_{1}&0\\\
0&\mathbf{C}_{2}\\\
\end{array}\right)\mathbf{B}^{\intercal}\right)^{-1}\mathbf{A}\par\end{array}\right)$
(5)

where $\mathbf{C}^{1}$ and $\mathbf{C}^{2}$ are image coordinate covariance
matrices respectively, $\mathbf{A}$ is defined as the inverted 3D location,
$\mathbf{B}$ defined as the inverted 2D point location. To simplify, we add a
parameter to normalize the 3D residuals which is proportional to the inverse
3D distance $\mathbf{X}$:

$e_{i}^{j}=(\mathbf{X}^{re}_{i}-(T_{j}+R_{j}\mathbf{X}^{ma}_{i}))\cdot\alpha/(\mathbf{X}^{ma})^{-1}$
(6)

where $\alpha$ is a parameter to constraint the geometry residuals inside a
certain scope for next step processing.

Considering both discrimination and calculation efficiency, we choose motion
classification, with finite combinations of motion clusters in $z$ direction
and $x$ direction respectively. The merits of this proposed distance
measurement is for the further motion statistics and clustering convenience
sake, with any motion residuals lying within a predefined intervals counted as
this motion pattern.

We set the minimal geometry residual $e_{int}^{z}$, $e_{int}^{x}$ as the unit
of the finite motion category pattern. Then the distance measurement can be
converted into a 2D motion pattern table with $e_{int}$ the interval, as shown
in Fig (1).

Figure 1: Illustration of dynamic object motion patterns. The center cell
represents static status.

###  3.2 Motion Constraint Analysis

Given adjacent frame pairs, the associated extracted feature points should
view the same 3D locations. So its neighboring keypoints shall move together
if the density of features is high enough.

According to the conclusion of [5], motion smoothness causes a small
neighborhood around a true match to view the same 3D location. Likewise, it
also causes a (small) neighborhood around a object keypoint to drop into the
same motion pattern table as shown in (1), which is also the primitive idea of
the proposed approach. $re$ refers to reference frame, $ma$ refers to matched
frame. We set $f_{re}$ as one of the $n$ features in region $re$. Thus
$f_{re}^{ma}$ represents $f_{re}$’s nearest neighbor happens to be the feature
in region $ma$.

Therefore our assumption gives

$p(f^{ma}_{re}|f_{re}^{false})=\beta m/M$ (7)

where $M$ is all the location possibilities once the $f_{re}$’s neighbors drop
into different local regions from this keypoint, $m$ is the number of features
in region $ma$ and $\beta$ is a factor to accommodate violations of this
assumption caused by repeated structures like a row of windows.

Let $p_{true}=p(f^{ma}_{re}|T^{re\rightarrow ma})$ be the probability that,
given ${re,ma}$ view the same location, feature $f_{re}$’s nearest neighbor is
in region $ma$. Also we set the event of motion consistency with
$T^{re\rightarrow ma}$ the same motion of all keypoints in one local region
and $F^{re\rightarrow ma}$ not the same motion. So we can derive:

$\displaystyle\begin{split}p_{true}=p(f^{ma}_{re}|T^{re\rightarrow
ma})&=p(f^{true}_{re}|T^{re\rightarrow
ma})+p(f^{false}_{re},f^{ma}_{re}|T^{re\rightarrow ma})\\\
&=p(f^{true}_{re}|T^{re\rightarrow ma})+\\\ &p(f^{false}_{re}|T^{re\rightarrow
ma})p(f^{ma}_{re}|f^{false}_{re},T^{re\rightarrow ma})\\\ &=t+(1-t)\beta
m/M\\\ \end{split}$ (8)

where $t$ represents the probability of all $f_{re}$’s neighbors are
consistent with $f_{re}$.

Likewise, let $p_{false}=p(f^{ma}_{re}|F^{re\rightarrow ma})$ and similar to
(8),

$\displaystyle p_{false}=\beta(1-t)m/M$ (9)

According to equations (8), (9) we can approximate the distribution of
$S_{i}$, the number of points in a neighborhood of keypoint $x_{i}$, with a
pair of binomial distribution:

$S_{i}\sim\begin{cases}B(n,p_{true}),&\text{if }x_{i}\text{ is true}\\\
B(n,p_{false}),&\text{if }x_{i}\text{ is false}\end{cases}$ (10)

where $S_{i}$ here refers to the neighborhood scores that support the motion
of the target keypoint $k_{i}$. The two distinct distributions make $S$ the
good indicator for considering a local region with evident motions different
from background. In the next section, we will use this property to solve
dynamic point filtering by counting the neighborhood scores and merge similar
motion patterns.

###  3.3 Approach Description

The proposed algorithm takes a presupposed number of approximately equally
sized grid cells $N$. Thus a motion statistics tensor is created as shown in
Fig (2) (a).

####  3.3.1 Assignment with Consensus of motion patterns

Enough number of keypoints can provide statistically better quality motion
cluster. Referring to Fig (2), each layer of the tensor represents the motion
distribution of the associated grid cell. So we can just calculate the
residuals of the location difference
$\mathbf{x}_{j}^{ma}-\mathbf{x}_{j}^{ref}$ where $j$ is the keypoint index
within this grid and $\mathbf{x}\in{\mathbb{R}^{3}}$.

Figure 2: Fig. (a) illustrates the motion clustering tensor
$\mathcal{X}\in{\mathbb{R}^{z\times x\times N}}$ where $N$ is the number of
grid cells. We classify the dynamic objects into 2D arrays
$S\in{\mathbb{R}^{z\times x}}$ for $z$ and $x$ directions respectively. Fig.
(b) demonstrates the merging process between all the cells that share similar
motions.

####  3.3.2 Assignment with Quadtree

The counting of grid cells with the most feature points falling in sounds easy
but will cause the motion segmentation not accurate due to the fixed size of
grid cell, which in some cases, may lead to failure by not considering the
partial object cased in the cell. So hereby we proposed quadtree structure to
further split the grid cell into quads and repeat the assignment task.

The condition of subdividing grids is twofold:

  1. 1.

The motion pattern that wins the consensus has fewer than
$p_{min}\cdot{n_{i}}$, with $p_{min}$ the threshold of maximum point
proportion and $n_{i}$ the number of all keypoints detected within cell $i$.

  2. 2.

$n_{i}>n_{min}$ with $n_{min}$ the minimum required keypoints for statistics.
Our motion smoothness is valid only when there are enough number of points, as
illustrated in equation (7).

This subdividing process ends when one of the above conditions fails. Here we
display the pseudo-code for the whole procedure of assignment.

Algorithm 1 GMC Assignment

Input $\mathcal{F}_{r}$:reference frame, $\mathcal{F}_{m}$: frame to be
matched, $\mathbf{R}_{0},\mathbf{t}_{0}$: initial estimation of pose
transform.
Output $\mathcal{L}$: motion pattern tensor representing most likely motion
clusters for each sub-region of the frame.

1:Detect feature points with corresponding descriptors from $\mathcal{F}_{r}$
and $\mathcal{F}_{m}$: $\mathbf{s}_{r}$ and $\mathbf{s}_{m}$

2:Calculate the 3D location residuals of the feature points $\mathbf{e}$.

3:Generate an image-size 2D array $I$ with height $h$ and width $w$. The
indexes are the corresponding 2D key point coordinates $s_{m}$, and the values
are the the 3D location residuals $\mathbf{e}$. $\triangleright$ Eq. (6)

4:Divide $I$ by $N=G_{x}\times G_{y}$ grid cells. Create a 3D tensor
$\mathcal{X}$ with depth the grid number $N$, width $x$ and height $z$ the
motion pattern resolution. $\triangleright$ Fig. (1)

5:for each keypoint $k\in{\mathcal{K}}$ in array $I$ do

6: Acquire the grid cell index $g$ this key point belongs to;

7: push the 3D residuals $\mathbf{e}$ into the corresponding entry $s_{ijc}$
of $\mathcal{X}$. The count of this entry plus $1$.

8:end for

9:for $c=1$ to $N$ do

10: Find the maximum value $s_{max1}$ and second biggest value $s_{max2}$ of
the motion pattern $\mathcal{X}_{c}$.

11:end for

12:Run motion clustering process as explained in Algorithm (2).

13:repeat

14:from line 4 to line 12, with motion patterns shifted by half cell-width in
the $x$,$z$ and both $x$ and $z$ directions. (3 times)

15:until The cluster of motion converges.

####  3.3.3 Merging

After the assignment, we might have hundreds of motion entries. Thus an
additional updating and merging operation is needed, because we need a post-
processing step to enforce connectivity by reassigning disjoint pixels to
nearby clusters. Here we have an assumption based on the prior knowledge that
most regions of the background is static. So we hereby adopt a strong
constraint: cluster the motion patterns as few as possible and impose more
penalties to low motion areas to avoid trivial motion fragmentation. Thus we
adopt non-maximum suppression approach to make sure there is no duplicate
assignments in nearby neighborhoods.

We first eliminate the cluster with fewer than 10 features and sort the grids
in each motion pattern and choose the highest score grids the candidates. Then
we step by step merge clusters with the same label around the candidates.

The entire algorithm is summarized in Algorithm (2).

Algorithm 2 Merge Clusters

Input $\mathcal{L}$: labels representing different motion patterns with the
corresponding assignment.
Output New cluster assignment $\mathcal{L}\prime$.

1:repeat

2: for each motion pattern $m$ do Sort all the grid cells within $m$ in
decreasing order

3: for each grid cell assignment $l_{i}\in{l_{m}}$ do Find its neighbors with
the same labels.

4: if Can find then Merge $l_{i}$ and $l_{k}$ to create the new cluster
$\mathcal{L}\prime$

5: end if

6: end for

7: end for

8:until the distance between two cells is below threshold

9:Return $\mathcal{L}\prime$

###  3.4 Integration into SLAM System

Fig.(3) illustrates an overview pipeline of the SLAM system. First of all, the
new frame is passed into the system and motion estimation is operated as
normal. The, before the bundle adjustment (pose and structure refinement)
step, we insert the GMC filtering module which has been discussed in details
in section (3.3).

Figure 3: The pipeline of integrating GMC filtering approach into general
visual SLAM system.

##  4 Experiments

In this section we present extensive experimental outcomes to demonstrate the
superior performance of the proposed system. To benchmark the integration of
GMC with SLAM methods, we inject GMC approach into ORB-SLAM2 (RGBD version)
[17] and compare it with the original performance. The advantage of RGBD is
that we don’t need to triangulate the 3D points and calculate the covariance.
However, we can still use monocular or stereo version of SLAM with GMC. The
experimental results demonstrate the competitiveness of the proposed system as
well as the versatile nature of the fusion framework.

All the experiments are performed on a standard laptop running Ubuntu 16.04
with 8G RAM and an Intel Core i7-6700HQ CPU at 2.60 GHz.

###  4.1 Filtering out dynamic points

The input of our algorithm is the extracted features from two frames. As we
use motion coherence advantage, based on equation (8), (9), we need abundant
information from features to correctly partition moving regions and static
regions. In the experiment we use GFTT [24] features and set the nearest
distance as 10 decrease the threshold to extract features.

Figure 4: The demonstration of GMC filter on KITTI RAW dataset. Note that GMC
can well handle dynamic object with certain size due to the grid resolution.
To increase accuracy, you have to increase the number of features to be
extracted.

For more features, RANSAC process will take much longer time and the result
might be not robust. We adopt the grid based statistics matcher (GMS) [5] to
help reject outliers with proper matching time. As shown in Fig (4), we
evaluate our method in KITTI dataset with some middle traffic. It proves that
our method can label the dynamic features with high accuracy.

###  4.2 Accuracy Evaluation

We evaluate our results in TUM RGB-D dataset [25] with dynamic scenes and
accurate ground truth system capturing walking and sitting motions. In the
experiment, we take the grid cell number $20\times 15$ to balance the
efficiency and accuracy.

Table 1: Comparison on translational error RMSE and MAE of ORBSLAM with and without GMC (ATE)  Dataset | ORBSLAM+GMC | ORBSLAM
---|---|---
RMSE | MAE | RMSE | MAE
fr3_walking_xyz | 0.0873 | 0.0451 | 0.8121 | 0.5977
fr3_walking_static | 0.0157 | 0.0193 | 0.4115 | 0.3198
fr3_walking_rpy | 0.7301 | 0.4935 | 0.8802 | 0.7992
fr3_walking_half | 0.1029 | 0.0644 | 0.5273 | 0.4100
fr3_sitting_static | 0.0088 | 0.0593 | 0.0089 | 0.0071
Table 2: Comparison on translational error RMSE and MAE of ORBSLAM with and without GMC (RPE). (Unit: m)  Dataset | ORBSLAM+GMC | ORBSLAM
---|---|---
RMSE | MAE | RMSE | MAE
fr3_walking_xyz | 0.0977 | 0.0347 | 0.4344 | 0.2469
fr3_walking_static | 0.0137 | 0.0094 | 0.2120 | 0.0208
fr3_walking_rpy | 0.1943 | 0.0735 | 0.4501 | 0.1507
fr3_walking_half | 0.0338 | 0.0303 | 0.3441 | 0.0673
fr3_sitting_static | 0.0081 | 0.0066 | 0.0093 | 0.0070
Table 3: Comparison on rotational error RMSE and MAE of ORBSLAM with and without GMC (RPE)  Dataset | ORBSLAM+GMC | ORBSLAM
---|---|---
RMSE | MAE | RMSE | MAE
fr3_walking_xyz | 1.8765 | 0.9878 | 8.4355 | 6.7230
fr3_walking_static | 0.9818 | 0.2880 | 4.1566 | 0.6744
fr3_walking_rpy | 7.7448 | 2.1677 | 8.0703 | 2.8950
fr3_walking_half | 1.9705 | 0.8776 | 7.124 | 1.9094
fr3_sitting_static | 0.2934 | 0.2518 | 0.2890 | 0.2593

An extensive experimental validation is performed in terms of absolute
trajectory error (ATE), along with relative ose error (RPE) through root mean
squared error (RMSE) and median absolute error (MAE) over the entire
trajectory. As shown in Table (1), (2), (3) the proposed GMC aided system far
surpasses the pure vision-based approach under dynamic scenes. It can be seen
that the trajectory estimation from the GMC aided system is much closer to the
ground truth with lower drift and lower rate of failure cases.

###  4.3 Efficiency Evaluation

Table 4: Average runtime of the GMC aided system. ‘ORB+GMC’ means the GMC aided ORBSLAM system. Platform | Method | Tracking | Mapping | Total
---|---|---|---|---
Laptop | ORB+GMC | 116.7ms | 9.4ms | 126.1ms
ORBSLAM | 7.5ms | 9.2ms | 15.7ms

We evaluate the efficiency of the proposed system on both tracking and mapping
over all datasets. The average running time on two platforms is given
separately in Table (4). The added time on top of ORBSLAM can be divided into
two parts. One part is the GMC time which has complexity $O(N)$. The other
part is the additional time for feature extraction. To maintain the property
of motion coherence, we have to increase the number of features to be
detected.

Note that the proposed method is not optimized due to specifically for SLAM
system. We can use GPU parallel architecture and TBB parallel library to
accelerate the feature extraction and loop access of matrices. In this way the
efficiency of our proposed method can be further improved.

Therefore, the proposed GMC aided approach does not corrupt the instantaneity
of the original method.

##  5 Conclusions

We propose GMC, a statistical filter for dynamic objects during pose
estimation tasks, by partitioning of different motion patterns based on the
number of neighboring keypoints. A simple and fast system is developed based
on the algorithm which can be validated by experiment results.

## References

  * Alcantarilla et al [2012] Alcantarilla PF, Yebes JJ, Almazán J, Bergasa LM (2012) On combining visual slam and dense scene flow to increase the robustness of localization and mapping in dynamic environments. In: Robotics and Automation (ICRA), 2012 IEEE International Conference on, IEEE, pp 1290–1297
  * Badrinarayanan et al [2017] Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence (12):2481–2495
  * Beder and Steffen [2006] Beder C, Steffen R (2006) Determining an initial image pair for fixing the scale of a 3d reconstruction from an image sequence. In: Joint Pattern Recognition Symposium, Springer, pp 657–666
  * Bescós et al [2018] Bescós B, Fácil JM, Civera J, Neira J (2018) Dynslam: Tracking, mapping and inpainting in dynamic scenes. arXiv preprint arXiv:180605620
  * Bian et al [2017] Bian J, Lin WY, Matsushita Y, Yeung SK, Nguyen TD, Cheng MM (2017) Gms: Grid-based motion statistics for fast, ultra-robust feature correspondence. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 2828–2837
  * Bowman et al [2017] Bowman SL, Atanasov N, Daniilidis K, Pappas GJ (2017) Probabilistic data association for semantic slam. In: Robotics and Automation (ICRA), 2017 IEEE International Conference on, IEEE, pp 1722–1729
  * Engel et al [2018] Engel J, Koltun V, Cremers D (2018) Direct sparse odometry. IEEE transactions on pattern analysis and machine intelligence 40(3):611–625
  * Forster et al [2014] Forster C, Pizzoli M, Scaramuzza D (2014) Svo: Fast semi-direct monocular visual odometry. In: Robotics and Automation (ICRA), 2014 IEEE International Conference on, IEEE, pp 15–22
  * Forster et al [2017] Forster C, Zhang Z, Gassner M, Werlberger M, Scaramuzza D (2017) Svo: Semidirect visual odometry for monocular and multicamera systems. IEEE Transactions on Robotics 33(2):249–265
  * Ganti and Waslander [2018] Ganti P, Waslander SL (2018) Visual slam with network uncertainty informed feature selection. arXiv preprint arXiv:181111946
  * Geiger et al [2012] Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the kitti vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR)
  * He et al [2017] He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Computer Vision (ICCV), 2017 IEEE International Conference on, IEEE, pp 2980–2988
  * Klein and Murray [2007] Klein G, Murray D (2007) Parallel tracking and mapping for small ar workspaces. In: Mixed and Augmented Reality, 2007. ISMAR 2007. 6th IEEE and ACM International Symposium on, IEEE, pp 225–234
  * Li et al [2018] Li P, Qin T, et al (2018) Stereo vision-based semantic 3d object and ego-motion tracking for autonomous driving. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 646–661
  * Lianos et al [2018] Lianos KN, Schonberger JL, Pollefeys M, Sattler T (2018) Vso: Visual semantic odometry. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 234–250
  * Liu et al [2016] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision, Springer, pp 21–37
  * Mur-Artal and Tardos [2016] Mur-Artal R, Tardos JD (2016) Orb-slam2: an open-source slam system for monocular, stereo and rgb-d cameras. arXiv preprint arXiv:161006475
  * Pire et al [2017] Pire T, Fischer T, Castro G, De Cristóforis P, Civera J, Berlles JJ (2017) S-ptam: Stereo parallel tracking and mapping. Robotics and Autonomous Systems 93:27–42
  * Redmon et al [2016] Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788
  * Ren et al [2017] Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence (6):1137–1149
  * Rodriguez-Losada and Minguez [2007] Rodriguez-Losada D, Minguez J (2007) Improved data association for icp-based scan matching in noisy and dynamic environments. In: Robotics and Automation, 2007 IEEE International Conference on, IEEE, pp 3161–3166
  * Samet [1984] Samet H (1984) The quadtree and related hierarchical data structures. ACM Computing Surveys (CSUR) 16(2):187–260
  * Schönberger et al [2018] Schönberger JL, Pollefeys M, Geiger A, Sattler T (2018) Semantic visual localization. ISPRS Journal of Photogrammetry and Remote Sensing (JPRS)
  * Shi and Tomasi [1993] Shi J, Tomasi C (1993) Good features to track. Tech. rep., Cornell University
  * Sturm et al [2012] Sturm J, Engelhard N, Endres F, Burgard W, Cremers D (2012) A benchmark for the evaluation of rgb-d slam systems. In: Proc. of the International Conference on Intelligent Robot Systems (IROS)
  * Tan et al [2013] Tan W, Liu H, Dong Z, Zhang G, Bao H (2013) Robust monocular slam in dynamic environments. In: Mixed and Augmented Reality (ISMAR), 2013 IEEE International Symposium on, IEEE, pp 209–218
  * Tipaldi and Ramos [2009] Tipaldi GD, Ramos F (2009) Motion clustering and estimation with conditional random fields. In: Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, IEEE, pp 872–877
  * Wolf and Sukhatme [2005] Wolf DF, Sukhatme GS (2005) Mobile robot simultaneous localization and mapping in dynamic environments. Autonomous Robots 19(1):53–65
  * Yu et al [2018] Yu C, Liu Z, Liu XJ, Xie F, Yang Y, Wei Q, Fei Q (2018) Ds-slam: A semantic visual slam towards dynamic environments. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, pp 1168–1174