# Effective Scaling Approach to Frictionless Quantum Quenches in Trapped Bose
Gases

Tang-You Huang<EMAIL_ADDRESS>International Center of Quantum
Artificial Intelligence for Science and Technology (QuArtist)
and Department of Physics, Shanghai University, 200444 Shanghai, China
Department of Physical Chemistry, University of the Basque Country UPV/EHU,
Apartado 644, 48080 Bilbao, Spain  Michele Modugno<EMAIL_ADDRESS>Department of Physics, University of the Basque Country UPV/EHU, 48080 Bilbao,
Spain  IKERBASQUE, Basque Foundation for Science, 48013 Bilbao, Spain  Xi Chen
<EMAIL_ADDRESS>Department of Physical Chemistry, University of the
Basque Country UPV/EHU, Apartado 644, 48080 Bilbao, Spain

###### Abstract

We work out the effective scaling approach to frictionless quantum quenches in
a one-dimensional Bose gas trapped in a harmonic trap. The effective scaling
approach produces an auxiliary equation for the scaling parameter
interpolating between the noninteracting and the Thomas-Fermi limits. This
allows us to implement a frictionless quench by engineering inversely the
smooth trap frequency, as compared to the two-jump trajectory. Our result is
beneficial to design the shortcut-to-adiabaticity expansion of trapped Bose
gases for arbitrary values of interaction, and can be directly extended to the
three-dimensional case.

##  I introduction

Bose-Einstein condensates (BECs) and their related phenomena - such as
collective excitations, collapse, and nolinear dynamics, to mention a few -
have aroused great interest since their first experimental realization Cornell
and Wieman (2002); Dalfovo  _et al._ (1999). From the theoretical point of
view, weakly interacting BECs can be accurately described within the framework
of the Gross-Pitaevskii (GP) theory, which provides a remarkable agreement
with experimental observations. In most cases of classical hydrodynamics
Stringari (1996) or scaling transformations Castin and Dum (1996); Kagan  _et
al._ (1996), exact analytical solution can be found for the collective
dynamics and free expansion of BECs in time-dependent harmonic traps, both in
the noninteracting limit and the Thomas-Fermi (TF) regime Dalfovo  _et al._
(1999). In this vein, symmetries give birth to an intriguing property of self-
similarity, which allows to utilize the scaling approach for describing the
dynamics of ultracold atomic systems, for instance, the atomic gases in the
non-interacting and the hydrodynamic regimes Gritsev  _et al._ (2010);
Egusquiza  _et al._ (2011), Tonks-Girardeau (TG) gas of impenetrable bosons
Pedri  _et al._ (2003); Minguzzi and Gangardt (2005), superfluid Fermi gas
Egusquiza  _et al._ (2011); Schäfer (2010), and thermal cloud Bruun and Clark
(2000) in different geometries.

Besides, an effective scaling approach has further been proposed as an
approximate solution for the evolution of both bosonic and fermionic density
distributions, for describing the collective dynamics of a trapped Bose gas
Guéry-Odelin (2002), the expansion of Fermi gas Menotti  _et al._ (2002);
Diana  _et al._ (2006) and of quantum degenerate Bose-Fermi mixtures Hu  _et
al._ (2003). It consists in a self-similar evolution in the hydrodynamic
regime to be satisfied on average by integrating over the spatial coordinates,
reduce the complexity of numerical treatment. Recently, the accuracy of such
effective approach in reproducing the exact solution of quasi one-dimensional
(1D) and three-dimensional (3D) GP equations for arbitrary values of the
interactions has been discussed in Refs. Öhberg and Santos (2002a); Modugno
_et al._ (2018). Remarkably, it turns out that the space-averaged self-
similarity can provide an accurate description in several situations Viedma
and Modugno (2020).

In a slightly different but relevant topic, the concept of shortcuts to
adiabaticity (STA), originally proposed for fast schemes reproducing or
approaching slow adiabatic process Torrontegui  _et al._ (2013); Guéry-Odelin
_et al._ (2019), have extended further the control paradigms for frictionless
atomic cooling in a expanding harmonic trap Muga  _et al._ (2009); Chen  _et
al._ (2010); Schaff  _et al._ (2010, 2011); Del Campo (2011); Rohringer  _et
al._ (2015). In the context of inverse engineering, the scaling approach Muga
_et al._ (2009) and Lewis-Riesenfeld dynamical invariant Chen  _et al._ (2010)
bring out the various forms of the Ermakov equation for the scaling parameter,
capturing the character of the self-similar evolution. Along with it, the
harmonic trap frequency is thus inversely engineered for the propose, by
choosing an interpolation function of the scaling parameter with the
appropriate boundary conditions. This strategy can be applicable to other
ultracold atomic systems as a TG gas del Campo (2011), an anisotropic gas
containing quantum defects Papoular and Stringari (2015) and a Fermi gas Deng
_et al._ (2018). However, tracking back to a BEC described by the GP equation
in the mean-field approximation, one can realize that the original Ermakov
equation obtained in non-interacting case needs to be modified in the TF limit
or in the case of a time-dependent interaction Muga  _et al._ (2009);
Rohringer  _et al._ (2015) . To remedy it, the variational approximation
Pérez-García  _et al._ (1996) (which is equivalent to moment method García-
Ripoll  _et al._ (1999)), can be complemented by the concept of STA, for
studying the dynamics of BECs Li  _et al._ (2016); Huang  _et al._ (2020a, b),
valid for the range from zero to small atomic interaction, with the
implication on the quantum speed limits and quantum thermodynamics Li  _et
al._ (2018); Xu  _et al._ (2020). As a matter of fact, the accuracy of the
variational approximation depends on the presumed ansatz in terms of
nonlinearity Huang  _et al._ (2020a, b). Thus, the motivation of this work is
to fill the gap in more general theory on STA design for 1D Bose gas with
arbitrary interactions.

In this paper, we integrate the effective scaling approach into inverse
engineering for frictionless quantum quenches in trapped Bose gas, for
arbitrary values of the atomic interaction strength. Here we focus on the 1D
GP equation, but the result can be extended to 3D case. By assuming the
scaling solution in the hydrodynamic limit, we derive the Ermakov-like
equation for the scaling parameter interpolating between non-interacting and
the TF limit. With this, the frictionless quench is designed, and also
compared to the free expansion and two-jump trajectory of STA. Finally, the
numerical simulation is performed to check the stability of our method, and
the energetic cost of STA is discussed as well.

##  II Effective scaling approach

We start by considering a quasi-1D BEC confined in cigar-shaped trap,
characterized by a longitudinal frequency $\omega_{0}$ and a tight transverse
frequency $\omega_{\perp}\gg\omega_{0}$. Therefore, the system can be
effectively described by a wave function $\psi(x,t)$, whose dynamics is
governed by the following GP equation,

$i\frac{\partial\psi}{\partial t}=-\frac{1}{2}\frac{\partial^{2}\psi}{\partial
x^{2}}+\frac{1}{2}\omega^{2}(t)x^{2}\psi+g|\psi|^{2}\psi,$ (1)

that is written here in dimensionless form, for convenience. To this end, we
have used $l_{0}=\sqrt{\hbar/m\omega_{0}}$ as unit length ($m$ being the
particle mass), $\hbar\omega_{0}$ as unit energy, and $\omega_{0}^{-1}$ as
unit time. The interaction strength can be written in terms of the scattering
length $a_{s}$ as $g=2Na_{s}/\ell_{0}$, with $N$ being the number of atoms and
with the total density being normalized to one.

In order to elaborate the effective scaling approach, we apply the Madelung
transformation $\psi=\sqrt{n(x,t)}e^{i\phi(x,t)}$, such that the Lagrangian of
the system can be written as

$\mathcal{L}=-\left[\partial_{t}\phi+\frac{1}{2}(\bigtriangledown\phi)^{2}+\frac{1}{8}\left(\frac{\bigtriangledown
n}{n}\right)^{2}+V(x,t)+gn\right]n,$ (2)

where $V(x,t)=\omega^{2}(t)x^{2}/2$. The essence of the variational Lagrangian
formalism is to minimize the action $S={\iint}\mathcal{L}dxdt$ with respect to
the parameters $q_{i}=\\{n,\phi\\}$, that is, $\delta S/\delta q_{i}=0$. The
latter corresponds to the Eular-Lagrangian equations
$\partial\mathcal{L}/\partial
q_{i}-d(\partial\mathcal{L}/\partial\dot{q}_{i})/dx=0$, from which the
hydrodynamic equations are obtained as Dalfovo  _et al._ (1999)

$\displaystyle\frac{\partial n}{\partial
t}+\frac{\partial\left(n\bigtriangledown\phi\right)}{\partial x}=0,$ (3)
$\displaystyle\frac{\partial(\bigtriangledown\phi)}{\partial
t}+\partial_{x}\left(P(x,t)+\frac{1}{2}v^{2}+V(x,t)+gn\right)=0,$ (4)

where $P\left(x,t\right)=-(\partial^{2}_{x}\sqrt{n})/(2\sqrt{n})$ is the so-
called quantum pressure. By inserting the scaling solution $n=n_{0}(x/a)/a$
and $v=\dot{a}x/a$ Dalfovo  _et al._ (1999) into Eq. (4), one can obtain

$-\frac{1}{2}\frac{\ddot{a}}{a}x^{2}=P(x,t)+V(x,t)+gn-P(0,t)-gn(0,0).$ (5)

Multiplying the resulting equation by $n_{0}(\xi)$ after rescaling the spatial
as $\xi=x/a$, and integrating over the coordinates, we get the following
effective Ermakov-like equation Modugno  _et al._ (2018)

$\ddot{a}+\omega^{2}(t)a=\frac{A}{a^{3}}+\frac{B}{a^{2}},$ (6)

where $A$ and $B$ read

$\displaystyle
A=\frac{P(0,0)-E_{k}^{0}}{E_{v}^{0}},~{}~{}B=\frac{gn(0,0)-E_{int}^{0}}{E_{v}^{0}},$
(7)

with $E_{k}^{0}\equiv(1/2)\int[\partial_{x}\sqrt{n_{0}(x)}]^{2}dx$ being the
kinetic energy, $E_{v}^{0}\equiv\int V(x,0)n_{0}(x)dx$ the potential energy,
and $E_{int}^{0}\equiv(g/2)\int n_{0}^{2}(x)dx$ the interaction energy, all
evaluated at the initial time $t=0$. Here $n_{0}(x)\equiv n(x,0)$ represents
the ground state of the system in the harmonic trap. As shown in Ref. Modugno
_et al._ (2018), the parameters $A$ and $B$ satisfy the relation $A+B=1$ for
arbitrary (positive) interactions.

Remarkably, the Ermakov-like Eq. (6) permits to describe the dynamics of the
system in terms of an effective self-similar evolution, for arbitrary
interactions. Note that this equation is more accurate than the one obtained
by applying a Gaussian ansatz for arbitrary interactions,
$\ddot{a}+\omega^{2}(t)a=1/a^{3}+g/(\sqrt{2\pi}a^{2})$ Pérez-García  _et al._
(1996); Huang  _et al._ (2020a). Obviously, Eq. (6) reproduces the exact
scaling in the non-interacting ($g=0$) and Thomas-Fermi limits ($g\gg 1$).
Namely, for $g=0$ the above Eq. (6) corresponds to the original Ermakov
equation,

$\ddot{a}+\omega^{2}(t)a={1/a^{3}},$ (8)

with $n_{0}(x)=e^{-x^{2}/a^{2}}/\sqrt{\pi a^{2}}$ and $A=1$, $B=0$. This is
consistent with the results derived from variational control Pérez-García  _et
al._ (1996); Huang  _et al._ (2020a) and also from the scaling approach and
Lewis-Riesenfeld dynamical invariant Chen  _et al._ (2010); Muga  _et al._
(2009). In the opposite TF regime, the ground state density is
$n_{0}=[\mu-\omega(0)x^{2}/2]/g$, with $\mu=E_{int}^{0}+E_{v}^{0}$ being the
chemical potential (the kinetic energy can be safely neglected in this limit
Dalfovo  _et al._ (1999). Then, inserting $n_{0}$ into Eq. (7) yields ($A=0$,
$B=1$)

$\ddot{a}+\omega^{2}(t)a={1/a^{2}},$ (9)

which corresponds to the exact TF result Castin and Dum (1996); Kagan  _et
al._ (1996); Huang  _et al._ (2020a).

##  III Shortcuts to Adiabaticity

In this section, we use the Ermakov-like equation (6) to construct a STA
protocol for $\omega(t)$, for achieving a frictionless quench from the initial
trap frequency $\omega_{i}\equiv\omega(0)$ (with $\omega(0)=1$ fixed by our
notation choice) to a final value $\omega_{f}\equiv\omega(t_{f})$ within a
short time $t_{f}$, with $\gamma\equiv\sqrt{\omega_{i}/\omega_{f}}>1$. That
is, in a finite-time non-adiabatic expansion, the trap frequency is changed to
some lower final value, while keeping the populations of the initial and final
levels invariant, thus without generating friction and heating. The
frictionless cooling of ultracold atoms trapped in time-dependent harmonic
traps was originally investigated in two noninteracting and TF limits Muga
_et al._ (2009); Chen  _et al._ (2010). Later, the variational approximation
was used to design the same process in the weakly interacting regime Huang
_et al._ (2020a). Here, we propose a general approach based on the formulation
discussed in previous section, for arbitrary values of the interactions,
ranging from the non-interacting to the TF regime.

Figure 1: The designed trap frequency $\omega^{2}(t)$ (a) and the
corresponding width $a(t)$ (b) as a function of time, for $g=0.01$ (red-dashed
line), $g=1$ (blue solid line) and $g=100$ (black dashed line). (c) The final
states (thick lines) are compared to the corresponding stationary states (thin
lines). Parameters: $\omega_{f}=0.1$ and $t_{f}=1$.

To this end, we recast Eq. (6) in the form of the perturbative Kepler problem
in classical mechanics, in the presence of the effective potential

$\mathcal{U}(a)=\frac{A}{2a^{2}}+\frac{B}{a}+\frac{1}{2}\omega^{2}(t)a^{2},$
(10)

for a fictitious particle with unit mass, satisfying the Newton equation
$\ddot{a}=-\partial\mathcal{U}(a)/\partial a$, derived from Eq. (6). The total
energy of the particle reads

$E(a)=\frac{\dot{a}{\color[rgb]{1,0,1}{}^{2}}}{2}+\frac{A}{2a^{2}}+\frac{B}{a}+\frac{1}{2}\omega^{2}(t)a^{2}.$
(11)

The conditions for an adiabatic evolution are $\dot{a}=0$ and
$\partial\mathcal{U}(a)/\partial a=0$ Li  _et al._ (2016), yielding

$a^{4}\omega^{2}(t)-aB=A.$ (12)

Then, we define the time-averaged energy

$\mathcal{E}=\frac{1}{t_{f}}\int_{0}^{t_{f}}E(a)dt,$ (13)

that will be used for quantifying the energetic cost of STA, in the discussion
below. The initial boundary conditions read Li  _et al._ (2016):

$\displaystyle a(0)=a_{\mathrm{i}},~{}\dot{a}(0)=\ddot{a}(0)=0,$ (14)
$\displaystyle a(t_{f})=a_{\mathrm{f}},~{}\dot{a}(t_{f})=\ddot{a}(t_{f})=0,$
(15)

where $a_{\mathrm{i}}$ and $a_{\mathrm{f}}$ are the unique positive real
solutions of Eq. (12), at $t=0$ and $t=t_{f}$. These boundary conditions (14)
and (15) guarantee that the initial and final states are adiabatic
correspondences for designing STA protocols. Having fixed the boundary
conditions, the trajectory of $a(t)$ can be interpolated, by choosing a simple
polynomial ansatz of the form

$a(t)=a_{\mathrm{i}}-6(a_{\mathrm{i}}-a_{\mathrm{f}})s^{5}+15(a_{\mathrm{i}}-a_{\mathrm{f}})s^{4}-10(a_{\mathrm{i}}-a_{\mathrm{f}})s^{3},$
(16)

with $s=t/t_{f}$. Consequently, the trap frequency $\omega(t)$ is determined
by Eq. $(\ref{GEE})$. If an imaginary trap frequency is allowed, the harmonic
trap inverts to a parabolic repeller, instead of a trap, such that $t_{f}$ may
be formally made arbitrarily short. However, since in experimental
implementations there are always imperfections and limitations related, e.g.,
to the trap anharmonicity and to the laser power, this poses a constraint of
the amplitude of the frequency that can be physically achieved, namely
$|\omega^{2}(t)|\leq\delta$, with $\delta$ being a real number Huang  _et al._
(2020a); Stefanatos  _et al._ (2010); Stefanatos and Li (2012).

In Fig. 1 we show the results for a STA protocol of a frictionless quench from
$\omega_{i}=1$ to $\omega_{f}=0.1$, within a time $t_{f}=1$. Panels (a) and
(b) illustrate the the evolution of the trap frequency $\omega(t)$ and of the
width $a(t)$ for different values of the interactions, $g=0.01,1,100$. With
the designed protocol, we use the split-operator method to propagate
numerically the wave function to the final state $|\psi_{f}\rangle$, that is
then compared to the ground state of the final trap
$|\tilde{\psi}_{f}\rangle$, see Fig. 1(c) (the latter is computed by means of
a standard by imaginary-time evolution). The corresponding fidelity,
$F\equiv|\langle\tilde{\psi}_{f}|\psi_{f}\rangle|^{2}$, is shown in Fig. 2 as
a function of the interaction strength $g$, for different values of the trap
frequency. Notice that $F\equiv 1$ in both the non-interacting ($g\ll 1$) and
TF limits ($g\gg 1$), where the scaling ansatz is an exact solution.
Remarkably, the fidelity always stays very close to one even in the
intermediate regime, thanks to the accuracy of the effective scaling approach
Modugno  _et al._ (2018). There the deviations from $F=1$ are less than $1\%$,
and show a weak dependence on the trap frequency, i.e., the fidelity decreases
$\omega_{f}$ is decreased. This is due to the fact that the interaction
becomes dominant when the trap frequency is negligible. Notice also that the
behaviour of the fidelity presented here can be also intuitively understood in
terms of the stability of a particle in the presence of the effective
potential, see Eq. (10).

Figure 2:  The fidelity $F$ (see text) versus the interaction strength $g$,
for a frictionless quench (symbols) and a sudden quench (lines). Three final
frequencies are considered: $\omega_{f}=0.05,0.1,0.2$. Notice that, whereas in
the first case the final time is fixed to $t_{f}=1$, in case of sudden quench
it depends on the (final) frequency, see Eq. (22).

##  IV Sudden Quench

For completeness, here we also consider the case of a sudden quench of the
trap frequency:

$\omega(t)=\begin{cases}~{}\omega_{i},\qquad t=0\\\ ~{}\omega_{f},\qquad
0<t<t_{f}\end{cases}.$ (17)

In the non-interacting case, the conventional Ermakov equation (8) gives the
analytical solution

$a(t)=\sqrt{1+(\omega_{i}^{2}-\omega_{f}^{2})\sin^{2}(\omega_{f}t)/\omega_{f}^{2}},$
(18)

with initial boundary conditions $a(0)=1$, and $\dot{a}(0)=0$, which
corresponds to a collective oscillation with period $\tau=\pi/\omega_{f}$.
Therefore, in general the boundary conditions (14) and (15) cannot be not
attained at $t=t_{f}$, and this implies a heating/excitation of the system.

In the limit $\omega_{f}\to 0$, the case of a free expansion, Eq. (18) reduces
to $a(t)=(1+\omega_{i}^{2}t^{2})^{1/2}$. In more general cases, analytical
solutions for a sudden quenches are not available, but one can still solve Eq.
(6) numerically. Fig. 3, we demonstrate that Eq. (6) is accurate enough to
describe the dynamics in sudden quenches, when the values of nonlinear
interaction is changed from zero to infinity.

Figure 3:  (a) Dynamics of free expansion when the trap is suddenly switched
off, i.e. $\omega_{f}=0$. (b) Collective oscillation modes after quenching
trap frequency to $\omega_{f}=0.1$. For comparison, the center of the mass of
the wave packet, $a(t)=\sqrt{\langle x^{2}\rangle-\langle x\rangle^{2}}$ ,
calculated from numerical simulation, is denoted by symbol. Parameters:
$g=0.01$ (sold line; with red circle), $g=1$ (dashed line, with black
diamond), and $g=100$ (dotted line, with red square), and the initial trap
frequency is $\omega_{0}=1$.

Based on the above considerations, we can build a STA protocol with just two
quenches, as proposed for the compression of solitons in nonlinear fibers
Anderson  _et al._ (1994): an initial quench from $\omega_{i}$ to $\omega_{c}$
at $t=0$, and a second one from $\omega_{c}$ to $\omega_{f}$, at the final
time $t_{f}$ such that

$t_{f}=\pi/\left(2\omega_{c}\right),$ (19)

with $\omega_{c}=\sqrt{\omega_{i}\omega_{f}}$. Then, in the non-interacting
case we have [see again Eqs. (17) and (18)]

$a(t)=\sqrt{1+(\omega^{2}_{i}-\omega_{c}^{2})\sin^{2}(\omega_{c}t)/\omega_{c}^{2}},$
(20)

satisfying the above-mentioned conditions, $a(0)=1$, $a(t_{f})=\gamma$, and
$\dot{a}(0)=\dot{a}(t_{f})=0$ Chen and Muga (2010); Huang  _et al._ (2020a).
Thus, Eqs. (19) and (20) provide a simple exact solution for the shortcut with
just one intermediate frequency, the geometric mean of the initial and final
frequencies. Importantly, the solution can be generalized for Eq. (6), without
requiring a explicit form. The energy conservation in this perturbative Kepler
problem implies $U(a_{\mathrm{i}})=U(a_{\mathrm{f}})$, or, in an explicit
form,

$\omega_{c}=\sqrt{\frac{A}{a_{\mathrm{i}}^{2}a_{\mathrm{f}}^{2}}+\frac{2B}{a_{\mathrm{i}}a_{\mathrm{f}}(a_{\mathrm{i}}+a_{\mathrm{f}})}}.$
(21)

In this case, a simple expression for $t_{f}$ is not available, but it can be
written in the form of an integral:

$t_{f}=\int_{a_{\mathrm{i}}}^{a_{\mathrm{f}}}\frac{da}{\sqrt{2\left[U(a_{\mathrm{in}})-U(a)\right]}},$
(22)

where $a_{\mathrm{i}}$ and $a_{\mathrm{f}}$ are given by Eq. (12).
Accordingly, the trajectory of $a(t)$ can be obtained numerically from Eq.
(6). The corresponding fidelity is plotted in Fig. 2 along with the results of
the frictionless quench. This figure shows that the two-jump STA is less
accurate than the STA by using smooth polynomial function, when the final
frequency is smaller. Another important difference between the two approaches
is the fact that in the STA protocol discussed in the previous section the
final time is a free parameter, whereas in the present case it is fixed by the
trap frequencies and the interaction strength, see Eq. (22). We should
emphasize that the larger repulsive interaction somehow speed up the two-jump
STA e.g. $t_{f}=4.9658$ for $g=0.01$ while $t_{f}=4.5380$ for $g=100$, when
$\omega_{i}=1$ and $\omega_{f}=0.1$.

##  V Discussion

Figure 4:  Dependence of the the time-averaged energy $\mathcal{E}$ in Eq.
(13), on the short time $t_{f}$, where the parameters are the same as those in
Fig. 1.

In the STA protocol, there is a trade-off between speed and cost in shortcuts
to adiabaticity Abah  _et al._ (2019). In principle, the transient energy
excitation of the STA protocol, described by Eq. (11), is stipulated by the
time-energy uncertainty, which implies an increase of the energy for shorter
times. In detail, Fig. 4, illustrates the exponential scaling
$\mathcal{E}\propto 1/t^{2}_{f}$ for different regimes of atomic interaction
by using the STA protocols in Fig. 1, where the time-averaged energy
$\mathcal{E}$ is calculated by Eq. (13). Moreover, for the same $t_{f}$, the
energy excitation is higher when the interaction is lager. In this sense, the
given energetic cost gives the tight bound on the running time of STA.
Additionally, we can obtain other scaling exponential, i.e.
$t_{f}\propto(\omega_{i}/\omega_{f})^{1/2}$ in the non-interacting limit while
$t_{f}\propto(\omega_{i}/\omega_{f})^{3/2}$ in the TF limit Huang  _et al._
(2020a). Instead of using the polynomial and two-jump trajectories, one can
further apply the Pontryagin’s maximum principle in optimal control theory
Stefanatos  _et al._ (2010); Stefanatos and Li (2012) to design the time-
optimal STA with the bounded trap frequency. We expect that the atomic
interaction slows down the frictionless quenches, and the minimal time of STA
in the intermediate interaction regime is between the ones in non-interacting
and TF limits, see Ref. Huang  _et al._ (2020a). Moreover, this has the
fundamental implications on the quantum speed limit and the third law of
thermodynamics in request of absolute zero temperature Rezek  _et al._ (2009).

##  VI Conclusion

To conclude, we have employed the effective scaling approach to derive the
Ermakov-like equation (6) for the scaling parameter interpolating between the
noninteracting and the TF limits. By combining inverse engineering with the
appropriate boundary conditions, this provides a general way to design
accurate STA for 1D Bose gas, for arbitrary values of the atomic interaction
strength. Here we have considered the case of a quasi-1D condensate confined
in cigar-shaped trap, with a tight transverse frequency (that is, much larger
than the longitudinal one). These results can be easily generalized to the 3D
case in different interaction regimes, see Ref. Viedma and Modugno (2020).

In addition, the effective scaling approach is harnessed to design a STA
protocol for a trapped 1D Bose gas in arbitrary repulsive interacting regime,
where the previous methods, such as dynamical invariant or scaling approach,
can not work successfully. We emphasize that the method is similar to, yet
different from, the variational approximation Pérez-García  _et al._ (1996);
Huang  _et al._ (2020a). In the effective scaling approach, the scaling
solution in the hydrodynamic regime is used as ansatz, but the Gaussian-shaped
ground state in non-interacting limit is replaced as a preassumed ansatz in
variational approximation. In this sense, the effective scaling approach has
more reasonable accuracy, when the atomic interaction with the arbitrary value
is considered. Finally, some extensions are interesting for further
exploration, for instance, the soliton dynamics by quenching the interactions
of the BEC from repulsive to attractive Di Carli  _et al._ (2019); Huang  _et
al._ (2020b), or the expansion of a Bose gas in the crossover from TF to TG
regimes Öhberg and Santos (2002b).

## Acknowledgments

This work has been partially supported by NSFC (Grant No. 12075145), STCSM
(Grant No. 2019SHZDZX01-ZX04), and the Program for Eastern Scholar. M.M. and
X. C. acknowledge support by the Spanish Ministry of Science and the European
Regional Development Fund through PGC2018-101355-B-I00 (MCIU/AEI/FEDER, UE)
and the Basque Government through Grant No. IT986-16. X. C. acknowledges
support by the Ramon y Cajal program (Grant No. RYC-2017-22482), the EU FET
Open Grant Quromorphic (Grant No. 828826), and EPIQUS (Grant No. 899368). T.Y.
H. acknowledges support by the CSC fellowship (202006890071).

## References

  * Cornell and Wieman (2002) E. A. Cornell and C. E. Wieman, Rev. Mod. Phys. 74, 875 (2002).
  * Dalfovo  _et al._ (1999) F. Dalfovo, S. Giorgini, L. P. Pitaevskii, and S. Stringari, Rev. Mod. Phys. 71, 463 (1999).
  * Stringari (1996) S. Stringari, Phys. Rev. Lett. 77, 2360 (1996).
  * Castin and Dum (1996) Y. Castin and R. Dum, Phys. Rev. Lett. 77, 5315 (1996).
  * Kagan  _et al._ (1996) Y. Kagan, E. L. Surkov, and G. V. Shlyapnikov, Phys. Rev. A 54, R1753 (1996).
  * Gritsev  _et al._ (2010) V. Gritsev, P. Barmettler, and E. Demler, New Journal of Physics 12, 113005 (2010).
  * Egusquiza  _et al._ (2011) I. L. Egusquiza, M. Modugno, and M. A. Valle Basagoiti, Phys. Rev. A 84, 043629 (2011).
  * Pedri  _et al._ (2003) P. Pedri, L. Santos, P. Öhberg, and S. Stringari, Phys. Rev. A 68, 043601 (2003).
  * Minguzzi and Gangardt (2005) A. Minguzzi and D. M. Gangardt, Phys. Rev. Lett. 94, 240404 (2005).
  * Schäfer (2010) T. Schäfer, Phys. Rev. A 82, 063629 (2010).
  * Bruun and Clark (2000) G. M. Bruun and C. W. Clark, Phys. Rev. A 61, 061601(R) (2000).
  * Guéry-Odelin (2002) D. Guéry-Odelin, Phys. Rev. A 66, 033613 (2002).
  * Menotti  _et al._ (2002) C. Menotti, P. Pedri, and S. Stringari, Phys. Rev. Lett. 89, 250402 (2002).
  * Diana  _et al._ (2006) G. Diana, N. Manini, and L. Salasnich, Phys. Rev. A 73, 065601 (2006).
  * Hu  _et al._ (2003) H. Hu, X.-J. Liu, and M. Modugno, Phys. Rev. A 67, 063614 (2003).
  * Öhberg and Santos (2002a) P. Öhberg and L. Santos, Phys. Rev. Lett. 89, 240402 (2002a).
  * Modugno  _et al._ (2018) M. Modugno, G. Pagnini, and M. A. Valle-Basagoiti, Phys. Rev. A 97, 043604 (2018).
  * Viedma and Modugno (2020) D. Viedma and M. Modugno, Phys. Rev. Research 2, 033478 (2020).
  * Torrontegui  _et al._ (2013) E. Torrontegui, S. Ibáñez, S. Martínez-Garaot, M. Modugno, A. del Campo, D. Guéry-Odelin, A. Ruschhaupt, X. Chen, and J. G. Muga, in  _Advances in Atomic, Molecular, and Optical Physics_, Advances In Atomic, Molecular, and Optical Physics, Vol. 62, edited by E. Arimondo, P. R. Berman, and C. C. Lin (Academic Press, 2013) pp. 117–169.
  * Guéry-Odelin  _et al._ (2019) D. Guéry-Odelin, A. Ruschhaupt, A. Kiely, E. Torrontegui, S. Martínez-Garaot, and J. G. Muga, Rev. Mod. Phys. 91, 045001 (2019).
  * Muga  _et al._ (2009) J. Muga, X. Chen, A. Ruschhaupt, and D. Guéry-Odelin, Journal of Physics B: Atomic, Molecular and Optical Physics 42, 241001 (2009).
  * Chen  _et al._ (2010) X. Chen, A. Ruschhaupt, S. Schmidt, A. del Campo, D. Guéry-Odelin, and J. G. Muga, Phys. Rev. Lett. 104, 063002 (2010).
  * Schaff  _et al._ (2010) J.-F. Schaff, X.-L. Song, P. Vignolo, and G. Labeyrie, Phys. Rev. A 82, 033430 (2010).
  * Schaff  _et al._ (2011) J.-F. Schaff, X.-L. Song, P. Capuzzi, P. Vignolo, and G. Labeyrie, EPL (Europhysics Letters) 93, 23001 (2011).
  * Del Campo (2011) A. Del Campo, EPL (Europhysics Letters) 96, 60005 (2011).
  * Rohringer  _et al._ (2015) W. Rohringer, D. Fischer, F. Steiner, I. E. Mazets, J. Schmiedmayer, and M. Trupke, Scientific reports 5, 9820 (2015).
  * del Campo (2011) A. del Campo, Phys. Rev. A 84, 031606 (2011).
  * Papoular and Stringari (2015) D. J. Papoular and S. Stringari, Phys. Rev. Lett. 115, 025302 (2015).
  * Deng  _et al._ (2018) S. Deng, P. Diao, Q. Yu, A. del Campo, and H. Wu, Phys. Rev. A 97, 013628 (2018).
  * Pérez-García  _et al._ (1996) V. M. Pérez-García, H. Michinel, J. I. Cirac, M. Lewenstein, and P. Zoller, Phys. Rev. Lett. 77, 5320 (1996).
  * García-Ripoll  _et al._ (1999) J. J. García-Ripoll, V. M. Pérez-García, and P. Torres, Phys. Rev. Lett. 83, 1715 (1999).
  * Li  _et al._ (2016) J. Li, K. Sun, and X. Chen, Sci. Rep. 6, 38258 (2016).
  * Huang  _et al._ (2020a) T.-Y. Huang, B. A. Malomed, and X. Chen, Chaos: An Interdisciplinary Journal of Nonlinear Science 30, 053131 (2020a).
  * Huang  _et al._ (2020b) T.-Y. Huang, J. Zhang, J. Li, and X. Chen, Phys. Rev. A 102, 053313 (2020b).
  * Li  _et al._ (2018) J. Li, T. Fogarty, S. Campbell, X. Chen, and Th. Busch, New J. Phys. 20, 015005 (2018).
  * Xu  _et al._ (2020) T.-N. Xu, J. Li, T. Busch, X. Chen, and T. Fogarty, Phys. Rev. Research 2, 023125 (2020).
  * Stefanatos  _et al._ (2010) D. Stefanatos, J. Ruths, and J.-S. Li, Phys. Rev. A 82, 063422 (2010).
  * Stefanatos and Li (2012) D. Stefanatos and J.-S. Li, Phys. Rev. A 86, 063602 (2012).
  * Anderson  _et al._ (1994) D. Anderson, M. Lisak, B. Malomed, and M. Quiroga-Teixeiro, J. Opt. Soc. Am. B 11, 2380 (1994).
  * Chen and Muga (2010) X. Chen and J. G. Muga, Phys. Rev. A 82, 053403 (2010).
  * Abah  _et al._ (2019) O. Abah, R. Puebla, A. Kiely, G. D. Chiara, M. Paternostro, and S. Campbell, New Journal of Physics 21, 103048 (2019).
  * Rezek  _et al._ (2009) Y. Rezek, P. Salamon, K. H. Hoffmann, and R. Kosloff, EPL (Europhysics Letters) 85, 30008 (2009).
  * Di Carli  _et al._ (2019) A. Di Carli, C. D. Colquhoun, G. Henderson, S. Flannigan, G.-L. Oppo, A. J. Daley, S. Kuhr, and E. Haller, Phys. Rev. Lett. 123, 123602 (2019).
  * Öhberg and Santos (2002b) P. Öhberg and L. Santos, Phys. Rev. Lett. 89, 240402 (2002b).


# The Mapillary Traffic Sign Dataset for Detection and Classification on a
Global Scale

Christian Ertler  Jerneja Mislej  Tobias Ollmann  Lorenzo Porzi  Gerhard
Neuhold  Yubin Kuang
_{ christian, jerneja, tobias, lorenzo, gerhard<EMAIL_ADDRESS>
###### Abstract

Traffic signs are essential map features globally in the era of autonomous
driving and smart cities. To develop accurate and robust algorithms for
traffic sign detection and classification, a large-scale and diverse benchmark
dataset is required. In this paper, we introduce a traffic sign benchmark
dataset of $100$$K$ street-level images around the world that encapsulates
diverse scenes, wide coverage of geographical locations, and varying weather
and lighting conditions and covers more than $300$ manually annotated traffic
sign classes. The dataset includes $52$$K$ images that are fully annotated and
$48$$K$ images that are partially annotated. This is the largest and the most
diverse traffic sign dataset consisting of images from all over world with
fine-grained annotations of traffic sign classes. We have run extensive
experiments to establish strong baselines for both the detection and the
classification tasks. In addition, we have verified that the diversity of this
dataset enables effective transfer learning for existing large-scale benchmark
datasets on traffic sign detection and classification. The dataset is freely
available for academic research111www.mapillary.com/dataset/trafficsign.

Figure 1: Taxonomy overview. The sizes are relative to the number of samples
within the Mapillary Traffic Sign Dataset (MTSD).

Dataset | #Images | #Classes | #Signs | Attributes | Region | BBoxes | Unique
---|---|---|---|---|---|---|---
MTSD (TRAIN+VAL) MTSD |  $41,907$ $100,000$ | 313 |  11footnotemark: 1$206,388$ $325,172$ |  occluded, exterior, out-of-frame, dummy, ambiguous, included | world-wide | ✓ |  ✓ ✗44footnotemark: 4
TT100K [36] |  22footnotemark: 2 $100,000$ |  66footnotemark: 6 221 | $26,349$ | ✗ | China | ✓ | ✓
MVD [21] | $20,000$ |  22footnotemark: 22 | $174,541$ | ✗ | world-wide | ✓ | ✓
GTSDB [9] | $900$ | 43 | $852$ | ✗ | Germany | ✓ | ✗
RTSD [25] |  33footnotemark: 3$179,138$ | 156 |  33footnotemark: 3$104,358$ | ✗ | Russia | ✓ | ✗
STS [12] | $3777$ | 20 | $5582$ | ✗ | Sweden | ✓ | ✗
LISA [20] | $6610$ | 47 | $7855$ | ✗ | USA | ✓ | ✗
GTSRB [30] | ✗ | 43 | $39,210$ | ✗ | Germany | ✗ | ✗
BelgiumTS [31] | ✗ | 108 | $8851$ | ✗ | Belgium | ✗ | ✗
Table 1: Overview of traffic sign datasets. The numbers include only publicly
available images and annotations (_e.g_. we only report numbers for the
training and validation set for MTSD). _Unique_ refers to datasets where each
traffic sign bounding box corresponds to a unique traffic sign instance
(_i.e_. no sequences showing the same sign again and again). 11footnotemark:
1$66,138$ signs are within the taxonomy. 22footnotemark: 2 TT100K contains
only $10,000$ images containing traffic signs. 66footnotemark: 6 only 45
classes have more than 100 examples. 22footnotemark: 2 MVD contains only back
_vs._ front classes. 33footnotemark: 3video-frames covering only $15,630$
unique signs. 44footnotemark: 4signs within the partially annotated set
correspond to signs within the training set.

##  1 Introduction

Robust and accurate object detection and classification in diverse scenes is
one of the essential tasks in computer vision. With the development and
application of deep learning in computer vision, object detection and
recognition has been studied [5, 23, 16] extensively on general scene
understanding datasets [17, 4, 10]. In terms of fine-grained detection and
classification, there are also the datasets that focused on general
hierarchical object classes [10] or domain-specific datasets, _e.g_. on bird
species [33]. In this paper, we will focus on detection and fine-grained
classification of traffic signs.

Traffic signs are key map features for navigation, traffic control, and road
safety. Specifically, traffic signs encode information for driving directions,
traffic regulation, and early warning. For autonomous driving, accurate and
robust perception of traffic signs is essential for localization and motion
planning.

As an object class, traffic signs have specific characteristics in their
appearance. First of all, traffic signs are in general rigid and planar.
Secondly, traffic signs are designed to be distinctive from their
surroundings. In addition, there is limited variety in colors and shapes for
traffic signs. For instance, regulatory signs in European countries are
typically circular with a red border. To some degree, the aforementioned
characteristics limit the appearance variation and increase the distinctness
of traffic signs. However, traffic sign detection and classification is still
a very challenging problem due to the the following reasons:  (1) traffic
signs are easily confused with other object classes in street scenes (_e.g_.
advertisements, banners, and billboards);  (2) reflection, low light
condition, damages, and occlusion hinder the classification performance of a
sign class;  (3) fine-grained classification with small inter-class difference
is not trivial;  (4) the majority of traffic signs—when appearing in street-
level images—are relatively small in size, which requires efficient
architecture designs for small objects.

Traffic sign detection and classification has been studied extensively in the
computer vision community. Specifically, convolutional neural networks (CNN)
[24] have obtained great success for traffic sign classification in the German
Traffic Sign Benchmark [29]. Recent works on simultaneous detection and
classification of traffic signs have also achieved good results on well-
studied benchmark datasets [19, 36] using either the Viola-Jones framework
[32] or CNN-based methods [13]. However, these studies were done in relatively
constrained settings in terms of the benchmark dataset: the images and traffic
signs are collected in a specific country; the number of traffic sign classes
is relatively small; the images lack diversity in weather conditions, camera
sensors, and seasonal changes.

Extensive research is still needed for the task of detecting and classifying
traffic signs at a global scale and under varying capture conditions and
devices. In this paper, we present the following contributions:

  * •

We present the most diverse traffic sign dataset with $100$$K$ images from all
over the world. The dataset contains over $52$$K$ images that are fully
annotated, covering $313$ known traffic sign classes and other unknown
classes, resulting in over $250$$K$ signs in total. Additionally, the dataset
also includes about 48K images, where traffic signs are partially annotated by
automatically propagating labels between neighboring images.

  * •

We establish extensive baselines for detection and classification on the
dataset, shedding light on future research directions.

  * •

We study the impact of transfer learning using our traffic sign dataset and
other traffic sign datasets released in the past. Our results show that pre-
training on our dataset boosts the average precision ($\mathrm{AP}$) of the
binary detection task by $46\text{\,}\mathrm{\char 37\relax}$, thanks to the
completeness and diversity of our dataset.

Related Work. Traffic sign detection and recognition has been studied
extensively in the literature in the past. The German Traffic Sign Benchmark
Dataset (GTSBD) [29] is one of the first datasets that was created to evaluate
the classification branch of the problem. Following that, there have also been
other traffic sign datasets focusing on regional traffic signs, _e.g_. Swedish
Traffic Sign Dataset [11], Belgium Traffic Sign Dataset [19], Russian Traffic
Sign Dataset [26], and Tsinghua-Tencent Dataset (TT100K) in China [36]. For
generic traffic sign detection (where no class information of the traffic
signs is available), there has been work done in the Mapillary Vistas Dataset
(MVD) [21] (global) and BDD100K [35] (US only). A detailed overview and
comparison of publicly available traffic sign datasets can be found in
Footnote 4.

For general object detection, there has been substantial work on CNN-based
methods with two main directions, _i.e_. one-stage detectors [18, 22, 16] and
two-stage detectors [6, 5, 23, 3]. One-stage detectors are generally much
faster, trading off accuracy compared to two-stage detectors. One exception is
the one-stage RetinaNet [16] architecture that outperforms the two-stage
Faster-RCNN [23] thanks to a weighting scheme during training to suppress
trivial negative supervision. For simultaneous detection and classification,
one of the recent works [2] shows that decoupling the classification from
detection head boosts the accuracy significantly. Our work is related to [2]
as we also decouple the detector from the traffic sign classifier.

To handle the scale variation of objects in the scene, many efficient multi-
scale training and inference algorithms have been proposed and evaluated on
existing datasets. For multi-scale training, in [27, 28, 14], a few schemes
have been proposed to distill supervision information from different scales
efficiently by selective gradient propagation and crop generation. To enable
efficient multi-scale inference, feature pyramid networks (FPN) [15] were
proposed to utilize lateral connections in a top-down architecture to
construct effective multi-scale feature pyramid from a single image.

To develop the baselines presented in this paper, we have chosen Faster-RCNN
[23] with FPN [15] as the backbone. Given the aforementioned characteristics
of traffic sign imagery, we have also trained a separate classifier for fine-
grained classification as in [2]. We elaborate the details of our baseline
method in Section 4.

##  2 Mapillary Traffic Sign Dataset

In this section, we present a large-scale traffic sign dataset called
Mapillary Traffic Sign Dataset (MTSD) including $52,453$ images with fully
annotated traffic sign bounding boxes and corresponding class labels.
Additionally, it includes a set of $47,547$ nearby images with $87,358$
automatically generated labels, making it a total of $100,000$ images. In the
following sections we describe how the dataset was created and present our
traffic sign class taxonomy consisting of 313 classes.

###  2.1 Image Selection

Figure 2: Geographical distribution of the images within MTSD.

There are various conventions for traffic signs in different parts of the
world, leading to strong appearance differences. Even within a single country
or state, the distribution of signs is not uniform: some signs occur only in
urban areas, some only on highways, and others only in rural areas. With MTSD,
we present a dataset that covers this diversity uniformly. In order to do so,
a proper pre-selection of images for annotation is crucial. The requirements
for this selection step are:  (1) to have a uniform geographical distribution
of images around the world,  (2) to cover images of different quality,
captured under varying conditions,  (3) to include as many signs as possible
per image, and  (4) to compensate for the long-tailed distribution of
potential traffic sign classes.

_Mapillary_ 222www.mapillary.com/app is a street-level imagery platform
hosting images collected by members of their community. All images are
accessible to everyone via Mapillary’s public API under a CC-BY-SA license.
Derived data including traffic sign detections are also available for academic
research and approved applications.

In order to get a pool of pre-selected images satisfying the aforementioned
requirements, we sample images in a per-country manner with a greedy scheme.
The fraction of target images for each country is derived from the number of
images available in that country and its population count weighted by a global
target distribution over all continents (_i.e_. $20\text{\,}\mathrm{\char
37\relax}$ North America, $20\text{\,}\mathrm{\char 37\relax}$ Europe,
$20\text{\,}\mathrm{\char 37\relax}$ Asia, $15\text{\,}\mathrm{\char
37\relax}$ South America, $15\text{\,}\mathrm{\char 37\relax}$ Oceania,
$10\text{\,}\mathrm{\char 37\relax}$ Africa). We further make sure to cover
both rural and urban areas within each country by binning the sampled images
uniformly in terms of their geographical locations and sample random images
from each of the resulting bins. In the last step of our greedy image sampling
scheme, we prioritize images containing at least one traffic sign instance
according to the traffic sign detections given by the Mapillary API and make
sure to cover various image resolutions, camera manufacturers and scene
properties333Details on how scene properties are defined and derived are
included in the supplementary materials.. Additionally, we add a distance
constraint so that selected images are far away from each other in order to
avoid highly correlated images and traffic sign instances.

The heat map in Figure 2 shows the resulting geographical distribution of the
images within the dataset, covering almost all habitable areas of the world
with higher density in populous areas. Statistics of the final dataset can be
found in Section 3.

###  2.2 Annotation Process

The process of annotating an image including image selection approval, traffic
sign localization by drawing bounding boxes, and class label assignment for
each box is a complex and demanding task. To improve efficiency and quality,
we split it into 3 consecutive tasks, with each having its own quality
assurance process. All tasks were done by $15$ experts in image annotation
after being trained with explicit specifications for each task.

Image Approval.

Since initial image selection was done automatically based on the greedy
heuristics described in Section 2.1, the annotators needed to reject images
that did not fulfill our criteria for the dataset. In particular, we do not
include non-street level images or images that have been taken from unusual
places or viewpoints. Also we discarded images of very bad quality that could
not be used for training at all (_i.e_. extremely blurry or overexposed).
However, we still sample images of low quality in the dataset which include
recognizable traffic signs as these are good examples to evaluate recognition
on traffic signs in real-world scenarios.

Sign Localization.

In this task, the annotators were instructed to localize all traffic signs in
the images and annotate them with bounding boxes. In contrast to previous
traffic sign datasets where only specific types of traffic signs have been
annotated (_e.g_. TT100K [36] includes only standard circular and triangular
shaped signs), MTSD contains bounding boxes for all types of traffic related
signs including direction, information, highway signs, _etc_.

To speed up the annotation process, each image was initialized with bounding
boxes of traffic signs extracted from the _Mapillary_ API. The annotators were
asked to correct all existing bounding boxes to tightly contain the signs (or
reject them in cases of false positives) and to annotate all missing traffic
signs if their shorter sides were larger than
$10\text{\,}\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$. We
provide a statistical evaluation of the manual changes done by the annotators
in Section 3.3.

Sign Classification.

This task was done independently for each annotated traffic sign. Each traffic
sign (together with some image context) was shown to the annotators who were
asked to provide the correct class label. This is not trivial, since the
number of possible traffic sign classes is large. To the best of our
knowledge, there is no globally valid traffic sign taxonomy available; even
then, it would be impossible for the annotators to keep track of all the
different traffic sign classes.

To overcome this issue, we used a set of previously harvested template images
of traffic signs from Wikimedia Commons [1] and grouped them by similarity in
appearance and semantics. This set of templates (together with their grouping)
defines the possible set of traffic sign classes that can be selected by the
annotators. In fact, we store an identifier of the actual selected template,
which allows us to link the traffic sign instances to our flexible traffic
sign taxonomy without even knowing the final set of classes beforehand (see
Section 2.3).

Since it would still be too time-consuming to scroll through the entire list
of templates to choose the correct one out of thousands, we trained a neural
network (with the grouped template images) to predict the similarities between
an arbitrary image of a traffic sign instance and the templates. We used this
proposal network to assist the annotators in choosing the correct template by
pre-sorting the template list for each individual traffic sign. For cases in
which this strategy fails to provide a matching template, we provided a text-
based search for templates. For details about the annotation UI and how the
proposal network was used to assist the annotator we refer to the supplemental
material.

Additional Attributes.

In addition to the bounding boxes and the matching traffic sign templates, the
annotators were asked to provide additional attributes for each sign:
_occluded_ if the sign is partly occluded; _ambiguous_ if the sign is not
classifiable at all (_e.g_. too small, bad quality, heavy occlusion _etc_.);
_dummy_ if it looks like a sign but isn’t (_e.g_. car stickers, reflections,
_etc_.); _out-of-frame_ if the sign is cut off by the image border; _included_
if the sign is part of another bigger sign; and _exterior_ if the sign
includes other signs. Some of these attributes were assigned during
localization (if context information is needed). The rest was assigned during
classification. In Section 4 we describe how we use some of these attributes
to guide the training of our traffic sign detector.

Annotation Quality.

All annotations in MTSD were done by expert annotators going through a
thorough training process. Their work was monitored by a continuous quality
control (QC) process to quickly identify problems during annotations.
Moreover, our step-wise annotation process (_i.e_. approval followed by
localization followed by classification) ensures that each traffic sign was
seen by at least two annotators. The second annotator operating in the
classification step was able to reject false positive signs or to report
issues with the bounding box in which case the containing image was sent back
to the localization step.

In additional quality assurance (QA) experiments done by a 2 annotator on
$5$$K$ images including $26$$K$ traffic signs we found that

  * •

only $0.5\text{\,}\mathrm{\char 37\relax}$ of bounding boxes needed
correction.

  * •

the false negative rate was $0.89\text{\,}\mathrm{\char 37\relax}$
(corresponding to a total number of only $212$ missing signs; most of them
being very small).

  * •

the false positive rate was at $2.45\text{\,}\mathrm{\char 37\relax}$. Note
that is in the localization step before classification, where a second
annotator would have been asked to classify the sign and could potentially fix
false positives.

###  2.3 Traffic Sign Class Taxonomy

Traffic signs vary across different countries. For many countries, there
exists no publicly available and complete catalogue of signs. The lack of a
known set of traffic sign classes leads to challenges in assigning class
labels to traffic signs annotated in MTSD. The potential magnitude of this
unknown set of traffic signs is in the thousands as indicated by the set of
template images described in Section 2.2.

For MTSD, we did a manual inspection of the templates that have been chosen by
the annotators and selected a subset of them to form the final set of $313$
classes included in the dataset as visualized in Figure 1. This subset was
chosen and grouped such that there are no overlaps or confusion (visual or
semantic) among the classes. Templates with the same semantics and similar
appearance form a class. However, different groups of templates that share the
same semantics but are different in terms of appearance form different classes
as shown in Figure 3.

All these classes defined by disjoint sets of templates build up our traffic
sign class taxonomy. We map all annotated traffic signs in MTSD that have a
template selected within this taxonomy to a class label. We would like to
emphasize that our flexible traffic sign taxonomy allows us to incrementally
extend MTSD by adding more classes together with already annotated traffic
sign instances with known templates.

Figure 3: Example templates in traffic sign taxonomy. Each row represent a
traffic sign class based on semantics and appearance.

###  2.4 Partial Annotations

In addition to the fully-annotated images, we provide another set of images
with partially annotated bounding boxes and labels of traffic signs. Given the
fully-annotated images, the annotations of this set of images are generated
automatically. We achieve this by finding correspondences between the manual
annotations in the fully-annotated images and automatic detections in
geographically neighboring images from the Mapillary API. To find these
correspondences, we first use Structure from Motion (SfM) [7] to recover the
relative camera poses between the fully-annotated images and the partially
annotated images. With these estimated relative poses, we generate the
correspondences between annotated signs and automatically detected signs by
triangulating and verifying the re-projection errors for the centers of the
bounding boxes between multiple images. Having these correspondences, we
propagate the manually annotated class labels to the automatic detections in
the partially annotated images. Since there is no guarantee that all traffic
signs are detected through Mapillary’s platform, we obtained a set of images
with partially annotated bounding box annotations and labels. Note that, for
unbiased evaluation, we ensure that the extension is done only in the
geographical neighborhood of images in the training set (based on the split
discussed in Section 2.5). Example images can be found in Figure 4. A more
detailed description of how this set was created can be found in supplemental
material.

Figure 4: Example from the partially annotated set: The leftmost image is from
the fully annotated set. The other 3 images show the same sign from different
perspectives in the partially annotated set. Best viewed zoomed in.

We see this set of partially annotated images serving as a data source for
further research in semi-supervised learning for traffic sign detection and
recognition. In addition, the correspondence information between the traffic
sign observations will pave the ways for other learning tasks like semantic
matching with street-level objects.

###  2.5 Dataset Splits

As common practice with other datasets such as COCO [17], MVD [21] and PASCAL
VOC [4], we split MTSD into training, validation and test sets, consisting of
$36,589$, $5320$, and $10,544$ images, respectively. We provide the image data
for all images as well as the annotations for the training and validation set;
the annotations for the test set will not be released in order to ensure a
fair evaluation. Additionally, we provide a set of $47,547$ images with
partial annotations as discussed in Section 2.4 that can be used for training
as well.

Each split is created in a way to match the distributions described in Section
2.1. Especially, we ensure that the distribution of class instances is similar
for each split, to avoid that rare classes are under-represented in the
smaller sets (_i.e_. validation/test sets). The same holds true for the
additional sign attributes (_e.g_. _ambiguous_ , _etc_.).

##  3 Statistics

In this section, we provide statistics of image and traffic sign properties of
MTSD and also a comparison with previous datasets such as TT100K [36] and MVD
[21]. Furthermore, we provide statistical insights about annotator actions
during the creation of MTSD.

###  3.1 Image Properties

Figure 5: Distribution of camera devices used for image capture.

For a diverse dataset to reflect a real-world image capturing setting, it is
important to cover a broad range of different image qualities and other image
properties such as aspect ratio, focal length and other sensor-specific
properties. The image selection strategy described in Section 2.1 used for
MTSD ensures a good distribution over different capturing settings. In Figure
5, we show the distribution of camera manufacturers used for capturing the
images of MTSD. In total, the dataset covers over $200$ different sensor
manufacturers (we group the tail of the distribution for displaying purposes)
which results in a large variety of image properties similar to the properties
described in [21]. This is in contrast to the setup used for the TT100K [36]
which contains only images taken by a single sensor setup, making MTSD more
challenging in comparison.

Figure 6: Distribution of image sizes (square root of pixel area).

The diversity in camera sensors further results in a diverse distribution over
image resolutions as shown in Figure 6. MTSD covers a broad range of image
sizes starting from low-resolution images with
$1\text{\,}\mathrm{M}\mathrm{P}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\
$ going up to images of more than
$16\text{\,}\mathrm{M}\mathrm{P}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$.
Additionally, we include $1138$ 360-degree panoramas stored as standard images
with equi-rectangular projection. Besides the overall larger image volume
compared to other datasets, MTSD also covers a larger fraction of low-
resolution images, which is especially interesting for pre-training and
validating detectors applied on similar sensors e.g. built-in automotive
cameras. For comparison, TT100K only contains images of
$2048^{2}\text{\,}\mathrm{px}$ and even for this resolution the volume of
images is smaller than in MTSD.

###  3.2 Traffic Sign Properties

Figure 7: _Left:_ Number of traffic sign classes; _Middle:_ Number of signs
binned by size; _Right:_ Number of images binned by number of traffic sign
instances. Figure 8: Example images in MTSD from different geographical
locations under varying lighting and weather conditions. Top: Bounding box and
class label annotation of traffic signs (green boxes without neighboring
template indicts a _other-sign_); Bottom: Results from our detection and
classification baseline on the validation set (green colored: true positive,
red: missing detections)

The fully-annotated set of MTSD includes a total number of $257,543$ traffic
sign bounding boxes out of which $82,724$ have a class label within our
traffic sign taxonomy covering $313$ different traffic sign classes. The
remaining traffic signs sum up as $85,122$ ambiguous signs, $23,407$
directional signs, $9141$ information signs, $3416$ highway shields, $6451$
exterior signs, $1917$ barrier signs, $23,468$ signs without a selected
template, and $21,897$ signs with a template not included by our current
taxonomy (but potentially in future releases of the dataset).

The left plot in Figure 7 shows a comparison of the traffic sign class
distribution between MTSD and TT100K. Note that MVD is not included here since
it does not have labels of traffic sign classes. MTSD has approximately twice
as many traffic sign classes than TT100K; if we use the definition of a
trainable class in [36] (which are classes with at least $100$ traffic sign
instances within the dataset) this factor increases to approximately $3$
between TT100K and MTSD. This difference gets even higher if we consider the
instances from the partially annotated set of MTSD as well.

The plot in the middle of Figure 7 compares the areas of signs in terms of
pixels in the original resolution of the containing image. MTSD covers a broad
range of traffic sign sizes with an almost uniform distribution up to
$256^{2}\text{\,}\mathrm{px}$. MVD has a similar distribution with a lower
overall volume. In comparison to TT100K, MTSD provides a higher fraction of
extreme sizes which poses another challenge for traffic sign detection.

Finally, the plot on the right of Figure 7 shows the distribution of images
over the number of signs within the image. Besides the higher volume of
images, MTSD contains a larger fraction of images with a large number of
traffic sign instances (_i.e_. $>12$). One reason for this is that the
annotations in MTSD cover all types of traffic signs, whereas TT100K only
contains annotations for very specific types of traffic signs in China.

###  3.3 Annotator Interactions

| Count | Mean |
---|---|---|---
Images worked on | $52,608$ | - |
Signs worked on | $266,238$ | - |
Originated from detection | $128,601$ | $0.52$ |
IoU with original detection | - | $0.76$ |
New signs per image | - | $2.63$ |
Table 2: Statistics of manual annotator interactions.

To gather some insights about the work of the annotators, we analyzed their
interactions with the bounding boxes fetched from the _Mapillary_ API as
described in Section 2.2 and present the results in Table 2. We found that
$52\text{\,}\mathrm{\char 37\relax}$ of the bounding boxes annotated in MTSD
originated from an automatic detection already present. However, when
comparing the final boxes within MTSD to the original ones, we find an overlap
of only $76\text{\,}\mathrm{\char 37\relax}$ in terms of IoU, proving the
improvement of detection accuracy. Additionally, we found that the annotators
on average added approximately $3$ completely new bounding boxes that were
missing before in each image.

##  4 Traffic Sign Detection

The first task defined on MTSD is detecting traffic signs as bounding boxes
without inferring the specific class labels. The goal is to predict a set of
bounding boxes with corresponding confidence scores as traffic signs.

Metrics.

Given a set of detections with estimated scores for each image, we first
compute the matching between the detections and annotated ground truth within
each image separately. A detection can be successfully matched to a ground
truth if their Jaccard overlap ($\mathrm{IoU}$) [4] is $>0.5$; if multiple
detections match the same ground truth, only the detection with the highest
score is a match while the rest is not (_double detections_); each detection
will only be matched to one ground truth bounding box with the highest
overlap.

Having this matching indicator (TP vs. FP) for every detection, we define
define average precision ($\mathrm{AP}$) similar to COCO [17] (_i.e_.
$\mathrm{AP}^{\mathrm{IoU=}0.5}$ which resembles $\mathrm{AP}$ definition of
PASCAL VOC [4]) and compute precision as a function of recall by sorting the
matching indicators by their corresponding detection confidence scores in
descending order and accumulate the number of TPs and FPs. $\mathrm{AP}$ is
then defined as the area under the curve of this step function. Additionally,
we follow [17] and compute $\mathrm{AP}$ for traffic signs of different
scales: $\mathrm{AP}_{\mathrm{s}}$, $\mathrm{AP}_{\mathrm{m}}$, and
$\mathrm{AP}_{\mathrm{l}}$ refer to $\mathrm{AP}$ computed for boxes with area
$a<32^{2}$, $32^{2}<a<96^{2}$, and $a>96^{2}$, respectively.

Baseline and Results.

In Footnote 1, we show experimental results using a Faster R-CNN based
detector [23] with FPN [15] and residual networks [8] as the backbone.

During training we randomly sample crops of size $$1000$\times$1000$$ at full
resolution instead of down-scaling the image to avoid vanishing of small
traffic signs, as traffic signs can be very small in terms of pixels and MTSD
covers traffic signs from a broad range of scales in different image
resolutions. We use a batch size of 16 distributed over 4 GPUs during training
for the ResNet50 models; for the ResNet101 version, we use batches of size 8.
Unless stated otherwise, we train using stochastic gradient descent (SGD) with
an initial learning rate of $1\text{\times}{10}^{-2}$ and lower the learning
rate when the validation error plateaus. For inference, we down-scale the
input images such that their larger side does not exceed a certain number of
pixels (either $2048\text{\,}\mathrm{px}$ or $4000\text{\,}\mathrm{px}$) or
operate on full resolution if the original image is smaller.

Besides training on MTSD, we conduct transfer-learning experiments on TT100K
and MVD444We convert the segmentation of _traffic-sign–front_ instances to
bounding boxes by taking the minimum and maximum in the x, y axes. Note that
this conversion can be inaccurate if signs are occluded by other objects. to
test the generalization properties of the proposed dataset. We use the same
baseline as for the MTSD experiments and train it on both datasets, once with
ImageNet initialization and once with MTSD initialization. The models trained
with ImageNet initialization are trained to convergence. To ensure a fair
comparison, we fine-tune only for half the number of epochs when initializing
with MTSD weights. The results in Footnote 1 show that MTSD pre-training
boosts detection performance by a large margin on both datasets regardless of
the input resolution. This is a clear indication for the generalization
qualities of MTSD.

| Max 4000px | Max 2048px
---|---|---
| $\mathrm{AP}$ | $\mathrm{AP}_{\mathrm{s}}$ | $\mathrm{AP}_{\mathrm{m}}$ | $\mathrm{AP}_{\mathrm{l}}$ | $\mathrm{AP}$ | $\mathrm{AP}_{\mathrm{s}}$ | $\mathrm{AP}_{\mathrm{m}}$ | $\mathrm{AP}_{\mathrm{l}}$
MTSD
ResNet50 FPN | $87.3$ | $73.03$ | $91.91$ | $93.56$ | $80.22$ | $52.31$ | $88.87$ | $94.73$
ResNet101 FPN | $88.44$ | $74.00$ | $92.14$ | $93.70$ | $81.80$ | $56.55$ | $89.22$ | $94.82$
TT100K
[36] multi-scale11footnotemark: 1 | $91.79$ | $84.56$ | $96.40$ | $92.60$ | - | - | - | -
ResNet50 FPN | - | - | - | - | $91.27$ | $84.01$ | $95.87$ | $90.13$
\+ pre-trained on MTSD |  | - | - | - |  $97.60$ (+$6.33$) | $93.13$ | $99.03$ | $98.44$
MVD (traffic signs)
ResNet50 FPN | $72.90$ | $46.60$ | $79.93$ | $85.42$ | $64.00$ | $30.70$ | $75.28$ | $86.50$
\+ pre-trained on MTSD |  $76.31$ (+$3.41$) | $51.00$ | $83.49$ | $88.33$ |  $68.29$ (+$4.29$) | $33.60$ | $79.45$ | $89.53$
Table 3: Detection results on MTSD, TT100K and MVD. Numbers in brackets refer to absolute improvements when pre-training on MTSD in comparison to ImageNet. 11footnotemark: 1 They evaluate using multi-scale inference with scales $0.5$, $1$, $2$, and $4$. | $\mathrm{mAP}$ | $\mathrm{mAP}_{\mathrm{s}}$ | $\mathrm{mAP}_{\mathrm{m}}$ | $\mathrm{mAP}_{\mathrm{l}}$
---|---|---|---|---
MTSD
FPN50 + classifier | $81.1$ | $69.4$ | $85.0$ | $87.2$
FPN101 + classifier | $83.4$ | $76.4$ | $85.8$ | $87.3$
TT100K
[36] multi-scale | $81.6$ | $68.3$ | $86.5$ | $85.7$
FPN50 + classifier |  $89.9$ (+$8.3$) | $83.9$ | $93.0$ | $84.3$
\+ _det pre-trained_ |  $93.4$ (+$11.8$) | $88.2$ | $94.8$ | $93.6$
\+ _cls pre-trained_ |  $95.7$ (+$14.1$) | $91.3$ | $96.9$ | $96.7$
Table 4: Simultaneous detection and classification results. The numbers in
brackets are absolute improvements over [36]. _det pre-trained_ and _cls pre-
trained_ refer to experiments with additionally MTSD pre-trained detector and
classifier, respectively.

##  5 Simultaneous Detection and Classification

Figure 9: Network architecture of the baseline classifier. Figure 10: Failure
cases of the baseline classification network on MTSD.

The second task on MTSD is simultaneous detection and classification of
traffic signs (_i.e_. multi-class detection). It extends the detection task to
additionally predict a class label for each traffic sign instance that is in
our taxonomy. For the traffic signs that do not have a label within our
taxonomy we introduce a general class _other-sign_.

Metric.

The main metric for this task is mean average precision ($\mathrm{mAP}$) over
all $313$ classes; per-class $\mathrm{AP}$ is calculated as described in
Section 4. The matching between predicted and ground truth boxes is done in a
binary way by ignoring the class label. After that, we filter out all _other-
sign_ ground-truth instances and detections since we do not want to evaluate
on this general class.

Baseline.

A trivial baseline for this task would be to extend the binary detection
baseline from Section 4 to the multi-class setting by adding a $314$-way
classification head. However, preliminary experiments showed that a straight-
forward training of such a network does not yield acceptable performance. We
hypothesize that this is due to  (1) scale issues for small signs before RoI
pooling and,  (2) under-represented class variation within the training
batches given that the majority of traffic sign instances are _other-sign_.

To overcome the scale issue and to have better control over batch statistics
during training, we opted for a two-stage architecture with using our binary
detector in the first stage and a decoupled shallow classification network in
the second stage. This form of decoupling has been shown to improve the
detection and recognition accuracy [2]. The classification network consists of
seven $3\times 3$ convolutions (each followed by batch normalization) with 2
max-pooling layers after the $2^{nd}$ and $6^{th}$ convolution layer. The last
convolution is followed by spatial average pooling and a fully-connected stage
resulting in a $314$-way classification head with softmax activation ($313$
and _other-sign_) and a single sigmoid activation for foreground/background
classification. The network architecture is depicted in Figure 9.

We use image crops predicted by the detector (both foreground and background)
together with crops from the ground-truth as input during training and
optimize the network using cross-entropy loss. To balance the distribution of
traffic sign classes in a batch, we uniformly sample $128$ different classes
with 3 samples each class and add another $128$ background crops per batch. We
train the network with SGD for $30$ epochs starting with a learning rate of
$1\text{\times}{10}^{-2}$, lowered by a factor of $0.1$ after $10$ and $20$
epochs.

Results.

We show the results of our baseline in Table 4. Our classifier in combination
with ResNet101 binary detector reaches $83.4$ $\mathrm{mAP}$ over all $313$
classes; the ResNet50 variant is only about $2$ points lower. Figure 8 shows
visual examples of our baseline’s predictions and Figure 10 shows typical
failure cases of the classification network.

To verify our baseline, we train with the same setup on TT100K and compare the
results with the baseline in [36]555We convert their results to the format
used by MTSD and evaluate using our metrics.. Our two-stage approach
outperforms their baseline by $8.3$ points, even though the performances of
the binary detectors are similar (see Footnote 1). This validates that the
decoupled classifier (even with a very shallow network) is able to yield good
results. Moreover, the accuracy is improved further after pre-training the
classifier (and the detector) on MTSD before fine-tuning it on TT100K, which
further validate the generalization effectiveness of the MTSD.

##  6 Conclusion

In this work, we have introduced MTSD, a large-scale traffic sign benchmark
dataset that includes $100$$K$ images with full and partial bounding-box
annotations, covering $313$ traffic sign classes from all over the world. MTSD
is the most diverse traffic sign benchmark dataset in terms of geographical
locations, scene characteristics, and traffic sign classes. We have shown in
baseline experiments that decoupling detection and fine-grained classification
yields superior results on previous traffic sign datasets. Additionally, in
transfer-learning experiments, we show that MTSD facilitates fine-tuning and
improves accuracy substantially for traffic sign datasets in a narrow domain.

We see MTSD as the first step to drive the research efforts towards solving
fine-grained traffic sign detection and classification at a global scale. With
the partial annotated dataset, we would also like to pave the way for further
research in semi-supervised learning. In the future, we would like to extend
the dataset towards a complete traffic sign taxonomy globally. To achieve
this, we see the potential of applying zero-shot learning to efficiently model
the semantic and appearance attributes of traffic sign classes. With the
global taxonomy built, we can optimize the performance further with
hierarchical classification [22, 34].

## References

  * [1] Wikimedia commons.  https://commons.wikimedia.org.  Accessed: 2019-03-13.
  * [2] B. Cheng, Y. Wei, H. Shi, R. Feris, J. Xiong, and T. Huang.  Revisiting rcnn: On awakening the classification power of faster rcnn.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 453–468, 2018.
  * [3] J. Dai, Y. Li, K. He, and J. Sun.  R-fcn: Object detection via region-based fully convolutional networks.  In Advances in neural information processing systems, pages 379–387, 2016.
  * [4] M. Everingham, S. A. Eslami, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman.  The pascal visual object classes challenge: A retrospective.  International journal of computer vision, 111(1):98–136, 2015.
  * [5] R. Girshick.  Fast r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015.
  * [6] R. Girshick, J. Donahue, T. Darrell, and J. Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.
  * [7] R. Hartley and A. Zisserman.  Multiple View Geometry in Computer Vision.  Cambridge university press, 2003.
  * [8] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [9] S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing, and C. Igel.  Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark.  In International Joint Conference on Neural Networks, number 1288, 2013.
  * [10] A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, T. Duerig, et al.  The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale.  arXiv preprint arXiv:1811.00982, 2018.
  * [11] F. Larsson and M. Felsberg.  Using fourier descriptors and spatial models for traffic sign recognition.  In Scandinavian conference on image analysis, pages 238–249. Springer, 2011.
  * [12] F. Larsson, M. Felsberg, and P.-E. Forssen.  Correlating Fourier descriptors of local patches for road sign recognition.  IET Computer Vision, 5(4):244–254, 2011.
  * [13] J. Li, X. Liang, Y. Wei, T. Xu, J. Feng, and S. Yan.  Perceptual generative adversarial networks for small object detection.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1222–1230, 2017.
  * [14] Y. Li, Y. Chen, N. Wang, and Z. Zhang.  Scale-aware trident networks for object detection.  arXiv preprint arXiv:1901.01892, 2019.
  * [15] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie.  Feature pyramid networks for object detection.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2117–2125, 2017.
  * [16] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár.  Focal loss for dense object detection.  In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
  * [17] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick.  Microsoft coco: Common objects in context.  In European conference on computer vision, pages 740–755. Springer, 2014.
  * [18] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg.  Ssd: Single shot multibox detector.  In European conference on computer vision, pages 21–37. Springer, 2016.
  * [19] M. Mathias, R. Timofte, R. Benenson, and L. Van Gool.  Traffic sign recognition—how far are we from the solution?  In The 2013 international joint conference on Neural networks (IJCNN), pages 1–8. IEEE, 2013.
  * [20] A. Mogelmose, M. M. Trivedi, and T. B. Moeslund.  Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey.  IEEE Transactions on Intelligent Transportation Systems, 13(4):1484–1497, 2012.
  * [21] G. Neuhold, T. Ollmann, S. Rota Bulo, and P. Kontschieder.  The mapillary vistas dataset for semantic understanding of street scenes.  In Proceedings of the IEEE International Conference on Computer Vision, pages 4990–4999, 2017.
  * [22] J. Redmon and A. Farhadi.  Yolo9000: better, faster, stronger.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7263–7271, 2017.
  * [23] S. Ren, K. He, R. Girshick, and J. Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In Advances in neural information processing systems, pages 91–99, 2015.
  * [24] P. Sermanet and Y. LeCun.  Traffic sign recognition with multi-scale convolutional networks.  In IJCNN, pages 2809–2813, 2011.
  * [25] V. Shakhuro and A. Konushin.  Russian traffic sign images dataset.  Computer Optics, 40(2):294–300, 2016.
  * [26] V. I. Shakhuro and A. Konouchine.  Russian traffic sign images dataset.  Computer Optics, 40(2):294–300, 2016.
  * [27] B. Singh and L. S. Davis.  An analysis of scale invariance in object detection snip.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3578–3587, 2018.
  * [28] B. Singh, M. Najibi, and L. S. Davis.  Sniper: Efficient multi-scale training.  In Advances in Neural Information Processing Systems, pages 9333–9343, 2018.
  * [29] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel.  The german traffic sign recognition benchmark: A multi-class classification competition.  In IJCNN, volume 6, page 7, 2011.
  * [30] J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition.  Neural Networks, (0):–, 2012.
  * [31] R. Timofte, K. Zimmermann, and L. Van Gool.  Multi-view traffic sign detection, recognition, and 3d localisation.  Machine vision and applications, 25(3):633–647, 2014.
  * [32] P. Viola, M. Jones, et al.  Rapid object detection using a boosted cascade of simple features.  CVPR (1), 1:511–518, 2001.
  * [33] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie.  The caltech-ucsd birds-200-2011 dataset.  2011\.
  * [34] Z. Yan, H. Zhang, R. Piramuthu, V. Jagadeesh, D. DeCoste, W. Di, and Y. Yu.  Hd-cnn: hierarchical deep convolutional neural networks for large scale visual recognition.  In Proceedings of the IEEE international conference on computer vision, pages 2740–2748, 2015.
  * [35] F. Yu, W. Xian, Y. Chen, F. Liu, M. Liao, V. Madhavan, and T. Darrell.  Bdd100k: A diverse driving video database with scalable annotation tooling.  arXiv preprint arXiv:1805.04687, 2018.
  * [36] Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu.  Traffic-sign detection and classification in the wild.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2110–2118, 2016.