0.0pt\offinterlineskip\halign{\entry@#!@&&\entry<EMAIL_ADDRESS>0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{E\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
0.0pt\raise-28.55554pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
15.50096pt\raise 6.1111pt\hbox{{}\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\hbox{\hbox{\kern
0.0pt\raise-1.75pt\hbox{$\scriptstyle{\psi}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern 33.8125pt\raise 0.0pt\hbox{\hbox{\kern
0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
33.8125pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\raise
0.0pt\hbox{$\textstyle{E^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
41.56165pt\raise-26.99998pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx<EMAIL_ADDRESS>0.0pt\raise 0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{M\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
32.39583pt\raise-38.38885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
32.39583pt\raise-38.38885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{M^{\prime}}$}}}}}}}\ignorespaces}}}}\ignorespaces},\quad\vbox{\lx@xy@svg{\hbox{\raise
0.0pt\hbox{\kern
6.97916pt\hbox{\ignorespaces\ignorespaces\ignorespaces\hbox{\vtop{\kern
0.0pt\offinterlineskip\halign{\entry@#!@&&\entry<EMAIL_ADDRESS>0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
14.17424pt\raise 5.1875pt\hbox{{}\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\hbox{\hbox{\kern
0.0pt\raise-0.8264pt\hbox{$\scriptstyle{\chi}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern 31.02707pt\raise 0.0pt\hbox{\hbox{\kern
0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
0.0pt\raise-19.19443pt\hbox{{}\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\hbox{\hbox{\kern 0.0pt\raise-1.75pt\hbox{$\scriptstyle{\subset}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern 0.0pt\raise-28.55554pt\hbox{\hbox{\kern
0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
31.02707pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\raise
0.0pt\hbox{$\textstyle{C^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
38.7283pt\raise-19.19443pt\hbox{{}\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\hbox{\hbox{\kern
0.0pt\raise-1.75pt\hbox{$\scriptstyle{\subset}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern
38.7283pt\raise-26.99998pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx<EMAIL_ADDRESS>0.0pt\raise 0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{E\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
14.08429pt\raise-32.27776pt\hbox{{}\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\hbox{\hbox{\kern
0.0pt\raise-1.75pt\hbox{$\scriptstyle{\psi}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern
30.97916pt\raise-38.38885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
30.97916pt\raise-38.38885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{E^{\prime}}$}}}}}}}\ignorespaces}}}}\ignorespaces}.$
(258)

Before proceeding, we need some terminology concerning cones and operations on
them. These notions are often used in convex geometry.[102]

######  Definition C.2.

Given a finite dimensional vector space $V$ and a convex cone $C\subset V$,
denote its closure by $\bar{C}$ and its open interior by $\mathring{C}$. We
define the _convex dual_ (a.k.a.  _polar dual_) $C^{*}\subset V^{*}$ as the
set

$C^{*}=\\{u\in V^{*}\mid u\cdot v\geq 0\quad\text{for all}~{}v\in C\\}.$ (259)

We define the _strict convex dual_ $C^{\circledast}\subset V^{*}$ as the set

$C^{\circledast}=\\{u\in V^{*}\mid u\cdot v>0\quad\text{for
all}~{}v\in\bar{C}\setminus\\{0\\}\\}.$ (260)

The attribute _strict_ may be dropped from the description of
$C^{\circledast}$ when it is clear from context. It is easy to check the
following

######  Proposition C.3.

Consider a convex cone $C$.

  1. (i)

The convex dual $C^{*}$ is always closed and also convex. In addition,
$C^{**}=\bar{C}$. The strict convex dual $C^{\circledast}$ is always open and
convex.

  2. (ii)

$C^{*}\setminus\\{0\\}$ is non-empty iff $C$ is contained in a closed half
space. $C^{\circledast}$ is non-empty iff $C$ contains no affine line (it is
_salient_).

  3. (iii)

If $C$ is open and salient, then $C^{\circledast\circledast}=C$.

  4. (iv)

The inclusion of cones $C_{1}\subseteq C_{2}$ implies the reverse inclusion of
their duals, $C_{1}^{*}\supseteq C_{2}^{*}$ and $C_{1}^{\circledast}\supseteq
C_{2}^{\circledast}$.

  5. (v)

The convex dual of the intersection of closures of cones $C_{1}$ and $C_{2}$
is the convex union (convex hull of the union) of their duals,
$(\bar{C}_{1}\cap\bar{C}_{2})^{*}=C_{1}^{*}+C_{2}^{*}$, where the right hand
side is written as a Minkowski sum, which for cones coincides with the convex
hull of the union. The converse identity holds as well,
$(\bar{C}_{1}+\bar{C}_{2})^{*}=C_{1}^{*}\cap C_{2}^{*}$. Similarly, if $C_{1}$
and $C_{2}$ are open and salient, then $(C_{1}\cap
C_{2})^{\circledast}=C_{1}^{\circledast}+C_{2}^{\circledast}$ and
$(C_{1}+C_{2})^{\circledast}=C_{1}^{\circledast}\cap C_{2}^{\circledast}$.

We extend these operations to cone bundles by acting fiberwise. So, if $C\to
M$ is a cone sub-bundle of a vector bundle $E\to M$, then the _convex dual
cone bundle_ $C^{*}\to M$ is the cone sub-bundle of $E^{*}\to M$ such that
each fiber $C^{*}_{x}$ is the convex dual of the corresponding fiber $C_{x}$,
for $x\in M$. Similarly, we can define the _strict convex dual cone bundle_
$C^{\circledast}\to M$. The operations of intersection $(\cap)$ and convex
union $(+)$ are extended to cone bundles in the same way. Clearly if
$C^{\circledast}\to M$ is a smooth bundle, it is also a cone bundle. On the
other hand, $C^{*}\to M$ is usually not (since it is usually not open), though
for brevity we shall sometimes refer to it as a cone bundle anyway.

In the next definition, we make a slight break with the usual terminology
concerning covectors in Lorentzian geometry. A covector naturally defines a
codimension-$1$ subspace, its annihilator, of the tangent space. In the
presence of a metric, a covector can be canonically identified with a vector.
If this vector is timelike, the corresponding codimension-$1$ subspace is
called spacelike. However, a direct means of identifying covectors with
vectors is missing in general. On the other hand, the link between covectors
and codimension-$1$ tangent subspaces is metric independent and since the term
_spacelike_ still makes sense for tangent subspaces, we transfer it to
corresponding covectors naming them _spacelike_ as well.

######  Definition C.4.

Given a manifold $M$, a _chronal cone bundle_ on $M$ is a cone bundle $C\to M$
enveloped by the tangent bundle $TM\to M$ such that each fiber $C_{x}$, $x\in
M$, is a _proper cone_ (non-empty, open, convex, salient). The elements of $C$
are _future directed, timelike vectors_. The strict convex dual
$C^{\circledast}\to M$, enveloped by the cotangent bundle $T^{*}M\to M$, is
the corresponding _spacelike cone bundle_. The elements of $C^{\circledast}$
are _future oriented, spacelike covectors_. The corresponding cone bundles
$\bar{C}\to M$ and $\bar{C}^{\circledast}\to M$ are referred to, respectively,
as the _causal_ and _cocausal_ cone bundles. A morphism of two chronal or
spacelike cone bundles $C\to M$ and $C^{\prime}\to M^{\prime}$ is induced by
pushforward $\chi_{*}\colon TM\to TM^{\prime}$ (or $T^{*}M\to
T^{*}M^{\prime}$) of an open embedding $\chi\colon M\to M^{\prime}$.

Note that the open convex dual of a proper cone is again a proper cone.
Prototypical examples of chronal and spacelike cone bundles are the cone
bundles of timelike vectors and spacelike covectors on a Lorentzian manifold.

At this point, one may recall some standard notions of Lorentzian geometry, as
long as they are defined only in terms of timelike cones, and apply them to
the geometry of cone bundles. Many of the standard theorems translate as well,
some directly and others with some extra effort. We restrict ourselves to
those that are relevant to the issues at hand.

Fix a chronal cone bundle $C\to M$ on a spacetime manifold $M$. Note that any
chronal cone bundle is _time oriented_ , since by definition the fibers
consist of single cones rather than double cones like in the Lorentzian case.
By assumptions the cones are directed into the future.

######  Definition C.5.

A smooth curve $\gamma$ in $M$ is called _future directed, timelike_ if the
tangent to $\gamma$ is everywhere contained in $C$. The _chronal precedence
relation_ $I^{+}\subseteq M\times M$ (also $I^{+}_{C}$) is defined as

$I^{+}=\\{(x,y)\in M\times M\mid\exists\gamma,~{}\text{future directed,
timelike curve from $x$ to $y$}\\}.$ (261)

When $(x,y)\in I^{+}$, we say that $x$ _chronologically precedes_ $y$ and also
write $x\ll y$.

If we replace $C$ by $\bar{C}$ in the above definitions, we obtain _causal
curves_ and the _causal precedence relation_ $J^{+}\subseteq M\times M$,
denoted $x<y$. The inverse relations are written $I^{-}$ and $J^{-}$.

It is clear that $I^{+}$ is an open, transitive relation ($x\ll y$ and $y\ll
z$ implies $x\ll z$). Using this relation, we can define the usual causal
hierarchy.

######  Definition C.6.

  1. (i)

If $I^{+}$ is irreflexive ($x\not\ll x$ or, equivalently, no closed timelike
curves exist), then $C$ is _chronological_.

  2. (ii)

Given an open $N\subseteq M$, $I^{+}_{C|_{N}}$ and $I^{+}_{C}\cap(N\times N)$
are both relations on $N\times N$. We say that $N$ is _chronologically
compatible_ if both of these relations coincide. More conventionally, this
means that any two points of $N$ that can be joined by a timelike curve in $M$
can also be joined by a timelike curve in $N$.

  3. (iii)

An open $N\subseteq M$ is called _chronologically convex_ if any timelike
curve that joins any two points of $N$ must also lie in $N$, which is a
stronger condition than chronological compatibility.

  4. (iv)

The chronal cone bundle $C$ is said to be _strongly chronological_ if it is
chronological and for every $x\in M$ and every neighborhood $N\subseteq M$ of
$x$, there exists a smaller open neighborhood $L\subseteq N$ that is
chronologically convex.

  5. (v)

The chronal cone bundle $C$ is said to be _stably chronological_ if there
exists another chronal cone bundle $C^{\prime}$ that is itself chronological
and an open neighborhood of the closure of $C$, that is,
$\bar{C}\setminus\\{0\\}\subset C^{\prime}$. For spacelike cone bundles,
stable chronology is equivalent to the reverse inclusion
$\bar{C}^{\prime\circledast}\setminus\\{0\\}\subset C^{\circledast}$.

Each of these definitions has an obvious analog when the adjectives
_chronological_ or _timelike_ are replaced by _causal_.

Note that _stably chronological_ is equivalent to _stably causal_ , so these
terms will be used interchangeably. Moreover, the chronological chronal cone
bundle $C^{\prime}$ such that it contains a stably chronological chronal cone
bundle $C$ can itself be chosen to be stably chronological. Next we turn from
curves to surfaces.

######  Definition C.7.

Each of the following concepts may be prefaced with $C$\- or
$C^{\circledast}$\- to be more specific.

  1. (i)

An oriented codim-$1$ surface $S\subset M$ is called _future oriented,
spacelike_ if its oriented conormals are everywhere contained in
$C^{\circledast}$.

  2. (ii)

A codim-$1$ surface $S\subset M$ is called _achronal_ if it $(S\times S)\cap
I^{+}=\varnothing$, that is, no two points of $S$ are connected by a timelike
curve. Similarly, $S$ is _acausal_ when $(S\times S)\cap J^{+}=\varnothing$.

  3. (iii)

A codim-$1$ surface $S\subset M$ is called _Cauchy_ if it is acausal and every
inextensible causal curve intersects $S$ exactly once.

  4. (iv)

A chronal cone bundle $C\to M$ is called _globally hyperbolic_ if there exists
a Cauchy surface $S\subseteq M$. A spacelike cone bundle $C^{\circledast}\to
M$ is called _globally hyperbolic_ if $C\to M$ is.

Next we define some commonly used domains. Fix $S\subset M$ to be a
$C$-acausal codim-$1$ submanifold, such that either $S$ is closed or
$\bar{S}\subset M$ is a submanifold with boundary.

######  Definition C.8.

  1. (i)

The _future/past domain of influence_ $I^{\pm}(N)$ of a subset $N\subseteq M$
is the set of points $y\in M$ such that there exists $x\in N$ with either
$x\ll y$ ($+$) or $y\ll x$ ($-$). Let $I(N)=I^{+}(N)\cup I^{-}(N)\cup N$.

  2. (ii)

The _domain of dependence_ $D(S)$ is the largest open subset of $M$ for which
$S$ is a Cauchy surface. More commonly, $D(S)$ is the set of points $y\in M$
such that every inextensible timelike curve through $y$ intersects $S$. Let
$D^{\pm}(S)=D(S)\cap I^{\pm}(S)$.

  3. (iii)

An open subset $L\subseteq M$ is _lens-shaped_ with respect to $S$ if it can
be smoothly factored as $L\cong(-1,1)\times S$, with $t\colon L\to(-1,1)$
denoting the projection onto the first factor (the _temporal function_), such
that the level set $t=0$ is $S$ and all other level sets are spacelike as well
as share the same boundary as $S$ in $M$ (which may be empty).

The literature in relativity and Lorentzian geometry mostly makes use of the
notion of _global hyperbolicity_ as given above, but specialized to Lorentzian
cone bundles. On the other hand, the literature on hyperbolic PDE systems
(including symmetric and regular hyperbolic ones) mostly makes use of lens-
shaped domains. It is a non-trivial fact that these two notions coincide. The
argument is essentially that the temporal function of a lens-shaped domain
foliates it with Cauchy surfaces. Conversely, a globally hyperbolic cone
bundle admits a temporal function and a smooth factorization that turns it
into a lens-shaped domain (glossing over some details related to spatial
compactness). The original argument establishing the converse link in
Lorentzian geometry is due to Geroch.[97] However, his argument only
established the existence of a continuous temporal function. The details
necessary to establish the smooth version of the result are due to more recent
work of Bernal and Sanchez.[103, 104] The very recent result by Fathi and
Siconolfi,[105] using completely different methods, established the existence
of a smooth temporal function and factorization for a more general class of
cone bundles that is sufficient for our purposes.

######  Proposition C.9.

If a cone bundle $C\to M$ is globally hyperbolic, then there exists a smooth
temporal function $t\colon M\to\mathbb{R}$, whose level sets are all
diffeomorphic and are $C$-Cauchy surfaces. Hence, an open subset $D\subseteq
M$ is $C$-lens-shaped with respect to $S\subseteq D$ iff it is globally
$C|_{D}$-hyperbolic, with $S$ being $C|_{D}$-Cauchy.

## References

  * [1] P. A. M. Dirac, Lectures on Quantum Mechanics (Dover Publications, March 2001).
  * [2] M. Henneaux and C. Teitelboim, Quantization of Gauge Systems (Princeton University Press, August 1994).
  * [3] J. Lee and R. M. Wald, Journal of Mathematical Physics 31, 725 (March 1990).
  * [4] Č. Crnković and E. Witten, Covariant description of canonical formalism in geometrical theories, in Three hundred years of gravitation, eds. S. W. Hawking and W. Israel (Cambridge University Press, Cambridge, 1987) pp. 676–684.
  * [5] G. J. Zuckerman, Action principles and global geometry, in Mathematical aspects of string theory, ed. S. T. Yau (World Scientific Publishing, Singapore, 1987), pp. 259–284.
  * [6] R. E. Peierls, Proceedings of the Royal Society of London. Series A, Mathematical and Physical Sciences 214, 143 (1952).
  * [7] B. DeWitt, The Global Approach to Quantum Field Theory: I & II (Oxford University Press, USA, February 2003).
  * [8] G. Barnich, M. Henneaux and C. Schomblond, Physical Review D 44, R939 (August 1991).
  * [9] M. Henneaux, Physics Letters B 238, 299 (April 1990).
  * [10] M. Forger and S. V. Romero, Communications in Mathematical Physics 256, 375 (2005), arXiv:math-ph/0408008.
  * [11] E. Binz, J. Śniatycki and H. Fischer, Geometry of Classical Fields, Notas de Matemática, Vol. 154 (North-Holland, 1988).
  * [12] K. Fredenhagen and K. Rejzner, Communications in Mathematical Physics 314, 93 (July 2012), arXiv:1101.5112v1.
  * [13] K. Rejzner, Batalin-Vilkovisky formalism in locally covariant field theory, PhD thesis, Hamburg (November 2011).
  * [14] R. Brunetti, K. Fredenhagen and P. L. Ribeiro, Algebraic structure of classical field theory I: Kinematics and linearized dynamics for real scalar fields (2012).
  * [15] E. T. Whittaker, A treatise on the analytical dynamics of particles and rigid bodies, 1st edn. (CUP, 1904).
  * [16] H. Goldstein, Classical mechanics, 2nd edn. (Addison-Wesley, Reading, MA, 1980).
  * [17] R. Abraham and J. E. Marsden, Foundations of mechanics (Benjamin-Cummings, Reading, Mass., 1978).
  * [18] V. I. Arnold, Mathematical Methods of Classical Mechanics (Graduate Texts in Mathematics, Vol. 60), 2nd edn. (Springer, September 1997).
  * [19] J. M. Souriau, Structure des systèmes dynamiques (Dunod, Paris, 1970).
  * [20] L. Rosenfeld, Zeitschrift für Physik 65, 589 (1930).
  * [21] J. L. Lagrange, Mémoires de Classe des Sciences Mathématiques et Physiques de l’Institut National de France , 1 (1808).
  * [22] V. Glaser, H. Lehmann and W. Zimmermann, Il Nuovo Cimento 6, 1122 (1957).
  * [23] I. E. Segal, Journal of Mathematical Physics 1, 468 (1960).
  * [24] I. E. Segal, Mathematical problems of relativistic physics, Lectures in Applied Mathematics, Vol. 1960 (American Mathematical Society, Providence, RI, 1963).
  * [25] I. E. Segal, Representations of the canonical commutation relations, in Application of Mathematics to Problems in Theoretical Physics, ed. F. Lurçat (Gordon and Breach, New York, 1967), pp. 107–170.
  * [26] B. S. DeWitt, Dynamical theory of groups and fields, in Relativity, Groups and Topology, eds. C. DeWitt and B. S. DeWitt (Gordon and Breach, New York, 1964), pp. 587–820.
  * [27] O. Steinmann, Perturbation Expansions in Axiomatic Field Theory, Lecture Notes in Physics, Vol. 11 (Springer-Verlag, Berlin, 1971).
  * [28] D. M. Marolf, Green’s bracket algebras and their quantization, PhD thesis, The University of Texas at Austin. (1992).
  * [29] D. Marolf, Annals of Physics 236, 374 (February 1994), arXiv:hep-th/9308141.
  * [30] D. Marolf, Annals of Physics 236, 392 (February 1994), arXiv:hep-th/9308150.
  * [31] M. Duetsch and K. Fredenhagen, Communications in Mathematical Physics 243, 275 (June 2003), arXiv:hep-th/0211242.
  * [32] F. Brennecke and M. Duetsch, Reviews in Mathematical Physics 20, 119 (January 2008), arXiv:0705.3160.
  * [33] K. Rejzner, Reviews in Mathematical Physics 23, 1009 (October 2011), arXiv:1101.5126.
  * [34] T.-P. Hack and A. Schenkel, General Relativity and Gravitation 45, 877 (March 2013), arXiv:1205.3484.
  * [35] P. Bergmann and R. Schiller, Physical Review 89, 4 (January 1953).
  * [36] A. Komar, Physical Review 134, B1430 (June 1964).
  * [37] A. Ashtekar and A. Magnon, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences 346, 375 (November 1975).
  * [38] A. Ashtekar (2012), Private communication.
  * [39] A. Ashtekar, Physical Review Letters 46, 573 (March 1981).
  * [40] J. L. Friedman, Communications in Mathematical Physics 62, 247 (1978).
  * [41] J. L. Friedman and B. F. Schutz, The Astrophysical Journal 221, 937 (May 1978).
  * [42] J. L. Friedman (2012), Private communication.
  * [43] A. Trautman, Invariance of lagrangian systems, in General Relativity, Papers in honour of J. L. Synge, (Clarenden Press, Oxford, 1972), pp. 85–99.
  * [44] E. Noether, Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen, mathematisch-physikalische Klasse 1918, 235 (1918).
  * [45] P. Dedecker, On the generalization of symplectic geometry to multiple integrals in the calculus of variations, in Differential Geometrical Methods in Mathematical Physics, eds. K. Bleuler and A. Reetz, Lecture Notes in Mathematics, Vol. 570 (Springer, 1977) pp. 395–456.
  * [46] F. Takens, Symmetries, conservation laws and variational principles, in Geometry and Topology, eds. J. Palis and M. do Carmo, Lecture Notes in Mathematics, Vol. 597 (Springer, 1977) pp. 581–604.
  * [47] H. Goldschmidt and S. Sternberg, Annales de l’institut Fourier 23, 203 (1973).
  * [48] M. Henneaux, Annals of Physics 140, 45 (April 1982).
  * [49] M. Henneaux (2013), Private communication.
  * [50] E. Witten, Nuclear Physics B 276, 291 (October 1986).
  * [51] S. Sternberg, Some preliminary remarks on the formal variational calculus of Gel’fand and Dikii, in Differential Geometrical Methods in Mathematical Physics II, eds. K. Bleuler, A. Reetz and H. Petry, Lecture Notes in Mathematics, Vol. 676 (Springer, 1978) pp. 399–407.
  * [52] P. Deligne and D. S. Freed, Classical field theory, in Quantum fields and strings: a course for mathematicians, eds. P. Deligne, D. Kazhdan, P. Etingof, J. W. Morgan, D. S. Freed, D. R. Morrison, L. C. Jeffrey and E. Witten (AMS, Providence, RI, 1999), Providence, RI, pp. 137–225.
  * [53] Č. Crnković, Classical and Quantum Gravity 5, 1557 (December 1988).
  * [54] A. Ashtekar, L. Bombelli and O. Reula, The covariant phase space of asymptotically flat gravitational fields, in Mechanics, analysis and geometry: 200 years after Lagrange, eds. M. Francaviglia and D. Holm (North-Holland, Amsterdam, 1991), pp. 417–450.
  * [55] S. Hollands and D. Marolf, Classical and Quantum Gravity 24, 2301 (April 2007), arXiv:gr-qc/0611044.
  * [56] G. Barnich and F. Brandt, Nuclear Physics B 633, 3 (February 2002), arXiv:hep-th/0111246.
  * [57] P. Bergmann, I. Goldberg, A. Janis and E. Newman, Physical Review 103, 807 (August 1956).
  * [58] C. Baer, N. Ginoux and F. Pfaeffle, Wave Equations on Lorentzian Manifolds and Quantization, ESI lectures in mathematics and physics, Vol. 2 (European Mathematical Society, June 2007).
  * [59] C. Baer, Green-hyperbolic operators on globally hyperbolic spacetimes (October 2013).
  * [60] I. Khavkine, Characteristics, conal geometry and causality in locally covariant field theory (November 2012).
  * [61] N. Ginoux, Linear wave equations (Ch. 3), in Quantum Field Theory on Curved Spacetimes: Concepts and Methods, eds. C. Baer and K. Fredenhagen, Lecture Notes in Physics, Vol. 786 (Springer, 2009)
  * [62] H. Ringström, The Cauchy Problem in General Relativity, ESI Lectures in Mathematics and Physics, Vol. 6 (European Mathematical Society, 2009).
  * [63] S. Waldmann, Geometric wave equations (August 2012).
  * [64] R. M. Wald, Quantum Field Theory in Curved Spacetime and Black Hole Thermodynamics (Chicago Lectures in Physics), 1 edn. (University Of Chicago Press, November 1994).
  * [65] K. Sanders, Classical and Quantum Gravity 30, 115014 (November 2012), arXiv:1211.2469.
  * [66] H. Goldschmidt, Journal of Differential Geometry 1, 269 (1967).
  * [67] H. Goldschmidt, The Annals of Mathematics 86, 246 (1967).
  * [68] J.-F. Pommaret, Systems of partial differential equations and Lie pseudogroups (Gordon and Breach, New York, 1978).
  * [69] R. S. Palais, Seminar on the Atiyah-Singer Index Theorem, Annals of Mathematics Studies, Vol. 57 (Princeton University Press, 1965).
  * [70] R. Courant and D. Hilbert, Methods of Mathematical Physics, Volume 1 (John Wiley & Sons, 2008).
  * [71] I. M. Anderson, The variational bicomplex (1989).
  * [72] G. Giachetta, L. Mangiarotti and G. Sardanashvily, Journal of Mathematical Physics 42, 4272 (2001), arXiv:math/0006074.
  * [73] I. M. Anderson, Introduction to the variational bicomplex, in Mathematical aspects of classical field theory, eds. M. J. Gotay, J. E. Marsden and V. Moncrief, Contemporary Mathematics, Vol. 132 (American Mathematical Society, Providence, Rhode Island, 1992) pp. 51–73.
  * [74] I. Khavkine, Journal of Mathematical Physics 54, 111502 (November 2013), arXiv:1210.0802.
  * [75] R. Geroch, Partial differential equations of physics, in Proceedings of the Forty-Sixth Scottish Summer School in Physics, eds. G. S. Hall and J. R. Pulham (SUSSP Publ, Edinburgh, February 1996).  arXiv:gr-qc/9602055.
  * [76] K. Sanders, C. Dappiaggi and T.-P. Hack, Electromagnetism, local covariance, the Aharonov-Bohm effect and Gauss’ law (October 2013).
  * [77] C. Dappiaggi, Reviews in Mathematical Physics 23, 1035 (November 2011).
  * [78] R. M. Wald, General Relativity, 1st edn. (University Of Chicago Press, Chicago, June 1984).
  * [79] C. J. Fewster and D. S. Hunt, Reviews in Mathematical Physics 25, 1330003 (March 2013), arXiv:1203.0261.
  * [80] E. Calabi, On compact riemannian manifolds with constant curvature, in Differential Geometry, ed. C. B. Allendoerfer, Proceedings of Symposia in Pure Mathematics, Vol. 3 (AMS, Providence, RI, 1961), pp. 155–180.
  * [81] I. Khavkine, Cohomology with causally restricted supports (in preparation).
  * [82] G. Grubb, Distributions and operators, Graduate Texts in Mathematics, Vol. 252 (Springer, Berlin, 2009).
  * [83] P. J. Olver, Applications of Lie groups to differential equations, Graduate Texts in Mathematics, Vol. 107, second edn. (Springer-Verlag, New York, 1993).
  * [84] I. Kolař, P. W. Michor and J. Slovák, Natural Operations in Differential Geometry (Springer, 1993).
  * [85] D. Spring, Convex Integration Theory: Solutions to the H-Principle in Geometry and Topology, Monographs in Mathematics, Vol. 92 (Birkhäuser, 1998).
  * [86] W. M. Seiler, Involution: The Formal Theory of Differential Equations and its Applications in Computer Algebra, Algorithms and Computation in Mathematics, Vol. 24 (Springer, 2010).
  * [87] R. L. Bryant, S. S. Chern, R. B. Gardner, H. L. Goldschmidt and P. A. Griffiths, Exterior Differential Systems, Mathematical Sciences Research Institute Publications, Vol. 18 (Springer, 2011).
  * [88] A. M. Vinogradov and I. S. Krasilshchik (eds.), Symmetries and conservation laws for differential equations of mathematical physics, Translations of Mathematical Monographs, Vol. 182 (American Mathematical Society, Providence, RI, 1999).
  * [89] N. N. Tarkhanov, Complexes of Differential Operators, Mathematics and Its Applications, Vol. 340 (Springer Netherlands, Dordrecht, 1995).
  * [90] L. Hörmander, The Analysis of Linear Partial Differential Operators III, Grundlehren Der Mathematischen Wissenschaften, Vol. 274 (Springer, 1985).
  * [91] R. T. Smith, Bulletin of the American Mathematical Society 82, 297 (1976).
  * [92] S. W. Hawking and G. F. R. Ellis, The Large Scale Structure of Space-Time (Cambridge University Press, Cambridge, 1973).
  * [93] R. Beig, Concepts of hyperbolicity and relativistic continuum mechanics, in Analytical and Numerical Approaches to Mathematical Relativity, eds. J. Frauendiener, D. Giulini and V. Perlick, Lecture Notes in Physics, Vol. 692 (Springer, Berlin, 2006) pp. 101–116.
  * [94] J. D. Lawson, Forum Mathematicum 1, 273 (1989).
  * [95] K. H. Neeb, Inventiones Mathematicae 104, 467 (December 1991).
  * [96] R. Penrose, Techniques in Differential Topology in Relativity, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 7 (Society for Industrial and Applied Mathematics, Philadelphia, PA, January 1972).
  * [97] R. Geroch, Journal of Mathematical Physics 11, 437 (1970).
  * [98] A. García-Parrado and J. M. M. Senovilla, Classical and Quantum Gravity 22, R1 (May 2005), arXiv:gr-qc/0501069.
  * [99] U. Bannier, Communications in Mathematical Physics 118, 163 (March 1988).
  * [100] M. Rainer, Journal of Mathematical Physics 40, 6589 (January 1999), arXiv:gr-qc/9905106.
  * [101] M. Rainer, Classical and Quantum Gravity 17, 1935 (May 2000), arXiv:gr-qc/9911076.
  * [102] R. T. Rockafellar, Convex Analysis, Princeton Mathematical Series, Vol. 28 (Princeton University Press, 1996).
  * [103] A. N. Bernal and M. Sánchez, Communications in Mathematical Physics 257, 43 (May 2005), arXiv:gr-qc/0401112.
  * [104] A. Bernal and M. Sánchez, Letters in Mathematical Physics 77, 183 (August 2006), arXiv:gr-qc/0512095.
  * [105] A. Fathi and A. Siconolfi, Mathematical Proceedings of the Cambridge Philosophical Society 152, 303 (November 2011).


# Teacher-Student Knowledge Distillation for Radar Perception on Embedded
Accelerators

Steven Shaw,
Aptiv Technical Center
Carmel, Indiana, 46032
<EMAIL_ADDRESS>
&Kanishka Tyagi,
Aptiv Advance Research Center
Agoura Hills, California, 91301
<EMAIL_ADDRESS>
&Shan Zhang
Aptiv Advance Research Center
Agoura Hills, California, 91301
<EMAIL_ADDRESS>

###### Abstract

Many radar signal processing methodologies are being developed for critical
road safety perception tasks. Unfortunately, these signal processing
algorithms are often poorly suited to run on embedded hardware accelerators
used in automobiles. Conversely, end-to-end machine learning (ML) approaches
better exploit the performance gains brought by specialized accelerators. In
this paper, we propose a teacher-student knowledge distillation approach for
low-level radar perception tasks. We utilize a hybrid model for stationary
object detection as a teacher to train an end-to-end ML student model. The
student can efficiently harness embedded compute for real-time deployment. We
demonstrate that the proposed student model runs at speeds 100x faster than
the teacher model.

##  1 Introduction

With the steady advances in autonomous driving, advanced safety features using
one or more sensors are highly desirable. In order to avoid collisions and
unintended breaking maneuvers, it is crucial to detect potential road
obstacles accurately. Although camera and LiDAR-based object detection have
been studied in the literature [1, 2], it’s only recently that interest in
radar-based object detection using ML methods has begun, primarily because of
its low cost, long-range detection capability, and robustness to poor weather
conditions.

Traditionally, automotive radar-based object detection is performed through
peak detection using simple local thresholding methods such as the Constant
False-Alarm Rate (CFAR) algorithm [3]. With the breakthroughs of ML in
numerous applications [4, 5, 6, 7], radar-based object perception using ML has
attracted attention [8, 9, 10, 11, 12, 13]. In [8, 9], point cloud radar data
was used for object detection. Low-level radar spectra-based object perception
was studied in [10, 11, 12, 13].

Radar data can be represented in many forms, such as point cloud, compressed
data cube (CDC), or low-level spectra. Point cloud data is a detection level
representation retaining the least amount of the original information. CDC
[14] consists of a subsection of beam vectors (BVs) that exceeds a CFAR
threshold. Unlike point cloud and CDC, low-level spectra retains all of the
information returned from the radar. In this work, we propose an ML based
stationary object detection system using low-level spectra data. [8, 9, 10,
11, 12, 13], study large and mostly non-stationary objects such as
pedestrians, cars, bikes, and trucks were considered for perception tasks. In
contrast, our work focuses on detecting stationary debris objects in the
driving path.

Labeling low-level radar spectrum data is typically expensive. Large-scale
deep models in [14, 10, 11, 12, 13] often require a large amount of labeled
data. With limited low-level radar spectrum data, a hybrid system can be used
for perception tasks, e.g., object detection. However, many of the constituent
traditional signal processing techniques cannot make use of hardware
acceleration. Hybrid systems and large-scale deep models have computation
efficiency requirements which make practical deployment to embedded systems a
great challenge.

The essential factors for computing hardware deployed in next generation
electric automobiles are cost and power efficiency. To achieve these goals,
hardware accelerators such as matrix multiplication accelerators (MMA) are
becoming increasingly popular. However, with MMA, deploying hybrid systems or
even large-scale ML models on devices with limited resources, such as
automotive, is still challenging. One solution to avoid performance loss is to
take advantage of lightweight ML models that can efficiently run on MMA’s.
Since we typically develop hybrid systems or large-scale deep models for
perception tasks, knowledge distillation (KD) from these models (teacher
models) to student models can enable fast inference on embedded systems [15].

In this paper, we propose what we believe is the first teacher-student KD
approach for stationary object detection using low-level radar spectrum data,
where detection knowledge is transferred from a block-by-block (BB) hybrid
teacher model to a light weight end-to-end ML student model intended for
deployment to an embedded hardware accelerator.

Overall, our main contributions are as follows. First, customized modules in
the BB hybrid teacher model, i.e., the non-trainable blocks, effectively take
advantage of the radar-specific properties. Second, the BB hybrid teacher
model does automated labeling purely on low-level radar data and requires
minimum to no human labeling efforts. Third, the proposed student model is a
direct embedded implementation with low power and memory usage. Fourth, the
student-teacher model works entirely on low-level radar datasets for
stationary object detection problems. i.e., no other sensor is used to make
the detection decisions; last, new evaluation methods for the teacher-student
model is proposed and justified for its effectiveness.

##  2 Problem formulation

In our teacher-student framework shown in Figure 1, knowledge from a hybrid
teacher is transferred to an end-to-end deep learning-based student. Since we
are focusing on stationary object detection, the radar spectra can be reduced
to a range azimuth spectrum maps. Therefore, the input spectrum data for the
teacher and student models are range azimuth maps. The teacher model processes
these range azimuth maps, creating labeled training data. The student model is
then trained in a supervised fashion to mimic the same detection task that the
teacher model performed. The student can later be easily deployed on an
embedded platform and run much faster, more efficiently, and is product ready
for the automotive market. Following [16], [15], we modify the idea of KD by
changing the teacher model. In our work, the teacher is a BB hybrid model
containing trainable and non-trainable parametric blocks. By combining the
traditional radar knowledge and deep learning approach in a BB hybrid model,
we can significantly reduce the data needed for training [17] and the time-
consuming data labeling efforts in prior end-to-end deep learning studies [10,
11, 13].

Figure 1: Teacher-Student framework.

Figure 2: Cartesian and polar representation of debris detection for in-lane,
out-lane objects.

##  3 Proposed Algorithm

The teacher model contains sophisticated radar processing algorithms, some of
which are trainable (e.g. a multi-layer perceptron (MLP)) and some non-
trainable (e.g., interpolation, convolution based signal processing, feature
extraction, and post processing). The non-trainable algorithms cannot benefit
from the MMA core on an embedded devices, designed to run fast matrix
multiplication calculations as in Figure 3. In our study, we observed that the
teacher model’s non-trainable components take the majority of the processing
time, making it unsuitable to deploy on an embedded devices.

The teacher model takes as inputs a range azimuth map which is obtained by
processing low-level time series radar data using traditional radar signal
processing methods [3] and host vehicle speed information. The teacher model
uses the host vehicle speed for input spectrum interpolation. Data
interpolation is needed for the purpose of manual feature extraction. We
assume that we have $N$ number of range azimuth samples. We use
$\mathbf{X}_{i}\in\mathbb{R}^{464\times 256}$ to denote the range azimuth map
at time $i$, where $464$ is the number of range bins and $256$ is the number
of azimuth bins, and $i=1,\cdots,N$. The output of the teacher model is a
probability vector containing probabilities of in-lane objects at each range
bin over time. After applying a decision boundary on the probability vector,
we obtain a in-lane object prediction vector
$\mathbf{y}_{i}\in\mathbb{R}^{464\times 1}$ at time $i,i=1,\cdots,N$. We use
$\mathbf{Y}\in\mathbb{R}^{N\times 464}$ denote the output decisions of the
teacher model for all the data samples, where $\mathbf{Y}_{i,j}\in\\{0,1\\}$
is the detection decision at range bin $j,j=0,\cdots,463$, and
$\mathbf{Y}_{i,j}=1$ denotes that there is an obstacle at range bin $i$,
otherwise, $\mathbf{Y}_{i,j}=0$.

Figure 3: Teacher-Student embedded architecture. The timings for each block
(in milliseconds) demonstrate the advantage of student algorithm.

The teacher model contains radar processing tasks involving ego lane
determination using specified geometry information, interpolation of the
spectrum data, feature extraction with accumulated information across time,
and a MLP detection network, to produce labels for the student model. The
student model must learn all the mentioned tasks using only the input spectrum
data and provided labels. Further more, the student model is able to provide
high detection confidence without feature accumulation over time.

Although the raw input to the student model is the same shape as the teacher
model, the first step taken by the student is to is to narrow its focus to the
middle angular bins. We will denote the input range azimuth map for the
student model by $\tilde{\mathbf{X}}_{i}^{s}\in\mathbb{R}^{464\times
30},i=1,\cdots,N$, where $30$ is the number of azimuth bins we select. These
30 angular bins ensure to capture of the widest portion of the ego lane,
including a small buffer. Angular bin selection significantly improves the
training time.

Now, we present the network structure details of the student model. The
student model comprises of five convolutional layers with ReLU activations and
a Sigmoid activation for the final convolutional layer. Deploying fully
connected layers on an embedded MMA accelerators is typically limited to such
an extent as to not useful. Typically the solution is to utilize 1x1
convolution kernels to emulate fully connected layers. In our case, we use a
3x1 convolutional kernel in the final layer. Since objects usually span
multiple range bins, using kernels with extended receptive field in the range
direction allows the model to use the information from neighboring range bins
to make a better predictions. Figure 3 details the teacher-student paradigm
along with the architectural details of each block.

The collected radar dataset is extremely imbalanced. There are 464 range bins
but typically just one in-lane object which may only span a few range bins.
Therefore, we propose to use a weighted mean squared error (WMSE) to boost the
loss values where in-lane objects are located by the teacher model, which is
given as

$\text{WMSE}=\frac{1}{N}\sum_{i=1}^{N}w_{i}(\hat{\mathbf{Y}}_{i,j}-\mathbf{Y}_{i,j})^{2},$
(1)

where $\hat{\mathbf{Y}}\in\mathbb{R}^{N\times 464}$ is the predicted
probability score matrix, $\hat{\mathbf{Y}}_{i,j}$ is the predicted score and
$\mathbf{Y}_{i,j}$ is the ground truth for time frame $i$ and range bin $j$.
$w_{i}=1$ when $\mathbf{Y}_{i,j}=0$ and
$w_{i}=\frac{|\mathbf{Y}^{-}|}{|\mathbf{Y}^{+}|}$ when $\mathbf{Y}_{i,j}=1$,
$i=1,\cdots,N$ and $j=0,\cdots,463$. Also, $|\mathbf{Y}^{-}|$ denotes the
number of negative samples and $|\mathbf{Y}^{+}|$ denotes the number of
positive samples.

In the following, we show the training process of the student model. The input
range azimuth maps are processed by the teacher algorithm to produce
corresponding labels. The labels are used to train the student model in a
supervised setting. Knowledge distillation during training from the teacher to
student model is illustrated in Figure 1.

In order to discriminate between in- and out-of-lane objects, the student
model must learn the ego lane geometry which is a function range as shown in
Figure 2. However, learning this function using solely convolutional layers
presents a problem because convolutions are translationally invariant. To
address this, we implement a modified version of the CoordConv algorithm [18]
by appending two additional channels to label the range and azimuth
respectively. This allows the convolution kernels to utilize their location on
the range azimuth map in order to learn the lane width function. In our study,
we found that CoordConv also allows the student to better model the apparent
difference in signal strength between near and far objects.

For an input range-azimuth
$\widetilde{\mathbf{X}}_{i}^{s}\in\mathbb{R}^{464\times 30},i=1,\cdots,N$, we
add two more range CoordConv and azimuth CoordConv channels, denoted by
$\mathbf{R}\in\mathbb{R}^{464\times 30}$ and
$\mathbf{A}\in\mathbb{R}^{464\times 30}$, respectively. The improved input
data is denoted by $\mathbf{X}_{i}^{s}$ $\in$ $\mathbb{R}^{464\times 30\times
3},i=1,\cdots,N$. $\mathbf{X}_{i}^{s}=[\mathbf{x}_{i}^{s_{0}}\
\mathbf{x}_{i}^{s_{1}}\ \mathbf{x}_{i}^{s_{2}}]$ where
$\mathbf{x}_{i}^{s_{0}}=\widetilde{\mathbf{X}}_{i}^{s}$,
$\mathbf{x}_{i}^{s_{1}}=\mathbf{R}$, and $\mathbf{x}_{i}^{s_{2}}=\mathbf{A}$.
$\mathbf{R}$ and $\mathbf{A}$ are given as:

$\displaystyle\mathbf{R}[n,:]=\frac{n}{R_{\text{max}}},$ (2)
$\displaystyle\mathbf{A}[:,m]=2|\frac{m-\frac{A_{\text{max}}}{2}}{A_{\text{max}}}|,$
(3)

where $n,n=0,\cdots,463$ is the index for the range bin, $m,m=0,\cdots,29$ is
the index for the azimuth bin, $\mathbf{R}_{\text{max}}$ is the maximum range
index and $\mathbf{A}_{\text{max}}$ is the maximum azimuth index. $|\cdot|$
denotes absolute value operation.

The teacher model must look at multiple time steps before predicting the
debris detection probability with reasonable confidence. Also, the teacher
model interpolation step requires a minimum critical speed in order to make
predictions. In order to avoid both of these issues, the student model is only
trained with samples where the teacher was able to positively identify at
least one debris object at some range. The intuition is to not penalize the
student for extending the maximum distance at which debris can be detected as
well as allow the student to fill in the gaps where the teacher could not make
predictions due to slow host speed. The same intuitions underpinning the
selective training technique have implications on the evaluation metric as
discussed in the next section.

##  4 Experimental Results

Given the lack of a public low-level automotive radar dataset for early
stationary object detection, we have collected our dataset. Our data
collection vehicle has a high-definition radar mounted on the front bumper.
Since it is costly to collect enough open environment data with stationary
objects in-lane, the data we collected in a controlled environment is a set of
debris objects including a dehumidifier, tire with and without rim, wooden
pallet and stationary car next to repeated structures such as guardrails and
signposts. The objects were placed 300m from the host vehicle before the
beginning of each collection run. We also consider different rotations of each
stationary object in order to get a complete characterization. The data was
split into 70 % training, 20 % validation and 10 % testing along with five
fold validation. We used the Tensorflow library on a Windows i7 CPU to train
the machine learning model.

The setup of our experiment requires particular metrics for evaluating
performance. We have modified the recall metric and call it R-score. R-score
represents the percentage of true positives predicted by the student compared
to the corresponding number of positives predicted by the teacher. Our R-Score
can also allow a $+/-1$ range bin offset allowance. We use $R_{0}$ to denote
the score without range bin offset allowance, and $R_{1}$ to denote the score
with $+/-1$ variance in the range bin direction. Note $R_{0}$ score is
equivalent to recall, $\frac{tp}{tp+fn}$. $R_{0}$ and $R_{1}$ are given as

$R_{0}=\frac{\sum_{i=1}^{N}\sum_{j=1}^{M}\mathbf{Y}_{i,j}\hat{\mathbf{Y}}_{i,j}^{{}^{\prime}}}{\sum_{i=1}^{N}\sum_{j=1}^{M}\mathbf{Y}_{i,j}},$
(5)

$R_{1}=\frac{\sum_{i=1}^{N}\sum_{j=1}^{M}\mathbf{Y}_{i,j}\text{max}[\hat{\mathbf{Y}}_{i,j-1}^{{}^{\prime}},\hat{\mathbf{Y}}_{i,j}^{{}^{\prime}},\hat{\mathbf{Y}}_{i,j+1}^{{}^{\prime}}]}{\sum_{i=1}^{N}\sum_{j=1}^{M}\mathbf{Y}_{i,j}},$
(6)

where $\mathbf{Y}$ is the ground truth and $\hat{\mathbf{Y}}^{{}^{\prime}}$ is
the predicted labels of the student model after a decision boundary is
applied.

The student can extend the maximum range at which objects are detected. If all
teacher samples were used as ground truth, the extended range performance of
the student would be judged as false positives. Therefore, we have modified
the precision score, $\frac{tp}{tp+fp}$, and call it P-score. $P_{1}$
precision score allows a $+/-1$ range bin offset allowance for false positive
identification. $P_{0}$ precision score is evaluated without any offset
allowance. P-score and the true negative rate, specificity =
$\frac{tn}{tn+fp}$, are only evaluated on samples which the teacher model has
at least one positive prediction of an object at some range.

Table 1 shows results from the evaluation sets used in the experiment. The
student extended the maximum range of debris detection by an average of 47
meters in the evaluation sets. The student scored an average of 0.91 for the
$R_{1}$ recall score, an average of 0.93 for $P_{1}$ precision score, and an
average of 0.99 for specificity score. The student also reduced the processing
time on the desktop CPU benchmarks by an average factor 36 for the evaluation
sets. This speed up is even greater on the embedded device where the student
runs 100x faster than the teacher, see Figure 3.

Table 1: Teacher (Tea) and Student (Stu) evaluation results comparison showing first detection range, average (Avg) runtime, statistical metrics. Range shown in meters and time in milliseconds. $R_{0}$, $R_{1}$, $P_{0}$, $P_{1}$, and sp denote $R_{0}$ recall, $R_{1}$ recall, $P_{0}$ precision, $P_{1}$ precision, and specificity, respectively. | Bike | Tire | Shovel | Car | Toy Truck | Tire
---|---|---|---|---|---|---
Stu max range | 79 | 65 | 57 | 244 | 222 | 114
Tea max range | 8 | 5 | 26 | 215 | 168 | 76
Stu Avg Time | 9 | 11 | 8 | 9 | 8 | 8
Tea Avg Time | 323 | 318 | 329 | 299 | 311 | 325
$R_{0}$ | 0.83 | 0.88 | 0.90 | 0.98 | 1.00 | 0.82
$R_{1}$ | 0.9 | 0.88 | 1.00 | 1.00 | 1.00 | 0.82
$P_{0}$ | 0.43 | 0.35 | 0.70 | 0.64 | 0.69 | 0.43
$P_{1}$ | 0.86 | 0.78 | 1.00 | 0.99 | 0.96 | 1.00
$sp$ | 0.99 | 0.99 | 0.99 | 0.99 | 0.99 | 0.99

##  5 Conclusion

The student model was successful in learning necessary mappings from the
teacher. The student also exhibits many additional benefits over the teacher.
The teacher’s processing speed is dependent on the host vehicle’s speed. As
the host speed increases, the teacher produces more interpolated frames to
processes. In contrast, the student processing time is independent of host
speed allowing for constant runtime and memory usage, which are highly
desirable attributes for embedded deployments. The student also exceeds the
teacher’s performance in the low-speed regime, where the teacher fails to make
predictions. Through selective training, the student can take what it learns
from the teacher’s higher-speed examples and apply it to the low-speed
samples. The student model vastly simplifies the deployment complexity onto
embedded accelerators, all while running faster, using less memory and less
power, and extending the maximum range of detections.

## References

  * [1] Muhammad Azri Yahya, Shuzlina Abdul-Rahman, and Sofianita Mutalib.  Object detection for autonomous vehicle with lidar using deep learning.  In 2020 IEEE 10th International conference on system engineering and technology (ICSET), pages 207–212. IEEE, 2020.
  * [2] Rui Qian, Xin Lai, and Xirong Li.  3d object detection for autonomous driving: a survey.  Pattern Recognition, page 108796, 2022.
  * [3] Mark A Richards.  Fundamentals of radar signal processing.  McGraw-Hill Education, 2014.
  * [4] Shan Zhang, Aditya Vempaty, Susan E Parks, and Pramod K Varshney.  On classification of environmental acoustic data using crowds.  In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5880–5884. IEEE, 2017.
  * [5] Shan Zhang, Baocheng Geng, Pramod K Varshney, and Muralidhar Rangaswamy.  Fusion of deep neural networks for activity recognition: A regular vine copula based approach.  In 2019 22th International Conference on Information Fusion (FUSION), pages 1–7. IEEE, 2019.
  * [6] Nalin Kumar, Manuel Gerardo Garcia, Kanishka Tyagi, et al.  Material handling using machine learning system, January 27 2022.  US Patent App. 17/495,291.
  * [7] Kanishka Tyagi, Chinmay Rane, and Michael Manry.  Chapter 1 - supervised learning.  In Rajiv Pandey, Sunil Kumar Khatri, Neeraj kumar Singh, and Parul Verma, editors, Artificial Intelligence and Machine Learning for EDGE Computing, pages 3–22. Academic Press, 2022.
  * [8] Nicolas Scheiner, Florian Kraus, Fangyin Wei, Buu Phan, Fahim Mannan, Nils Appenrodt, Werner Ritter, Jurgen Dickmann, Klaus Dietmayer, Bernhard Sick, et al.  Seeing around street corners: Non-line-of-sight detection and tracking in-the-wild using doppler radar.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2068–2077, 2020.
  * [9] Daniel Niederlöhner, Michael Ulrich, Sascha Braun, Daniel Köhler, Florian Faion, Claudius Gläser, André Treptow, and Holger Blume.  Self-supervised velocity estimation for automotive radar object detection networks.  In 2022 IEEE Intelligent Vehicles Symposium (IV), pages 352–359. IEEE, 2022.
  * [10] Tokihiko Akita and Seiichi Mita.  Object tracking and classification using millimeter-wave radar based on lstm.  In 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pages 1110–1115. IEEE, 2019.
  * [11] Bence Major, Daniel Fontijne, Amin Ansari, Ravi Teja Sukhavasi, Radhika Gowaikar, Michael Hamilton, Sean Lee, Slawomir Grzechnik, and Sundar Subramanian.  Vehicle detection with automotive radar using deep learning on range-azimuth-doppler tensors.  In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pages 0–0, 2019.
  * [12] Guoqiang Zhang, Haopeng Li, and Fabian Wenger.  Object detection and 3d estimation via an fmcw radar using a fully convolutional network.  In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4487–4491. IEEE, 2020.
  * [13] Adriana-Eliza Cozma, Lisa Morgan, Martin Stolz, David Stoeckel, and Kilian Rambach.  Deephybrid: Deep learning on automotive radar spectra and reflections for object classification.  In 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), pages 2682–2687. IEEE, 2021.
  * [14] Florian Kastner, Markus Schoeler, Mirko Meuter, and Adrian Becker.  Methods and systems for processing radar reflections, 2021.  US20210018593A1.
  * [15] Lin Wang and Kuk-Jin Yoon.  Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021\.
  * [16] Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al.  Distilling the knowledge in a neural network.  arXiv preprint arXiv:1503.02531, 2(7), 2015.
  * [17] Valeriy Gavrishchaka, Zhenyi Yang, Rebecca Miao, and Olga Senyukova.  Advantages of hybrid deep learning frameworks in applications with limited data.  International Journal of Machine Learning and Computing, 8(6):549–558, 2018.
  * [18] Rosanne Liu, Joel Lehman, Piero Molino, Felipe Petroski Such, Eric Frank, Alex Sergeev, and Jason Yosinski.  An intriguing failing of convolutional neural networks and the coordconv solution.  Advances in neural information processing systems, 31, 2018.