[a]Thomas Stone

# Evaluating Parametric Integrals in the Minkowski Regime without Contour
Deformation

Stephen Jones  Anton Olsson

###### Abstract

We present selected examples demonstrating an alternative approach to contour
deformation for numerically computing loop integrals in the Minkowski regime.
This method focuses on identifying singular hypersurfaces (varieties of the
$\mathscr{F}$ polynomial) and mapping them to known points which can then be
resolved by employing blow-ups/sector decomposition techniques, thereby
avoiding the need for contour deformation. Using this technique, we achieve
improved convergence properties without the need for contour deformation,
which is known to significantly increase the complexity of the integrand by
introducing, for example, derivatives of the $\mathscr{F}$ polynomial and
complicated Jacobians. We highlight that while we have only tested the
approach on selected one-, two- and three-loop massless and one-loop massive
examples, it shows promise for practical applications, offering potential
benefits over the traditional approach. Evaluation times are compared with
existing contour deformation implementations to illustrate the performance of
this alternative method.

##  1 Introduction & Motivation

Many loop integrals appearing in state-of-the-art amplitude calculations are
analytically intractable; as a result of this, numerical methods and other
approximations have been developed to tackle these integrals [1, 2, 3, 4, 5,
6, 7, 8, 9, 10, 11]. Even with these tools, we often have trouble numerically
calculating in the so-called “Minkowski” regime, which we define below, due to
poles on the contour of integration. To remedy this, we deform the contour of
integration away from the real axis to avoid these poles in such a way that we
respect the causal $i\delta$ (Feynman) prescription. Methods have been
explored to remove the need for contour deformation for momentum-space
integrals (for example, in the context of loop-tree duality [12, 13, 14]) so
it is a natural question to ask whether it is possible to do the same in
Feynman parameter space.

As a reminder, we show how an integral in momentum space appears once it has
been cast into Feynman-parameterised form:

$\begin{gathered}I=\bigintsss\limits_{-\infty}^{+\infty}\left(\prod\limits_{l=1}^{L}\frac{\mathrm{d}^{D}k_{l}}{i\pi^{\frac{D}{2}}}\right)\prod\limits_{j=1}^{N}\frac{1}{P^{\nu_{j}}_{j}\left(\\{k\\},\\{p\\},m_{j}^{2}\right)}\\\
\Downarrow\\\ I=\frac{\left(-1\right)^{\nu}\Gamma\left(\nu-
LD/2\right)}{\prod_{j=1}^{N}\Gamma\left(\nu_{j}\right)}\\!\bigintsss\limits_{\mathbb{R}_{\geq
0}^{N}}\left(\prod\limits_{j=1}^{N}\mathrm{d}x_{j}x_{j}^{\nu_{j}-1}\right)\frac{\mathscr{U}\\!\left(\mathbf{x}\right)^{\nu-(L+1)D/2}}{\left(\mathscr{F}\\!\left(\mathbf{x},\mathbf{s}\right)-i\delta\right)^{\nu-
LD/2}}\delta\left(1-\sum\limits_{j=1}^{N}x_{j}\right)\end{gathered}$ (1)

where $D$ is the spacetime dimension, $L$ is the number of loops and $N$ is
the number of propagators (denoted by $P_{j}$ which are of the standard
“$p^{2}-m^{2}+i\delta$” momentum-space form) and we allow for dots
($\nu_{j}\in\mathbb{N}$). After Feynman parameterisation, we have an integral
of the Feynman parameters $\\{x_{j}\\}$ over the positive hyper-quadrant
$\mathbb{R}_{\geq 0}^{N}$ with an integrand containing the homogeneous
Symanzik polynomials $\mathscr{U}\\!\left(\mathbf{x}\right)$ and
$\mathscr{F}\\!\left(\mathbf{x},\mathbf{s}\right)$ of degree $L$ and $L+1$
respectively.

We note that there are several equivalent algorithms to construct the Symanzik
polynomials, including constructing directly from cutting propagators in the
Feynman diagram associated to the relevant integral (where this diagram
exists). It is also instructive to emphasise that $\mathscr{F}$ is dependent
on the kinematics of the problem (denoted by $\mathbf{s}$) whereas
$\mathscr{U}$ depends solely on the Feynman parameters and is strictly non-
negative in the integration domain (and hence, does not need an $i\delta$
prescription). We define the (pseudo-)Euclidean regime to be the region of
kinematic space where $\mathscr{F}\\!\left(\mathbf{x},\mathbf{s}\right)\geq 0$
everywhere in the domain of integration (vanishing only on the boundary) and
the Minkowski regime is henceforth defined to be the remaining region of
kinematic space which is not in the (pseudo-)Euclidean regime (e.g.
$\mathcal{M}=\\{(s,t)\in\mathbb{R}^{2}\ |\ s>|t|\land t<0\\}$ for on-shell
massless $2\to 2$ scattering). In this region, $\mathscr{F}$ can take
different signs within the domain of integration and hence, in this regime, a
prescription is required to deform the contour in a way that obeys causality.
The causal $i\delta$ prescription in momentum space induces
$\mathscr{F}-i\delta$ in Feynman parameter space and this is needed when
$\mathscr{F}=0$ within the domain of integration (that is to say, within
$\mathbb{R}_{>0}^{N}$ but strictly not on the boundary).

The standard way this is implemented in Feynman parameter space is as follows
[15, 16, 17, 18]:

$\mathscr{F}\left(\mathbf{z}\right)=\mathscr{F}\left(\mathbf{x}\right)-i\sum\limits_{j}\tau_{j}\frac{\partial\mathscr{F}\left(\mathbf{x}\right)}{\partial
x_{j}},\qquad\tau_{j}=\lambda_{j}x_{j}\left(1-x_{j}\right)\frac{\partial\mathscr{F}\left(\mathbf{x}\right)}{\partial
x_{j}}$ (2)

such that $\mathscr{F}$ is given a small imaginary part with the deformation
away from the real axis parameterised by $\\{\lambda_{j}\\}$. Many techniques,
including neural networks [19], have been used to optimise this choice of
$\\{\lambda_{j}\\}$ but overall this can still slow down numerical integration
massively (due to increased variance of the integrand leading to cancellations
between postive and negative quantities of similarly large magnitude). There
are even cases where this procedure fails entirely and this is related to the
presence of a leading Landau singularity for arbitrary kinematics (see, for
example, [20]). This happens when, for all kinematic points in the Minkowski
regime, $\mathcal{M}$, there exists (at least) one point where
$\mathscr{F}\left(\mathbf{x}\right)$ and all
$\frac{\partial\mathscr{F}\left(\mathbf{x}\right)}{\partial x_{j}}$ are
simultaneously zero so the construction in Eq. 2 breaks down. We encounter an
example of this in section 2.3.

For the reasons described above, it would be beneficial to avoid this
procedure of contour deformation in Feynman parameter space where possible;
that motivation is the focus of the work presented in these proceedings.

###  1.1 The Idea

The central idea is to construct transformations of the Feynman parameters
$\\{x_{j}\\}$ which map zeroes of the $\mathscr{F}$ polynomial to the boundary
of integration as illustrated in an example in Fig. 1.

$\Rightarrow$

Figure 1: A one-loop example hypersurface defined by $\mathscr{F}=0$ before
(left) and after (right) transformation.

These transformations look like a) positive parameter rescalings (e.g.
$x_{j}\rightarrow\alpha x_{j}$ or $x_{j}\rightarrow x_{i}x_{j}$) and b) the
introduction of hierarchies between Feynman parameters which generate split
integrals to cover the entire original parameter space (e.g. $x_{j}\rightarrow
x_{i}+x_{j}$). Transformations of type a) are justified by the projective
invariance of the integral and the Cheng-Wu theorem [21, 22, 23] and
transformations of type b) are justified by inserting the Heaviside identity,
$\theta\left(x_{a}-x_{b}\right)+\theta\left(x_{b}-x_{a}\right)=1$, under the
integral sign. These transformations are very reminiscent of sector
decomposition [24] (and indeed, this work is greatly inspired by that
technique) but we stress that sector decomposition as used for the numerical
computation of Feynman integrals in the physics literature is a method for
dealing with singularities on the boundary of the domain of integration
whereas this technique maps singularities within the domain to the boundary
where they can then be dealt with using the sector decomposition algorithm
(for which we use pySecDec [1]).

The result of these transformations is that we only ever integrate manifestly
non-negative integrands (for the transformations which make $\mathscr{F}$ non-
positive, we extract an overall minus sign from $\mathscr{F}$ and factor it
outside the integral along with the $i\delta$ all to the appropriate power
which generates the physically-correct imaginary part). We stitch everything
together as follows:

$I=\sum\limits_{n_{+}=1}^{N_{+}}I^{+}_{n_{+}}+(-1-i\delta)^{-\left(\nu-
LD/2\right)}\sum\limits_{n_{-}=1}^{N_{-}}I^{-}_{n_{-}}$ (3)

where $I^{+(\\!-\\!)}_{n_{+(\\!-\\!)}}$ corresponds to an integral where the
transformed $\mathscr{F}$ polynomial is non-negative (non-positive). As we
will demonstrate in a number of specific examples, it can be much faster to
numerically integrate the manifestly non-negative integrands of the integrals
$\\{I^{+}_{n_{+}},I^{-}_{n_{-}}\\}$. We begin by considering a selection of
integrals with massless internal lines.

##  2 Massless Integrals

Our improvements on timings for numerical integration are best illustrated in
our application of the method to massless integrals (by which we refer to
integrals whose internal propagators have no masses; however, we do allow for
off-shell external legs). To make the technique concrete, we will explicitly
go through a simple one-loop massless box and then demonstrate the efficiency
increases on two-loop and three-loop cases. The Feynman diagrams of the
specific example integrals we consider are shown in Fig. 2.

$p_{1}$$p_{2}$$p_{3}$$p_{4}$$x_{1}$$x_{0}$$x_{2}$$x_{3}$

$p_{1}$$p_{2}$$p_{3}$$p_{4}$

$p_{1}$$p_{3}$$p_{2}$$p_{4}$

Figure 2: A selection of massless integrals to which the method has been
applied. Thin blue lines denote on-shell external legs, thick green lines
denote off-shell external legs and black lines denote massless internal
propagators. For the one-loop example, we also explicitly associate Feynman
parameters (in red) to each propagator to simplify the connection between the
diagram and its cut-constructed $\mathscr{U}$ and $\mathscr{F}$ polynomials.

###  2.1 One-Loop Off-Shell Box

We begin by considering the one-loop off-shell box. For this example,

$\displaystyle\mathscr{U}$ $\displaystyle=x_{0}+x_{1}+x_{2}+x_{3}$ (4)
$\displaystyle\mathscr{F}$
$\displaystyle=-sx_{0}x_{2}-tx_{1}x_{3}-p_{1}^{2}x_{0}x_{1}$ (5)

where $s$ and $t$ are the usual Mandelstam variables.

If we restrict to
${\mathcal{M}_{\mathrm{phys}}=\\{(s,t,p_{1}^{2})\in\mathbb{R}^{3}\ |\ s>0\land
t<0\land p_{1}^{2}>0\\}}$, there will be zeroes of $\mathscr{F}$ within the
integration volume for $(x_{0},x_{1},x_{2},x_{3})\in\mathbb{R}_{>0}^{4}$. We
show in Fig. 3 the series of transformations which resolves the singularities
due to $\mathscr{F}$.

Figure 3: A flowchart depicting the series of transformations which resolves
the singularities for the one-loop off-shell box.

To be consistent, we must also generate $\mathscr{U}^{+}_{1}$,
$\mathscr{U}^{-}_{1}$ and $\mathscr{U}^{-}_{2}$ by applying the corresponding
transformations to $\mathscr{U}$ as well as generating the absolute values of
the corresponding Jacobian determinants: $\mathscr{J}^{+}_{1}$,
$\mathscr{J}^{-}_{1}$ and $\mathscr{J}^{-}_{2}$.

Having calculated these objects, we have a set of manifestly non-negative
integrands with the same structure, $\
\sim\\!\\!\\!\mathscr{J}^{\pm}_{n_{\pm}}\left(\mathscr{U}^{\pm}_{n_{\pm}}\right)^{2\epsilon}\\!\left(\mathscr{F}^{\pm}_{n_{\pm}}\right)^{-2-\epsilon}$,
which we can combine to express our initial integral into the following sum of
three easier integrals:

$I=I_{1}^{+}+(-1-i\delta)^{-2-\epsilon}\left(I^{-}_{1}+I^{-}_{2}\right)$ (6)

and we have verified this construction numerically integrates to the known
analytic result using pySecDec.

###  2.2 Two-Loop Non-Planar Box

We now consider a slightly more complicated example: a two-loop non-planar
box. After cut-construction, we find that
$\mathscr{F}=-sx_{1}x_{2}x_{5}-tx_{0}x_{1}x_{3}-ux_{0}x_{2}x_{4}$ and
furthermore, applying the momentum conservation constraint $s+t+u=0$ allows us
to eliminate $u$ resulting in

$\mathscr{F}=-sx_{1}x_{2}x_{5}-tx_{0}x_{1}x_{3}+(s+t)x_{0}x_{2}x_{4}.$ (7)

It is clear to see that even with $s$ and $t$ of the same sign, $\mathscr{F}$
can be zero within $\mathbb{R}_{>0}^{6}$ and, in fact, a (pseudo-)Euclidean
region does not exist for this integral – the entire kinematic space is in the
Minkowski regime. To test our method, we consider two distinct subsets of the
Minkwoski regime:

$\displaystyle\mathcal{M}_{1}:=\\{\ (s,t)\in\mathbb{R}^{2}\ |\ s>|t|\ \land\
t<0\ \\}$ (8) $\displaystyle\mathcal{M}_{2}:=\\{\ (s,t)\in\mathbb{R}^{2}\ |\
0<s<|t|\ \land\ t<0\ \\}.$ (9)

We find that, to cover all the space in the regime defined by
$\mathcal{M}_{1}$, we need six integrals combined in the form:

$I=\left(I_{1}^{+}+I_{2}^{+}+I_{3}^{+}\right)+(-1-i\delta)^{-2-2\epsilon}\left(I^{-}_{1}+I^{-}_{2}+I^{-}_{3}\right)$
(10)

which agrees with the analytic result [25] when integrated numerically. For
the regime $\mathcal{M}_{2}$, we similarly find a total of six (different)
integrals. We compare the timings for integrating up-to-and-including finite
order in $\epsilon$ with the standard contour deformation setup in pySecDec
versus our method in Fig. 4 and we find significant improvements for a point
in $\mathcal{M}_{1}$ (4(a)) and $\mathcal{M}_{2}$ (4(b)).

(a) A kinematic point in $\mathcal{M}_{1}$

(b) A kinematic point in $\mathcal{M}_{2}$

Figure 4: Timing vs. precision comparisons avoiding contour deformation for
the two-loop non-planar box. Orders of magnitude improvements are observed for
points in both $\mathcal{M}_{1}$ (a) and $\mathcal{M}_{2}$ (b).

###  2.3 Three-Loop Non-Planar Box

The final example of massless integrals is the three-loop non-planar box with
an $\mathscr{F}$ polynomial given by

$\mathscr{F}=-s\left(x_{1}x_{4}-x_{0}x_{5}\right)\left(x_{3}x_{6}-x_{2}x_{7}\right)-t\left(x_{1}x_{2}-x_{0}x_{3}\right)\left(x_{5}x_{6}-x_{4}x_{7}\right).$
(11)

As previously mentioned, this an example of a case where the integral has a
leading Landau singularity for arbitrary kinematics [20] and so the standard
“out-the-box” implementation of contour deformation will inevitably fail. As
explained in [20], this Landau singularity can be resolved by first
linearising the singular hypersurface and then by considering all permutations
of hierarchies for the Feynman parameters; finally, after taking into account
symmetry, we are left with six independent integrals. For the regime
$\mathcal{M}_{\mathrm{phys}}=\\{(s,t)\in\mathbb{R}^{2}\ |\ s>|t|\land t<0\\}$,
we find that two of these six integrals would usually require contour
deformation. Their $\mathscr{F}$ polynomials are

$\displaystyle\mathscr{F}_{a}=x_{1}x_{3}x_{5}x_{7}\left[-sx_{0}x_{2}+|t|\left(x_{0}+x_{4}\right)\left(x_{2}+x_{4}\right)\right]$
(12)
$\displaystyle\mathscr{F}_{b}=x_{1}x_{3}x_{5}x_{7}\left[sx_{6}\left(x_{0}+x_{2}+x_{6}\right)-|t|\left(x_{0}+x_{6}\right)\left(x_{2}+x_{6}\right)\right].$
(13)

Through a more involved combination of splits and rescalings, we can express
each of these integrals in terms of four others, resulting in a total of
twelve integrals to compute (none of which require contour deformation):

$I=\sum\limits_{n_{+}=1}^{8}I^{+}_{n_{+}}+(-1-i\delta)^{-2-3\epsilon}\sum\limits_{n_{-}=1}^{4}I^{-}_{n_{-}}.$
(14)

Having verified the numerical result of this construction against the known
analytic result [26, 27], we compared timings between numerically computing
the six split integrals (post-resolution of the Landau singularity) using the
usual contour deformation procedure and numerically computing the twelve
integrals required to avoid contour deformation and these are shown in Fig. 5.

Figure 5: Timing comparison to obtain the leading pole ($1/\epsilon^{4}$)
coefficient of the three-loop non-planar-box.

We were not able to go beyond the leading pole ($1/\epsilon^{4}$) for the
standard contour deformation setup whereas we could expand deeper in
$\epsilon$ with our method for avoiding the procedure.

For the point $(s=1,t=-1/5)\in\mathcal{M}_{\mathrm{phys}}$, we state the
precisions obtained after $\mathcal{O}\\!\left(\text{mins}\right)$ with
pySecDec as well as the analytic result for comparison:

$\displaystyle I_{\text{CD}}$ $\displaystyle=\
\left[8.340\textbf{55}-52.36\textbf{08}i\right]\epsilon^{-4}\ +\
\mathcal{O}(\epsilon^{-3})$ (15) $\displaystyle I_{\text{Avoid CD}}$
$\displaystyle=\ \left[8.3400403920\textbf{28}\ \ -\ \
52.35987755983\textbf{47}i\right]\epsilon^{-4}\ +\ \mathcal{O}(\epsilon^{-3})$
(16) $\displaystyle I_{\text{analytic}}$ $\displaystyle=\
\left[8.34004039223768-52.35987755984493i\right]\epsilon^{-4}\ +\
\mathcal{O}(\epsilon^{-3})$ (17)

where the boldface digits denote the reported error on the final two stated
digits preceding them.

##  3 Massive Integrals

Integrals with massive internal propagators (henceforth, “massive integrals”)
appear in a variety of phenomenologically-relevant amplitudes; for example,
QCD corrections to processes involving massive quarks or electroweak
corrections which contain a number of different mass scales resulting in
integrals which often have no known analytic solution. In this scenario,
numerical methods are essential – the question therefore arises whether the
method presented in this work extends to this class of integrals. The
essential difference is that the $\mathscr{F}$ polynomial is modified by an
extra term proportional to the $\mathscr{U}$ polynomial:

$\mathscr{F}\left(\mathbf{x},\mathbf{s}\right)\ =\
\mathscr{F}_{0}\left(\mathbf{x},\mathbf{s}\right)\ +\
\mathscr{U}\left(\mathbf{x}\right)\sum\limits_{j=1}^{N}m_{j}^{2}x_{j}$ (18)

where $\mathscr{F}_{0}\left(\mathbf{x},\mathbf{s}\right)$ is the polynomial
for the corresponding massless integral. This modification means that each
individual Feynman parameter $x_{j}$ may appear quadratically in $\mathscr{F}$
as opposed to the massless case where, in each term, $x_{j}$ can appear
linearly at most. The effect of this is that viable transformations can be
difficult to deduce even for the most trivial of massive integrals. This
motivates us to consider whether geometrical intuition can guide us in the
right direction.

We have successfully employed the method on the massive bubble (which we
describe in detail in Section 3.1) and triangle (both shown in Fig. 6) and
current work involves the investigation of the sunrise integral (also in Fig.
6) which is known to be the simplest Feynman integral which cannot be
expressed in terms of multiple polylogarithms but rather elliptic integrals.

$p$$m_{1}$$m_{2}$

$p_{1}$$p_{2}$$p_{3}$$m$

$p$$m_{1}$$m_{3}$$m_{2}$

Figure 6: The massive integrals to which the method has been successfully
applied (left & centre) and the massive sunrise integral (right) which we are
currently investigating. The colour scheme matches Fig. 2 and, additionally,
purple lines denote massive internal propagators with their masses given in
red.

###  3.1 Massive Bubble

The $\mathscr{F}$ polynomial of the massive bubble is given by

$\mathscr{F}=-p^{2}x_{1}x_{2}+\left(x_{1}+x_{2}\right)\left(m_{1}^{2}x_{1}+m_{2}^{2}x_{2}\right).$
(19)

We define
$\beta^{2}:=\frac{p^{2}-\left(m_{1}+m_{2}\right)^{2}}{p^{2}-\left(m_{1}-m_{2}\right)^{2}}\in\left[0,1\right)$
and scale out the dimension of $\mathscr{F}$ with the transformations
$x_{i}\rightarrow\frac{x_{i}}{m_{i}}$ to instead analyse the dimensionless
polynomial

$\widetilde{\mathscr{F}}=x_{1}^{2}+x_{2}^{2}-2\frac{1+\beta^{2}}{1-\beta^{2}}x_{1}x_{2}.$
(20)

It is illuminating to plot the variety of $\widetilde{\mathscr{F}}$ and this
is given in Fig. 7.

$x_{2}=\frac{1\pm\beta}{1\mp\beta}x_{1}$ Figure 7: Plot of the variety (in
orange) of the $\widetilde{\mathscr{F}}$ polynomial for the massive bubble
which separates three regions in the positive quadrant of $\mathbb{R}^{2}$.

We see that there are three distinct regions separated by the codimension-1
zero hypersurface of $\widetilde{\mathscr{F}}$, two of which are defined by
$\widetilde{\mathscr{F}}$ being positive, the remaining region defined by
$\widetilde{\mathscr{F}}$ being negative.

We can construct transformations which directly send the variety to the
integration boundary. For example, for the upper-left region of the plot, we
demand of the transformed variables {$y_{i}$} that the $y_{2}$-axis coincides
with the $x_{2}$-axis and the $y_{1}$-axis coincides with the solution line
$x_{2}=\frac{1+\beta}{1-\beta}x_{1}$. Along with the constraint that points
within this region get mapped to the positive quadrant in the new $y_{i}$
variables, this uniquely defines the transformation:

$y_{1}\overset{!}{=}x_{1},y_{2}\overset{!}{=}x_{2}-\frac{1+\beta}{1-\beta}x_{1}\Rightarrow
x_{1}\rightarrow y_{1},x_{2}\rightarrow y_{2}+\frac{1+\beta}{1-\beta}y_{1}.$
(21)

We have verified that the resulting construction
$I=I_{1}^{+}+I_{2}^{+}+(-1-i\delta)^{-\epsilon}I^{-}_{1}$ replicates the
analytic result by integrating it both numerically with pySecDec and
analytically (as we can easily calculate the constituent integrals in the sum
symbolically, for example, using Mathematica).

##  4 Conclusions & Outlook

As we have shown, this method allows us to avoid contour deformation (and its
associated computational inefficiencies) in regions of kinematic space which
would ordinarily require it. We demonstrate that the time to evaluate multi-
loop integrals numerically can be improved by a factor of one thousand or more
in selected cases and the ability to expand deeper in the dimensional
regulator $\epsilon$ has also been observed for a particularly pathological
three-loop example. We have also conveyed that geometry can play a role in
deriving the necessary transformations of the Feynman parameters to enable the
avoidance of contour deformation, for example, in massive integrals where they
are considerably more difficult to identify.

Current and future work is directed towards utilising this intuition attained
by visualising the problem geometrically to move away from arbitrary shift and
rescaling transformations (derived from a trial-and-error approach) and
towards a general algorithm to define canonical transformations. There is
potential insight to be gained from techniques in the fields of algebraic and
tropical geometry by directly considering the variety of the Symanzik
polynomial $\mathscr{F}$ and using computational tools for the resolution of
singularities [28].

Understanding how to apply this method to two- and even three-loop massive
integrals could lead to very significant time improvements, perhaps even
better than the orders of magnitude improvements we have already presented in
these proceedings for massless integrals. Moreover, applying the method to
further multi-loop massless integrals may allow for (currently-impossible)
numerical cross-checks of analytic or semi-analytic results.

The long-term objective of this work is to implement this technique in
numerical loop integration packages like pySecDec to be used in the efficient
calculation of QFT amplitudes where, along with the integration-by-parts
reduction to master integrals, multi-loop integral evaluation remains a major
bottleneck.

## Acknowledgments

We thank the members of the pySecDec collaboration, Gudrun Heinrich, Matthias
Kerner, Vitaly Magerya, and Johannes Schlenk, for discussions and insightful
input. This research was supported in part by the Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) under grant 396021762
- TRR 257, and by the UK Science and Technology Facilities Council under
contract ST/T001011/1. SJ is supported by a Royal Society University Research
Fellowship (Grant URF/R1/201268).

## References

  * [1] G. Heinrich, S. Jones, M. Kerner, V. Magerya, A. Olsson and J. Schlenk, _Numerical scattering amplitudes with pysecdec_ , _Computer Physics Communications_ 295 (2024) 108956.
  * [2] A. Smirnov, N. Shapurov and L. Vysotsky, _Fiesta5: Numerical high-performance feynman integral evaluation_ , _Computer Physics Communications_ 277 (2022) 108386.
  * [3] M. Borinsky, H.J. Munch and F. Tellander, _Tropical feynman integration in the minkowski regime_ , _Computer Physics Communications_ 292 (2023) 108874.
  * [4] Z. Capatti, V. Hirschi, D. Kermanschah, A. Pelloni and B. Ruijl, _Manifestly causal loop-tree duality_ , 2020.
  * [5] W.J.T. Bobadilla, _Lotty – the loop-tree duality automation_ , 2021.
  * [6] M. Czakon, _Automatized analytic continuation of mellin–barnes integrals_ , _Computer Physics Communications_ 175 (2006) 559–571.
  * [7] J. Gluza, K. Kajda and T. Riemann, _AMBRE: A Mathematica package for the construction of Mellin-Barnes representations for Feynman integrals_ , _Comput. Phys. Commun._ 177 (2007) 879 [0704.2423].
  * [8] X. Liu and Y.-Q. Ma, _Amflow: A mathematica package for feynman integrals computation via auxiliary mass flow_ , _Computer Physics Communications_ 283 (2023) 108565.
  * [9] M. Hidding, _Diffexp, a mathematica package for computing feynman integrals in terms of one-dimensional series expansions_ , 2020.
  * [10] T. Armadillo, R. Bonciani, S. Devoto, N. Rana and A. Vicini, _Evaluation of feynman integrals with arbitrary complex masses via series expansions_ , _Computer Physics Communications_ 282 (2023) 108545.
  * [11] S. Borowka, T. Gehrmann and D. Hulme, _Systematic approximation of multi-scale feynman integrals_ , _Journal of High Energy Physics_ 2018 (2018) .
  * [12] S. Buchta, G. Chachamis, I. Malamos, I. Bierenbaum, P. Draggiotis and G. Rodrigo, _The loop-tree duality at work_ , 2014.
  * [13] Z. Capatti, V. Hirschi, D. Kermanschah, A. Pelloni and B. Ruijl, _Numerical loop-tree duality: contour deformation and subtraction_ , _Journal of High Energy Physics_ 2020 (2020) .
  * [14] D. Kermanschah, _Numerical integration of loop integrals through local cancellation of threshold singularities_ , _Journal of High Energy Physics_ 2022 (2022) .
  * [15] D.E. Soper, _Techniques for qcd calculations by numerical integration_ , _Phys. Rev. D_ 62 (2000) 014009.
  * [16] T. Binoth, J.-P. Guillet, G. Heinrich, E. Pilon and C. Schubert, _An algebraic/numerical formalism for one-loop multi-leg amplitudes_ , _Journal of High Energy Physics_ 2005 (2005) 015–015.
  * [17] C. Anastasiou, S. Beerli and A. Daleo, _Two-loop qcd amplitude for $gg\rightarrow h$, $h$ in the minimal supersymmetric standard model_, _Phys. Rev. Lett._ 100 (2008) 241806.
  * [18] S. Borowka, J. Carter and G. Heinrich, _Numerical evaluation of multi-loop integrals for arbitrary kinematics with secdec 2.0_ , _Computer Physics Communications_ 184 (2013) 396.
  * [19] R. Winterhalder, V. Magerya, E. Villa, S. Jones, M. Kerner, A. Butter et al., _Targeting multi-loop integrals with neural networks_ , _SciPost Physics_ 12 (2022) .
  * [20] Gardi, Herzog, Jones and Ma, _To appear_ , 2024.
  * [21] H. Cheng and T.T. Wu, _Expanding Protons: Scattering at High Energies_ (1987).
  * [22] E. Panzer, _Feynman integrals and hyperlogarithms_ , 2015.  10.18452/17157.
  * [23] S. Weinzierl, _Feynman integrals_ , 2022.
  * [24] G. Heinrich, _Sector decomposition_ , _International Journal of Modern Physics A_ 23 (2008) 1457–1486.
  * [25] J. Tausk, _Non-planar massless two-loop feynman diagrams with four on-shell legs_ , _Physics Letters B_ 469 (1999) 225–234.
  * [26] J. Henn, B. Mistlberger, V.A. Smirnov and P. Wasser, _Constructing d-log integrands and computing master integrals for three-loop four-particle scattering_ , _Journal of High Energy Physics_ 2020 (2020) .
  * [27] P. Bargieła, F. Caola, A. von Manteuffel and L. Tancredi, _Three-loop helicity amplitudes for diphoton production in gluon fusion_ , _Journal of High Energy Physics_ 2022 (2022) .
  * [28] H. Hironaka, _Resolution of singularities of an algebraic variety over a field of characteristic zero: I_ , _Annals of Mathematics_ 79 (1964) 109.


#  EyeDAS: Securing Perception of Autonomous Cars Against the Stereoblindness
Syndrome

###### Abstract

The ability to detect whether an object is a 2D or 3D object is extremely
important in autonomous driving, since a detection error can have life-
threatening consequences, endangering the safety of the driver, passengers,
pedestrians, and others on the road. Methods proposed to distinguish between 2
and 3D objects (e.g., liveness detection methods) are not suitable for
autonomous driving, because they are object dependent or do not consider the
constraints associated with autonomous driving (e.g., the need for real-time
decision-making while the vehicle is moving). In this paper, we present
EyeDAS, a novel few-shot learning-based method aimed at securing an object
detector (OD) against the threat posed by the stereoblindness syndrome (i.e.,
the inability to distinguish between 2D and 3D objects). We evaluate EyeDAS’s
real-time performance using 2,000 objects extracted from seven YouTube video
recordings of street views taken by a dash cam from the driver’s seat
perspective. When applying EyeDAS to seven state-of-the-art ODs as a
countermeasure, EyeDAS was able to reduce the 2D misclassification rate from
71.42-100% to 2.4% with a 3D misclassification rate of 0% (TPR of 1.0). We
also show that EyeDAS outperforms the baseline method and achieves an AUC of
over 0.999 and a TPR of 1.0 with an FPR of 0.024.

##  1 Introduction

After years of research and development, automobile technology is rapidly
approaching the point at which human drivers can be replaced, as commercial
cars are now capable of supporting semi-autonomous driving. To create a
reality that consists of commercial semi-autonomous cars, scientists had to
develop the computerized driver intelligence required to: (1) continuously
create a virtual perception of the physical surroundings (e.g., detect
pedestrians, road signs, cars, etc.), (2) make decisions, and (3) perform the
corresponding action (e.g., notify the driver, turn the wheel, stop the car).

While computerized driver intelligence brought semi-autonomous driving to new
heights in terms of safety safety , recent incidents have shown that semi-
autonomous cars suffer from the stereoblindness syndrome: they react to 2D
objects as if they were 3D objects due to their the inability to distinguish
between these two types of objects. This fact threatens autonomous car safety,
because a 2D object (e.g., an image of a car, dog, person) in a nearby
advertisement that is misdetected as a real object can trigger a reaction from
a semi-autonomous car (e.g., cause it to stop in the middle of the road), as
shown in Fig. 1. Such undesired reactions may endanger drivers, passengers,
and nearby pedestrians as well. As a result, there is a need to secure semi-
autonomous cars against the perceptual challenge caused by the stereoblindness
syndrome.

The perceptual challenge caused by the stereoblindness syndrome stems from
object detectors’ (which obtain data from cars’ video cameras)
misclassification of 2D objects. One might argue that the stereoblindness
syndrome can be addressed by adopting a sensor fusion approach: by cross-
correlating data from the video cameras with data obtained by sensors aimed at
detecting depth (e.g., ultrasonic sensors, radar). However, due to safety
concerns, a "safety first" policy is implemented in autonomous vehicles, which
causes them to consider a detected object as a real object even when it is
detected by a single sensor without additional validation from another sensor
nassi2020phantom ; yan2016can . This is also demonstrated in Fig. 1 which
shows how a Tesla’s autopilot triggers a sudden stop due to the misdetection
of a 2D object as a real object, despite the fact that Teslas are equipped
with radar, a set of ultrasonic sensors, and a set of front-facing video
cameras.

In addition, while various methods have used liveness detection algorithms to
detect whether an object is 2D/3D, the proposed methods do not provide the
functionality required to distinguish between 2D/3D objects in an autonomous
driving setup, because they are object dependent (they cannot generalize
between different objects, e.g., cars and pedestrians) and do not take into
account the real-time constraints associated with autonomous driving. As a
result, there is a need for dedicated functionality that validates the
detections of video camera based object detectors and considers the
constraints of autonomous driving.

In this paper, we present EyeDAS, a committee of models that validates objects
detected by the on-board object detector. EyeDAS aims to secure a single
channel object detector that obtains data from a video camera and provides a
solution to the stereoblindness syndrome, i.e., distinguishes between 2 and 3D
objects, while taking the constrains of autonomous driving (both safety and
real-time constraints) into account. EyeDAS can be deployed on existing
advanced driver-assistance systems (ADASs) without the need for additional
sensors.

EyeDAS is based on few-shot learning and consists of four lightweight
unsupervised models, each of which utilizes a unique feature extraction method
and outputs a 3D confidence score. Finally, a meta-classifier uses the output
of the four models to determine whether the given object is a 2 or 3D object.

We evaluate EyeDAS using a dataset collected from seven YouTube video
recordings of street views taken by a dash cam from the driver’s seat
perspective; the 2D objects in the dataset were extracted from various
billboards that appear in the videos. When applying EyeDAS to seven state-of-
the-art ODs as a countermeasure, EyeDAS was able to reduce the 2D
misclassification rate from 71.42-100% to 2.4% with a 3D misclassification
rate of 0% (TPR of 1.0). We also show that EyeDAS outperforms the baseline
method and achieves an AUC of over 0.999 and a TPR of 1.0 with an FPR of
0.024.

Figure 1: Two well-known incidents that demonstrate how Teslas misdetect 2D
objects as real objects.

In this research we make the following contributions: (1) we present a
practical method for securing object detectors against the stereoblindness
syndrome that meets the constraints of autonomous driving (safety and real-
time constraints), and (2) we show that the method can be applied using few-
shot learning, can be used to detect whether an inanimate object is a 2D or 3D
object (i.e., distinguishes between a real car from an advertisement
containing an image of a car), and can generalize to different types of
objects and between cities.

The remainder of this paper is structured as follows: In Section 2, we review
related work. In Section 3, we present EyeDAS, explain its architecture,
design considerations, and each expert in the committee of models. In Section
4, we evaluate EyeDAS’s performance under the constraints of autonomous
driving, based on various YouTube video recordings taken by a dash cam from
several places around the world. In Section 5 we discuss the limitations of
EyeDAS, and in Section 6, we present a summary.

##  2 Related Work

The ability to detect whether an object is a 2D or 3D object is extremely
important in autonomous driving, since a detection error can have life-
threatening consequences, endangering the safety of the driver, passengers,
pedestrians, and others on the road. Without this capability, Tesla’s
autopilot was unintentionally triggered, causing the car to: (1) continuously
slam on the brakes in response to a print advertisement containing a picture
of a person that appeared on a bus ad-bus-incident , and (2) stop in response
to a billboard advertisement that contained a stop sign billboard-incident .
Moreover, attackers can exploit the absence of this capability and
intentionally trigger: (1) Tesla’s autopilot to suddenly stop the car in the
middle of a road in response to a stop sign embedded in an advertisement on a
digital billboard nassi2020phantom , and (2) Mobileye 630 to issue false
notifications regarding a projected road sign nassi2021spoofing .

The need to detect whether an object is 2 or 3D is also important for
authentication systems (e.g., face recognition systems) where the identity of
a user can be spoofed using a printed picture of the user. Various methods
have been suggested for liveness detection, however the two primary
disadvantages of the proposed methods are that they: (1) fail to generalize to
other objects (e.g., distinguish between a real car and a picture of car),
since they mainly rely on dedicated features associated with humans
pan2008liveness  (e.g., eye movements 7047885 , facial vein map wang2017face
), which makes them object dependent; or (2) have high false negative rates
for pictures of objects that were not taken from a specific orientation,
angle, or position (e.g., they fail to detect liveness if the picture of the
person was taken from the back). As a result, these methods are not suitable
for autonomous driving.

##  3 EyeDAS

The requirements for a method used to secure the perception of autonomous cars
against stereoblindness syndrome are as follows. The method must be capable
of: (1) operating under the constraints of autonomous driving, and (2)
securing an object detector that obtains data from a single video camera,
because a few commercial ADASs, including Mobileye 630 PRO, rely on a single
video camera without any additional sensors, and (3) utilizing just a small
amount of training data; the fact that there may be just a small amount of 2D
objects in each geographical area necessitates a method with high performance
and minimum training so that it can be generalized to different types of
objects and between geographical locations.

###  3.1 Architecture

Fig. 2 provides an overview of EyeDAS; whenever an object is detected by the
vehicle’s image recognition model, it is tracked during $t$ consecutive frames
sampled at a frequency of $f$ frames per second (FPS), cropped from each frame
and serially passed to EyeDAS. EyeDAS then predicts whether the object is a 2D
(e.g., an image of a person) or 3D object (e.g., a real person).

Let $x=(x^{1},...,x^{t-1},x^{t})$ be a time series of $t$ identical RGB
objects cropped from $t$ consecutive frames where each object is centered. To
predict whether an object is 2D or 3D, we could build a supervised machine
learning model which receives $x$, which consists of images of an object to
classify, and predicts whether the object detected is 2D or 3D. However, such
an approach would make the machine learning model reliant on specific features
and thus would not generalize to objects extracted when the vehicle is
traveling in different locations or at different speeds, or when the vehicle
is approaching the object from different angles or distances.

To avoid this bias, we utilize the committee of experts approach used in
machine learning applications hu2002handbook , in which there is an ensemble
of models, each of which has a different perspective of interpreting the
incoming data. By combining different perspectives, we (1) create a more
resilient classifier that performs well even in cases where one aspect fails
to capture the evidence, and (2) reduce the false alarm rate by focusing the
classifier on just the relevant input features; EyeDAS consists of an ensemble
of unsupervised models (experts), each of which outputs a 3D confidence score,
and a supervised model (meta-classifier), which produces the final outcome
(decision) given the set of confidence scores.

Although each of the proposed unsupervised models (experts) focuses on a
different perspective, they have a common property: given $x$, which consists
of images of an object to classify as 2D or 3D, each model measures a
_difference_ between each two consecutive elements in the series; in this
study, we show that the combination of a proper feature extraction method
together with the proper _distance_ metric (applied between each two
consecutive elements) is a good basis for building such a classifier. In
addition, decisions based on a _distance_ observed between two consecutive
elements in a given series allows EyeDAS to generalize; this approach
minimizes dependency on object types or geographical locations. In addition,
EyeDAS finds the optimal balances between time to decision and classification
accuracy; we compare EyeDAS utilizing $t>1$ object frames to a state-of-the-
art image classifier that is designed to process a single frame at a time.

Figure 2: EyeDAS’s architecture. When an object is detected, (i) a time series
of the cropped object images is transferred to EyeDAS, (ii) four types of
unique features are processed by four unsupervised models (i.e., experts),
resulting in four 3D confidence scores, and (iii) the meta-classifier model
interprets the confidence scores and makes the final decision regarding the
object (2D or 3D).

From the perspective of a software module like EyeDAS, a 3D object itself is
not expected to _change_ significantly within a short period of time, even in
cases in which the 3D object is a human, animal, or vehicle (i.e., a real
object). Therefore, it is not trivial to find distance metrics that can detect
statistically significant differences that: (1) allow accurate differentiation
between 2D and 3D objects by considering just the object, and (2) can be
computed quickly. Therefore, we suggest a time-efficient approach that
considers objects entangled with their close surrounding background.

Each of the proposed unsupervised models utilizes a unique feature extraction
method and a corresponding _distance_ metric; these models are designed to
process image time series of any size $t$ ($t>1$). This property is crucial,
since the exact time point at which there is a significant difference between
two images (in cases in which the object detected is 3D) is unpredictable. In
addition, the images to analyze should be represented in such a way that
ensures that a statistically significant difference can be efficiently
obtained for 3D objects, while the error rate for 2D objects is minimized. In
other words, considering the objects entangled with their close surrounding
background, we are interested in feature extraction methods whose outcomes for
2D objects and 3D objects are statistically distinguishable within a short
period of time.

###  3.2 Proposed Models

Our committee consists of four unsupervised models, each focusing on a
different perspective (see Fig. 2 for a description of the architecture and
the models’ arguments); each model receives the time series of consecutive
images of an object to classify, extracts features from each image, measures a
type of _difference_ between the features extracted from two consecutive
images, and finally outputs a 3D confidence score. Additional details on the
four models are provided below:

Blurring Model (B) - This model utilizes the automatic focus (auto-focus)
capability of video cameras commonly used to build 3D cameras
zheng2009parallax . Unlike 2D object images, for 3D object images, the
blurring effect differs and is applied alternately on the object and its
surrounding background during the auto-focus process (illustrated in Fig. 4);
from a software module perspective, this is reflected in a large amount of
contrast, which is observed when examining the blurring maps extracted from
the raw images. Thus, the blurring model extracts a blurring map from each raw
image received, using the method described in golestaneh2017spatially .
Finally, using the structural image similarity measure proposed in
wang2004image , the blurring model outputs the maximum value obtained from
calculating the differences between each two consecutive blurring maps.

Sharpness Model (S) - This model utilizes the possible image sharpness-level
instability observed in 3D object images due the objects’ movements; the
sharpness model extracts the sharpness level (a numeric value) from each raw
image received, using the method described in bansal2016blur .

Finally, the sharpness model outputs the maximum value obtained from
calculating the differences between each two consecutive sharpness levels.

Color Model (C) - This model utilizes the possible movement expected in the
surrounding environment of 3D objects; this movement is reflected in a large
difference observed in the color distributions of the raw images. Thus, the
color model first applies the k-means clustering algorithm likas2003global  to
the pixel values of each raw image and extracts the maximum cluster size
corresponding to each raw image. Finally, the color model outputs the maximum
value obtained from calculating the differences between each two consecutive
cluster sizes.

We found that K=2 is a sufficient clustering parameter, providing a balance
between the runtime and contribution to the objects’ classification accuracy;
this is due to the fact that when there is movement in the surrounding
environment (in the cases of 3D objects), the pixel values are expected to
change significantly, possibly reflecting the size of the largest cluster
obtained from a two-color clustering algorithm.

Edge Model (E) - This model is based on the Sobel edge detector
kanopoulos1988design , a gradient-based method for estimating the first order
derivatives of the image separately for the horizontal and vertical axes;
these derivatives are not expected to change significantly for static objects
like 2D objects. The edge model operates similarly to the blurring model,
except for one thing: given the raw images, the model extracts the edging maps
instead of the blurring maps for comparison (i.e., using the image similarity
measure wang2004image ); extraction is performed using the method described in
gao2010improved .

Meta-Classifier -  To make a prediction as to whether or not an object is a 2D
or 3D object, we combine the knowledge of the unsupervised models described
above in a final prediction; we utilize a gradient boosting (GB) based binary
classifier ke2017lightgbm  trained on the outputs of the models.

We choose the GB algorithm, since it can capture nonlinear function
dependencies; in our case, the above unsupervised models complement each
other, and their output scores form a nonlinear relationship.

Figure 3: Examples of EyeDAS models’ views on 2D and 3D objects and the
corresponding _distances_ calculated as 3D confidence scores.

##  4 Evaluation

The experiments described in this section were designed in recognition of the
needs of autonomous driving in the real world; to ensure safety, a solution
needs to both accurately distinguish between 2D and 3D objects and make a fast
decision.

The experiments are aimed at evaluating: (1) the performance of each expert in
the committee, (2) the performance of the entire committee, (3) improvement in
the false positive rate of ODs when EyeDAS is applied for validation, (4) the
practicality of EyeDAS in real-time environments (in terms of computational
resources and speed), and (5) the ability to generalize between objects (i.e.,
humans, animals and vehicles) and geographical locations (i.e., cities).

All the experiments described in this section, including the speed and memory
benchmark (described in Section 4.4), were conducted on an Intel 2.9 GHz Intel
Core i7-10700 and 32GB RAM. The machine’s operating system was Windows 10.

###  4.1 Experiment Setup

Dataset. Since some ADASs (e.g., Mobileye 630 PRO) rely on a single channel
camera, all of the data collected for our experiments was obtained from
single-channel cameras; we utilized seven YouTube video recordings 111 New
York City (USA, NY) San Francisco (USA, CA) Dubai (UAE) Miami (USA, FL) London
(UK) Los Angeles (USA, CA) George Town (Singapore) of street views taken by a
dash cam from the driver’s seat perspective. The distribution of the dataset
represents the real distribution of 2D and 3D objects encountered by
autonomous driving in the real world: 2,000 RGB objects (i.e., humans,
animals, and vehicles) were extracted from driving view videos taken in seven
cities: New York (NY, USA), San Francisco (CA, USA), Dubai (United Arab
Emirates), Miami (FL, USA), London (UK), Los Angeles (CA, USA), and George
Town (Singapore), of which approximately 95% are 3D objects and 5% are 2D
objects extracted from billboards.

The objects were extracted and cropped using the highest performing OD
described by Redmon et al. redmon2018yolov3 ; in our experiments, each input
instance $x$ associated with an object to classify contains up to five images
taken at 200 millisecond intervals, starting at the time point at which an
object was detected by the OD. Input instance $x_{i}$ is labeled as ’True’ if
the instance represents a 3D object and ’False’ if the instance represents a
2D object.

Figure 4: Illustration of the camera’s auto-focus stabilization process as
reflected on the raw images, the corresponding blurring maps and the
_distances_ calculated using the image similarity measure proposed by Wang et
al. wang2004image .

Training. We denote $TR_{3D}$ and $TR_{2D}$ as two training sets representing
3D and 2D objects respectively. To avoid an unbalanced training set, we extend
$TR_{2D}$ by utilizing known image data augmentation techniques
shorten2019survey  to randomly selected instances from $TR_{2D}$; we apply the
rotation technique. The rotation degrees are randomized uniformly between 90∘
and 180∘.

Given the outputs calculated by the unsupervised models for each input
instance (i.e., the 3D confidence scores), the final meta-classifier training
was performed; the best hyperparameters for training were selected using the
grid search algorithm bergstra2013making  using the random shuffling split
method and 10-fold cross-validation. We vary the number of estimators in the
set of $\\{25,30,...,40\\}$ trees, while we change the maximum depth of the
trees in the set of $\\{2,3,...,5\\}$. To select the best set of
hyperparameters, we evaluated the meta-classifier’s performance in terms of
accuracy.

Except for the experiment whose results are demonstrated in Fig. 7, in all of
the experiments described in this section $TR_{3D}=150$ and $TR_{2D}=70$
(i.e., 220 instances were used for training, and the remaining 1,780 instances
were used for testing). Except for the experiment whose results are
demonstrated in Fig. 6, in all of the experiments presented in this section, a
decision was made within 200 milliseconds from the moment the object was
detected by the OD (i.e., $t=2$),

###  4.2 Results

Table 1:  2D misclassification rates using state-of-the-art object detectors.
Figure 5: The values of the ROC curve and AUC metrics for different model
combinations (B: blurring model, S: sharpness model, C: color model, E: edge
model).

Performance. In Fig. 5, we present the receiver operating characteristic (ROC)
plot and the area under the ROC (AUC) for different combinations of the
blurring (B), sharpness (S), color (C), and edge (E) models; the combination
of all four is our proposed method. The ROC plot shows the true positive rate
(TPR) and false positive rate (FPR) for every possible prediction threshold,
and the AUC provides an overall performance measure of a classifier (AUC=0.5:
random guessing, AUC=1: perfect performance).

In our case, there are two critical trade-offs that must be considered: (1)
the time elapsed from the moment an object is detected until a decision is
made, and (2) the classification threshold. A lower threshold will decrease
the FPR but often decrease the TPR as well. In our case, it is critical that
EyeDAS predicts 3D objects with high accuracy (i.e., every time), because the
vast majority of objects passed to EyeDAS will be 3D objects (real objects);
even a very high TPR (TPR<1) would make the solution impractical. Therefore,
in Fig. 6, we present EyeDAS’s performance as a function of the time elapsed
in milliseconds (ms) from the object’s detection until a decision is made,
considering a threshold value at which the TPR=1. As can be seen, the FPR
decreases as more frames are available to use to classify an object.

In Fig. 7 we present EyeDAS’s FPR for TPR=1 when 44, 88, 132, 176 and 220
instances were randomly selected for training. In each case, the distribution
of 3D and 2D was approximately 66.7% and 33.3% respectively, and the data
augmentation technique described above was applied to avoid an unbalanced
training set. As can be seen, the best performance was achieved when 220
instances were used for training.

In Table 2, we provide the TPR and FPR of the models when the threshold is set
at 0.5 and for the threshold value at which the TPR=1. As can be seen, the use
of all of the proposed models (B+S+C+E) in combination outperforms all other
model combinations.

In Table 3, we compare EyeDAS’s performance to that of two other approaches:
(1) baseline models based on the state-of-the-art pre-trained image
classifiers (i.e., VGG16, VGG19 and Resnet50 mascarenhas2021comparison ); we
utilize the known transfer learning technique by re-training these models, and
(2) an optimized model similar to EyeDAS, except that it is based on a single
expert model which considers the raw images as is (i.e., it computes the image
similarity based distance wang2004image  between the raw images directly,
without extracting any features as EyeDAS does). In both cases, 220 instances
were randomly selected for training; the distribution of 3D and 2D images is
approximately 66.7% and 33.3% respectively, and the data augmentation
technique described above was applied to avoid an unbalanced training set.

Figure 6: Performance as a function of the time elapsed (in milliseconds) from
the moment the object was detected. Figure 7: Performance when 44, 88, 132,
176, and 220 instances were randomly selected for training. Table 2:  EyeDAS’s
TPR and FPR with different thresholds. Table 3:  Comparison to other
approaches.

Each baseline model was re-trained by (1) freezing all the layers except for
the output layer which was replaced by two trainable fully connected layers,
(2) randomly picking 50 instances from the training set to serve as the
validation set, (3) pre-processing the input data (i.e., image resizing and
scaling), and finally (4) minimizing the _categorical_crossentropy_ loss
function on the validation set using the Adam optimizer. The first new layer
contained 128 neurons and attached with the relu activation function, and the
second layer (output layer) contained two neurons and attached with the
softmax activation function. Increasing the first layer’s neurons count beyond
128 resulted in poorer results due to overfitting. As can be seen, EyeDAS
outperforms all of the abovementioned models.

Securing ODs with EyeDAS. To determine how effective EyeDAS is as part of a
system, we evaluated 2D misclassification rates on seven state-of-the-art ODs
redmon2018yolov3 ; Ren_2017 ; howard2017mobilenets ; Liu_2016 . The results
are presented in Table 1; we present the 2D misclassification rates obtained
for each detector before and after applying EyeDAS as a countermeasure and the
impact of the different thresholds. The results show that for most ODs, when
the detector mistakenly classified a 2D object as real (i.e., 3D), EyeDAS
provided effective mitigation, even for the threshold value at which the
TPR=1.

Ablation study. In order for the committee of experts approach to be
effective, there must be some disagreements between the models; although there
is a clear indication that each model (i.e., blurring, sharpness, color, and
edge) contributes a unique perspective on the final prediction, we perform an
ablation study to provide further evidence that each model has a unique
contribution. For that, we utilize the SHAP (SHapley Additive exPlanations)
framework to explain the model’s predictions; each input feature is assigned a
score (i.e., a Shapley value) which represents its contribution to the model’s
outcome. In our case, we are interested in the contributions of each expert
(i.e., each input feature to our meta-classifier) to the final prediction. In
Table 4, we present the measure of disagreement between each possible sub-
committee of experts: given an input instance $x$, we say that a sub-committee
disagrees on $x$ if there is at least one expert whose Shapley value is
negative and at least one expert whose Shapley value is positive.
Interestingly, the full combination of experts (B+S+C+E) has the highest
number of disagreements and results in the lowest FPR at TPR=1.

Table 4:  Disagreements between the models.

###  4.3 Generalization

We also evaluate how EyeDAS’s approach generalizes to different geographical
locations and even different types of objects.

Table 5:  Evaluation results for different combinations of locations (NY: New
York, GT: George Town, MI: Miami, SF: San Francisco, LA: Los Angeles, DU:
Dubai, LO: London).

Generalization to other geographical locations. To evaluate EyeDAS’s
geographical location generalization ability, we trained and tested the models
on complementary groups of location types (i.e., cities). For training, we
took minimum-sized city type combinations in which there are at least 56 2D
objects and at least 120 3D objects, since we observed that less than that
amount is insufficient for training the models and thus no meaningful
conclusions could be derived. In Table 5, we present the evaluation results
obtained for different geographical location (i.e., cities) combinations. As
can be seen, EyeDAS is not dependent on the geographical location in which the
training data was collected, as it is capable of using the knowledge gained
during training to distinguish between 2 and 3D objects.

Table 6:  Evaluation results for different combinations of object types (HU:
Humans, VE: Vehicles, AN: Animals).  Figure 8: The ROC curve and AUC measure
as a function of the different object type combinations used for training.

Generalization to other types of objects. To evaluate EyeDAS’s object type
generalization ability, we trained and tested the models on complementary
groups of object types. We focused on humans (HU), animals (AN), and vehicles
(VE). As previously done, for training we used minimum-sized object type
combinations in which there are at least 56 2D objects and at least 120 3D
objects. In Table 6, we present the evaluation results obtained for each
object type combination, and in Fig. 8, we present the ROC and the AUC. As can
be seen, EyeDAS is independent of the type of the object appeared in the
training set, as it is capable of using the knowledge gained during training
to distinguish between 2 and 3D objects.

###  4.4 Speed and Memory Performance

We performed a speed benchmark experiment to assess EyeDAS ’s performance. We
found that EyeDAS took an average of 19 ms to process a single object frame,
with a standard deviation of 1 ms. This translates to an approximately 220 ms
time to decision from the time the object is detected. We also note that the
meta-classifier is relatively small (40 kilo bytes) and only utilizes 0.01% of
the CPU per object frame.

##  5 Limitations

Despite the high performance achieved, EyeDAS has some limitations. First,
technical factors may influence the performance of EyeDAS. For example, EyeDAS
may perform poorly when image resolution is low or images are taken in low
lighting. However, adequate lighting conditions are expected on roads on which
autonomous vehicles can typically drive. Second, for 3D objects, if both the
detected object and its close surrounding background are stationary (e.g., the
object does not move or its surrounding background does not change), then
EyeDAS may perform poorly. However, if the vehicle is moving or the camera’s
auto-focus process is operating, then EyeDAS ’s errors will likely to decrease
significantly even for 3D stationary objects. If the vehicle is not moving,
the concern for the safety of passengers does not exist.

##  6 Summary

In this paper, we proposed a novel countermeasure which can be used to secure
object detectors (ODs) against the stereoblindness syndrome; this syndrome can
have life-threatening consequences, endangering the safety of an autonomous
car’s driver, passengers, pedestrians, and others on the road.

Designed in recognition of the needs of autonomous driving in the real world,
the proposed method is based on few-shot learning, making it very practical in
terms of collecting the required training set.

As presented in the previous sections, EyeDAS demonstrates excellent
performance and specifically its: (1) ability to maintain a zero
misclassification rate for 3D objects, (2) ability to improve the performance
of ODs, (3) practicality in real-time environments (e.g., computational
resources and speed), and (4) ability to generalize between objects (i.e.,
humans, animals, and vehicles) and geographical locations (i.e., cities).

The alternative approaches evaluated obtained poorer results, since they are
reliant on specific features and thus could not generalize to objects
extracted when the vehicle is traveling in different locations or at different
speeds, or when the vehicle is approaching the object from different angles or
distances.

## References

  * (1) “Tesla updates autopilot safety numbers; almost 9x safer than average driving.” [Online]. Available: https://electrek.co/2019/10/23/tesla-autopilot-safety-9x-safer-than-average-driving/
  * (2) B. Nassi, Y. Mirsky, D. Nassi, R. Ben-Netanel, O. Drokin, and Y. Elovici, “Phantom of the adas: Securing advanced driver-assistance systems from split-second phantom attacks,” in _Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security_ , 2020, pp. 293–308.
  * (3) C. Yan, W. Xu, and J. Liu, “Can you trust autonomous vehicles: Contactless attacks against sensors of self-driving vehicle,” _DEF CON_ , vol. 24, 2016\.
  * (4) “Tesla autopilot amusingly confuses ben fordhams bus poster ad as real pedestrian.” [Online]. Available: https://www.bandt.com.au/tesla-autopilot-amusingly-confuses-ben-fordhams-bus-poster-ad-as-real-pedestrian/
  * (5) “This billboard that confuses tesla autopilot is a good reminder of why self-driving is still a long way off.” [Online]. Available: https://jalopnik.com/this-billboard-that-confuses-tesla-autopilot-is-a-good-1846698527
  * (6) B. Nassi, D. Nassi, R. Ben-Netanel, and Y. Elovici, “Spoofing mobileye 630’s video camera using a projector,” in _Workshop on Automotive and Autonomous Vehicle Security (AutoSec)_ , vol. 2021, 2021, p. 25.
  * (7) G. Pan, Z. Wu, and L. Sun, “Liveness detection for face recognition,” _Recent advances in face recognition_ , pp. 109–124, 2008.
  * (8) O. V. Komogortsev, A. Karpov, and C. D. Holland, “Attack of mechanical replicas: Liveness detection with eye movements,” _IEEE Transactions on Information Forensics and Security_ , vol. 10, no. 4, pp. 716–725, 2015.
  * (9) S.-Y. Wang, S.-H. Yang, Y.-P. Chen, and J.-W. Huang, “Face liveness detection based on skin blood flow analysis,” _symmetry_ , vol. 9, no. 12, p. 305, 2017\.
  * (10) Y. H. Hu and J.-N. Hwang, “Handbook of neural network signal processing,” 2002\.
  * (11) K. C. Zheng, A. Colburn, A. Agarwala, M. Agrawala, D. Salesin, B. Curless, and M. F. Cohen, “Parallax photography: creating 3d cinematic effects from stills,” in _Proceedings of Graphics Interface 2009_ , 2009, pp. 111–118.
  * (12) S. A. Golestaneh and L. J. Karam, “Spatially-varying blur detection based on multiscale fused and sorted transform coefficients of gradient magnitudes.” in _CVPR_ , 2017, pp. 596–605.
  * (13) Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE transactions on image processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * (14) R. Bansal, G. Raj, and T. Choudhury, “Blur image detection using laplacian operator and open-cv,” in _2016 International Conference System Modeling & Advancement in Research Trends (SMART)_. IEEE, 2016, pp. 63–67.
  * (15) A. Likas, N. Vlassis, and J. J. Verbeek, “The global k-means clustering algorithm,” _Pattern recognition_ , vol. 36, no. 2, pp. 451–461, 2003.
  * (16) N. Kanopoulos, N. Vasanthavada, and R. L. Baker, “Design of an image edge detection filter using the sobel operator,” _IEEE Journal of solid-state circuits_ , vol. 23, no. 2, pp. 358–367, 1988.
  * (17) W. Gao, X. Zhang, L. Yang, and H. Liu, “An improved sobel edge detection,” in _2010 3rd International conference on computer science and information technology_ , vol. 5. IEEE, 2010, pp. 67–71.
  * (18) G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * (19) J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” 2018.
  * (20) C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” _Journal of big data_ , vol. 6, no. 1, pp. 1–48, 2019.
  * (21) J. Bergstra, D. Yamins, and D. Cox, “Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures,” in _International conference on machine learning_. PMLR, 2013, pp. 115–123.
  * (22) S. Mascarenhas and M. Agarwal, “A comparison between vgg16, vgg19 and resnet50 architecture frameworks for image classification,” in _2021 International Conference on Disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON)_ , vol. 1. IEEE, 2021, pp. 96–99.
  * (23) S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , Jun 2017.
  * (24) A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” _arXiv preprint arXiv:1704.04861_ , 2017.
  * (25) W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” _ECCV_ , 2016.