# There does not exist a strongly regular graph with parameters
$(1911,270,105,27)$

Jack H. Koolen  School of Mathematical Sciences, University of Science and
Technology of China, 96 Jinzhai Road, Hefei, 230026, Anhui, PR China. CAS Wu
Wen-Tsun Key Laboratory of Mathematics, University of Science and Technology
of China, 96 Jinzhai Road, Hefei, Anhui, 230026, PR China  Brhane
Gebremichel111Corresponding author. School of Mathematical Sciences,
University of Science and Technology of China, 96 Jinzhai Road, Hefei, 230026,
Anhui, PR China.

###### Abstract

In this paper we show that there does not exist a strongly regular graph with
parameters $(1911,270,105,27)$.

††E-mail addresses<EMAIL_ADDRESS>(J.H. Koolen),
<EMAIL_ADDRESS>(B. Gebremichel).

Keywords : strongly regular graph, join of graphs.

Mathematics Subject Classification: 05C50, 05E30.

##  1 Introduction

In this paper we show the following result:

######  Theorem 1.1.

There does not exist a strongly regular graph with parameters
$(1911,270,105,27)$.

This was the largest open case of a set of feasible parameters of a strongly
regular graph with smallest eigenvalue $-3$. We conjecture:

######  Conjecture 1.2.

Let $G$ be a primitive strongly regular graph with parameters
$(n,k,\lambda,\mu)$ and smallest eigenvalue $-3$. Then either
$\mu\in\\{6,9\\}$ or $n\leqslant 276$.

On this moment, there are twelve cases of parameter sets of putative primitive
strongly regular graphs with smallest eigenvalue $-3$, $n>276$ and
$\mu\not\in\\{6,9\\}$ which are open. They are in Table 1 below (cf. [8]).

To show the main result we find large cliques that intersect in many vertices.
We are sure that the method we use in this paper can be generalized to the
larger class of amply regular graphs (for a definition, see [6]).

$(v,k,\lambda,\mu)$ | $\theta_{0},[\theta_{1}]^{m(\theta_{1})},[\theta_{2}]^{m(\theta_{2})}$ | $(v,k,\lambda,\mu)$ | $\theta_{0},[\theta_{1}]^{m(\theta_{1})},[\theta_{2}]^{m(\theta_{2})}$
---|---|---|---
$(288,105,52,30)$ | $105,[25]^{27},[-3]^{260}$ | $(476,133,60,28)$ | $133,[35]^{34},[-3]^{441}$
$(300,117,60,36)$ | $117,[27]^{26},[-3]^{273}$ | $(540,147,66,30)$ | $147,[39]^{35},[-3]^{504}$
$(351,140,73,44)$ | $140,[32]^{26},[-3]^{324}$ | $(550,162,75,36)$ | $162,[42]^{33},[-3]^{516}$
$(375,102,45,21)$ | $102,[27]^{34},[-3]^{340}$ | $(575,112,45,16)$ | $112,[32]^{46},[-3]^{528}$
$(405,132,63,33)$ | $132,[33]^{30},[-3]^{374}$ | $(703,182,81,35)$ | $182,[49]^{37},[-3]^{665}$
$(441,88,35,13)$ | $88,[25]^{44},[-3]^{396}$ | $(1344,221,88,26)$ | $221,[65]^{56},[-3]^{1287}$
Table 1: List of putative primitive strongly regular graphs with smallest
eigenvalue $-3$ for $n>276$, unknown whether they exist or not.

This paper is organized as follows: In the next section we give the
preliminaries. In Section 3 we give restrictions on graphs with large cliques
and smallest eigenvalue at least $-3$. In Section 4 we find large cliques in a
putative strongly regular graph with parameters $(1911,270,105,27)$ and apply
the restrictions given in Section 3 to show the main result.

##  2 Preliminaries

###  2.1 Graphs

In this paper all the graphs are finite, undirected and simple. For
definitions, we do not define, see [2]. The eigenvalues of a graph are the
eigenvalues of its adjacency matrix $A(G)$ indexed by $V(G)$, such that
$A_{xy}=1$ if $xy\in E(G)$ and $0$ otherwise. The smallest eigenvalue of a
graph is denoted by $\lambda_{\min}(G)$.

Let $G=(V(G),E(G))$ be a graph. The valency $k_{x}$ of a vertex $x$ of $G$ is
the number of neighbours of $x$, i.e. the vertices $y\in V(G)$ such that
$xy\in E(G)$. A graph $G$ is $k$-regular if $k_{x}=k$ for all vertices $x\in
V(G)$. A graph $G$ is strongly regular with parameters $(n,k,\lambda,\mu)$ if
$G$ has $n$ vertices, is $k$-regular and any two distinct vertices have
exactly $\lambda$ (resp. $\mu$) common neighbours if they are adjacent (resp.
non-adjacent). The strongly regular graph $G$ is called primitive if $G$ and
its complement are both connected.

###  2.2 Interlacing

If $M$ (resp. $N$) is a real symmetric $m\times m$ (resp. $n\times n$) matrix
with
$\theta_{1}(M)\geqslant\theta_{2}(M)\geqslant\cdots\geqslant\theta_{m}(M)$
(resp.
$\theta_{1}(N)\geqslant\theta_{2}(N)\geqslant\cdots\geqslant\theta_{n}(N)$)
the eigenvalues of $M$ (resp. $N$) in nonincreasing order. Assume $m\leqslant
n$. Then we say that the eigenvalues of $M$ _interlace_ the eigenvalues of
$N$, if $\theta_{n-m+i}(N)\leqslant\theta_{i}(M)\leqslant\theta_{i}(N)$ for
$i=1,\ldots,m$.

The following result is a special case of interlacing.

######  Lemma 2.1 (cf. [5, Theorem 9.1.1]).

Let $B$ be a real symmetric $n\times n$ matrix and $C$ be a principal
submatrix of $B$ of order $m$, where $m<n$. Then the eigenvalues of $C$
interlace the eigenvalues of $B$.

As an easy consequence of Lemma 2.1, we have the following proposition.

######  Proposition 2.2.

Let $G$ be a graph and $H$ a proper induced subgraph of $G$. Denote by
$\theta_{\min}(G)$ (resp. $\theta_{\min}(H)$) the smallest eigenvalue of $G$
(resp. $H$). Then $\theta_{\min}(G)\leqslant\theta_{\min}(H)$.

Let $G=(V,E)$ be a graph and $\pi:=\\{V_{1},\ldots,V_{r}\\}$ be a partition of
$V$. We say $\pi$ is an _equitable partition_ with respect to $G$ if the
number of neighbours in $V_{j}$ of a vertex $u$ in $V_{i}$ is a constant
$q_{ij}$, independent of $u$, only dependent on $i$ and $j$. For an equitable
partition $\pi$ with respect to $G$, the quotient matrix $Q$ of $G$ with
respect to $\pi$ is defined as $Q=(q_{ij})_{1\leqslant i,j\leqslant r}$.

######  Lemma 2.3 (cf.[5, Theorem 9.3.3]).

Let $G$ be a graph. If $\pi$ is an equitable partition of $G$ and $Q$ is the
quotient matrix with respect to $\pi$ of $G$, then every eigenvalue of $Q$ is
an eigenvalue of $G$.

###  2.3 Terwilliger graphs

A _Terwilliger graph_ is a non-complete graph such that, for any two vertices
$x$ and $y$ at distance $2$, the subgraph induced by common neighbours of $x$
and $y$ forms a clique of order $c$ (for some fixed $c\geqslant 0$).

######  Lemma 2.4 (cf. [1, Corollary 1.16.6 (ii)]).

There is no strongly regular Terwilliger graph with parameters
$(n,k,\lambda,\mu)$ for $k<50(\mu-1)$.

######  Lemma 2.5.

If a strongly regular graph with parameters $(1911,270,105,27)$ exist, then it
contains an induced quadrangle.

######  Proof.

Suppose there exist a strongly regular graph $G$ with parameters
$(1911,270,105,27)$ which does not contain an induced quadrangles. Then $G$ is
a Terwilliger graph. As $\mu=27$ then, by Lemma 2.4, the valency of $G$ is at
least $1300$, which is a contradiction as $k=270$. This shows the lemma. ∎

###  2.4 Join of graphs

Let $G_{1}$ and $G_{2}$ be two graphs such that $V(G_{1})\cap
V(G_{2})=\emptyset$. The _join_ of $G_{1}$ and $G_{2}$, denoted by
$G_{1}\nabla G_{2}$, has as vertex set $V(G_{1})\cup V(G_{2})$ and edge set
$E(G_{1})\cup E(G_{2})\cup\\{\\{x_{1},x_{2}\\}\mid x_{1}\in V(G_{1}),x_{2}\in
V(G_{2})\\}$. The following lemma is a consequence of Section $2.3.1$ of [2].

######  Lemma 2.6.

Let $G_{i}$ be a $k_{i}$-regular graph with $n_{i}$ vertices, for $i=1,2$,
such that $V(G_{1})\cap V(G_{2})=\emptyset$. Then the smallest eigenvalue
$\lambda_{\min}(G_{1}\nabla G_{2})$ of the join $G_{1}\nabla G_{2}$ satisfies

$\lambda_{\min}=\min\\{\lambda_{\min}(G_{1}),\lambda_{\min}(G_{2}),\lambda_{\min}(Q)\\}$

where

$\displaystyle Q=\begin{pmatrix}k_{1}&n_{2}\\\ n_{1}&k_{2}\end{pmatrix}.$

The following lemma was inspired by Cao, Koolen, Munemasa, Yoshino [3].

######  Lemma 2.7.

Let $G$ be a $k$-regular graph on $n$ vertices with smallest eigenvalue
$\lambda_{\min}(G)\leqslant-1$. Consider $K_{t}\nabla G$ for some positive
integer $t$. Then $\lambda_{\min}(K_{t}\nabla G)=\lambda_{\min}(G)$ if and
only if $(\lambda_{\min}(G)-k)(\lambda_{\min}(G)+1-t)\geqslant nt$.

######  Proof.

As $\lambda_{\min}(K_{t})\geqslant-1$, by Lemma 2.6 we find
$\lambda_{\min}(G)=\lambda_{\min}(K_{t}\nabla G)$ if and only if

$\displaystyle\lambda_{\min}\begin{pmatrix}t-1&n\\\
t&k\end{pmatrix}\geqslant\lambda_{\min}(G)$

if and only if

$\displaystyle\det\begin{pmatrix}t-1-\lambda_{\min}(G)&n\\\
t&k-\lambda_{\min}(G)\end{pmatrix}\geqslant 0,\text{ as }t\geqslant 1.$

This show the lemma. ∎

##  3 Large cliques

Let $G$ be a strongly regular graph with parameters $(n,k,\lambda,\mu)$ and
smallest eigenvalue $-m$. Let $C$ be a clique of $G$ of order $c$. Then

$c\leqslant 1+\frac{k}{m}.$ (1)

The inequality (1) is called the _Delsarte bound_. Moreover, if
$c=1+\frac{k}{m}$, then $C$ is called a _Delsarte clique_.

Let $H(a,t)$ be the graph with $1+a+t$ vertices, consisting of a complete
graph $K_{a+t}$ and a vertex adjacent to exactly $a$ vertices of $K_{a+t}$.

In [6], Greaves, Koolen and Park obtained the following lemma.

######  Lemma 3.1.

Let $G$ be a graph with smallest eigenvalue $\lambda=\lambda_{\min}(G)$.
Assume that $G$ contains an induced $H(a,t)$. Then we have

$(a-\lambda(\lambda+1))(t-(\lambda+1)^{2})\leqslant\lambda(\lambda+1))^{2}.$
(2)

In this paper we need the following consequence of Lemma 3.1.

######  Lemma 3.2.

Let $G$ be a graph with smallest eigenvalue at least $-3$. Let $C$ be a
complete subgraph of $G$ with order $c$. Let $x$ be a vertex of $G$ not in $C$
with exactly $t$ neighbours in $C$. Then $t\leqslant t_{\min}$ or $t\geqslant
t_{\max}$ where $t_{\min}$ and $t_{\max}$ are as in the Table 2.

$c$ | $t_{\min}$ | $t_{\max}$ | $c$ | $t_{\min}$ | $t_{\max}$
---|---|---|---|---|---
$29$ | $8$ | $23$ | $31$ | $7$ | $26$
$30$ | $8$ | $24$ | $32$ | $7$ | $27$
Table 2: Values of $t_{\min}$ and $t_{\max}$

Using Lemma, 3.1 Greaves et al. [6] derived a method restricting the order of
maximal cliques in a strongly regular graph.

######  Lemma 3.3 (cf. [6, Lemma 3.7]).

Let $G$ be a strongly regular graph with parameters $(v,k,\lambda,\mu)$ having
smallest eigenvalue $-m$. Let $C$ be a maximal clique of $G$ of order $c$. If
$\mu>m(m-1)$ and $c>\frac{{\mu}^{2}}{\mu-m(m-1)}-m+1$, then

$((c+m-3)(k-c+1)-2(c-1)(\lambda-c+2))^{2}-(k-c+1)^{2}(c+m-1)(c-(m-1)(4m-1))\geqslant
0.$ (3)

We denote the polynomial on the left hand side of the inequality (3) by
$M_{G}(c)$.

######  Lemma 3.4.

If a strongly regular graph $G$ with parameters $(1911,270,105,27)$ exists,
then any clique in $G$ has order at most $32$.

######  Proof.

Let $G$ be a strongly regular graph with parameters $(1911,270,105,27)$. Then,
it has smallest eigenvalue $-3$. Let $C$ be a maximal clique in $G$ of order
$c$. If $c>\frac{27^{2}}{27-6}-3+1=32\frac{5}{7}$, then by Lemma 3.4, we have

$M_{G}(c)=672c^{3}-80784c^{2}+1468512c+3277200\geq 0$

as $\mu=27>6$. It is easily checked that $M_{G}(26)<0$ and $M_{G}(97)<0$. This
means that $c\geqslant 98$. This gives a contradiction, as the Delsarte bound
gives $c\leqslant 1+\frac{k}{m}=1+\frac{270}{3}=91$. So we obtain that any
clique has order at most $32$. ∎

######  Lemma 3.5.

Let $G$ be strongly regular graph with parameters $(1911,270,105,27)$. Assume
there are two cliques $C_{1}$ and $C_{2}$ such that $V(C_{1})\neq V(C_{2})$,
each of order at least $29$, intersecting in at least $22$ vertices. Then one
of the following holds:

  1. (1)

There is at least one vertex $z$ in the symmetric difference
$V(C_{1})\triangledown V(C_{2})$ which is adjacent to all vertices in
$V(C_{1})\triangledown V(C_{2})\backslash\\{z\\}$,

  2. (2)

$C_{1}$ and $C_{2}$ intersect in exactly $27$ vertices and both are maximal
with order $29$.

######  Proof.

Let $H$ be the induced subgraph on $V(C_{1})\triangledown V(C_{2})$. If $H$ is
complete, then we are in Case (1). So we may assume $H$ is not complete. This
means that $|V(C_{1})\cap V(C_{2})|=:t\in\\{22,23,\ldots,27\\}$ as $\mu=27$.

Assume $t=22$. Let $C^{\prime}_{1}$ (resp. $C^{\prime}_{2}$) be a sub clique
of $C_{1}$ (resp. $C_{2}$) such that $V(C^{\prime}_{1})\supseteq V(C_{1})\cap
V(C_{2})$, $V(C^{\prime}_{2})\supseteq V(C_{1})\cap V(C_{2})$ and
$|V(C^{\prime}_{1})|=|V(C^{\prime}_{2})|=29$. Let $K$ be the induced subgraph
on $V(C^{\prime}_{1})\cup V(C^{\prime}_{2})$. By Proposition 2.1 we see that
$K$ has smallest eigenvalue at least $-3$. Let $\pi=\\{V(C^{\prime}_{1})\cap
V(C^{\prime}_{2}),V(C^{\prime}_{1})\triangledown V(C^{\prime}_{2})\\}$ of
$V(C^{\prime}_{1})\cup V(C^{\prime}_{2})$ be a partition of $K$ with quotient
matrix

$\displaystyle Q=\begin{pmatrix}21&14\\\ 22&\alpha+6\end{pmatrix}.$

By Lemma 2.3, we see that the smallest eigenvalue of $Q$ is at least $-3$.
This implies that $24\alpha\geqslant 92$, as $\det(Q+3\mathbf{I})\geqslant 0$.
So, $\alpha\geqslant\frac{23}{6}$. This means that there are at least
$\lceil\frac{7\times 23}{6}\rceil=27$ edges between
$V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ and
$V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$. If Case (1) of the lemma does
not happen, then all vertices $V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$
have at most 5 neighbours in $V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$,
as $\mu=27$.

We need to consider two cases. There exist a vertex $x\in
V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ such that $x$ has $5$
neighbours in $V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$ or there is no
vertex $x\in V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ with $5$
neighbours in $V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$.

In the first case, let $y_{1},y_{2},\ldots,y_{5}$ be the $5$ neighbours of $x$
in $V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$. Then
$y_{1},y_{2},\ldots,y_{5}$ have each at most $5$ neighbours in
$V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ and hence there is an edge
$zu$ between $V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ and
$V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$, such that
$z\in(V(C^{\prime}_{1})\backslash V(C^{\prime}_{2}))\backslash\\{x\\}$ and
$u\in(V(C^{\prime}_{2})\backslash
V(C^{\prime}_{1}))\backslash\\{y_{1},y_{2},\ldots,y_{5}\\}$. Then $u$ and $x$
are at distance two and they have at least $28$ common neighbours, a
contradiction. Now assume that all vertices of $V(C^{\prime}_{1})\backslash
V(C^{\prime}_{2})$ (resp. $V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$)
have at most 4 neighbours in $V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$
(resp. $V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$). As there are at least
$27$ edges between $V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ and
$V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$, then there exist $x\in
V(C^{\prime}_{1})\backslash V(C^{\prime}_{2})$ and $y\in
V(C^{\prime}_{2})\backslash V(C^{\prime}_{1})$ such that $x$ and $y$ are at
distance two and they have at least $8+22=30$ common neighbours, a
contradiction. This shows that, if $t=22$, then we are in Case (1) of the
lemma.

In similar fashion, it can be shown that, if $t\in\\{23,23,\ldots,26\\}$, then
we are in Case (1) of the lemma.

Now assume $t=27$. If $|V(C_{1})|\geqslant 30$ and $|V(C_{2})|\geqslant 29$,
then the quotient matrix $Q$ of $\pi=\\{V(C_{1})\cap
V(C_{2}),V(C_{1})\backslash(V(C_{2}),V(C_{2})\backslash(V(C_{1})\\}$ satisfies

$\displaystyle\begin{pmatrix}26&t_{1}&t_{2}\\\ 27&t_{1}-1&0\\\
27&0&t_{2}-1\end{pmatrix},\text{ where }t_{1}+27=|V(C_{1})|\text{ and
}t_{2}+27=|V(C_{2})|,$

or we are in Case (1) of the lemma.

As the smallest eigenvalue of $Q$ is at least $-3$ we obtain that

$29(t_{1}+2)(t_{2}+2)-27(t_{1}(t_{2}+2)+t_{2}(t_{1}+2))\geqslant 0$

This means

$-25t_{1}t_{2}+4(t_{1}+t_{2})+116\geqslant 0,$

and hence

$25(t_{1}-\frac{4}{25})(t_{2}-\frac{4}{25})<117$

But, as $t_{1}\geqslant 3$ and $t_{2}\geqslant 2$ we have
$25(t_{1}-\frac{4}{25})(t_{2}-\frac{4}{25})>130$, a contradiction. This shows
the lemma. ∎

##  4 On the local graph of $G$

For a graph $G$ and $x\in V(G)$ let $\Delta(x)$ be the induced subgraph on
$\\{y\in V(G)\mid x\sim y\\}$. The graph $\Delta(x)$ is called the local graph
of $G$ with respect to $x$.

######  Lemma 4.1 (cf. [4, 7]).

Let $G$ be a primitive strongly regular graph with parameters
$(v,k,\lambda,\mu)$. Let $x$ be a vertex of $G$ and $\Delta(x)$ be the local
graph of $G$ with respect to $x$. Let $\bar{C}=\\{y_{1},y_{2},\ldots,y_{c}\\}$
be an independent set of $\Delta(x)$ of order $\bar{c}$. Then

$\binom{\bar{c}}{2}(\mu-1)\geqslant\bar{c}(\lambda+1)-k$ (4)

holds.

For distinct non-adjacent vertices $w_{1},w_{2}\in\Delta(x)$ define
$C(w_{1},w_{2}):=\\{z\sim x\mid z\sim w_{1},z\sim w_{2}\\}$ and
$c(w_{1},w_{2}):=\text{number of elements of }C(w_{1},w_{2})$.

######  Lemma 4.2.

Assume a strongly regular graph $G$ with parameters $(1911,270,105,27)$ exists
such that $G$ has an induced quadrangle, say $x\sim u\sim y\sim v\sim x$. Then
there is no independent set $S$ of order $5$ inside $\Delta(x)$ such that
$u,v\in S$.

######  Proof.

Assume that $\Delta(x)$ contains an independent set
$S=\\{u_{1},u_{2},\ldots,u_{5}\\}$ of order $5$. Then by Lemma 4.1 we have
$260=\binom{5}{2}26=5\times 106-270=260$. So we have equality in (4). This
means that $c(u_{i},u_{j})=26$ in $\Delta(x)$ for all $1\leqslant i<j\leqslant
5$. As $C(u,v)\leqslant 25$ in $\Delta(x)$, we obtain that $u$ and $v$ are not
both elements in an independent set $S$ of order $5$ in $\Delta(x)$. This
shows the lemma ∎

######  Lemma 4.3.

Assume a strongly regular graph $G$ with parameters $(1911,270,105,27)$ exists
such that $G$ has an induced quadrangle, say $x\sim u\sim y\sim v\sim x$. Let
$U=\\{u,v,w_{1},w_{2}\\}$ be an independent set of $\Delta(x)$. Let
$A_{i}=\\{a_{2i-1},a_{21}\\}$ for $i=1,2,\ldots,6$ such that
$A_{i}\in\binom{U}{2}$, $A_{i}\neq A_{j}$ if $1\leqslant i<j\leqslant 6$ and
$A_{1}=\\{u,v\\}$. Then $c(u,v)\in\\{24,25\\}$ and
$\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})\in\\{154,155\\}$. Then
one of the following hold:

  1. (1)

$c(u,v)\in\\{24,25\\}$ and
$\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})=154$. Then, without
loss of generality,

$(c(a_{1},a_{2}),c(a_{3},a_{4}),\ldots,c(a_{11},a_{12}))\in\\{(24,26,26,\ldots,26),(25,25,26,\ldots,26)\\}.$

Moreover, any vertex $w$ of $\Delta(x)$ has at most $2$ neighbours in $U$.

  2. (2)

$c(u,v)=25$ and $\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})=155$.
Then

$(c(a_{1},a_{2}),c(a_{3},a_{4}),\ldots,c(a_{11},a_{12}))=(25,26,26,\ldots,26).$

In this case there is a unique vertex $z$ of $\Delta(x)$ with exactly $3$
neighbours in $U$ and any other vertex $w$ of $\Delta(x)$ has at most $2$
neighbours in $U$.

######  Proof.

As $k=270$ and $\lambda=105$, we have $4(\lambda+1)-k=154$. This means that
the number of vertices $w\in\Delta(x)$ such that $w$ is adjacent to at least
two vertices in $U$ is $154$, by Lemma 4.1. This implies

$\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})\geqslant 154.$

As $c(u,v)\leqslant 25$ and $c(u_{1},u_{2})\leqslant 26$ for all
$\\{u_{1},u_{2}\\}\in\binom{U}{2}$ we find that

$154\leqslant\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})\leqslant
155.$

We also find that $c(u,v)\geq 154-5\times 26=24$.

If $c(u,v)=24$, then $c(a_{2i-1},a_{2i})=26$ for $i=2,3,\ldots,6$ and any
vertex $w$ in $\Delta(x)$ has at most two neighbours in $U$.

If $c(u,v)=25$ and
$\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})=154$, then there exists
at most one $i$ in $\\{2,3,\ldots,6\\}$ such that $c(a_{2i-1},a_{2i})=25$ and
for the other $i$’s in $\\{2,3,\ldots,6\\}$ we have $c(a_{2i-1},a_{2i})=26$.

If $c(u,v)=25$ and
$\sum_{\\{u_{1},u_{2}\\}\in\binom{U}{2}}c(u_{1},u_{2})=155$, then
$c(a_{2i-1},a_{2i})=26$ for all $i=2,3,\ldots,6$ and there exists a unique
vertex $z$ of $\Delta(x)$ such that $z$ has exactly $3$ neighbours in $U$. Any
other vertex $w$ of $\Delta(x)$ has at most $2$ neighbours in $U$. This shows
the lemma. ∎

As a consequence of Lemma 4.3 we have the following.

######  Lemma 4.4.

Assume a strongly regular graph $G$ exists with parameters $(1911,270,105,27)$
such that $G$ has an induced quadrangle, say $x\sim u\sim y\sim v\sim x$. Let
$W:=\\{w\sim x\mid w\neq u,w\neq v,w\not\sim u,w\not\sim v\\}$ and let
$Z:=\\{z\in W\mid\text{there exists a vertex }z^{\prime}\in
W\backslash\\{z\\}\text{ such that }z^{\prime}\not\sim z\\}$. Let $\Gamma_{W}$
be the subgraph induced on $W$. For $w\in W$, let $k_{w}$ be the valency of
$w$ in $\Gamma_{W}$. For $z\in Z$, let $K_{z}$ (resp. $\tilde{K}_{z}$) be the
subgraph induced on $\\{w\in W\mid w\neq z,w\not\sim z\\}$ (resp. $\\{w\in
W\mid w\neq z,w\not\sim z\\}\cup\\{x\\}$). Then the following hold:

  1. (1)

The graph $K_{z}$ and $\tilde{K}_{z}$ are complete and $K_{z}$ has at least
$28$ vertices,

  2. (2)

$|W|\in\\{82,83\\}$,

  3. (3)

For $w\in W$ we have $k_{w}\in\\{53,54,|W|-1\\}$. Moreover, if $|W|=82$, then
$k_{w}\in\\{53,|W|-1\\}$ and any two distinct non-adjacent vertices
$z,z^{\prime}\in Z$ have $c(z,z^{\prime})=26$.

######  Proof.

(1): If $K_{z}$ is not complete, then there would be an independent set $U$ of
order $5$ containing $u$ and $v$, a contradiction with Lemma 4.1. As $K_{z}$
is a subgraph of $\Delta(x)$, it is clear that $\tilde{K}_{z}$ is complete as
well. We will show later that $K_{z}$ has at least $28$ vertices.

(2): We have $|W|=k-2(\lambda+1)+c(u,v)$. As $k=270$, $\lambda=105$ and
$c(u,v)\in\\{24,25\\}$, we find $|W|\in\\{82,83\\}$. This shows (2).

(3): Let $w\in W$. If $w\not\in Z$, then $k_{w}=|W|-1$. So, let $w\in Z$ and
$w^{\prime}\in Z\backslash\\{z\\}$ such that $w\not\sim w^{\prime}$. We have
$c(u,v)\in\\{24,25\\}$.

If $c(u,v)=24$, then $|W|=82$, and $c(a_{1},a_{2})=26$ for all
$\\{a_{1},a_{2}\\}\in\binom{\\{u,v,w,w^{\prime}\\}}{2}$ and
$\\{a_{1},a_{2}\\}\neq\\{u,v\\}$, by Lemma 4.3(1). In particular, we have
$c(w,w^{\prime})=26$ and $k_{w}=k_{w^{\prime}}=105-2\times 26=53$. In this
case, $K_{w}$ has $82-53-1=28$ vertices.

If $c(u,v)=25$, then $|W|=83$. We have $c(u,w),c(v,w)\in\\{25,26\\}$. If one
of $c(u,w)$ or $c(v,w)$ is equal to $25$, then
$k_{w}=105-c(u,w)-c(v,w)=105-25-26=54$, by Lemma 4.3(1). If
$c(u,w)=c(v,w)=26$, then $54=105-c(u,w)-c(v,w)+1\geqslant
105-c(u,w)-c(v,w)=53$, by Lemma 4.4(2). This implies $k_{w}$ has at least
$83-54-1=28$ vertices. This shows the lemma. ∎

######  Lemma 4.5.

Assume a strongly regular graph $G$ with parameters $(1911,270,105,27)$ exists
containing an induced quadrangle, say $x\sim u\sim y\sim v\sim x$. Let
$W:=\\{w\sim x\mid w\neq u,w\neq v,w\not\sim u,w\not\sim v\\}$. Then $|W|\neq
83$.

######  Proof.

Let $\Gamma_{W}$ be the subgraph of $G$ induced on $W$. In Lemma 4.4(3), we
have seen that the valency $k_{w}$ in $\Gamma_{W}$ of a vertex $w\in W$
satisfies $k_{w}\in\\{82,54,53\\}$.

Let $Y:=\\{w\in W\mid k_{w}\in\\{54,82\\}\\}\cup\\{x\\}$.

######  Claim 4.6.

The induced subgraph of $\Gamma_{W}$ on $Y$ is complete.

Proof of Claim 4.6. Clearly $x$ is adjacent to all the other vertices in $Y$.
Let $w\in W$ such that $k_{w}\in\\{82,54\\}$. If $k_{w}=82$, then $w$ is
adjacent to all other vertices of $\Gamma_{W}$. So, we only need to show that,
if $w$ and $w^{\prime}$ are distinct vertices in $W$ such that
$k_{w}=k_{w^{\prime}}=54$, then $w\sim w^{\prime}$.

Let $w$ be such that $k_{w}=54$. Let $w^{\prime}\in W\backslash\\{w\\}$ be
such that $w\not\sim w^{\prime}$. There are, without loss of generality, two
cases namely, $c(u,w)=25$ and $c(v,w)=26$ or, $c(u,w)=c(v,w)=26$. In the first
case $c(u,w^{\prime})=26=c(w^{\prime},v)$ and
$k_{w^{\prime}}=105-c(u,w^{\prime})-(v,w^{\prime})=53$, by Lemma 4.3(1). In
the second case there exists a vertex $z\sim x$ such that $u\sim z\sim v$ and
$z\sim w$. This means that $c(u,w^{\prime})=26=c(v,w^{\prime})$ and
$k_{w^{\prime}}=105-c(u,w^{\prime})-c(v,w^{\prime})=53$, by Lemma 4.3(2). ∎

By Lemma 3.4, a clique of $G$ has order at most $32$, so $|Y|\leqslant 32$.
This means that $|\\{w\in W\mid k_{w}=53\\}|\geqslant 83-31=52$. So there are
two distinct non-adjacent vertices $z$ and $z^{\prime}$ in $W$ with
$k_{z}=k_{z^{\prime}}=53$. We have $c(z,z^{\prime})\in\\{25,26\\}$. There is
no vertex $\hat{z}$ that is adjacent to $u,v$ and one of $z$ and $z^{\prime}$.
This means that, if $c(z,z^{\prime})=26$, then there exists a vertex
$\tilde{z}$ adjacent to $z,z^{\prime}$ and one of $u$ and $v$. This means
$|C(z,z^{\prime})\cap W|=25$ holds whether $c(z,z^{\prime})=25$ or
$c(z,z^{\prime})=26$. This means that the cliques $K_{z}$ and $K_{z^{\prime}}$
both have exactly $29$ vertices, where $K_{z}$ is defined as in Lemma 4.4.

Assume that there are two distinct vertices in $C(z,z^{\prime})$ that are
adjacent to all vertices in $K_{z}$. Then they have at least $28$ common
neighbours and hence are adjacent. This implies there are at most $2$ vertices
in $C(z,z^{\prime})$ such that they are adjacent to all vertices in
$\tilde{K}_{z}$ and at most $2$ vertices in $C(z,z^{\prime})$ such that they
are adjacent to all vertices in $\tilde{K}_{z^{\prime}}$, where $K_{z}$ and
$\tilde{K}_{z}$ are as defined in Lemma 4.4. So there exists a vertex $w\in
C(z,z^{\prime})$ that is not adjacent to all vertices of $\tilde{K}_{z}$ and
not adjacent to all vertices of $\tilde{K}_{z^{\prime}}$. We find
$k_{w}\in\\{53,54\\}$. Either $w$ is adjacent to all other vertices in
$C(z,z^{\prime})$ or there exist $w^{\prime}\in
C(z,z^{\prime})\backslash\\{w\\}$ such that $w\not\sim w^{\prime}$.

######  Claim 4.7.

$|G(w)\cap G(x)\cap G(z)\cap G(z^{\prime})|\geqslant 22$.

Proof of Claim 4.7. We may assume that there exists a vertex $w^{\prime}\in
C(z,z^{\prime})\backslash\\{w\\}$ such that $w\not\sim w^{\prime}$, as
otherwise we are done. We have $|G(x)\cap G(z)\cap
G(z^{\prime})|=c(z,z^{\prime})=25$, $|G(z)\cap G(z^{\prime})|=27$ and
$|G(w)\cap G(z)\cap G(z^{\prime})|\geqslant 24$. The last statement follows
from Lemma 4.3, as $w\sim z\sim w^{\prime}\sim z^{\prime}\sim w$ is an induced
quadrangle of $G$. Now $|G(w)\cap G(x)\cap G(z)\cap G(z^{\prime})|\geqslant
25+24-27=22$. This shows Claim 4.7. ∎

Claim 4.7 implies that $w$ has at least $21$ neighbours in $W\cap
C(z,z^{\prime})$. So, without loss of generality, the vertex $w$ has at least
$\frac{53-25}{2}=14$ neighbours in $K_{z}$ and hence at least $15$ vertices in
$\tilde{K}_{z}$. By Table 2, we find that $w$ has at least $24$ neighbours in
$\tilde{K}_{z}$ and hence at least $23$ neighbours in $K_{z}$. This means that
$w$ has at most $54-21-23=10$ neighbours in $K_{z^{\prime}}$, by Claim 4.7. By
Table 2, we find that $w$ has at most $8$ neighbours in
$\tilde{K}_{z^{\prime}}$.

Now we consider $K_{w}$. Then $K_{w}$ has at least $28$ vertices, and
$\tilde{K}_{w}$ and $\tilde{K}_{z^{\prime}}$ intersect in at least $30-8=22$
vertices.

Now consider a maximal clique $C_{1}$ of $G$ containing $\tilde{K}_{w}$ and a
maximal clique $C_{2}$ of $G$ containing $\tilde{K}_{z^{\prime}}$. We have
$w\sim z\in V(K_{z^{\prime}})$. This means that $C_{2}$ does not contain
$C_{1}$. Also $K_{w}\cap K_{z}\neq\emptyset$, so $C_{1}$ does not contain
$C_{2}$. This means that $C_{1}\neq C_{2}$. As $|V(C_{1})|\geqslant 29$ and
$|V(C_{2})|\geqslant 30$, by Lemma 3.5, we find that $C_{1}$ or $C_{2}$ is not
maximal, a contradiction. This shows the lemma.

∎

######  Lemma 4.8.

Assume a strongly regular graph $G$ with parameters $(1911,270,105,27)$ exists
containing an induced quadrangle, say $x\sim u\sim y\sim v\sim x$. Let
$W:=\\{w\sim x\mid w\neq u,w\neq v,w\not\sim u,w\not\sim v\\}$. Then $|W|\neq
82$.

######  Proof.

Let $\Gamma_{W}$ be the subgraph of $G$ induced on $W$. In Lemma 4.4(3), we
have seen that the valency $k_{w}$ in $\Gamma_{W}$ of a vertex $w\in W$
satisfies $k_{w}\in\\{82,53\\}$. There are two distinct vertices
$z,z^{\prime}\in W$ such that $z\not\sim z^{\prime}$ and hence
$k_{z}=k_{z^{\prime}}=53$, as $\Gamma_{W}$ is not complete.

If there exists a vertex $w^{\prime}\in W$ such that $k_{w^{\prime}}=81$, then
consider the induced subgraphs $\hat{K}_{z}$ (resp. $\hat{K}_{z^{\prime}}$) on
$\\{w\in W\mid w\neq z,w\not\sim z\\}\cup\\{w^{\prime}\\}\cup\\{x\\}$ (resp.
$\\{w\in W\mid w\neq z^{\prime},w\not\sim
z^{\prime}\\}\cup\\{w^{\prime}\\}\cup\\{x\\}$). As $u$ and $v$ does not lie in
an independent set of order $5$ inside $\Delta(x)$, we see that $\hat{K}_{z}$
and $\hat{K}_{z^{\prime}}$ are complete. Now the proof follows the proof of
Lemma 4.5 in this case, as $|V(\hat{K}_{z})|=30=|V(K)_{z^{\prime}}|$. We leave
the details for the reader.

So we may assume that all vertices of $\Gamma_{W}$ have valency $53$ and any
two distinct non-adjacent vertices in $\Gamma_{W}$ have exactly $26$ common
neighbours. Note that, by Lemma 3.4, any clique $C$ in $\Gamma_{W}$ has at
most $31$ vertices, as the induced subgraph of $G$ on $\\{x\\}\cup V(C)$ is
complete.

Consider the join $K_{4}\nabla\Gamma_{W}$. Then
$\lambda_{\min}(K_{4}\nabla\Gamma_{W})\geqslant-3$ by Lemma 2.7, as
$\lambda_{\min}(\Gamma_{W})\geq\lambda_{\min}(G)=-3$.

Let $w$ be a vertex of $\Gamma_{W}$. Let $K_{w}$ be the subgraph on $\\{z\in
W\mid z\neq w,z\not\sim w\\}$. As before $K_{w}$ is complete and has $28$
vertices. Then, in $K_{4}\nabla\Gamma_{W}$, we consider the clique
$K_{4}\nabla K_{w}$. By Table 2, we find that any vertex $z$ of
$K_{4}\nabla\Gamma_{W}$ outside $K_{4}\nabla K_{w}$ has at most $7$ neighbours
or at least $27$ neighbours in $K_{4}\nabla K_{w}$. This means that $z$ has at
most $3$ neighbours or at least $23$ neighbours in $K_{w}$.

Let $w,w^{\prime}$ be two distinct non-adjacent vertices in $\Gamma_{W}$.
There are at most $3$ vertices in $C(w,w^{\prime})$ that are adjacent to all
vertices in $K_{w}$, and similarly there are at most $3$ vertices in
$C(w.w^{\prime})$ that are adjacent to all vertices in $k_{w^{\prime}}$ as
$|V(K_{w})|=28=|V(K_{w^{\prime}})|$. So there exist a vertex $z\in
C(w,w^{\prime})$ that is not adjacent to some vertex $p$ (resp. $p^{\prime}$)
in $K_{w}$ (resp. $K_{w^{\prime}}$). As $z$ has at least $22$ neighbours in
$C(w,w^{\prime})$, as in Claim 4.7 of Lemma 4.5, without loss of generality,
we see that $z$ has at most $\lfloor\frac{53-22}{2}\rfloor=15<23$ neighbours
in $K_{w}$. So $z$ has at most $3$ neighbours in $K_{w}$. So the two cliques
$K_{z}$ and $K_{w}$ intersect in at least $25$ vertices.

Now consider the induced subgraphs $\tilde{K}_{z}$ and $\tilde{K}_{w}$ on
$\\{x\\}\cup V(K_{z})$ and $\\{x\\}\cup V(K_{w})$ respectively. Then
$\tilde{K}_{z}$ and $\tilde{K}_{w}$ both have $29$ vertices and intersect in
at least $26$ vertices. As $z^{\prime}\in V(K_{z})\backslash V(K_{w})$ and
$p^{\prime}\in V(K_{w})\backslash V(K_{z})$ we see that $\tilde{K}_{z}$ and
$\tilde{K}_{w}$ must intersect in precisely $27$ vertices and both are
maximal, by Lemma 3.5. We have $z^{\prime}\in V(K_{z})\backslash V(K_{w})$ and
$p^{\prime}\in V(K_{w})\backslash V(K_{z})$ and, $K_{z}$ and $K_{w}$ intersect
in exactly $26$ vertices.

Now consider $K_{z^{\prime}}$. The clique $K_{z^{\prime}}$ and $K_{w}$
intersects in exactly two vertices under which $p^{\prime}$. Consider any
vertex $q$ of the clique $K_{p^{\prime}}$. Then

  1. (1)

$q$ has $26$ neighbours in $K_{z^{\prime}}$,

  2. (2)

$q$ has $26$ neighbours in $K_{w}$, or

  3. (3)

$q$ has at most $25$ neighbours in $K_{z^{\prime}}$ and at most $25$
neighbours in $K_{w}$.

There are at most $5$ vertices in case (1), and at most $5$ vertices in case
(2). This means that there exists a vertex $q$ in the clique $K_{p^{\prime}}$
which has at most $25$ neighbours in $K_{z^{\prime}}$ and at most $25$
neighbours in $K_{w}$. Without loss of generality, we may assume that $q$ has
at most $\frac{26}{2}=13$ neighbours in $K_{z^{\prime}}$. Then $q$ has at most
$3$ neighbours in $K_{z^{\prime}}$ and, $K_{q}$ and $K_{z^{\prime}}$ intersect
in at least $25$ vertices. This implies the induced subgraphs $\tilde{K}_{q}$
and $\tilde{K}_{z^{\prime}}$, on $\\{x\\}\cup V(K_{q})$ and $\\{x\\}\cup
V(K_{z^{\prime}})$ respectively, have both $29$ vertices and they intersect in
$27$ vertices, by Lemma 3.5. But this means that $K_{q}$ and $K_{z^{\prime}}$
intersect in exactly $26$ vertices and, $K_{q}$ and $K_{z^{\prime}}$ intersect
in exactly $4$ vertices.

We obtain that every vertex in $V(K_{q})\backslash V(K_{w})$ has at least $23$
neighbours in $V(K_{w})$. So this means that there are at least $24\times 21$
edges between $V(K_{q})\backslash V(K_{w})$ and $V(K_{w})\backslash V(K_{q})$.

This implies that the number of edges between $V(K_{z^{\prime}})\cup V(K_{w})$
and $V(K_{p^{\prime}})$ is at most $2\times 26\times 26-2\times 24\times
21=344$. On the other hand every vertex in $K_{p^{\prime}}$ has valency $53$,
so the number of edges between $V(K_{z^{\prime}})\cup V(K_{w})$ and
$V(K_{p^{\prime}})$ is exactly $26\times 28=728>344$, a contradiction. This
shows the lemma. ∎

Now we give the proof of the main theorem:

Proof of Theorem 1.1. By Lemma 2.5, we see that $G$ has an induced quadrangle,
say $x\sim u\sim y\sim v\sim x$. Let $W=\\{w\sim x\mid w\neq u,w\neq
v,w\not\sim u,w\not\sim v\\}$. Then, by Lemma 4.4, we have
$|W|\in\\{82,83\\}$. By Lemmas 4.5 and 4.8, we see that this is not possible.
This finishes the proof. ∎

## Acknowledgments

J.H. Koolen is partially supported by the National Natural Science Foundation
of China (No. 12071454), Anhui Initiative in Quantum Information Technologies
(No. AHY150000), and the project ”Analysis and Geometry on Bundles” of
Ministry of Science and Technology of the People’s Republic of China.

Brhane Gebremichel is supported by a Chinese Scholarship Council at University
of Science and Technology of China, China.

## References

  * [1] A. E. Brouwer, A. M. Cohen, and A. Neumaier.  Distance-Regular Graphs.  Springer-Verlag, Berlin Heidelberg, 1989.
  * [2] A. E. Brouwer and W. H. Haemers.  Spectra of Graphs.  Springer, New York, 2012.
  * [3] M.-Y. Cao, J. H. Koolen, A. Munemasa, and K. Yoshino.  Maximality of Seidel matrices and switching roots of graphs.  Graphs Combin., 2021.
  * [4] A. L. Gavrilyuk.  On the Koolen-Park inequality and Terwilliger graphs.  Elec. J. Combin., 17:#R125, 2010.
  * [5] C. Godsil and G. Royle.  Algebraic Graph Theory.  Springer-Verlag, Berlin, 2001.
  * [6] G. R. Greaves, J. H. Koolen, and J. Park.  Augmenting the Delsarte bound: a forbidden interval for the order of maximal cliques in strongly regular graphs.  European J. Combin., 97:103384, 2021.
  * [7] J. H. Koolen and J. Park.  Shilla distance-regular graphs.  European J. Combin., 31:2064–2073, 2010.
  * [8] Z. Qiao and Y. Pan.  A note on optimistic strongly regular graphs.  J. University of Science and Technology of China, 47(3):197–203, 2017.


# Density-Guided Label Smoothing for Temporal
Localization of Driving Actions

Tunc Alkanat*, Erkut Akdag*,†, Egor Bondarev, Peter H.N. de With
VCA Group, Department of Electrical Engineering, Eindhoven University of
Technology
P.O. Box 513, Eindhoven 5612AZ, The Netherlands
* equal technical contribution
† corresponding author
{t.alkanat, e.akdag, e.bondarev<EMAIL_ADDRESS>

###### Abstract

Temporal localization of driving actions plays a crucial role in advanced
driver-assistance systems and naturalistic driving studies. However, this is a
challenging task due to strict requirements for robustness, reliability and
accurate localization. In this work, we focus on improving the overall
performance by efficiently utilizing video action recognition networks and
adapting these to the problem of action localization. To this end, we first
develop a density-guided label smoothing technique based on label probability
distributions to facilitate better learning from boundary video-segments that
typically include multiple labels. Second, we design a post-processing step to
efficiently fuse information from video-segments and multiple camera views
into scene-level predictions, which facilitates elimination of false
positives. Our methodology yields a competitive performance on the A2 test set
of the naturalistic driving action recognition track of the 2022 NVIDIA AI
City Challenge with an $F_{1}$ score of 0.271.

##  1 Introduction

Efficient, fast, and safe transportation is one of the important foundations
of modern society and a major driving factor of the economy. Every day,
billions of people rely on various forms of transit, including road
transportation. However, frequent use of road transportation also has
drawbacks. For instance, the annual road traffic deaths were estimated at 1.25
million in 2013 [1]. Immense research is being conducted on advanced driver-
assistance systems (ADAS) to enhance the safety and comfort of road
transportation.

Figure 1: Diagram of joint task of action classification and localization.
From top to bottom: example images of the phone call, reaching behind, and
pick up from floor action classes. The graph shows an ideal output, where not
only classes but also the start and end times ($t_{s}[i]$, $t_{e}[i]$) of
actions are visible.

The ultimate aim of ADAS is to completely automate road transportation, while
ensuring safety by eliminating human factors of road accidents. However, this
automation task is difficult due to the complex nature of the problem,
variability of environmental conditions, and the strict requirements for high
robustness and reliability. In the current state, ADAS requires human
supervision to ensure safety while gradually addressing increasingly complex
driving tasks. This approach requires continuous attention of the driver,
since over-confidence by the driver in such systems may cause deadly
accidents. Consequently, the need for real-time monitoring of the driver’s
alertness arises and novel computer vision methodologies are being researched
to satisfy this requirement in an automated fashion.

For a practical implementation of driving action recognition, it is necessary
both to correctly classify the specific action of the driver and to temporally
localize this action, since the videos in the wild are untrimmed and may
include multiple classes at once. This problem is usually referred to as
temporal action localization in literature. Many previous works on this
subject are inspired by two-stage, proposal-generation and classification
methodologies that were initially devised for the problem of object detection
[2, 3]. Alternatively, the straightforward approach of using a temporal
sliding window has also been popular in literature [4, 5]. In the sliding-
window approach, action classification is performed for every overlapping
segment of the input video, and the resulting class probabilities for each
segment are fused in a post-processing step. While this approach provides
solid overall performance, the localization accuracy usually suffers from
segments that include multiple action classes. To alleviate this shortcoming,
in this work, we propose density-guided label smoothing to improve
localization performance. The associated loss function considers the
distribution of class labels in each training video segment, thereby enabling
better control over the performance trade-off of classification and temporal
localization sub-tasks. Furthermore, we design a post-processing step for
multi-camera fusion and prediction distillation. This design includes the
phases of: (1) efficient late-fusion of the predictions across multiple
cameras, (2) preservation of sharp temporal changes in class probabilities,
and (3) elimination of temporally overlapping detections. To summarize, the
main contributions of our work are as follows.

  * •

A smoothed multi-label training loss which facilitates better learning from
temporal boundary segments. The improved performance comes without any
additional computational overhead during inference.

  * •

An efficient post-processing step that eliminates false positives and improves
the overall performance.

  * •

An evaluation of the proposed method on the A2 test set of the naturalistic
driving action recognition track of the 2022 NVIDIA AI City Challenge. Our
method shows promising results with an $\text{F}_{1}$ score of 0.271.

##  2 Related Work

This section presents a literature overview of video feature extraction,
temporal action localization, and driving action recognition datasets.

Video feature extraction. Extracting features from given videos is a crucial
first step to many advanced scene understanding tasks, such as action
recognition and video anomaly detection. As such, this problem has been
extensively studied. Early approaches to the problem focus on handcrafted
motion features, such as Hidden Markov Models [6, 7], sparse coding [8, 9],
histogram of oriented gradients (HOGs) [10, 11, 12], and appearance features
[13, 12] Besides handcrafted feature extraction, deep learning-based methods
have been recently proposed for video action recognition tasks. Especially
convolutional neural network-based (CNN) architectures [14] have provided
solid performance. Various 2D-CNN methods [15, 16, 17], 3D-CNN methods, such
as C3D [18], I3D [19] and ResNet3D [20], and combinations of both [21, 22] are
used for spatio-temporal feature extraction. Similarly, the SlowFast [23]
network processes videos at both low and high frame rates to simultaneously
capture short-term and long-term temporal information by employing two
pathways.

Temporal action localization. Revealing the temporal locations of important
events in untrimmed videos is a challenging task. Early approaches [24, 5, 25]
apply temporal sliding windows as an exhaustive search solution to the
localization, which is followed by a support vector machine (SVM) to classify
actions within each window position. Besides sliding-window approaches, object
detection-inspired [26] proposal generation methodologies have been employed.
For instance, in [2], authors use dilated convolutions to encode temporal
context and perform multi-stream feature fusion to improve action
localization. In [3], Zeng  _et al_. concentrate on capturing the context
information and characterize the correlations between distinct actions by
implementing Graph Convolutional Networks (GCNs).

Figure 2: Overview of the methodology. Left: multi-class, multi-view
structured mini-batching ensures a training without bias. Every stream is a
camera view. Middle: the network is trained using cross-entropy loss, where
the target probabilities are computed individually for every segment according
to the class density within that segment. Right: localization and post-
processing concatenates the class probabilities of streams into the scene
class probabilities by fusing streams. Then, the 1-D probability signal is
analyzed for peaks for every class in a scene. Significant peaks are
considered predictions and are further refined by eliminating the overlapping
predictions. Note that the modules shown with a dashed border are used for
training only.

Driving action recognition datasets. Quality of datasets is of utmost
importance, especially for the data-hungry deep learning-based approaches.
Some datasets that comprise of in-vehicle camera footage have been made
publicly available for research on naturalistic driving studies. For instance,
the Driver Anomaly Detection (DAD) dataset [27] consists of 31 subjects that
perform various activities in a real car. The dataset is recorded
synchronously from the front and top view. This dataset includes approximately
538 and 145 minutes of video recording for normal driving and anomalous
driving, respectively. Another important dataset in literature is called
Drive&Act [28]. This dataset includes twelve distraction-related actions. The
data include 9.6 million frames captured by six cameras and three imaging
modalities. The distracted driver dataset (DAD) is published in another study
[29] which involves 44 drivers. The dataset includes 14478 frames distributed
over the following classes: safe driving, phone right/left, text right/left,
adjusting the radio, drinking, hair/makeup, reaching behind, and talking to
the passenger.

From the related work study, we adopt the SlowFast feature extraction along
with the sliding-window approach and enhance it with a loss function that
improves temporal localization.

##  3 Methodology

Our methodology is depicted in Figure 2. It consists of three subsequent
parts: (1) feature extraction, (2) segment-level classification, and (3)
localization and post-processing.

###  3.1 Feature Extraction

Our method relies on the temporal sliding-window approach and its associated
video segment features to detect and localize distracted driving behaviors. We
adopt this approach in order to benefit from the mature video action
recognition literature, to exploit pre-training on large-scale datasets in the
field, and to be able to capture long-term, complex motion cues. Without loss
of generality, we use the SlowFast [23] video recognition network as our
backbone for feature extraction. The multi-branch architecture of the SlowFast
network simultaneously captures both long-term and short-term relations, by
applying temporal sub-sampling to the given video segments to extract one low
(fast pathway) and one high (slow pathway) frame rate input sequence, as in
the original paper. The resulting fast pathway focuses on long-term actions
and vice versa. We choose this architecture for feature extraction, since the
given distracted driving classes are of varying continuity. For instance, the
action classes of eating and drinking are usually of intermittent nature,
while the phone call (right/left) and reaching behind actions show a temporal
continuity. Multiple temporal resolutions considered by the SlowFast network
enable robust feature extraction for both cases.

The SlowFast backbone receives all video segments from the given training
videos. Formally, assume that
$V_{t}^{i}\in\mathbb{R}^{H\times{}W\times{}T_{c}}$ denotes a video segment
that consists of $T_{c}$ frames at the time interval $[t,t+T_{c}-1]$ of the
$i^{th}$ video with horizontal and vertical resolutions of $H$ and $W$,
respectively. We remove the classification head of the SlowFast architecture
to extract the high-level and descriptive features from both the slow and fast
pathways. We then combine the branch features by averaging. The resulting
feature vector, $f_{t}^{i}=S(V_{t}^{i})\in\mathbb{R}^{N_{f}}$, is used further
as the segment features, where $S$ is the functional form of the SlowFast
backbone and $N_{f}$ is the resulting feature size.

Figure 3: Bar chart depicting the effects of the temperature parameter $\beta$
and $f(x,y_{s})$ on the computed density-guided smooth labels, shown as the
bar height. For this example, $N_{c}=3$ and $T_{c}=64$ and $\beta$ is set to
10, 20, and 30, from top to bottom. The number of frames for each class is
given along the horizontal axis.

###  3.2 Segment-level Classification

SlowFast network is deployed with weights pre-trained on the Kinetics-400
video action recognition dataset [30]. This large-scale dataset includes 400
classes of various human actions and, thus, ensures that the network focuses
on important spatio-temporal aspects of the data related specifically to the
driver in the scene. We adapt the pre-trained SlowFast network to the problem
at hand by adding fully-connected layers to facilitate the target-specific
learning. Explicitly, the segment-level features extracted from the SlowFast
backbone, $f_{t}^{i}$, are input to two fully-connected layers that generate
probabilities for each class, $p_{t}^{i}(y)$, where $y\in[1,N_{c}]$ denotes
the class labels. The activation functions of the first and the second fully-
connected layer are set to sigmoid and softmax, respectively.

Data Imbalance. Training data imbalance usually leads to performance
degradation in classification tasks. To address this problem, we train the
model with a sampler that ensures a mini-batch to have equal amount of samples
per class and per camera view. Since we use frame-level labels for training
and the video segments may contain more than one class, the label with the
maximum number of frames in a given segment is regarded as the label for the
entire video segment for the mini-batch sampling. On the other hand, ensuring
a balanced sampling of data from each camera view improves stability of
training and results in a final model that is unbiased against input streams.

Boundary Segments. One important pitfall of the temporal sliding-window
approach lies in how multi-label video segments are utilized during training.
Such segments present a challenge to the network if the training is posed as
single-label classification. In such a case, the common approach is to either
use the most commonly observed frame-level label as the segment-level label
during training, or to discard heterogeneous segments altogether. However, we
conjecture that the overall performance of the network can be improved if the
information contained within the boundary segments are better utilized. For
this, we develop the density-guided label smoothing technique.

####  3.2.1 Density-guided Label Smoothing

Label smoothing is a commonly used regularization technique in machine
learning. The aim is to prevent over-confidence of the model in its
predictions and thus to improve the overall generalization. In the work [31],
the mathematical formulation of the label smoothing is given as

$q^{\prime}(k|x)=(1-\epsilon)\delta_{k,y}+\epsilon u(k),$ (1)

where $y$ is the ground-truth label for a given training sample $x$,
$q^{\prime}(k|x)$ is the smoothed ground-truth distribution over labels,
$\delta_{k,y}$ is the Dirac delta function which equals unity for $k=y$ and
zero otherwise, $u(k)$ is a distribution over labels, and $\epsilon$ controls
the amount of smoothing. Usually, the $u(k)$ distribution is selected as a
uniform distribution with a probability $1/N_{c}$.

The label smoothing improves generalization of the model and increases
tolerance to label noise. Overall, this approach yields faster convergence and
better performance for many classification tasks. Here, starting from this
idea, we introduce density-guided label smoothing to facilitate better
learning from the boundary segments and achieve a superior overall
localization performance. Our technique analyzes the distribution of frame-
level labels in each considered segment to compute the smoothed labels. The
number of frame-level labels in a segment can be represented by

$f(x,y_{s})=\sum_{m=1}^{T_{c}}\delta_{y_{x_{m}},y_{s}},$ (2)

where $y_{x_{m}}$ is the frame-level label of mth frame of the sample video
segment, $x$, and $f(x,y_{s})$ denotes the total count of frames with the
label $y_{s}$ in $x$. Then, we derive the density-guided smooth labels for a
given sample $x$ as

$q^{\prime\prime}(k|x)=\frac{e^{\frac{1}{\beta}f(x,k)}}{\displaystyle\sum_{j=1}^{N_{c}}e^{\frac{1}{\beta}f(x,j)}}.$
(3)

In the above equation, we apply the generalized form of the softmax function
with the temperature parameter $\beta$. The generalized softmax has several
important properties beneficial for the task: (1) its values are guaranteed to
be within [0; 1], regardless of the values of $f(x,y_{s})$, (2) it sums up to
unity when summed for all values of $k$, and (3) via the parameter $\beta$, it
enables the control over smooth vs. sharp transitions of the label change.

A bar chart depicting the effects of the temperature parameter and
$f(x,y_{s})$ on target labels is presented in Figure 3. As can be observed
from the figure, the temperature parameter is effective in controlling the
target values to get the desired balance between multi-labels and can function
as regular label smoothing for the classes that are absent in a given segment.

Loss Function. We train our network with the cross-entropy loss for
classification. Including the density-guided label smoothing, the complete
loss function is given as

$\mathcal{L}(x)=-\sum_{k}p_{t}^{i}(k)q^{\prime\prime}(k|x),$ (4)

where $(i,t)\in B$ defines a unique video segment in a mini-batch $B$. After
training, we supply test video segments to our network, extract segment class
probabilities, $p_{t}^{i}(y)$, which are then further subject to the
subsequent post-processing.

###  3.3 Post-processing

Unlike most object detection-inspired models, the sliding-window-based
approaches require a post-processing step to construct the final predictions.
Our post-processing pipeline consists of three consecutive parts, (1) fusion
of the stream probabilities, (2) peak detection and thresholding, and (3)
elimination of overlapping predictions.

1\. Late-fusion of the Stream Probabilities. Our methodology can be applied to
both multi-camera and single-camera scenarios. However, while the single-
camera case is straightforward, the multi-camera setting requires the fusion
of individual stream probabilities into frame-level scene probabilities. In
other words, the class probabilities of all synchronized camera views should
be combined (fused) to extract class probabilities of the multi-camera scene
under consideration. Note that, every scene in the dataset has multiple
streams (camera views). For this fusion, we average all the segment-level
class probabilities associated with a video segment that contains the
considered frame. This step computes the frame-level probabilities by
combining the segment-level probabilities which are derived by the network.
Here, we extend the previous works and use the output of the softmax
activation function instead of raw class scores. Consequently, each of the
resulting normalized class probabilities exhibits the inter-class correlation.
As a result, the final likelihood value assigned to a combination of a class
label and a specific scene frame includes the information regarding the
likelihood of other class probabilities for that frame.

2\. Peak Detection and Thresholding. Raw predictions of temporally localized
and classified actions can be obtained by detecting consistent peaks in the
frame-level scene probabilities for each class and scene separately. To detect
consistent peaks in this 1-D signal, first, we apply a temporal median
filtering to eliminate noise, which preserves sharp edges that are helpful in
accurate localization of actions. Second, we determine the highest peak of the
filtered signal of probabilities. If the observed peak is higher than a pre-
defined probability threshold ($\tau$), we consider that peak as a detected
relevant action. Finally, we determine the fastest positive and negative
changes that precede and succeed the highest peak. The resulting time
instances at which the fastest changes in probability occur are then
considered as the start and end times of the detected action. Figure 4
visualizes the step of peak detection.

3\. Elimination of the Overlapping Predictions (EOP). Due to the similar
appearance of some actions, it is possible that more than one action are
assigned high probabilities for the same frame in individual scenes. However,
since the target dataset includes only one action for a time instance in all
scenes, overlapping predictions result in an increased false-positive rate and
consequently decrease the overall precision of the methodology. To overcome,
we first determine the intersection-over-union (IoU) for all possible pairs of
the same-scene predictions. Then, among the full set of predictions with an
IoU overlap greater than a pre-defined threshold ($o_{max}$), we only retain
the prediction with the highest peak in the final output.

##  4 Experiments

###  4.1 Track3 Dataset

The 2022 NVIDIA AI City Challenge presents naturalistic driving data [32]
containing 90 video clips (14 hours in total) recorded from different angles
by three synchronized cameras mounted in a car. Each video is about 10 minutes
long with a resolution of 1920$\times$1080 pixels. The dataset consists of 15
drivers with and without appearance blocks (e.g. sunglasses, hat) performing
18 different tasks (e.g. phone call, eating, and reaching back) once, in
random order throughout a video clip. The whole dataset is split into three:
A1, A2, and B, each containing five drivers. While the A1 dataset is provided
with the ground-truth labels of the start time, end time, and the type of
distracting behaviors, the A2 dataset is provided without labels. The main
target of the challenge is to classify the distracted behavior activities
executed by the driver in a given time frame on the A2 testing dataset, by
employing the A1 dataset for training. Organizers reserve the B dataset for
later testing to consider the final awards for the top performers.

###  4.2 Implementation Details

1\. Feature Extraction. We adopt the SlowFast model, pre-trained on the
Kinetics-400 video action recognition dataset, to our task of obtaining the
feature vectors. As a data-processing step, we re-size and crop each video
frame to 256$\times$256 pixels and feed the SlowFast model accordingly, by
repeating this process for each 64-frame segment of the video clip with a
temporal stride 1. The size of output feature vector, $N_{f}$, is 2,304 for
each segment.

2\. Segment-level Classification. The extracted feature vectors $f_{t}^{i}$
are given as an input to the first layer of two fully-connected layers, where
the sigmoid activation function is applied to a hidden layer size of 64. The
last layer generates probabilities for each action class by utilizing the
softmax activation function. We adopt the Adam optimizer with initial learning
rate 5$\times{10}^{-5}$ and weight decay 5$\times{10}^{-4}$. As the loss
function, the cross-entropy loss with density-guided label smoothing is used
in our proposed model. The temperature parameter, $\beta$, is set to 5. We
employ the GTX-1080 GPU for the training process and set the batch size to
270, which handles 15 training samples for each action class within three
camera views (e.g. rear, dash, and right).

3\. Localization and Post-processing. We apply the temporal median filtering
with a filter size of 301 frames. Minimum height and width for the peak
detection are set empirically to 0.05, and 200 frames, respectively.

###  4.3 Evaluation Metrics

Evaluation for 2022 NVIDIA AI City Challenge Track3 is based on the model
action identification performance, measured by the $F_{1}$ score. For
computing the $F_{1}$ score, a true positive (TP) action identification is
considered when an action is correctly identified as starting within one
second of the start time and ending within one second of the end time of the
action. A false positive (FP) is an identified action that is not a TP action.
Finally, a false negative (FN) action is a ground-truth action that is not
correctly identified. The $F_{1}$ score is the harmonic mean of recall $R_{c}$
and precision $P_{r}$, specified by

$F_{1}=2\times\frac{P_{r}\times R_{c}}{P_{r}+R_{c}}.$ (5)

The precision is calculated as the number of true positives divided by the
total number of true positives and false positives, which is given by:

$P_{r}=\frac{\text{TP}}{\text{TP}+\text{FP}}.$ (6)

The recall is calculated as the number of true positives divided by the total
number of true positives and false negatives, which is given by:

$R_{c}=\frac{\text{TP}}{\text{TP}+\text{FN}}.$ (7)

Although the final results of the challenge are announced based on the $F_{1}$
score, the evaluation system also presents recall and precision metrics.

###  4.4 Experimental Results

We evaluate the proposed methodology on the 2022 NVIDIA AI City Challenge
Track3 A2 test dataset. Some of the experimental results are summarized in
Fig. 4. In this figure, example images are shown from the A2 test dataset,
from top left to the bottom, of texting (right), pick-up from the floor
(driver), pick-up from the floor (passenger), and hand-on-head action classes.
The example pictures, including the sunglasses appearance block, are presented
only for the rear-view camera, since all camera views are synchronized. From
top right to the bottom, we display graphs including the start and end time of
each class after localization and post-processing steps. The vertical axis of
the each graph indicates the computed concatenated stream probabilities of
corresponding frames, while the horizontal axis shows the frame numbers within
the video clip per class. As shown in Fig. 4, the proposed methodology can
predict the first and last action classes’ start and end times with
confidence. Although other peaks appear in the second and third graphs, the
proposed algorithm still delivers the result correctly with the help of the
stages on peak detection & thresholding and overlapping-prediction
elimination. Overall, visual inspection of the corresponding video data
verifies that most of the predictions of our model were accurately localized
and classified.

We achieve 0.2710 $F_{1}$ score and rank the 9th place among 27 teams in
Track3, as can be seen from Tab. 2. In addition to the leaderboard results, we
were able to test only a subset of our contributions on the evaluation system.
For instance, Tab. 1 shows the improvement of performing the overlapping-
prediction elimination step within the localization and post-processing stage.

Table 1: Improvement of overlapping-prediction elimination step on the general leaderboard. Method |  $F_{1}$ score | Precision | Recall
---|---|---|---
Baseline | 0.2636 | 0.2706 | 0.2570
BL + EOP | 0.2710 | 0.3206 | 0.2346
Table 2: Final results based on $F_{1}$ score on 2022 NVIDIA AI City Challenge Track3 from top-15 on the public leaderboard. Rank | Team ID | Team Name | $F_{1}$
---|---|---|---
1 | 72 | VTCC-UTVM | 0.3492
2 | 43 | Stargazer | 0.3295
3 | 97 | CybercoreAI | 0.3248
4 | 15 | OPPilot | 0.3154
5 | 78 | SIS Lab | 0.2921
6 | 16 | BUPT-MCPRL2 | 0.2905
7 | 106 | Winter is Coming | 0.2902
8 | 124 | HSNB | 0.2849
9 | 54 | VCA | 0.2710
10 | 95 | Tahakom | 0.2706
11 | 1 | SCU_Anastasiu | 0.2558
12 | 148 | union | 0.2301
13 | 76 | Starwar | 0.2160
14 | 85 | Aespa winter | 0.1440
15 | 69 | SEEE-HUST | 0.1348
Figure 4: Experimental results of our methodology. From top left to bottom:
example images of texting by the right hand, pick up from the floor on the
driver side, pick up from the floor from the passenger side, hand on head
action classes. From top right to bottom: graphs show the predicted start and
end times ($t_{s}(i)$, $t_{e}(i)$) for each class of the video clip. After
obtaining the concatenated stream probabilities for each frame within the
video clip, we apply the localization and post-processing steps to locate each
class’s start and end time.

###  4.5 Discussion

Considering the correctly identified distracting activity as starting within
one second of the start time and ending within one second of the end time of
the activity, which is defined as part of the evaluation criteria, makes this
detection task significantly difficult. One of the related challenges is that
accurately defining the ground-truth labels of the start and end times can be
subjective for some activities. For instance, in a drinking activity, some may
annotate the start time as touching the bottle, while others may annotate the
moment the liquid enters the mouth, or even the moment of reaching out to the
bottle. This ambiguity of timing makes the definition of the action label
uncertain possibly leading to false model prediction because those examples
may differ by two to three seconds. Another challenge is that the range of
possible distracted behaviour types is broader than the given set of classes,
e.g. listening to instructions, organizing the objects on the passenger seat,
and taking off the mask. As a consequence, these actions can be easily
mismatched with the defined classes of the dataset.

As a possible improvement for alleviating some ambiguity, the audio modality
of three camera view recordings may be exploited as well. Audible information
can be introduced as additional data without large extra cost. The model based
on the proposed methodology can then learn better by jointly analyzing the
audio and video datasets.

##  5 Conclusion

In this paper, we proposed a methodology that localizes and classifies
distracted driver behavior from in-vehicle video data. This approach relies on
the fusion of CNN-based video segment features. To enable better learning, we
have developed a novel technique called the density-guided label smoothing,
which improves the localization performance. Furthermore, we have designed a
post-processing step, which efficiently combines data from multiple cameras
and eliminates overlapping predictions by retaining only the most confident
prediction. This filtering of the predictions improves the precision of the
methodology by eliminating false positives. Performance evaluation of our
methodology on the A2 test set of the naturalistic driving action recognition
track of the 2022 NVIDIA AI City Challenge yields competitive results.

## Acknowledgements

This work is supported by the European ITEA project SMART on intelligent
traffic flow systems and the NWO Efficient Deep Learning (EDL) RMR project.

## References

  * [1] World Health Organization.  Global status report on road safety 2015.  World Health Organization, 2015.
  * [2] Yu-Wei Chao, Sudheendra Vijayanarasimhan, Bryan Seybold, David A Ross, Jia Deng, and Rahul Sukthankar.  Rethinking the faster r-cnn architecture for temporal action localization.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1130–1139, 2018.
  * [3] Runhao Zeng, Wenbing Huang, Mingkui Tan, Yu Rong, Peilin Zhao, Junzhou Huang, and Chuang Gan.  Graph convolutional networks for temporal action localization.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7094–7103, 2019.
  * [4] Dan Oneata, Jakob Verbeek, and Cordelia Schmid.  The lear submission at thumos 2014, 2014.
  * [5] Limin Wang, Yu Qiao, Xiaoou Tang, et al.  Action recognition and detection by combining motion and appearance features.  THUMOS14 Action Recognition Challenge, 1(2):2, 2014.
  * [6] Timothy Hospedales, Shaogang Gong, and Tao Xiang.  A markov clustering topic model for mining behaviour in video.  In 2009 IEEE 12th International Conference on Computer Vision, pages 1165–1172. IEEE, 2009.
  * [7] Louis Kratz and Ko Nishino.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models.  In 2009 IEEE conference on computer vision and pattern recognition, pages 1446–1453. IEEE, 2009.
  * [8] Bin Zhao, Li Fei-Fei, and Eric P Xing.  Online detection of unusual events in videos via dynamic sparse coding.  In CVPR 2011, pages 3313–3320. IEEE, 2011.
  * [9] Xuan Mo, Vishal Monga, Raja Bala, and Zhigang Fan.  Adaptive sparse representations for video anomaly detection.  IEEE Transactions on Circuits and Systems for Video Technology, 24(4):631–645, 2013.
  * [10] Rizwan Chaudhry, Avinash Ravichandran, Gregory Hager, and René Vidal.  Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 1932–1939. IEEE, 2009.
  * [11] Rensso Victor Hugo Mora Colque, Carlos Caetano, Matheus Toledo Lustosa de Andrade, and William Robson Schwartz.  Histograms of optical flow orientation and magnitude and entropy to detect anomalous events in videos.  IEEE Transactions on Circuits and Systems for Video Technology, 27(3):673–682, 2016.
  * [12] Weixin Li, Vijay Mahadevan, and Nuno Vasconcelos.  Anomaly detection and localization in crowded scenes.  IEEE transactions on pattern analysis and machine intelligence, 36(1):18–32, 2013.
  * [13] Yang Cong, Junsong Yuan, and Ji Liu.  Sparse reconstruction cost for abnormal event detection.  In CVPR 2011, pages 3449–3456. IEEE, 2011.
  * [14] Chun-Fu Richard Chen, Rameswar Panda, Kandan Ramakrishnan, Rogerio Feris, John Cohn, Aude Oliva, and Quanfu Fan.  Deep analysis of cnn-based spatio-temporal representations for action recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6165–6175, 2021.
  * [15] Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool.  Temporal segment networks: Towards good practices for deep action recognition.  In European conference on computer vision, pages 20–36. Springer, 2016.
  * [16] Ji Lin, Chuang Gan, and Song Han.  Temporal shift module for efficient video understanding. corr abs/1811.08383 (2018), 1811.
  * [17] Quanfu Fan, Chun-Fu Richard Chen, Hilde Kuehne, Marco Pistoia, and David Cox.  More is less: Learning efficient video representations by big-little network and depthwise temporal aggregation.  Advances in Neural Information Processing Systems, 32, 2019.
  * [18] Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri.  Learning spatiotemporal features with 3d convolutional networks.  In Proceedings of the IEEE international conference on computer vision, pages 4489–4497, 2015.
  * [19] Joao Carreira and Andrew Zisserman.  Quo vadis, action recognition? a new model and the kinetics dataset.  In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017.
  * [20] Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh.  Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?  In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 6546–6555, 2018.
  * [21] Chenxu Luo and Alan L Yuille.  Grouped spatial-temporal aggregation for efficient action recognition.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5512–5521, 2019.
  * [22] Swathikiran Sudhakaran, Sergio Escalera, and Oswald Lanz.  Gate-shift networks for video action recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1102–1111, 2020.
  * [23] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He.  Slowfast networks for video recognition.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6202–6211, 2019.
  * [24] Svebor Karaman, Lorenzo Seidenari, and Alberto Del Bimbo.  Fast saliency based pooling of fisher encoded dense trajectories.  In ECCV THUMOS Workshop, volume 1, page 5, 2014.
  * [25] Jun Yuan, Bingbing Ni, Xiaokang Yang, and Ashraf A Kassim.  Temporal action localization with pyramid of score distribution features.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3093–3102, 2016.
  * [26] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  Advances in neural information processing systems, 28, 2015.
  * [27] Okan Kopuklu, Jiapeng Zheng, Hang Xu, and Gerhard Rigoll.  Driver anomaly detection: A dataset and contrastive learning approach.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 91–100, 2021.
  * [28] Manuel Martin, Alina Roitberg, Monica Haurilet, Matthias Horne, Simon Reiß, Michael Voit, and Rainer Stiefelhagen.  Drive&act: A multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2801–2810, 2019.
  * [29] Hesham M Eraqi, Yehya Abouelnaga, Mohamed H Saad, and Mohamed N Moustafa.  Driver distraction identification with an ensemble of convolutional neural networks.  Journal of Advanced Transportation, 2019, 2019.
  * [30] Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al.  The kinetics human action video dataset.  arXiv preprint arXiv:1705.06950, 2017.
  * [31] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016.
  * [32] Mohammed Shaiqur Rahman, Archana Venkatachalapathy, Anuj Sharma, Jiyang Wang, Senem Velipasalar Gursoy, David Anastasiu, and Shuo Wang.  Synthetic distracted driving (syndd1) dataset for analyzing distracted behaviors and various gaze zones of a driver, 2022.