Thus, it remains only to check that the regularity in time follows from our
control over the Hausdorff distance. Fix $x^{\prime}$ and $t_{1},t_{2}$, and
write $x_{1}=(x^{\prime},f(x^{\prime},t_{1}))$,
$x_{2}=(x^{\prime},f(x^{\prime},t_{2}))$. Then for
$\varepsilon=|f(x^{\prime},t_{1})-f(x^{\prime},t_{2})|=|x_{1}-x_{2}|$,
$B_{\varepsilon}(x_{1})$ contains the point
$\tilde{x}\in\partial\Omega_{T(x_{2})}$ which minimizes the distance to
$x_{1}$. By Lemma 5.11, after possibly shrinking our neighborhood,
$|x_{1}-\tilde{x}|\leq C|t_{1}-t_{2}|$. In particular,
$|\tilde{x}^{\prime}-x^{\prime}|\leq C|t_{1}-t_{2}|$, so
$|f(\tilde{x}^{\prime},t_{2})-f(x^{\prime},t_{2})|\leq C|t_{1}-t_{2}|$, for
some larger $C$ given by the $C^{1}$ spatial regularity of $f$. Then

$|f(x^{\prime},t_{1})-f(x^{\prime},t_{2})|\leq|f(x^{\prime},t_{1})-f(\tilde{x}^{\prime},t_{2})|+|f(\tilde{x}^{\prime},t_{2})-f(x^{\prime},t_{2})|\leq
C|t_{1}-t_{2}|$

which completes the proof.

∎

###  5.2. Singular points

Now we proceed to analysis of the singular set, with the goal of controlling
singular points in dimension. First, we will show that the blowup profile at
singular points varies continuously along the spacetime interface. The main
tool will be a uniform approximation result that we prove in the appendix,
which will allow us to make use of the uniform-in-time spatial continuity of
$\eta$.

######  Proposition 5.13.

$x\mapsto D^{2}w(x,T(x))$ is continuous on $\Sigma$.

###### Proof.

First, by Lemma 2.3 and Lemma 5.1, we have that $w$ is locally spacetime
Lipschitz and $\eta$ is $C^{0,\alpha}$ in space locally uniformly in time. We
also have that $T$ is locally $C^{0,\alpha}$ on $\mathcal{O}$. Thus, for some
$C>0$, we can restrict to a spacetime neighborhood of the interface where all
of these norms are bounded by $C$ ( in the case of $T$, in the sense of the
neighborhood’s projection into space).

Let $\varepsilon>0$. By Lemma 6.13, there exists a scale depending only on the
modulus of continuity of $1-\eta$ in (1.8), such that the quadratic blowup
uniformly approximates $w$ near singular points at that scale; concretely,
there is a $\delta=\delta(\varepsilon/3,C)>0$ such that if $x_{0}$ is a
singular point in our neighborhood with blowup $q_{0}(x)=\frac{1}{2}x\cdot
D^{2}w(x_{0})x$ (recentered at 0), then

$\|\delta^{-2}w(x_{0}+\delta
x,T(x_{0}))-q_{0}\|_{C^{1}(B_{1})}<\frac{\varepsilon}{3}$

In particular, if $x_{1}$ is another singular point in the same neighborhood
with blowup $q_{1}$, then we have

(5.7)
$\|q_{0}-q_{1}\|_{L^{\infty}(B_{1})}\leq\frac{2\varepsilon}{3}+\delta^{-2}\|w(x_{0}+\delta
x,T(x_{0}))-w(x_{1}+\delta x,T(x_{1}))\|_{L^{\infty}(B_{1})}$

From the regularity of $T$ and $w$ on our chosen neighborhood, it follows that

(5.8)
$\|q_{0}-q_{1}\|_{L^{\infty}(B_{1})}\leq\frac{2\varepsilon}{3}+C_{1}\delta^{-2}|x_{1}-x_{0}|^{\alpha}$

for some $C_{1}$ depending only on $C$. Then it is clear that for
$|x_{1}-x_{0}|$ sufficiently small, we have
$\|q_{0}-q_{1}\|_{L^{\infty}(B_{1})}<\varepsilon$. By equivalence of norms on
${\mathbb{R}}^{d\times d}$, this gives $|D^{2}w(x_{0})-D^{2}w(x_{1})|\leq
O(\varepsilon)$, and we conclude. ∎

We remark that $D^{2}w(x,T(x))$ also exists in a one-sided sense for $x\in R$,
with $D^{2}w(x,T(x))=\nu(x)\nu(x)^{T}$ for $\nu$ as in Proposition 5.6. Thus,
in light of that proposition, $D^{2}w(x,T(x))$ is continuous on $R$. However,
due to the jump in rank, there is no possibility of continuity from $R$ to
$\Sigma^{k}$ when $k<d-1$.

Using the continuous dependence of the blowup, we can subsequently apply a
Whitney extension argument to obtain that singular points are contained in
$C^{1}$ manifolds. Following the approach in [PSU12], we introduce the
following lemma:

######  Lemma 5.14 (Whitney’s extension theorem).

Let $K\subset{\mathbb{R}}^{d}$ be compact, and suppose we have a function
$f:K\to{\mathbb{R}}$ and a family of degree $m$ polynomials $p_{x}$ indexed
over $K$. If

  1. (i)

$p_{x_{0}}(x_{0})=f(x_{0})$ for each $x_{0}\in K$

  2. (ii)

$|D^{k}p_{x_{0}}(x_{1})-D^{k}p_{x_{1}}(x_{1})|=o(|x_{0}-x_{1}|^{m-k})$ for
$x_{0},x_{1}\in K$ and $0\leq k\leq m$.

Then $f$ extends to a $C^{m}$ function on ${\mathbb{R}}^{d}$ such that
$f(x)=p_{x_{0}}(x)+o(|x-x_{0}|^{m})$ for all $x_{0}\in K$.

######  Proposition 5.15.

Near a point in $\Sigma^{k}$, $\Sigma$ is locally contained in a $C^{1}$
manifold of dimension $k$. In particular, $\Sigma$ is contained in countably
many $C^{1}$ submanifolds of dimension $d-1$.

###### Proof.

We fix a compact $K\subset\Sigma$ for the proof. We will apply Lemma 5.14 to
extend the zero function on $K$, with second order Taylor polynomial
$q_{x_{0}}$ at $x_{0}\in K$ given by the quadratic blowup of $w$ at each
point. Specifically, this results in

$q_{x_{0}}(x)=\frac{1}{2}(x-x_{0})\cdot D^{2}w(x_{0},T(x_{0}))(x-x_{0})\hbox{
where }x_{0}\in K,x\in{\mathbb{R}}^{d}$

The extension will give us a $C^{2}$ function $f$ on ${\mathbb{R}}^{d}$ such
that $\nabla f\equiv 0$ on $K$, after which the implicit function theorem will
imply that in a neighborhood of $x_{0}\in K$, the set $\\{\nabla f=0\\}$ is
contained in a $C^{1}$ manifold of dimension $\dim\ker D^{2}f(x_{0})=\dim\ker
D^{2}w(x_{0},T(x_{0}))$.

Thus, we proceed to verify the assumptions of the lemma. First, we note that
by Lemma 5.1, $\eta$ is uniformly $C^{0,\alpha}$ in space in a spacetime
neighborhood of the interface as it passes through $K$ in space. It follows by
our uniform approximation result for the quadratic blowup at singular points,
Lemma 6.13, that there is a modulus of continuity $\sigma$ such that

(5.9) $\|r^{-2}w(x_{0}+r(x-x_{0}),T(x_{0}))-\frac{1}{2}(x-x_{0})\cdot
D^{2}w(x_{0},T(x_{0}))(x-x_{0})\|_{C^{1}_{x}(B_{1})}\leq\sigma(r)$

for any $x_{0}\in K$. Then, if we apply this estimate to $x_{0},x_{1}\in K$
with $T(x_{0})\leq T(x_{1})$, we get

$|q_{x_{0}}(x_{1})-q_{x_{1}}(x_{1})|=|\frac{1}{2}(x_{1}-x_{0})\cdot
D^{2}w(x_{0},T(x_{0}))(x_{1}-x_{0})|\leq|x_{0}-x_{1}|^{2}\sigma(|x_{0}-x_{1}|)$

directly from (5.9) and the fact that $w(x_{1},T(x_{0}))=0$. On the other
hand, if $T(x_{0})>T(x_{1})$, we have that
$|q_{x_{1}}(x_{0})|\leq|x_{0}-x_{1}|^{2}\sigma(|x_{0}-x_{1}|)$ from the above,
and

$|q_{x_{1}}(x_{0})-q_{x_{0}}(x_{1})|=|\frac{1}{2}(x_{1}-x_{0})\cdot(D^{2}w(x_{1},T(x_{1}))-D^{2}w(x_{0},T(x_{0})))(x_{1}-x_{0})|\leq
o(|x_{1}-x_{0}|^{2})$

by the continuity of the Hessian from Proposition 5.13. This completes the
$k=0$ case of (ii) in Lemma 5.14.

The verification of the $k=1$ case of (ii) is similar, using the derivative
bound from (5.9). In general, we have $\nabla
q_{x_{0}}(x)=D^{2}w(x_{0},T(x_{0}))(x-x_{0})$. Then, when $T(x_{0})\leq
T(x_{1})$, we get

$|\nabla q_{x_{0}}(x_{1})-\nabla
q_{x_{1}}(x_{1})|=|D^{2}w(x_{0},T(x_{0}))(x_{1}-x_{0})|\leq|x_{1}-x_{0}|\sigma(|x_{1}-x_{0}|)$

directly from (5.9) and the fact that $\nabla w(x_{1},T(x_{0}))=0$. On the
other hand, if $T(x_{0})>T(x_{1})$, we have that

$|\nabla q_{x_{1}}(x_{0})+\nabla
q_{x_{0}}(x_{1})|=|\frac{1}{2}(D^{2}w(x_{0},T(x_{0}))-D^{2}w(x_{1},T(x_{1})))(x_{1}-x_{0})|\leq
o(|x_{1}-x_{0}|)$

again by Proposition 5.13, giving that

$|\nabla q_{x_{0}}(x_{1})-\nabla q_{x_{1}}(x_{1})|=|\nabla
q_{x_{0}}(x_{1})|\leq|\nabla q_{x_{0}}(x_{1})+\nabla q_{x_{1}}(x_{0})|+|\nabla
q_{x_{1}}(x_{0})|\leq o(|x_{1}-x_{0}|)$

Finally, the $k=2$ case of (ii) in Lemma 5.14 is exactly continuity of the
Hessian, from Proposition 5.13. Thus, the conditions of the lemma are
satisfied, which completes the proof.

∎

Figure 3. A cylindrical patch with a nearly cylindrical hole. As the hole
contracts, singular points are expected to occur near the axis (dotted). Due
to variation in the diameter of the hole, however, singular points may occur
at different times. Proposition 5.15 confirms that we have the expected
spatial regularity for the set $\Sigma\subset{\mathbb{R}}^{d}$ of points which
are singular at any time.

We stress that this result is for $\Sigma$, and not for the corresponding
subset $\mathrm{Graph}_{T}(\Sigma)$ of the spacetime interface, where we use
the notation introduced in (2.10). Since $T$ is only known to be Hölder
continuous on $\Sigma$, we obtain the weaker result that
$\mathrm{Graph}_{T}(\Sigma)$ is locally contained in $C^{0,\alpha}$ manifolds
of dimension $d-1$. Nevertheless, we are able to apply this to establish
control in Hausdorff dimension over the interface. The Hausdorff dimension of
the spatial interface has previously been studied in [MPQ17] for a similar
problem, where the obstacle problem satisfied by $w$ was applied to conclude
that $\partial\Omega_{t}$ has locally finite $(d-1)$-dimensional Hausdorff
measure for each $t$. Using the Lipschitz regularity of $T$ near regular
points and our spatial control over singular points, we can study the
spacetime interface $\mathrm{Graph}_{T}(\mathcal{O})$ for the first time and
show that it has the expected Hausdorff dimension $d$. We summarize the
consequences of Proposition 5.15 with the following statements.

######  Corollary 5.16.

We have, for $\alpha$ as in Theorem 4.2:

  1. (i)

$\partial\Omega_{t}$ has finite $(d-1)$-dimensional Hausdorff measure.

  2. (ii)

$\Sigma$ has locally finite $(d-1)$-dimensional Hausdorff measure. In
particular, $\Sigma_{t}$ has zero $(d-1)$-dimensional Hausdorff measure for
all but countably many $t$ in $(0,\infty)$, and for a.e. $t\in(0,\infty)$,
$\Sigma_{t}$ has Hausdorff dimension at most $d-1-\alpha$.

  3. (iii)

$\mathrm{Graph}_{T}(\mathcal{O})$ has Hausdorff dimension $d$, and decomposes
as $\mathrm{Graph}_{T}(R)\cup\mathrm{Graph}_{T}(\Sigma)$, where the first set
is relatively open with locally finite $d$-dimensional Hausdorff measure, and
the second set has locally finite $(d-\alpha)$ Hausdorff measure.

###### Proof.

We use the fact that $\rho\in
L^{\infty}_{t}BV_{x}([0,\tau];{\mathbb{R}}^{d})$, and thus $\Omega_{t}$ is a
set of finite perimeter. In particular, we can consider the reduced boundary
$\partial^{*}\Omega_{t}$, which has finite $(d-1)$ Hausdorff measure and
contains $R_{t}$, by the local regularity of the boundary at those points. On
the other hand, $\Sigma_{t}$ is locally contained in a $C^{1}$ manifold of
dimension $(d-1)$. In fact, since $\Sigma_{t}$ is compact, $\Sigma_{t}$ is
contained in a bounded $C^{1}$ manifold, which then also has finite $(d-1)$
measure, so we have

$\mathcal{H}^{d-1}(\partial\Omega_{t})\leq\mathcal{H}^{d-1}(\partial^{*}\Omega_{t})+\mathcal{H}^{d-1}(\Sigma_{t})<\infty$

Since $\Sigma$ is locally contained in $C^{1}$ manifolds of dimension $d-1$,
the $(d-1)$-dimensional Hausdorff measure on $\Sigma$ is locally finite. In
particular, it is also $\sigma$-finite, which implies that there cannot be
uncountably many $t$ for which $\Sigma_{t}$ has positive $(d-1)$ measure. The
improvement to $(d-1-\alpha)$ dimension at a.e. time follows from a geometric
measure theory lemma of [FROS20]. Since $\Sigma$ has dimension $d-1$ and $T\in
C^{0,\alpha}_{\operatorname{\textup{loc}}}(\mathcal{O})$, Corollary 7.8 of
that paper directly gives the result.

For the final statement, we use the general result that the graph of a
$C^{0,\alpha}$ function on a set of Hausdorff dimension $s$, for
$\alpha\in(0,1]$ and $s\geq 0$, has Hausdorff dimension at most $s+1-\alpha$.
In particular, since $T$ is locally Lipschitz on $R$, which is open in
${\mathbb{R}}^{d}$, and locally $C^{0,\alpha}$ on $\Sigma$, which is Hausdorff
dimension at most $d-1$, we get the local control in Hausdorff measure for the
graphs of those sets. Then we can write $\mathrm{Graph}_{T}(\mathcal{O})$ as a
countable union of sets of Hausdorff dimension at most $d$, so we conclude. ∎

We remark that a natural open question is whether the space-time interface has
locally finite $d$-dimensional Hausdorff measure near singular points. This
would follow, for example, if $T$ were known to be uniformly Lipschitz near
$\Sigma$.

###  5.3. Speculative Results

We finish our treatment of the singular set by noting that stronger generic
control over $\Sigma_{t}$ is possible with slightly stronger regularity than
currently known: namely, when $T$ is Lipschitz.

######  Proposition 5.17.

If $T\in C^{0,1}_{\operatorname{\textup{loc}}}(\mathcal{O})$, then
$\Sigma_{t}$ has $(d-2)$-Hausdorff measure 0 for a.e. $t\in(0,\infty)$.

###### Proof.

Here we follow an argument by [Mon03], originally applied to a hitting time
for the constant Laplacian obstacle problem with a time-varying condition on
the fixed boundary. Since $T$ is Lipschitz, Proposition 4.6 in [Mon03] implies
that for any compact subset $K$ of $\Sigma$,

$\limsup_{x,y\in K,|x-y|\to 0}\frac{|T(x)-T(y)|}{|x-y|}=0$

Then from the coarea formula, we have

$\int_{\Sigma^{d}}|\nabla
T_{|_{\Sigma^{d}}}|\,d\mathcal{H}^{d-1}=\int_{0}^{\infty}\mathcal{H}^{d-2}(T_{|_{\Sigma^{d-1}}}^{-1}(t))\,dt=\int_{0}^{\infty}\mathcal{H}^{d-2}(\Sigma^{d-1}_{t})\,dt$

Then the integrand on the left is 0, so $\Sigma^{d-1}_{t}$ has
$(d-2)$-Hausdorff measure 0 for a.e. $t$. On the other hand, $\Sigma^{k}_{t}$
has $(d-2)$-Hausdorff measure 0 for $k<d-2$ and all $t$, while
$\Sigma^{d-2}_{t}$ has positive $(d-2)$-Hausdorff measure for at most
countably many $t$, so the result follows. ∎

We finish our treatment of the regular set by investigating the regularity
improvement possible under the assumption that no singular points occur at
some time. As suggested by Remark 5.7, an assumption of this form is required
to go beyond the regularity established in Proposition 5.10. The idea here
will be to apply the regularity of $\nu$ from Proposition 5.6 in conjunction
with global Schauder estimates to prove time regularity of $p$ and higher
spatial regularity of $T$.

As a preliminary step, we show the relationship between $\nabla p$ and $\nabla
T$.

######  Lemma 5.18.

Suppose that for some open $U\subset R$, we have that $\nabla p$ is continuous
in spacetime on $(U\times(t_{0},t_{1}))\cap\overline{\\{(x,t):\rho(x,t)=1\\}}$
for some $t_{0},t_{1}$ with $\inf T(U)<t_{0}<t_{1}<\sup T(U)$. Then $T$ is
continuously differentiable on $U\cap T^{-1}((t_{0},t_{1}))$ with $\nabla
T(x)=-\frac{\nabla p(T(x),x)}{|\nabla p(T(x),x)|^{2}}$.

###### Proof.

Let $e$ be a vector with positive component in the inward normal direction to
$\Omega_{T(x)}$ at $x$; that is, with $e\cdot\nu(x)<0$. Then, we have

$\nabla
w(T(x),x+he)=\operatorname{sgn}_{+}(T(x)-T(x+he))\int_{T(x+he)}^{T(x)}\nabla
p(t,x+he)\,dt$

If we divide both sides by $h$ and let $h\to 0$, then the left side converges
to $(\nu(x)\cdot e)\nu(x)$, from the quadratic blowup. If the right side is
nonzero, we rewrite it as

$\frac{T(x)-T(x+he)}{h}\nabla p(T(x),x)+\frac{1}{h}\int_{T(x+he)}^{T(x)}\nabla
p(t,x+he)-\nabla p(T(x),x)\,dt$

Using the Lipschitz continuity of $T$ from Proposition 5.10 and the spacetime
continuity of $\nabla p$, the second term vanishes as $h\to 0$. As we have
already seen, $\nabla p$ cannot vanish on the interface due to the Hopf lemma,
so in the limit, we get

$(\nu(x)\cdot e)\nu(x)=-\partial_{e}T(x)\nabla p(T(x),x)$

Here, $\partial_{e}T(x)$ refers to the one-sided derivative of $T$ at $x$ in
direction $e$. Since $\nabla p(T(x),x)$ has the same direction as $-\nu(x)$,
we get that $\partial_{e}T(x)=\frac{\nu(x)\cdot e}{|\nabla p(T(x),x)|}$.

Then, it is an elementary result that a continuous function on ${\mathbb{R}}$
with continuous left derivative is differentiable. Applying it here, we get
that $T$ has all two-sided directional derivatives, and we can read from the
formula that we must have

$\nabla T(x)=\frac{\nu(x)}{|\nabla p(T(x),x)|}=-\frac{\nabla
p(T(x),x)}{|\nabla p(T(x),x)|^{2}}$

∎

######  Proposition 5.19.

If for some interval $(t_{0},t_{1})$, we have that $\Sigma_{t}$ is empty for
every $t\in(t_{0},t_{1})$, then $T\in
C^{1,1/2-\varepsilon}_{\operatorname{\textup{loc}}}(\Omega_{t_{1}}\setminus\overline{\Omega_{t_{0}}})$
for every $\varepsilon\in(0,\frac{1}{2})$.

###### Proof.

From the previous lemma, we need to show spacetime continuity of $\nabla p$ to
establish differentiability of $T$. We do this using the $C^{1,\alpha}$ global
Schauder estimates, which applied to a function $u$ on a $C^{1,\alpha}$ domain
$\Omega$, give that

$\|u\|_{C^{1,\alpha}(\overline{\Omega})}\leq C(\|\Delta
u\|_{L^{\infty}(\Omega)}+\|u\|_{C^{1,\alpha}(\partial\Omega)})$

for some $C$ which depends only on $\alpha$ and $\Omega$. In our case, $C$
will actually be locally uniform in $t$ for $\Omega_{t}$, since the global
Schauder estimates are proved by patching interior and boundary estimates, and
we can cover the boundary with finitely many balls in which it evolves as a
uniformly $C^{1,1}$ graph for some time interval.

Then, specifically, we will apply the Schauder estimate to $p(t)-p(s)$ on
$\Omega_{s}$, for $t>s$, to bound $\|\nabla p(t)-\nabla
p(s)\|_{L^{\infty}(\Omega_{s})}$ in terms of $|t-s|$. Since $n\in
C^{0,1-}_{t}L^{\infty}_{x}$ by Lemma 2.2, the work will be in controlling
$p(t)$ on $\partial\Omega_{s}$. Intuitively, since $p(t)$ and its tangential
derivative vanish on $\partial\Omega_{t}$, we expect that if the free boundary
has not rotated too much between times $s$ and $t$, then these should be close
to 0 on $\partial\Omega_{s}$. We make this quantitative using Proposition 5.6.

First, since we have a locally uniform in time bound on $\nabla p$, we get by
the radial supersolution that $D(\partial\Omega_{s},\partial\Omega_{t})\leq
C|t-s|$ for some locally uniform in time constant, where $D$ denotes Hausdorff
distance as in Definition 6.5. Then, since $p(t)$ vanishes on
$\partial\Omega_{t}$, we integrate along shortest-distance paths and use the
gradient bound again to conclude that
$\|p(t)-p(s)\|_{L^{\infty}(\partial\Omega_{s})}=\|p(t)\|_{L^{\infty}(\partial\Omega_{s})}\leq
C|t-s|$ for some uniform $C$.

Next, we bound the tangential part of $\nabla p(t)$ on $\partial\Omega_{s}$.
Recalling our definition of $\nu(x)$ as the outward unit normal to
$\Omega_{T(x)}$ at $x$, we denote the projection onto the tangential part as
$P^{\perp}_{\nu(x)}$. Then for $x\in\partial\Omega_{s}$, we let
$\tilde{x}\in\partial\Omega_{t}$ be the distance minimizer so that
$|x-\tilde{x}|\leq C|t-s|$, and we have

$\displaystyle|P^{\perp}_{\nu(x)}\nabla p(x)|$
$\displaystyle\leq|(P^{\perp}_{\nu(\tilde{x})}-P^{\perp}_{\nu(x)})\nabla
p(\tilde{x})|+|P^{\perp}_{\nu(x)}(\nabla p(\tilde{x})-\nabla p(x))|$

where all pressures are at time $t$, and we use that the tangential derivative
of $p(t)$ on $\partial\Omega_{t}$ vanishes. The first term is controlled by
the continuity of $\nu$ and our uniform bound on the pressure gradient, so by
Proposition 5.6, it contributes $C|t-s|^{1/2}$. The second term is controlled
by the $C^{1,\alpha}$ regularity of $p$, so it contributes $C|t-s|^{\alpha}$.
Thus, choosing $\alpha>\frac{1}{2}$, we have
$\|p(t)-p(s)\|_{C^{1}(\partial\Omega_{s})}\leq C|t-s|^{1/2}$.

Finally, we improve this to Hölder by interpolation. Specifically, since
$p(\tau)$ is $C^{1,\alpha}$ on $\Omega_{\tau}$, uniformly in $\tau$, for any
$\alpha\in(0,1)$, we have

$\frac{|\nabla p(t,x)-\nabla p(t,y)|}{|x-y|^{\alpha}}\leq C(\alpha)$

We want the Hölder seminorm of $\nabla p(t)$ on $\partial\Omega_{s}$ to be
small, so at small scales, we rearrange this to

$\frac{|\nabla p(t,x)-\nabla p(t,y)|}{|x-y|^{\beta}}\leq
C(\alpha)|x-y|^{\alpha-\beta}$

for $\beta<\alpha$ to be chosen. For large scales, we use the $L^{\infty}$
bound for the tangential part of $\nabla p(t)$ on $\partial\Omega_{s}$, which
gives $\frac{C|t-s|^{1/2}}{|x-y|^{\beta}}$ on the right hand side. Optimizing,
the critical scale is $|x-y|\sim|t-s|^{1/2\alpha}$, and the $C^{1,\beta}$
seminorm will scale as $C(\alpha)|t-s|^{\frac{\alpha-\beta}{2\alpha}}$. In
particular, this shows that by choosing $\alpha$ close to 1 and $\beta$ close
to 0, we can get arbitrarily close to $\frac{1}{2}$, so for each
$\varepsilon>0$, we have some $\beta>0$ and some $C$ such that

$\|p(t)-p(s)\|_{C^{1,\beta}(\partial\Omega_{s})}<C|t-s|^{1/2-\varepsilon}$

Then, combining this with the regularity of the nutrient from Lemma 2.2, we
get $\|p(t)-p(s)\|_{C^{1,\beta}(\overline{\Omega}_{s})}\leq
C|t-s|^{1/2-\varepsilon}$ from the boundary Schauder estimate. We conclude
that for points $(x,t)$ and $(y,s)$ in the region with $t>s$, we have

(5.10) $|\nabla p(x,t)-\nabla p(y,s)|\leq|\nabla p(x,t)-\nabla p(y,t)|+|\nabla
p(y,t)-\nabla p(y,s)|\leq C|x-y|^{\alpha}+C|t-s|^{1/2-\varepsilon}$

for any fixed $\alpha\in(0,1)$ and $\varepsilon>0$, which proves the spacetime
continuity of $\nabla p$.

Then, by the lemma, $\nabla T(x)=-\frac{\nabla p(x,T(x))}{|\nabla
p(x,T(x))|^{2}}$, where we have $|\nabla p(x,T(x))|$ locally uniformly bounded
away from 0 by the Hopf lemma. In particular, it follows from (5.10) and the
Lipschitz continuity of $T$ that

$|\nabla p(x,T(x))-\nabla p(y,T(y))|\leq C|x-y|^{1/2-\varepsilon}$

so we obtain that $\nabla T\in
C^{0,1/2-\varepsilon}_{\operatorname{\textup{loc}}}$ directly from its
formula. ∎

This implies that the free boundary has $C^{2,1/2-\varepsilon}$ regularity at
the relevant times. Subsequently, the regularity of $\nu$ in Proposition 5.6
can be improved using second order approximations to the free boundary,
leading to a minor improvement in the Hölder exponent.

##  6\. Appendix: Obstacle problem with $C^{0,\alpha}$ source

In this section, we collect several known facts about obstacle problems with
Hölder continuous data. Many results for the model obstacle problem with
constant source carry over to the Hölder continuous case with minor
modifications, as noted in [Caf98] and [Wei99]. We will cite several results
from [Bla00], [Mon03], and [CSV18], which offer careful treatments of this
topic.

Let us consider solutions $u$ to the obstacle problem

(6.1) $\Delta u=(1+f)\chi_{\\{u>0\\}}$

on $B_{1}$, where $f$ is known a priori to vanish on the free boundary
$\Gamma(u)=\partial\\{u>0\\}$. When $f$ is sufficiently regular, this equation
has similar local behavior to the model case where $f\equiv 0$; in particular,
we make the assumption $f\in C^{0,\alpha}(B_{1})$ for some $\alpha\in(0,1)$.
Regarding notation, we write $\Omega(u)=\\{u>0\\}$, $\Lambda(u)=\\{u=0\\}$,
and in many cases we will refer to a tuple $(u,f)$ as the solution to (6.1).
We take $\lambda=\inf_{B_{1}}1+f,\mu=\sup_{B_{1}}1+f$, and we will have the
standing assumption that $\lambda>\frac{1}{2}$, which holds if $0\in\Gamma(u)$
and $[f]_{C^{0,\alpha}(B_{1})}$ is sufficiently small.

On the free boundary, we have $u=0,\nabla u=0$, so we should expect $u$ to
have quadratic growth away from the free boundary in the positive set. Of
course, $u$ is not regular enough to admit a second order Taylor expansion due
to the jump in the second derivatives along the free boundary, but
nevertheless, we recover several results to the same effect:

######  Lemma 6.1 (Quadratic nondegeneracy, [Bla00] Thm. 2.1).

If $0\in\overline{\Omega(u)}$, then for all $r<1$,

$\sup_{B_{r}}u\geq\frac{\lambda}{2d}r^{2}$

######  Lemma 6.2 (Quadratic bound, [Bla00] Thm. 2.4).

If $0\in\Gamma(u)$, then for all $r<\frac{1}{2}$,

$\sup_{B_{r}}u\leq C(d)\mu r^{2}$

######  Lemma 6.3 (Regularity up to the free boundary, [Bla00] Thm. 2.3).

If $0\in\Gamma(u)$, then $\|u\|_{C^{1,\beta}(B_{1})}\leq C(d,\beta)\mu$ for
all $\beta\in(0,1)$.

We adopt the following notation for the quadratic rescalings:

(6.2) $u_{r}(x)=r^{-2}u(rx)$ (6.3) $u_{0}(x)=\lim_{r\to 0^{+}}u_{r}(x),\hbox{
provided this limit exists }$

The combined results of lemmas 6.1, 6.2, and 6.3 can be used to derive Lemma
2.10: the $C^{1,\beta}$ compactness of the quadratic blowup sequence
$(u_{r})$. As we discuss in the main paper with Lemma 5.2, this compactness
improves to convergence of the blowup sequence when $f\in C^{0,\alpha}$, and
we classify points as regular or singular based on the blowup profile. In
general, regular points can be identified at finite scales, using criteria
such as Lemma 5.3.

The quadratic blowup proves to be a key tool in understanding local behavior
of the free boundary. For regular points, we use comparison and stability
results to show flatness of the free boundary, which leads to regularity of
the boundary. For singular points, we use monotonicity formulas and
compactness results to show that they can be locally contained in $C^{1}$
manifolds. Since the treatment of these cases diverges considerably, we will
split them into the next two sections.

###  6.1. Regular points

In this section, we review several results from [Bla00] which connect the
regularity of the free boundary at regular points and the scale at which this
regularity is achieved with the regularity of the source term and the scale at
which the zero set becomes large. The regularity of the free boundary can be
summarized as follows:

######  Lemma 6.4 ([Bla00] Thm. 7.2).

If $f\in C^{0,\alpha}$ with $\alpha\in(0,1]$, then in a neighborhood of a
regular point, the free boundary is a $C^{1,\alpha}$ graph.

In light of this result, we allow the case $\alpha=1$ for the rest of this
subsection.

For the main paper, we require a quantified version of this lemma, in order to
apply it uniformly to the family of obstacle problems $w(\cdot,t)$. Thus, we
will retrace Blank’s approach in this section while keeping track of its
dependencies.

######  Definition 6.5.

Let $S\subset{\mathbb{R}}^{d}$ be a compact set. We define the modulus of
flatness,

$\theta(r)=\sup_{0<\rho\leq r}\sup_{x\in S}\inf_{L}\frac{D(L\cap
B_{\rho}(x),S\cap B_{\rho}(x))}{\rho}$

where the inner infimum is over all hyperplanes $L$ containing $x$, and $D$
denotes Hausdorff distance:

$D(A,B)=\max(\sup_{x\in A}\operatorname{dist}(x,B),\sup_{y\in
B}\operatorname{dist}(y,A))$

We say that $S$ is $\delta$-Reifenberg flat if there exists $R$ such that
$\theta(r)\leq 2\delta$ for all $r<R$, and Reifenberg vanishing if
$\theta(r)\to 0$.

######  Lemma 6.6 ([Bla00] Theorem 6.7).

Let $S$ be a compact Reifenberg vanishing set with modulus of flatness
$\theta$ satisfying $\int_{0}^{1}\frac{\theta(r)}{r}\,dr<\infty$. Then there
exist constants $C_{0},C_{1}$ such that if
$\int_{0}^{\rho}\frac{\theta(r)}{r}\,dr<C_{0}$, then there exists a coordinate
system in which $S\cap B_{\rho/2}$ is the graph of a $C^{1}$ function $g$,
such that $\nabla g$ is continuous with modulus of continuity
$C_{1}\int_{0}^{r}\frac{\theta(s)}{s}\,ds$.

######  Lemma 6.7 ([Bla00] Theorem 7.1).

Suppose that $u$ solves the obstacle problem $\Delta u=f\chi_{\\{u>0\\}}$ in
$B_{1}$ with $\lambda\leq f\leq\mu$ and $f$ Dini continuous with modulus
$\sigma$. Then if $0$ is a regular point and the free boundary is
$\delta$-Reifenberg flat in $B_{3/4}$ for some sufficiently small $\delta$,
then the modulus of flatness of the free boundary inside $B_{1/2}$ is
controlled by $C\sigma(r)$.

In particular, these two results imply the $C^{1,\alpha}$ regularity of the
free boundary near regular points when $f\in C^{0,\alpha}$ with $f(0)=1$, at a
scale depending on $[f]_{C^{0,\alpha}}(B_{1})$ and the scale at which the
$\delta$ Reifenberg flatness is achieved. For this, we have another result
from Blank

######  Lemma 6.8 ([Bla00] Theorem 6.4).

Let $\varepsilon\in(0,\frac{1}{4})$, and suppose we have $f$ with $\lambda\leq
f\leq\mu$ and $u$ a solution to the obstacle problem $\Delta
u=f\chi_{\\{u>0\\}}$ in $B_{1}$. If $\mu-\lambda$ is sufficiently small, then
there exist constants $r_{0},\tau,\delta\in(0,1)$ depending on
$d,\mu,\lambda,\varepsilon$ for which the following holds:

If for some $t\leq r_{0}$, we have

$\frac{|B_{t}\cap\\{u=0\\}|}{|B_{t}|}>\varepsilon,$

then $\overline{B_{\tau t}}\cap\partial\\{u>0\\}$ is $\delta$-Reifenberg flat.

Moreover, as $\mu-\lambda\to 0$, $\delta\to 0$. In particular, if $f$ is
continuous, $\delta$ can be taken to be arbitrarily small (with all parameters
now also depending on the modulus of continuity of $f$).

To be more precise, suppose $u$ solves the obstacle problem $\Delta
u=f\chi_{\\{u>0\\}}$ in $B_{1}$ with $f$ taking values in $[\lambda,\mu]$, and
write $u_{c}$ for the solution to the obstacle problem $\Delta
u_{c}=c\chi_{\\{u_{c}>0\\}}$ such that $u_{c}|_{\partial B_{1}}=u$. Then
$\\{u_{\lambda}=0\\}\subset\\{u=0\\}\subset\\{u_{\mu}=0\\}$. Moreover, if $0$
is a regular point for $u$, then there exists $c\in[\lambda,\mu]$ such that
$0$ is a regular point for $u_{c}$. Then Blank’s uniform stability theorem for
regular points ([Bla00] Theorem 5.4) gives that in $B_{1/2}$, there is a
universal $C$ such that for any $c^{\prime}$, we have
$d(FB(u_{c}),FB(u_{c^{\prime}}))\leq C|c-c^{\prime}|$ where $FB(v)$ denotes
the free boundary of $v$. Then we get flatness of $FB(u)$ by trapping it
between $FB(u_{\lambda})$ and $FB(u_{\mu})$ and using the stability and
$C^{1,\alpha}$ regularity of the constant-source free boundaries. As we zoom
in, the $C^{1,\alpha}$ seminorm goes to 0, and if $f$ is continuous,
$|\mu-\lambda|\to 0$, so we can get $\delta$-Reifenberg flatness with
arbitrarily small $\delta$. In particular, the $C^{1,\alpha}$ seminorm is
uniformly bounded, depending only on the scale at which the density of the
zero set is sufficiently large, and the rate at which $|\mu-\lambda|\to 0$
depends only on the modulus of continuity of $f$. Thus, we can replace the
hypothesis of $\delta$-Reifenberg flatness in Lemma 6.6, to conclude:

######  Lemma 6.9.

Suppose that $(u,f)$ solve (6.1) with $f\in C^{0,\alpha}$ for
$\alpha\in(0,1]$. Then there exist $r_{0},\varepsilon_{0}$ such that if $0$ is
a free boundary point and
$\frac{|B_{t}\cap\\{u=0\\}|}{|B_{t}|}>\varepsilon_{0}$ for some $t\leq r_{0}$,
then there exists $r=r(t,[f]_{C^{0,\alpha}})$ such that the free boundary is a
$C^{1,\alpha}$ graph in $B_{r}$, with $C^{1,\alpha}$ seminorm controlled by
$[f]_{C^{0,\alpha}(B_{1})}$ and $r$.

###  6.2. Singular points

In this section, we use monotonicity formulas to study the continuity of the
blowup limit at singular points and the rate of convergence for the blowup
sequence. First, we have the following result:

######  Lemma 6.10 ([CSV18], Thm 5).

Restricted to the singular set, $D^{2}u$ is continuous with a logarithmic
modulus of continuity. In particular, in a neighborhood of a singular point,
the singular set is contained in a $C^{1,\log}$ manifold of dimension
$\dim\ker D^{2}u$.

The result of [CSV18] is obtained using an epiperimetric inequality to control
the Weiss monotonocity formula, introduced in [Wei99]. For simplicity, we will
instead consider the related Monneau monotonicity formula, at the cost of the
explicit logarithmic modulus of continuity. The following result is part of
the proof of [Mon03] Theorem 1.9.

######  Lemma 6.11.

Define

(6.4) $\Xi_{u}^{q}(r)=r^{-(d+3)}\int_{\partial B_{r}}(u-q)^{2}=\int_{\partial
B_{1}}(u_{r}-q)^{2}$

where $u$ solves (6.1) with $0$ as a singular point, and $q$ is a quadratic
form $q(x)=\frac{1}{2}x\cdot Qx$ with $Q\geq 0,trQ=1$. Then

$\frac{d}{dr}\Xi_{u}^{q}(r)\geq-Cr^{\alpha-1}$

where $C=C([f]_{C^{0,\alpha}})$. In particular, the limit $\Xi_{u}^{q}(0+)$
exists.

As a corollary, we can show that near a singular point, $u$ approximates a
global solution at a uniform scale. This extends Lemma 13 of [Caf98] to the
$C^{0,\alpha}$ source case.

We break the proof into two steps. First, we will prove the following slightly
weaker claim:

######  Lemma 6.12.

Let $u$ solve (6.1) with 0 as a singular point. Then for every
$\varepsilon>0$, there exists $\delta=\delta(\varepsilon,[f]_{C^{\alpha}})$
such that $\|u_{\delta}-q\|_{C^{1}(B_{1})}<\varepsilon$, for some $q$ of the
form $q(x)=\frac{1}{2}x\cdot Qx$ with $Q$ a positive semidefinite matrix of
trace 1. Here, we use the notation defined in (6.2).

###### Proof.

Suppose for contradiction that we can find a sequence $v^{k}$ solving $\Delta
v^{k}=(1+f^{k})\chi_{\\{v^{k}>0\\}}$ on $B_{1}$ such that the result fails
along the sequence $v^{k}_{1/k}$. That is, for each $k$, and every $q$ of the
form in the statement of the lemma,

(6.5) $\|v^{k}_{1/k}-q\|_{C^{1}(B_{1})}>\varepsilon$

Then the sequence $v^{k}_{1/k}$, defined on the expanding balls $B_{k}$,
converges along a subsequence on all compact sets in $C^{1}$, to some
$v^{\infty}$. Since the $f^{k}$ are uniformly $C^{\alpha}$, the sequence
$f^{k}(\frac{x}{k})$ converges locally uniformly to 0. It follows that
$v^{\infty}$ is a nonnegative solution to $\Delta
v^{\infty}=\chi_{\\{v^{\infty}>0\\}}$ on ${\mathbb{R}}^{d}$ with
$v^{\infty}(0)=0$.

Next, we show that the zero set of $v^{\infty}$ has empty interior. Suppose
otherwise, and we have some $B_{r}(x)\subset B_{1}$ such that
$v^{\infty}\equiv 0$ on $B_{r}(x)$. This implies that $v^{k}_{1/k}$ is $o(1)$
on $\partial B_{r}(x)$ as $k\to\infty$, and an application of the
nondegeneracy bound Lemma 6.1 yields that $v^{k}_{1/k}\equiv 0$ on
$B_{r/2}(x)$ for $k$ sufficiently large.

Then, since the zero set of $v^{\infty}$ has empty interior, all points in the
zero set are singular free boundary points, and $v^{\infty}$ is twice
differentiable at those points. In particular, we get that $v^{\infty}$ solves
$\Delta v^{\infty}=1$. As noted in the remarks after Lemma 13 of [Caf98],
this, along with the quadratic growth estimate, Lemma 6.2, implies that
$v^{\infty}$ is a quadratic polynomial of the type in the statement above.
Thus, taking $q=v^{\infty}$, we get a contradiction to (6.5) for $k$
sufficiently large, which completes the proof. ∎

######  Lemma 6.13.

For every $\varepsilon>0$, there exists
$\delta=\delta(\varepsilon,[f]_{C^{0,\alpha}(B_{1})})$ such that
$\|u_{\delta}-u_{0}\|_{C^{1}(B_{1})}<\varepsilon$, where we use the notation
of (6.2). Equivalently, we have
$\|u-u_{0}\|_{C^{1}(B_{\delta})}<o(\delta^{2})$ as $\delta\to 0$, where the
little $o$ depends only on $[f]_{C^{0,\alpha}}(B_{1})$.

###### Proof.

Let $q$ be the quadratic form given by the previous lemma. We apply Monneau’s
monotonicity formula, $\Xi_{u}^{q}$, defined in (6.4). The derivative bound
from Lemma 6.11 gives

$\Xi_{u}^{q}(0)-\Xi_{u}^{q}(\delta)\leq C\delta^{\alpha}$

We have $\|u_{\delta}-q\|_{C^{1}(B_{1})}<\varepsilon$ from the lemma, so it
follows that $\|u_{\delta}-q\|_{L^{2}(\partial B_{1})}<C\varepsilon$ for some
dimensional constant. Then $\|u_{0}-q\|_{L^{2}(\partial B_{1})}^{2}\leq
C\varepsilon^{2}+C\delta^{\alpha}$. But now we recall that $u_{0},q$ are both
quadratic forms, and so by equivalence of norms on ${\mathbb{R}}^{d\times d}$,
there exists a dimensional constant for which $\|u_{0}-q\|_{C^{1}(B_{1})}\leq
C\|u_{0}-q\|_{L^{2}(\partial B_{1})}$. Combining our estimates for
$\|u_{0}-q\|_{C^{1}(B_{1})}$ and $\|u_{\delta}-q\|_{C^{1}(B_{1})}$, we
conclude. ∎

## References

  * [AB79] D. G. Aronson and P. Bénilan.  Régularité des solutions de l’équation des milieux poreux dans $r^{n}$.  C. R. Acad. Sci. Paris Sér. A–B., 288(2), 1979.
  * [BCMP73] C. Baiocchi, V. Comincioli, E. Magenes, and G. A. Pozzi.  Free boundary problems in the theory of fluid flow through porous media: Existence and uniqueness theorems.  Annali di Matematica Pura ed Applicata, 97(1):1–82, Dec 1973.
  * [BKMP03] H.M. Byrne, J.R. King, D.L.S. McElwain, and L. Preziosi.  A two-phase model of solid tumour growth.  Applied Mathematics Letters, 16(4):567–573, 2003.
  * [Bla00] Ivan Alexander Blank.  Sharp results for the regularity and stability of the free boundary in the obstacle problem.  New York University, 2000.
  * [BPS22] Giulia Bevilacqua, Benoît Perthame, and Markus Schmidtchen.  The aronson–bénilan estimate in lebesgue spaces.  Annales de l'Institut Henri Poincaré C, Analyse non linéaire, 40(2):259–286, July 2022.
  * [Caf98] Luis A Caffarelli.  The obstacle problem revisited.  Journal of Fourier Analysis and Applications, 4(4):383–402, 1998\.
  * [CF80] Luis A Caffarelli and Avner Friedman.  Regularity of the free boundary of a gas flow in an n-dimensional porous medium.  Indiana University Mathematics Journal, 29(3):361–391, 1980.
  * [CG15] Pierre Cardaliaguet and P. Jameson Graber.  Mean field games systems of first order.  ESAIM: Control, Optimisation and Calculus of Variations, 21(3), 2015\.
  * [CJK07] Sunhi Choi, David Jerison, and Inwon Kim.  Regularity for the one-phase hele-shaw problem from a lipschitz initial surface.  American journal of mathematics, 129(2):527–582, 2007.
  * [CJK09] Sunhi Choi, David Jerison, and Inwon Kim.  Local regularization of the one-phase hele-shaw flow.  Indiana University mathematics journal, pages 2765–2804, 2009.
  * [CSV18] Maria Colombo, Luca Spolaor, and Bozhidar Velichkov.  A logarithmic epiperimetric inequality for the obstacle problem.  Geometric and Functional Analysis, 28:1029–1061, 2018.
  * [DGN21] Hongjie Dong, Francisco Gancedo, and Huy Q Nguyen.  Global well-posedness for the one-phase muskat problem.  arXiv preprint arXiv:2103.02656, 2021.
  * [DP21] Noemi David and Benoît Perthame.  Free boundary limit of a tumor growth model with nutrient.  Journal de Mathématiques Pures et Appliquées, 155:62–82, 2021\.
  * [FROS20] Alessio Figalli, Xavier Ros-Oton, and Joaquim Serra.  Generic regularity of free boundaries for the obstacle problem.  Publications mathématiques de l’IHÉS, 132:181–292, 2020\.
  * [FS19] Alessio Figalli and Joaquim Serra.  On the fine structure of the free boundary for the classical obstacle problem.  Inventiones mathematicae, 215(1):311–366, 2019.
  * [GDSK22] Alexander Golden, Ilija Dukovski, Daniel Segrè, and Kirill S Korolev.  Growth instabilities shape morphology and genetic diversity of microbial colonies.  Physical Biology, 19(5):056005, 2022.
  * [GKM22] Nestor Guillen, Inwon Kim, and Antoine Mellet.  A hele-shaw limit without monotonicity.  Archive for Rational Mechanics and Analysis, 243(2):829–868, Feb 2022.
  * [GPŚG19] Piotr Gwiazda, Benoît Perthame, and Agnieszka Świerczewska-Gwiazda.  A two-species hyperbolic–parabolic model of tissue growth.  Communications in Partial Differential Equations, 44(12):1605–1618, 2019.
  * [Jac21] Matt Jacobs.  Existence of solutions to reaction cross diffusion systems, 2021.
  * [Jac23] Matt Jacobs.  Lagrangian solutions to the porous media equation and reaction diffusion systems, 2023.
  * [JKT23] Matt Jacobs, Inwon Kim, and Jiajun Tong.  Tumor growth with nutrients: Regularity and stability.  Communications of the American Mathematical Society, 3(4):166–208, May 2023.
  * [Kit97] So Kitsunezaki.  Interface dynamics for bacterial colony formation.  Journal of the Physical Society of Japan, 66(5):1544–1550, 1997\.
  * [KZ21] Inwon Kim and Yuming Paul Zhang.  Porous medium equation with a drift: Free boundary regularity.  Archive for Rational Mechanics and Analysis, 242(2):1177–1228, 2021\.
  * [LN07] YanYan Li and Louis Nirenberg.  On the Hopf lemma.  Available at https://arxiv.org/abs/0709.3531v1, 2007.
  * [Mon03] R. Monneau.  On the number of singularities for the obstacle problem in two dimensions.  The Journal of Geometric Analysis, 13(2):359–389, 2003.
  * [MPQ17] Antoine Mellet, Benoît Perthame, and Fernando Quirós.  A hele–shaw problem for tumor growth.  Journal of Functional Analysis, 273(10):3061–3093, 2017.
  * [MRCS10] Bertrand Maury, Aude Roudneff-Chupin, and Filippo Santambrogio.  A macroscopic crowd motion model of gradient flow type, 2010.
  * [MRCS14] Bertrand Maury, Aude Roudneff-Chupin, and Filippo Santambrogio.  Congestion-driven dendritic growth.  Discrete Contin. Dyn. Syst, 34(4):1575–1604, 2014.
  * [MVS02] Judith Müller and Wim Van Saarloos.  Morphological instability and dynamics of fronts in bacterial growth models with nonlinear diffusion.  Physical Review E, 65(6):061111, 2002.
  * [OS22] Takayoshi Ogawa and Senjo Shimizu.  Maximal regularity for the cauchy problem of the heat equation in bmo.  Mathematische Nachrichten, 295(7):1406–1442, 2022.
  * [PQV14] Benoît Perthame, Fernando Quirós, and Juan Luis Vázquez.  The hele–shaw asymptotics for mechanical models of tumor growth.  Archive for Rational Mechanics and Analysis, 212(1):93–127, Apr 2014.
  * [PSU12] Arshak Petrosyan, Henrik Shahgholian, and Nina Uraltseva.  Regularity of Free Boundaries in Obstacle-Type Problems.  American Mathematical Society, 2012.
  * [PT08] Luigi Preziosi and Andrea Tosin.  Multiphase modelling of tumour growth and extracellular matrix interaction: mathematical tools and applications.  Journal of Mathematical Biology, 58(4-5):625–656, October 2008\.
  * [RBE+10] Jonas Ranft, Markus Basan, Jens Elgeti, Jean-François Joanny, Jacques Prost, and Frank Jülicher.  Fluidization of tissues by cell division and apoptosis.  Proceedings of the National Academy of Sciences, 107(49):20863–20868, 2010.
  * [SS17] Sylvia Serfaty and Joaquim Serra.  Quantitative stability of the free boundary in the obstacle problem.  Analysis & PDE, 11, 2017.
  * [Váz07] Juan Luis Vázquez.  The porous medium equation: mathematical theory.  Oxford University Press, 2007.
  * [Wei99] George S. Weiss.  A homogeneity improvement approach to the obstacle problem.  Inventiones Mathematicae, 138(1):23–59, 1999.


# Smartphone Impostor Detection with Behavioral Data Privacy and Minimalist
Hardware Support

Guangyuan Hu  Princeton UniversityPrincetonNJUSA<EMAIL_ADDRESS>,  Zecheng
He  Princeton UniversityPrincetonNJUSA<EMAIL_ADDRESS>and  Ruby B. Lee
Princeton UniversityPrincetonNJUSA<EMAIL_ADDRESS>
(2021)

###### Abstract.

Impostors are attackers who take over a smartphone and gain access to the
legitimate user’s confidential and private information. This paper proposes a
defense-in-depth mechanism to detect impostors quickly with simple Deep
Learning algorithms, which can achieve better detection accuracy than the best
prior work which used Machine Learning algorithms requiring computation of
multiple features. Different from previous work, we then consider protecting
the privacy of a user’s behavioral (sensor) data by not exposing it outside
the smartphone. For this scenario, we propose a Recurrent Neural Network (RNN)
based Deep Learning algorithm that uses only the legitimate user’s sensor data
to learn his/her normal behavior. We propose to use Prediction Error
Distribution (PED) to enhance the detection accuracy. We also show how a
minimalist hardware module, dubbed SID for Smartphone Impostor Detector, can
be designed and integrated into smartphones for self-contained impostor
detection. Experimental results show that SID can support real-time impostor
detection, at a very low hardware cost and energy consumption, compared to
other RNN accelerators.

Impostor Detection, Deep Learning, Hardware Support, Security, User
Authentication, Sensors

††copyright: rightsretained††journalyear: 2021

##  1\. Introduction

Smartphone theft is one of the biggest threats to smartphone users (Wang et
al., 2012). Impostors are defined as adversaries who take over a smartphone
and perform actions allowed only for the legitimate smartphone owners.
Impostor attacks breach the confidentiality, privacy and integrity of the
sensitive personal information stored in the smartphone, and accessible online
through the smartphone. As powerful attackers may already know, or can bypass,
the legitimate smartphone user’s password or personal identification number
(PIN), can a defense-in-depth mechanism be provided to detect impostors
quickly before further damage is done? Can the detection be done implicitly,
using behavioral biometrics, like how a user moves or uses the phone?

The ubiquitous inclusion of motion sensors, e.g., the 3-axis accelerometer and
the 3-axis gyroscope, in smartphones provides a great opportunity to capture a
user’s large and small motion patterns. In the literature, implicit smartphone
user authentication with sensors is primarily modeled as a binary
classification problem (Lee and Lee, 2017; Ehatisham-ul Haq et al., 2017; Lee
and Lee, 2015; Yang et al., 2014; Zheng et al., 2014; Nickel et al., 2012;
Frank et al., 2012). While these can also be leveraged as implicit impostor
detection systems, the problem is these past work require both the legitimate
user’s sensor data and other users’ sensor data for model training. This
causes serious privacy issues as users must provide their sensitive behavioral
data to a centralized training system. The privacy protection of a user’s
behavioral biometric data in implicit impostor detection (conversely, implicit
user authentication) has not been investigated in the past, which we do in
this paper.

The privacy of a user’s biometric data can be preserved if it does not need to
be sent to the cloud for training with other users’ data. Hence, we propose
using only the legitimate user’s data to train a Recurrent Neural Network
(RNN) to learn the user’s normal behavior. A large deviation of the currently
observed behavior from the model’s prediction indicates that the smartphone is
not being used by the legitimate user, and hence it is probably being used by
an impostor. To achieve high detection accuracy, we further propose comparing
the model’s prediction error distributions. We show that this can
significantly improve the detection accuracy in one-class deep learning
scenarios like ours.

To reduce the attack surface and the time taken for impostor detection, we
design a small and energy-efficient hardware module for impostor detection on
the smartphone. While previous ML/DL accelerators try to maximize the
performance of one or a few specific ML or DL algorithms (e.g. RNN), which is
only a part of the detection process, our goal is to support the end-to-end
detection process and provide sufficient performance at low cost and low power
consumption.

Our Smartphone Impostor Detector (SID) module is flexible in that it supports
not only the best deep learning algorithms we found for impostor detection in
both scenarios (with and without other users’ data for training) but also
other Machine learning (ML) and Deep Learning (DL) algorithms. Furthermore, it
accelerates the computation of empirical probability distributions and
statistical tests. SID reuses functional units where possible to reduce
hardware size and cost. It is also scalable for higher performance if more
parallel datapath tracks are implemented. Programmability provides flexibility
in selecting algorithms and adjusting trade-offs in security, privacy,
usability and costs, e.g., execution time, memory requirements and energy
consumption. Our key contributions are:

  * $\bullet$

We propose privacy-preserving smartphone detection algorithms that detect
impostors and implicitly authenticate users, while protecting their behavioral
data privacy.

  * $\bullet$

We show that we can significantly improve the accuracy of a deep learning
algorithm (LSTM or other RNN) for detecting abnormal users (i.e., impostors)
by comparing prediction error distributions.

  * $\bullet$

When both the user and non-users’ data are used for centralized (2-class)
model training, we show that a simple deep learning algorithm, MLP, can
outperform the previous best implicit user authentication algorithm, KRR with
honed feature selection.

  * $\bullet$

We propose a new light-weight hardware module, SID, that is versatile,
scalable and efficient for performing impostor detection without preprocessing
or postprocessing on the CPU or other devices. Sensor data do not need to be
stored and transmitted through the network, thus significantly reducing user
data exposure.

##  2\. Threat Model and Assumptions

Our threat model considers powerful attackers who can bypass the conventional
explicit authentication mechanisms, e.g. password or personal identification
number (PIN). For example, the PIN/password may not be strong enough. The
attacker can actively figure out the weak pin/password by guessing or social
engineering. Another example is the attacker taking the phone after the
legitimate user has entered his password.

We assume the smartphone has common, built-in motion sensors, e.g. the
accelerometer and the gyroscope. We assume that the sensor readings are always
available. We explicitly consider protecting the privacy of smartphone users’
behavioral data. We assume that due to privacy concerns, many smartphone users
are not willing to send their sensor data to a centralized authentication
service for joint model training with other users’ data. We consider both
scenarios, where training is done with or without other users’ data.

While this paper assumes a single legitimate user of a smartphone, our
detection methodology can be extended to allow multiple legitimate users to
share a smartphone by deploying multiple models trained for different users.

##  3\. Algorithms for Impostor Detection

For impostor detection, we explicitly consider three important factors: attack
detection capability (security), usability and user data privacy of the
solution. We first show that Deep Learning algorithms can work better than the
best past work using sensors and Machine Learning, e.g. (Lee and Lee, 2017),
in the conventional 2-class classification scenario (Section 3.1). We then
explore the privacy-preserving scenario (Section 3.2) where only the
legitimate user’s data is used for training.

###  3.1. Two-class Algorithms and Metrics

A good impostor detection solution needs to be able to detect suspicious
impostors while not affecting the usability of the smartphone owner. At the
center of this trade-off is selecting an appropriate algorithm for impostor
detection. One of our key takeaways is that choosing the right algorithm (and
model) is more important for achieving security and performance goals than
increasing the model size or adding hardware complexity to accelerate a model.

Previous work on implicit smartphone user authentication mostly leverage
(user, non-user) binary classification techniques (Frank et al., 2012; Zheng
et al., 2014; Lee and Lee, 2017). This scenario requires both data from the
real user and other users for training, and we call it the Impostor Detection-
as-a-Service (IDaaS) scenario in this paper. For a specific customer, the data
from him/herself are labeled as benign or negative (the user), while all data
from other customers are labeled as malicious or positive (non-users). We
select certain classification-based algorithms that give the best accuracy for
impostor detection (security) and legitimate user recognition (usability) in
the non-privacy-preserving IDaaS scenario. We treat them as benchmarks when
comparing with our privacy-preserving impostor detection algorithms.

Among the many Machine Learning (ML) algorithms we investigated, we report the
results on Support Vector Machine (SVM) and Kernel Ridge Regression (KRR). SVM
is a powerful and commonly used linear model, which can establish a non-linear
boundary using the kernel method. KRR alleviates over-fitting by penalizing
large parameters and also achieves the highest detection rate in the
literature (Lee and Lee, 2017) while requiring the computation of 14
heuristically chosen features. Surprisingly, we show that even a simple Deep
Learning algorithm (Multi-layer perceptron, MLP) without heuristic and tedious
hand-crafting, can outperform it. MLP is a family of feed-forward neural
network models, consisting of an input layer, an output layer and one or more
hidden layers in between.

Metrics. While security is commonly measured as FNR, the percentage of actual
attacks that are not detected, we use the inverse term TPR, the percentage of
attacks that are detected. Similarly, while usability is commonly measured in
FPR, the percentage of normal user attempts that are incorrectly detected as
attacks, we use the inverse term TNR, which is the percentage of normal user
attempts detected as normal. TPR and TNR enable us to have metrics where
higher is always better.

Eq (1) gives the formula for TPR and TNR, as well as for the other metrics
commonly used in comparing the ML/DL models: accuracy, recall, precision and
F1. Accuracy is the percentage of correctly identified samples over all
samples. Recall, i.e., TPR, is the percent of all attacks that are detected
whereas precision is the percent of all reported attacks that are real
attacks. F1 is the harmonic mean of recall and precision.

(1)

$\begin{aligned} &TNR=\frac{TN}{TN+FP}\\\ &TPR=Recall,R=\frac{TP}{TP+FN}\\\
&Accuracy=\frac{TN+TP}{TN+FP+TP+FN}\\\ &Precision,P=\frac{TP}{TP+FP}\\\
&F_{1}\ Score=\frac{2\times Recall\times
Precision}{Recall+Precision}\end{aligned}$

###  3.2. Protecting Behavioral Data Privacy

The above binary classification approaches can only be applied to the IDaaS
scenario where the data from other users are available. However, smartphone
users may not be willing to send their sensor data to a centralized service
for training, due to behavioral data privacy concerns. Therefore, we need to
consider another important scenario where the smartphone user only has his/her
own data for training. We call this the local anomaly detection (LAD)
scenario.

We consider two representative algorithms for one-class learning, i.e. One-
Class SVM (OCSVM) and Long Short-Term Memory (LSTM), an RNN algorithm. We
propose enhancing the LSTM-based deep learning models with the comparison of
reference and actual Prediction Error Distributions (PEDs). We show that
generating and comparing the prediction error distributions is the key to a
successful detection for this LAD scenario.

OCSVM. OCSVM is an extension of normal SVM, by separating all the data points
from the origin in the feature space and maximizing the distance from this
hyperplane to the origin. Intuitively, the OCSVM looks for the minimum support
of the normal data, and recognizes points outside this support as anomalies.

LSTM. Different from the above discussed stateless models (SVM, KRR, OCSVM,
etc.), the LSTM model has two hidden states, $h_{t}$ and $c_{t}$, which can
remember the previous input information. An LSTM cell updates its hidden
states ($h_{t}$, $c_{t}$) using the previous states ($h_{t-1}$, $c_{t-1}$) and
the current input $x_{t}$ as described in Eq (2), where the W’s and U’s are
weight matrices, and the b’s are bias vectors. $tanh^{\prime}s$ and
$\sigma^{\prime}s$ are activation functions.

(2)

$\begin{aligned} &cand_{t}=tanh(W_{c}\times x_{t}+U_{c}\times
h_{t-1}+b_{c})\\\ &f_{t}=\sigma(W_{f}\times x_{t}+U_{f}\times
h_{t-1}+b_{f})\\\ &i_{t}=\sigma(W_{i}\times x_{t}+U_{i}\times
h_{t-1}+b_{i})\\\ &o_{t}=\sigma(W_{o}\times x_{t}+U_{o}\times
h_{t-1}+b_{o})\\\ &c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot cand_{t}\\\
&h_{t}=o_{t}\odot tanh(c_{t})\end{aligned}$

We use an LSTM-based model as an outlier detector (Malhotra et al., 2015), by
training it to predict the next sensor reading, and investigating the
prediction errors. The intuition is that an LSTM model trained on only the
normal user’s data predicts better for his/her behavior than for other users’
behavior. The deviations of the actual monitored behavior from the predicted
behavior indicate anomalous behavior. Typically, a threshold value is used to
decide if the prediction error is normal or not.

LSTM + Comparing Prediction Error Distributions (PEDs). Inspired by our
previous work in another domain (He et al., 2019), our intuition is that a
single prediction error may vary significantly, but the probability
distribution of the errors is more stable. Therefore, comparing the observed
PED and a reference PED from the real user’s validation data is more stable
than comparing the average prediction error with a pre-calculated threshold.
With PED, very accurate DL prediction from the LSTM model is not essential.

As we do not need to assume the prior distribution of PED, non-parametric
tests are powerful tools to determine if two distributions are the same. The
Kolmogorov-Smirnov (KS) test is a statistical test that determines whether two
i.i.d sets of samples follow the same distribution. The KS statistic for two
sets with $n$ and $m$ samples is:

(3) $\displaystyle D_{n,m}=sup_{x}|F_{n}\left(x\right)-F_{m}\left(x\right)|$

where $F_{n}$ and $F_{m}$ are the empirical distribution functions of two sets
of samples respectively, i.e. $F_{n}(t)=\frac{1}{n}\sum_{i=1}^{n}1_{x_{i}\leq
t}$, and $sup$ is the supremum function. The null hypothesis that the two sets
of samples are i.i.d. sampled from the same distribution, is rejected at level
$\alpha$ if:

(4) $\displaystyle D_{n,m}>c\left(\alpha\right)\sqrt{\frac{n+m}{nm}}$

where $c\left(\alpha\right)$ is a pre-calculated value and can be found in the
standard KS test lookup table.

Table 1. Impostor detection in the IDaaS secnario, using binary classification
models, achieves 97%-98% accuracy.

Models | 64-reading Window
---|---
| TNR
---
(%)
| TPR/Recall
---
(%)
| Accuracy
---
(%)
P | F1
KRR | 88.91 | 82.66 | 85.78 | 0.87 | 0.83
SVM |  99.26 |  97.57 |  98.42 |  0.99 |  0.98
MLP-50 | 98.31 | 92.70 | 95.51 | 0.98 | 0.94
MLP-100 | 98.60 | 94.65 | 96.63 | 0.98 | 0.96
MLP-200 | 98.41 | 95.72 | 97.06 | 0.98 | 0.97
MLP-500 |  98.68 |  95.47 |  97.07 |  0.99 |  0.96
MLP-50-25 | 98.13 | 94.49 | 96.31 | 0.98 | 0.96
MLP-100-50 | 98.44 | 95.45 | 96.95 | 0.98 | 0.96
MLP-200-100 |  98.47 |  95.72 |  97.10 |  0.98 |  0.97

###  3.3. Algorithm Experimental Settings

We evaluate the algorithms for impostor detection using the WALK subset in the
Human Activities and Postural Transitions (HAPT) dataset (Reyes-Ortiz et al.,
2016) at UCI (Dheeru and Karra Taniskidou, 2017). The HAPT dataset contains
smartphone sensor readings. The smartphone is worn on the waist of each of 30
participants of various ages from 19 to 48. Each reading consists of the
3-axial measurements of both the linear acceleration and angular velocity, so
it could be treated as a 6-element vector. The sensors are sampled at 50Hz. We
select 25 out of the 30 users in the HAPT dataset as the registered users
while the other 5 users act as unregistered users. To investigate the
feasibility of user versus impostor classification, the samples from the
correct user are labeled negative for impostor detection while all the data
from the other 24 registered users and the 5 unregistered users are labeled as
positive.

In the IDaaS scenario, each data sample used in both training and testing
contains 64 consecutive readings from the same user. At 50Hz sampling
frequency, 64 readings correspond to 1.28 seconds which is the latency to
detect an impostor. Models are trained for each registered user using his/her
data and randomly picked sensor data of the other 24 registered users. We make
sure that the training samples have no overlap with the testing samples. The
samples from unregistered users are used to examine whether unseen attackers
can be successfully detected.

In the LAD scenario, the training data is only from the real user. The testing
samples still include the data from the real user and the other users. We test
for window sizes of 64, which is the same size as the IDaaS scenario, and 200,
which corresponds to a longer detection latency of 4s but shows an improvement
in the detection accuracy (Table 2). An LSTM-based model is trained to
minimize its average prediction error for each registered user. In the testing
of LSTM-based models, prediction errors for consecutive readings in each
sample form a testing PED.

Table 2. Impostor detection accuracy in the LAD secnario, using one-class
models. The numbers next to LSTM-th and PED-LSTM-Vote, i.e. 50 to 500, are the
size of hidden states in the LSTM cell. We test some common levels of
$\alpha$, i.e. 0.15, 0.10, 0.05, 0.025, and present the best choices for
different window sizes in this table.

64-reading Window | 200-reading Window
---|---
Models |  | TNR
---
(%)
| TPR/Recall
---
(%)
| Accuracy
---
(%)
Avg P | F1 | Models |  | TNR
---
(%)
| TPR/Recall
---
(%)
| Accuracy
---
(%)
Avg P | Avg F1
OCSVM | 64.24 | 74.19 | 69.22 | 0.59 | 0.65 | OCSVM | 50.02 | 75.81 | 62.92 | 0.55 | 0.62
LSTM-th | 50 | 79.37 | 65.13 | 72.25 | 0.59 | 0.60 | LSTM-th | 50 | 72.43 | 67.04 | 69.74 | 0.57 | 0.60
100 | 78.76 | 66.72 | 72.74 | 0.61 | 0.62 | 100 | 72.20 | 69.27 | 70.73 | 0.58 | 0.62
200 | 78.50 | 69.64 | 74.07 | 0.62 | 0.64 | 200 | 67.88 | 71.60 | 69.74 | 0.58 | 0.62
500 | 79.14 | 70.29 | 74.71 | 0.63 | 0.65 | 500 | 67.57 | 74.42 | 70.99 | 0.60 | 0.65
PED-LSTM -Vote ($\alpha=0.10$) | 50 | 85.55 | 83.60 | 84.58 | 0.84 | 0.84 |  PED-LSTM -Vote ($\alpha=0.05$) | 50 | 82.16 | 91.96 | 87.06 | 0.92 | 0.85
100 | 87.80 | 85.68 | 86.74 | 0.86 | 0.85 | 100 | 84.98 | 93.20 | 89.09 | 0.93 | 0.86
200 | 89.27 | 85.00 |  87.13 | 0.85 | 0.87 | 200 | 88.49 | 92.00 |  90.24 | 0.92 | 0.89
500 | 87.02 | 83.86 | 85.44 | 0.84 | 0.86 | 500 | 87.14 | 91.16 | 89.15 | 0.91 | 0.88

###  3.4. Algorithm Evaluation

We evaluate each of the 25 registered users against each of the 30 users, i.e.
750 test pairs in total, and we report the average metrics of all pairs. Table
1 and Table 2 show the results of different algorithms in the IDaaS and the
LAD scenarios, respectively. Table 1 shows that in the IDaaS scenario, the SVM
model outperforms the other models, including KRR with 14 manually designed
features (Lee and Lee, 2017), on all evaluated metrics. A simple deep learning
model, MLP, performs almost as well, achieving accuracy $>97\%$. Larger
models, e.g. MLP-500 and models with more layers, e.g. MLP-200-100, also
slightly improve the accuracy.

Table 2 shows the approaches we evaluated for the LAD scenario, for 2 window
sizes of 64 (left) and 200 (right) sensor measurements. For each LSTM
algorithm, we also tested different hidden state sizes, from 50 to 500. LSTM-
th compares the average prediction error in a window with a threshold obtained
from the validation set, while PED-LSTM-Vote compares the empirically-derived
PEDs. We randomly choose 20 samples of prediction errors from the validation
set and use them to represent the reference PEDs. In the testing phase, the
prediction error distribution of each testing sample is compared to all the
reference distributions. The PED-LSTM-Vote models consider a sample as
abnormal if more than half of the D statistics of the KS tests are larger than
the fixed threshold in Eq (4). Table 2 shows the results for the
$\alpha$-values that give the best detection accuracy, i.e. $\alpha=0.10$ for
a 64-reading window and $\alpha=0.05$ for a 200-reading window.

In Table 2, the one-class SVM (OCSVM) achieves an average accuracy of 62-69%,
thirty percent worse than the 2-class SVM model trained with positive data
involved. The LSTM-th models (64-reading window) have an accuracy between 72%
and 75%, only slightly better than the one-class SVM model, regardless of the
hidden state size. If PED and statistical KS test are leveraged, we see a
significant improvement in the detection accuracy up to 87.1% and 90.2% for a
64-reading window and a 200-reading window, respectively.

However, the overhead in execution time may increase. In Section 5, we discuss
such security-performance trade-offs, which are essential to algorithm
selection in practice.

###  3.5. Insights from Algorithm Performance

The results in Section 3.4 show that in the IDaaS scenario, detection in 1.28s
with very high sensitivity levels (95%-99%) can be achieved for accuracy,
security (TPR) and usability (TNR) when SVM or MLP models are used. In the
data-privacy preserving LAD scenario, the detection accuracy, using our LSTM-
based models enhanced by collecting error distributions, can reach 87.13% for
the same detection latency of 1.28s. If the user allows a detection latency of
4 seconds which is usually not long enough for an impostor to perform
malicious operations on the smartphone after stealing it, the accuracy can be
increased to 90.24%. Although the accuracy is not perfect, it is comparable to
the state-of-the-art one-class smartphone authentication using various
handcrafted features and complex model fusion (Kumar et al., 2018) in the
literature. Also, our privacy-preserving one-class model (PED-LSTM-Vote-200)
achieves better detection accuracy, F1 score, TPN and TNR results than the
state-of-the-art two-class KRR model with hand-crafted features (Lee and Lee,
2017) for this data set when both are using a 64-reading window.

A key contribution we make is to show that it is the Prediction Error
Distributions and KS test that provide the significant increase in detection
capability. While tuning the size of deep learning models, e.g. LSTM, has
little impact on accuracy, the KS test for PED comparison increases the
overall accuracy by +12.4% for the 64-reading window and +19.2% for the
200-reading window. Therefore, we provide the hardware support for generating
empirical PEDs and computing the KS statistic in Section 4.4.

##  4\. Hardware Detection Module

Our goal is to design a small but versatile hardware module that can be
integrated into a smartphone to perform the entire impostor detection, without
needing another processor or accelerator. This not only eliminates the network
and cloud attack vectors but also reduces the cost to move data and the
contention with other applications for computing on the CPU or the GPU.
Ideally, the hardware module can read the latest sensor measurements from a
buffer so that the main memory does not need to be involved. Our design goals
are:

  * $\bullet$

Suitability for smartphones and other battery and resource-constrained
devices,

  * $\bullet$

Reduced attack surface for better security,

  * $\bullet$

Flexibility for different ML/DL algorithms and trade-offs of security,
usability, privacy, execution time, storage and energy consumption,

  * $\bullet$

Scalability for more performance in the future if needed.

Figure 1. A SID module implementing vector and matrix operations with 4
parallel datapath tracks.

Unlike prior work on implementing deep learning models in hardware (Liu et
al., 2015; Wang et al., 2018; Gao et al., 2018), we are interested in neither
the highest performance nor the lowest energy consumption. Rather we want to
investigate what performance is sufficient with minimum hardware that can
achieve an important security goal (like impostor detection), while still
being flexible for future needs, such as different algorithms or more
performance. To reduce the attack surface, SID should be able to support
detection without subsequent processing on another device like the CPU. This
includes collecting and comparing empirical probability distributions to
enhance DL models. While our primary goal is to perform the best algorithms
for impostor detection, namely, MLP and SVM for the IDaaS scenario, and PED-
LSTM-Vote for the LAD scenario, we also want SID to be flexible enough to
support other ML/DL algorithms as well. For performance scalability, we design
SID to allow more parallel data tracks to be implemented, if desired. An
innovative aspect of our design is that the SID macro instructions
implementing the selected ML/DL algorithms do not even have to be changed when
the number of parallel tracks is increased and performance increased. This is
in line with our goal of minimal hardware.

###  4.1. Functional Operations Supported

We first consider what operations are needed by the DL (and ML) algorithms we
want to implement. These are first the PED-LSTM algorithm, and also the MLP
and SVM algorithms, which are the best impostor detection algorithms for the
two scenarios considered in the previous section. Table 3 shows the operations
needed for these different DL/ML algorithms. The instructions from Vargmax to
Vsqrt (at the bottom of Table 3) are needed only by the KRR algorithm (Lee and
Lee, 2017), the previous highest performing method, to compute the 7 features
for the accelerometer and the gyroscope each, i.e., the average, maximum,
minimum and variance of the sensor data and three frequency domain features:
the main frequency and its amplitude and the second frequency. We decide not
to implement these operations, since they are not needed by the other higher-
performing algorithms, while needing significant extra hardware.

###  4.2. Programming Model and Macro-instruction

The programming model of SID is to execute macro-instructions, where each
macro instruction is a whole vector or matrix operation. The number of
iterations is automatically determined by the hardware, in a Finite State
Machine (FSM), based on the hardware’s knowledge of the number of parallel
data tracks available in the implementation. The macro-instruction supplies
the type of operation needed, and the dimensions of the vector or matrix.

The format of a SID macro instruction is shown in Table 4. The Mode field
specifies one of the operation modes in Table 3. Three memory addresses can be
specified in a macro-instruction: Addr_x and Addr_y for up to two operands,
and Addr_z for the result. Instead of implementing vector machines with vector
registers of fixed length, we use memory to store the vector or matrix
operands and results. This design is less expensive than vector registers. It
is also more efficient since it supports the flexible-size inputs and outputs
and operates seamlessly with our automatic hardware control of the execution
of a vector or matrix operation. This is one way (memory not vector registers)
to use minimal hardware and achieve scalability (macro-instruction with FSM
control).

Each macro-instruction initializes the FSM state of the control unit to
indicate the number of iterations of the specified operation. Each cycle, the
FSM updates the number of uncomputed iterations, according to the number of
parallel tracks, to decide when a macro-instruction finishes (details in the
following paragraphs). Thus, the same SID software program can run on SID
hardware modules with a different number of parallel tracks, without
modification. This achieves our performance scalability goal.

The Length and Width fields can initialize three state registers, reg_length,
reg_width and reg_width_copy, which define the control FSM state. The FSM can
be configured by instructions in two ways: the one-dimension iteration and the
matrix-vector iteration. For the one-dimension iteration in a vector
operation, the value of reg_length is initialized by the length field of the
instruction. During execution, reg_length is decreased every cycle by
N(track), which is the number of parallel datapath tracks, until reg_length is
no larger than N(track) and the next instruction will be fetched.

When an instruction for a matrix-vector operation is fetched, the length field
initializes reg_length and the width field initializes both reg_width and
reg_width_copy. A matrix-vector multiplication macro-instruction computes the
product of a matrix of size width-by-length with a vector containing length
elements. The SID module performs loop tiling by computing a tile of width
rows by N(track) columns in the matrix before moving to the next tile. When
the last tile of columns in the matrix is computed, the next instruction can
be fetched.

Table 3. Computation primitives needed by different ML/DL models and
statistical testing.

Operations | Description | IDaaS | LAD |
---|---|---|---|---
KRR | MLP |  | SVM
---
Basic
| SVM w/
---
Gaussian
Kernel
| OCSVM w/
---
Gaussian
Kernel
LSTM | KS-test |  | PED-LSTM
---
-Vote
| Support
---
Status
Vadd | Element-wise addition of two vectors | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Yes
Vsub | Element-wise subtraction of two vectors |  |  |  | ✓ | ✓ | ✓ | ✓ | ✓ | Yes
Vmul | Element-wise multiplication of two vectors | ✓ |  |  | ✓ | ✓ | ✓ | ✓ | ✓ | Yes
Vsgt | Element-wise set-greater-than of two vectors | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |  | ✓ | Yes
Vsig | Sigmoid function of a vector | ✓ | ✓ |  |  |  | ✓ |  | ✓ | Yes
Vtanh | Tanh function of a vector |  | ✓ |  |  |  | ✓ |  | ✓ | Yes
Vexp | Exponential function of a vector |  |  |  | ✓ | ✓ |  |  |  | Yes
MVmul | Multiplication of a matrix and a vector | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |  | ✓ | Yes
VSsgt |  | Set-greater-than to compare a scalar and
---
a vector’s elements
|  |  |  |  |  | ✓ | ✓ | Yes
Vmaxabs | Find the maximum absolute value of a vector | ✓ |  |  |  |  |  | ✓ | ✓ | Yes
Vsqnorm | Squared L2-norm of a vector |  |  |  | ✓ | ✓ | ✓ |  | ✓ | Yes
Vargmax | Find the index of the maximum in a vector | ✓ |  |  |  |  |  |  |  | No
Vmin | Find the minimum in a vector | ✓ |  |  |  |  |  |  |  | No
Vmax2 | Find the second largest number in a vector | ✓ |  |  |  |  |  |  |  | No
VFFT | Compute the Fourier transform of a vector | ✓ |  |  |  |  |  |  |  | No
Vsqrt |  | Compute the square root of each
---
element in a vector
✓ |  |  |  |  |  |  |  | No

###  4.3. Parallel Datapaths and Functional Unit Reuse

Figure 1 shows a SID implementation with four parallel datapath tracks. Each
track consists of a Look-up Table (LUT), a multiplier (MUL) and an adder
(ADD), which are put into three consecutive pipelined execution stages (EXE0,
EXE1 and EXE2). We also have a small local scratchpad memory in the last
execution stage (EXE2) for faster access to intermediate results during a
macro-instruction. The control path shows 6 pipeline stages: fetch, decode
instruction, 3 execution stages and write the result back to memory.

Each macro-instruction is decoded into the FSM control mechanism in the Decode
stage of SID ’s pipeline. This design is scalable since the hardware is aware
of the number of parallel datapath tracks that are implemented and can perform
automatic control of the FSM for vector and matrix operations, and any loop
iterations required. For performance, the control by an FSM avoids using
branch instructions for frequent jump-backs in loop-control, as is needed in
general-purpose processors, which can take up a large portion of processor
throughput for the simple loops needed to implement vector and matrix
computation.

Table 4. SID Macro-instruction with Scalable FSM Control

127 | 124 | 123 | 110 | 109 | 96 | 95 | 64 | 63 | 32 | 31 | 0
---|---|---|---|---|---|---|---|---|---|---|---
Mode | Length | Width | Addr_x | Addr_y | Addr_z
4 bits | 14 bits | 14 bits | 32 bits | 32 bits | 32 bits

We discuss two optimizations: we reduce the number of memory accesses with
local storage and we minimize the hardware design cost by reusing functional
units.

When computing the matrix-vector multiplication in the MVmul mode, we use a
local scratchpad memory and loop tiling to save the latency of storing and
accessing partial sums from memory and also reduce the memory traffic. In the
Vmaxabs mode which finds the maximum absolute value in a vector by doing a
comparison of input elements in the EXE2 stage, a temporary maximum is stored
in the local scratchpad to reduce external memory accesses; it gets updated
every cycle. In the Vsqnorm mode which computes the squared L2-norm of a
vector, the local scratchpad memory stores the partial sum of x[i]2 (x is the
input vector), which are computed by the multipliers and adders in the EXE1
and EXE2 stages.

When computing non-linear functions like sigmoid (Vsig), tanh (Vtanh) and
exponential (Vexp), we avoid implementing complex non-linear functional units,
but instead use the flexible look-up table (LUT) to look up the slope and
intercept for linear approximation. An added benefit of our approach over
prior work, e.g. (Chen et al., 2014), is that we place the LUTs before the
multipliers and adders in the three consecutive execution stages so that no
extra multipliers or adders are needed. The ELE0 (LUT) stage of SID outputs a
slope, k[i], and an intercept, b[i], for each input value. The interpolation
is then computed in the later two stages as z[i] = k[i] $\times$ x[i] + b[i]
with z[i] being the value of the non-linear function for input x[i]. Also,
instead of having another adder tree stage for the MVmul mode, we save on
hardware cost by reusing the adders in the EXE2 stage to sum the products
computed in the EXE1 stage and the partial sum read from the local scratchpad.
The new partial sum is written back to the local scratchpad memory if the
computation is not finished.

Integration in smartphone SOC. Our SID anomaly detection module can be
integrated close to the sensors to reduce the attack surface and also to save
the overhead of memory accesses. (If software processing was used, the sensor
measurements would have to be stored to memory first, then read back from
memory to the CPU or GPU for software impostor detection.) Modern smartphones
have already implemented the interface to write the collected sensor
measurements to a cache memory for efficient signal processing (Codrescu,
2015). The SID module can leverage a similar interface (viz., ”Sensor Inputs”
in Figure 1). A valid incoming sensor input can reset the program counter of
SID to the beginning of the detection program.

###  4.4. Support for Empirical Distribution Representation and Comparison

Another novel contribution of this work is to show that empirical probability
distributions can be collected and compared efficiently using the multipliers
and adders already needed for the ML/DL algorithms. To the best of our
knowledge, we are the first to describe the following simple and efficient
hardware support for collecting error distributions and comparing them with
the KS test.

Figure 2. An example of five-step KS test.

We add two operations for this KS test, but these general-purpose operations
can be useful for other ML/DL algorithms and statistical tests as well. The
first is a vector-scalar comparison (VSsgt described in Table 3). The second
operation is Vmaxabs, which can be used to find the maximum absolute value in
a vector.

We illustrate with an example in Figure 2, showing a five-step workflow. The
grey dotted boxes represent inputs, which include the reference prediction
error distribution (PED), the test PED, the threshold and the output. The
reference PED is collected in the training phase and is represented by
reference histogram bin boundaries and a reference cumulative histogram. The
test PED is collected online and represented by a series of observed test
errors.

Step ①: compare an observed error with reference bin boundaries. The output of
this step is a vector of “0”s and “1”s. “1” means that the corresponding bin
boundary is greater than the observed error, and “0” otherwise. This uses the
VSsgt operation. Step ②: accumulate all binary vectors from ①. The accumulated
vector, namely the “test cumulative histogram”, represents the cumulative
histogram of the observed test errors using the reference bins. Step ③ and
step ④: find the largest difference in the reference and test cumulative
histograms. Step ③ is a vector subtraction and step ④ is the Vmaxabs
operation, to find the maximum absolute value in a vector. This gives us the
right hand side of Eq (3) without dividing by $n$, the number of data points
in the testing error distribution. Hence, we have computed $n\times D_{n,m}$
at the end of step ④. Step ⑤: compare the largest difference with a threshold.
We do the equivalent comparison as Eq (4) by multiplying both sides by $n$. In
our experiments, $m$, which is the number of data points in the reference
error distribution, and $n$ are hyper-parameters that are decided during
training and always set to be the same. In the example of Figure 2, both $n$
and $m$ are 5. The test histogram is considered abnormal if the largest
difference is larger than the threshold, $T$.

##  5\. Evaluation

We evaluate the cost in terms of execution time (performance) and memory usage
for the detection algorithms from Section 3 implemented on the SID module, and
discuss accuracy-cost trade-offs. We also show that SID has lower energy
consumption and needs much fewer hardware resources than other hardware
implementations.

Figure 3. Execution time and accuracy of detection algorithms. The red line
stands for the average accuracy. The maximum execution time for inferencing,
using any of the ML/DL models shown, is 4 ms, much smaller than the 20 ms
interval between new sensor measurements.

###  5.1. Accuracy vs. Execution Time

New sensor measurements, e.g., motion sensors like the accelerometer and
gyroscope, are available every 20 ms in most smartphones. Hence, we cannot
detect any impostor in less than this time.

Figure 3 compares different machine learning and deep learning algorithms with
respect to accuracy and execution time on SID. The algorithms to the left of
the black dashed line are used in the IDaaS scenario. Although the SVM
algorithm achieves slightly higher accuracy than MLP-200-100 (98.4% versus
97.1%), it needs a longer execution time for doing inferencing.

The algorithms used in the LAD scenario are to the right of the dashed line.
We measure the execution time of the best algorithm in Section 3.4, i.e. PED-
LSTM-Vote, and the baseline algorithm, LSTM-th, for comparison. We choose the
best LSTM size, which is 200, and consider the cases of both the 64-reading
window and the 200-reading window. Figure 3 shows that while the KS test
technique increases the detection accuracy, it also needs additional execution
time. However, the execution time (less than 4 ms) of all algorithms is always
much smaller than 20ms, so the performance of SID is more than adequate to
achieve impostor detection with the highest accuracy of PED-LSTM-Vote.
Depending on the size of the window (64-reading or 200-reading), the detection
time from attack to detection by PED-LSTM-Vote is 1.28s or 4s (see Section
3.5) plus the last 4 ms execution time for the LSTM-PED-Vote inference
algorithm. Hence, the execution time of SID is negligible (and more than
sufficient) compared to the time to collect sufficient consecutive sensor
readings.

###  5.2. Accuracy vs. Memory Usage

Figure 4 compares models used in the IDaaS and LAD scenarios, in terms of
their accuracy and the model size. In the IDaaS scenario (left), we see that a
2-layer MLP-200-100 can achieve a slightly higher detection accuracy with a
smaller model size than MLP-500. The SVM model has a very small improvement on
the accuracy over MLP-200-100 but incurs the highest cost in terms of memory
usage as it has to store support vectors. Hence, MLP-200-100 appears to be the
best for the cost (execution time + memory usage) versus accuracy trade-off
for the IDaaS scenario.

In the LAD scenario (right) which is better for protecting the privacy of a
user’s sensor data, we evaluate the additional memory usage of the KS test
technique compared to the baseline LSTM-th algorithm where the size of the
LSTM cell is still 200. We find that PED-LSTM-Vote uses only 1.6% and 4.8%
more space, for the 64-reading window and 200-reading window, respectively.
They are better choices in the memory-versus-accuracy trade-off as the
improvement in accuracy is significant.

Figure 4. Model size and accuracy of detection algorithms. The red line stands
for the average accuracy.

###  5.3. Hardware Design Complexity

We implement an FPGA prototype of the SID module, in order to show how small a
hardware module can be to support impostor detection. Our implementation has
four parallel tracks and a 256-byte scratchpad memory. The size of the
datapath RAM is 1.75MB and the size of the instruction RAM is 128KB. We use
32-bit fixed-point numbers since prior work (Liu et al., 2015) has shown
significant accuracy degradation with 16-bit numbers for certain models. SID
would be even smaller if we use lower-precision numbers. The platform board is
Xilinx Zynq-7000 SoC ZC706 evaluation kit. The hardware implementation is
generated with Vivado 2016.2.

In Table 5, we compare SID implementing LSTM-PED-Vote to two FPGA
implementations of Recurrent Neural Network (RNN) accelerators, C-LSTM (Wang
et al., 2018) and DeltaRNN (Gao et al., 2018). The C-LSTM algorithm represents
some matrices in LSTM as multiple block-circulant matrices, to reduce the
storage and computation complexity. DeltaRNN ignores minor changes in the
input of the Gated Recurrent Unit (GRU) RNN to reuse the old computation
result and thus reduce the computation workload. These accelerators are
capable of RNN inference, which is found to be important for the LAD scenario,
and consist of multiple submodules or pipeline stages. More hardware resources
are required because each of the submodules is designed to optimize and
compute one specific operation in the inference data flow, e.g., one matrix-
vector multiplication or non-linear activation, and the functional units are
not reused for different operations. Also, the accelerators have a higher
level of parallelism than the SID module. However, they lack the support for
generating and comparing empirical prediction error distributions, which we
have shown is indispensable to achieve better accuracy.

Table 5 shows a major difference between our minimalist SID module and the
performance-oriented accelerators. The FPGA resource usage of Slice LUTs,
Slice Flip-flops and DSPs of SID are one or two orders of magnitude less than
the other two RNN implementations. We measure the FPGA power consumption using
the TI Fusion Digital Power Designer tool while the power of C-LSTM is also
measured with a TI Fusion Power device and DeltaRNN with a Voltcraft energy
monitor. SID’s power consumption is an order of magnitude less, making it more
suitable for a smartphone.

While we have used an FPGA implementation to prototype SID, and enable
comparisons with existing FPGA accelerators, further power reduction is
achievable using an ASIC implementation in real smartphone products.

Table 5. SID hardware requirements and power compared to performance-oriented
RNN accelerators

| C-LSTM | DeltaRNN | SID
---|---|---|---
Functionality |  | Supports LSTM
---
only
| Supports Gated
---
Recurrent Unit
only
| Supports LSTM,
---
GRU, other ML/DL
models and
statistical tests
FPGA | Xilinx Virtex-7 | Kintex-7 XC7Z100 |  | XC7Z045
---
FFG900
Quantization | 16-bit fixed point | 16-bit fixed point | 32-bit fixed point
Slice LUT | 406,861 (49.07X) | 261,357 (31.52X) | 8,292 (1X)
Slice Flip-flop | 402,876 (106.07X) | 119,260 (31.40X) | 3,798 (1X)
DSP | 2675 (167.19X) | 768 (48X) | 16 (1X)
BRAM | 966 (1.98X) | 457.5 (0.94X) | 489 (1X)
Clock Freq | 200MHz | 125MHz | 115MHz
Power (W) | 22 |  | Zynq MMP: 7.3
---
FPGA: 15.2
| Idle: 0.12
---
Running: 0.62

##  6\. Related Work

User-behavior-based implicit smartphone authentication leveraging smartphone
and smartwatch sensors (Lee and Lee, 2017), touchscreen input (Frank et al.,
2012), and tapping behaviors (Zheng et al., 2014), have been investigated in
the literature. These works require other users’ data for model training
(similar to our IDaaS scenario) and may raise behavioral data privacy
concerns. There have been preliminary works on authenticating smartphone users
with only that user’s data, which might fit under our LAD scenario. For
example, multi-motion sensors (Shen et al., 2017), fusion of swiping and phone
movement patterns (Kumar et al., 2018) and keystroke (Kazachuk et al., 2016;
Liatsis and Tran, 2017) have been used for one-class smartphone user
authentication. However, these works leveraged manually-crafted features and
deep learning models are not found to outperform the conventional machine
learning models. In contrast, we show the superiority of deep learning
algorithms, without needing tedious feature extraction, for both the IDaaS and
LAD scenarios.

Many accelerators for a single machine learning (ML) algorithm have been
proposed, e.g. SVM (Papadonikolakis and Bouganis, 2012)(Papadonikolakis and
Bouganis, 2010), k-neatest neighbors (Schumacher et al., 2008) and
k-means(Chen and Chien, 2011). More recent work have also been proposed for
deep neural networks (DNNs). EIE (Han et al., 2016) exploit data sparsity
during inference to improve performance and energy efficiency. The sparsity in
training is exploited in (Rhu et al., 2018). Minerva (Reagen et al., 2016)
presents a framework to reduce power consumption by finding the optimal data
quantization. Weight sharing in CNN is supported by (Song et al., 2018) with
software exploration and a dedicated accelerator. Some hardware accelerators
support multiple machine learning models. (Majumdar et al., 2011) evaluates
the acceleration of four models in an embedded system. MAPLE (Majumdar et al.,
2012) accelerates the vector and matrix operations found in five
classification workloads. PuDianNao (Liu et al., 2015) highlights the non-
vector operations and data locality in seven ML techniques. These works
support their chosen ML/DL algorithms and exploit different properties of DNN
models to improve parallelism, performance or energy efficiency, while we aim
for sufficient performance for real problems at a low cost. They also do not
consider other analysis techniques such as the collection and comparison of
empirical probability distributions with KS tests, as we do. To the best of
our knowledge, we are the first to explore delivering versatility and
sufficient hardware performance for detecting anomalous behavior, with reduced
energy consumption, rather than shooting for maximum hardware performance or
maximum energy efficiency.

##  7\. Conclusions

We study how sensors in a smartphone can be used to detect smartphone
impostors and theft using machine learning and deep learning algorithms. A key
contribution is showing that we can detect impostors while preserving the
user’s privacy by using LSTM-based models enhanced by comparing Prediction
Error Distributions (PEDs). In both the IDaaS and LAD scenarios, our deep
learning algorithms have better detection accuracy than the studied machine
learning algorithms including the current best ML algorithm (Lee and Lee,
2017) which required many hand-crafted features.

To reduce the attack surface, we design a low-cost hardware module, SID, to
support the best impostor detection algorithms we found in both scenarios. It
is versatile enough to support other ML/DL algorithms as well. It has an
innovative hardware implementation for collecting and comparing empirical
probability distributions that we use to represent user behavior. This enables
users of SID to trade-off security with data privacy in choosing one of the
two scenarios, as well as to trade-off accuracy with overhead. Our FPGA
implementation shows that SID provides sufficient performance with minimal
cost. Compared to other model-specific accelerators, SID provides more
functional versatility and uses less hardware resources and energy which are
one to two orders of magnitude less than performance-oriented FPGA
accelerators.

###### Acknowledgements.

This work is supported in part by NSF SaTC #1814190, SRC Hardware Security
Task 2844.002 and a Qualcomm Faculty Award for Prof. Lee. We also thank the
reviewers for their feedback.

## References

  * (1)
  * Chen and Chien (2011) T. Chen and S. Chien. 2011.  Flexible Hardware Architecture of Hierarchical K-Means Clustering for Large Cluster Number.  _IEEE Transactions on Very Large Scale Integration (VLSI) Systems_ 19, 8 (Aug 2011), 1336–1345.  https://doi.org/10.1109/TVLSI.2010.2049669
  * Chen et al. (2014) Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam. 2014.  DianNao: A Small-footprint High-throughput Accelerator for Ubiquitous Machine-learning. In _Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems_ _(ASPLOS ’14)_. ACM, New York, NY, USA, 269–284.  https://doi.org/10.1145/2541940.2541967
  * Codrescu (2015) Lucian Codrescu. 2015\.  Architecture of the Hexagon 680 DSP for mobile imaging and computer vision. In _2015 IEEE Hot Chips 27 Symposium (HCS)_. IEEE, 1–26.
  * Dheeru and Karra Taniskidou (2017) Dua Dheeru and Efi Karra Taniskidou. 2017.  UCI Machine Learning Repository.  (2017).  http://archive.ics.uci.edu/ml
  * Ehatisham-ul Haq et al. (2017) Muhammad Ehatisham-ul Haq, Muhammad Awais Azam, Jonathan Loo, Kai Shuang, Syed Islam, Usman Naeem, and Yasar Amin. 2017\.  Authentication of smartphone users based on activity recognition and mobile sensing.  _Sensors_ 17, 9 (2017), 2043.
  * Frank et al. (2012) Mario Frank, Ralf Biedert, Eugene Ma, Ivan Martinovic, and Dawn Song. 2012.  Touchalytics: On the applicability of touchscreen input as a behavioral biometric for continuous authentication.  _IEEE transactions on information forensics and security_ 8, 1 (2012), 136–148.
  * Gao et al. (2018) Chang Gao, Daniel Neil, Enea Ceolini, Shih-Chii Liu, and Tobi Delbruck. 2018.  DeltaRNN: A Power-efficient Recurrent Neural Network Accelerator. In _Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays_. ACM, 21–30.
  * Han et al. (2016) Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally. 2016.  EIE: efficient inference engine on compressed deep neural network. In _2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA)_. IEEE, 243–254.
  * He et al. (2019) Zecheng He, Aswin Raghavan, Guangyuan Hu, Sek Chai, and Ruby Lee. 2019.  Power-Grid Controller Anomaly Detection with Enhanced Temporal Deep Learning. In _18th IEEE International Conference on Trust, Security and Privacy in Computing and Communications_.
  * Kazachuk et al. (2016) Maria Kazachuk, Alexander Kovalchuk, Igor Mashechkin, Igor Orpanen, Mikhail Petrovskiy, Ivan Popov, and Roman Zakliakov. 2016.  One-class models for continuous authentication based on keystroke dynamics. In _International Conference on Intelligent Data Engineering and Automated Learning_. Springer, 416–425.
  * Kumar et al. (2018) Rajesh Kumar, Partha Pratim Kundu, and Vir V Phoha. 2018\.  Continuous authentication using one-class classifiers and their fusion. In _2018 IEEE 4th International Conference on Identity, Security, and Behavior Analysis (ISBA)_. IEEE, 1–8.
  * Lee and Lee (2015) Wei-Han Lee and Ruby B Lee. 2015.  Multi-sensor authentication to improve smartphone security. In _2015 International Conference on Information Systems Security and Privacy (ICISSP)_. IEEE, 1–11.
  * Lee and Lee (2017) Wei-Han Lee and Ruby B Lee. 2017.  Implicit smartphone user authentication with sensors and contextual machine learning. In _Dependable Systems and Networks (DSN), 2017 47th Annual IEEE/IFIP International Conference on_. IEEE, 297–308.
  * Liatsis and Tran (2017) Panos Liatsis and Quang Duc Tran. 2017.  One-class classification in multimodal biometrie authentication. In _2017 International Conference on Infocom Technologies and Unmanned Systems (Trends and Future Directions)(ICTUS)_. IEEE, 37–42.
  * Liu et al. (2015) Daofu Liu, Tianshi Chen, Shaoli Liu, Jinhong Zhou, Shengyuan Zhou, Olivier Teman, Xiaobing Feng, Xuehai Zhou, and Yunji Chen. 2015.  PuDianNao: A Polyvalent Machine Learning Accelerator. In _Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems_ _(ASPLOS ’15)_. ACM, New York, NY, USA, 369–381.  https://doi.org/10.1145/2694344.2694358
  * Majumdar et al. (2012) Abhinandan Majumdar, Srihari Cadambi, Michela Becchi, Srimat T Chakradhar, and Hans Peter Graf. 2012.  A massively parallel, energy efficient programmable accelerator for learning and classification.  _ACM Transactions on Architecture and Code Optimization (TACO)_ 9, 1 (2012), 6.
  * Majumdar et al. (2011) Abhinandan Majumdar, Srihari Cadambi, and Srimat T Chakradhar. 2011.  An energy-efficient heterogeneous system for embedded learning and classification.  _IEEE embedded systems letters_ 3, 1 (2011), 42–45.
  * Malhotra et al. (2015) Pankaj Malhotra, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal. 2015.  Long short term memory networks for anomaly detection in time series. In _Proceedings_. Presses universitaires de Louvain, 89.
  * Nickel et al. (2012) Claudia Nickel, Tobias Wirtl, and Christoph Busch. 2012\.  Authentication of smartphone users based on the way they walk using k-nn algorithm. In _2012 Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing_. IEEE, 16–20.
  * Papadonikolakis and Bouganis (2012) M. Papadonikolakis and C. Bouganis. 2012.  Novel Cascade FPGA Accelerator for Support Vector Machines Classification.  _IEEE Transactions on Neural Networks and Learning Systems_ 23, 7 (July 2012), 1040–1052.  https://doi.org/10.1109/TNNLS.2012.2196446
  * Papadonikolakis and Bouganis (2010) Markos Papadonikolakis and Christos-Savvas Bouganis. 2010.  A heterogeneous fpga architecture for support vector machine training. In _2010 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines_. IEEE, 211–214.
  * Reagen et al. (2016) Brandon Reagen, Paul Whatmough, Robert Adolf, Saketh Rama, Hyunkwang Lee, Sae Kyu Lee, José Miguel Hernández-Lobato, Gu-Yeon Wei, and David Brooks. 2016\.  Minerva: Enabling low-power, highly-accurate deep neural network accelerators. In _2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA)_. IEEE, 267–278.
  * Reyes-Ortiz et al. (2016) Jorge-L Reyes-Ortiz, Luca Oneto, Albert Samà, Xavier Parra, and Davide Anguita. 2016.  Transition-aware human activity recognition using smartphones.  _Neurocomputing_ 171 (2016), 754–767.
  * Rhu et al. (2018) Minsoo Rhu, Mike O’Connor, Niladrish Chatterjee, Jeff Pool, Youngeun Kwon, and Stephen W Keckler. 2018.  Compressing DMA engine: Leveraging activation sparsity for training deep neural networks. In _2018 IEEE International Symposium on High Performance Computer Architecture (HPCA)_. IEEE, 78–91.
  * Schumacher et al. (2008) Tobias Schumacher, Robert Meiche, Paul Kaufmann, Enno Lübbers, Christian Plessl, and Marco Platzner. 2008\.  A Hardware Accelerator for k-th Nearest Neighbor Thinning.. In _ERSA_. Citeseer, 245–251.
  * Shen et al. (2017) Chao Shen, Yuanxun Li, Yufei Chen, Xiaohong Guan, and Roy A Maxion. 2017.  Performance analysis of multi-motion sensor behavior for active smartphone authentication.  _IEEE Transactions on Information Forensics and Security_ 13, 1 (2017), 48–62.
  * Song et al. (2018) Mingcong Song, Kan Zhong, Jiaqi Zhang, Yang Hu, Duo Liu, Weigong Zhang, Jing Wang, and Tao Li. 2018\.  In-Situ AI: Towards Autonomous and Incremental Deep Learning for IoT Systems. In _2018 IEEE International Symposium on High Performance Computer Architecture (HPCA)_. IEEE, 92–103.
  * Wang et al. (2018) Shuo Wang, Zhe Li, Caiwen Ding, Bo Yuan, Qinru Qiu, Yanzhi Wang, and Yun Liang. 2018.  C-lstm: Enabling efficient lstm using structured compression techniques on fpgas. In _Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays_. ACM, 11–20.
  * Wang et al. (2012) Yong Wang, Kevin Streff, and Sonell Raman. 2012.  Smartphone security challenges.  _Computer_ 45, 12 (2012), 52–58.
  * Yang et al. (2014) Lei Yang, Yi Guo, Xuan Ding, Jinsong Han, Yunhao Liu, Cheng Wang, and Changwei Hu. 2014.  Unlocking smart phone through handwaving biometrics.  _IEEE Transactions on Mobile Computing_ 14, 5 (2014), 1044–1055.
  * Zheng et al. (2014) Nan Zheng, Kun Bai, Hai Huang, and Haining Wang. 2014\.  You are how you touch: User verification on smartphones via tapping behaviors. In _2014 IEEE 22nd International Conference on Network Protocols_. IEEE, 221–232.