# Stability of Large Amplitude Viscous Shock Wave for 1-D Isentropic Navier-
Stokes System in the Half Space ††thanks: Received date, and accepted date
(The correct dates will be entered by the editor).

Lin  Chang School of Mathematics Science, Beihang University, Beijing, China,
( changlin23@buaa.edu.cn).

###### Abstract

In this paper, the asymptotic-time behavior of solutions to an initial
boundary value problem in the half space for 1-D isentropic Navier-Stokes
system is investigated. It is shown that the viscous shock wave is stable for
an impermeable wall problem where the velocity is zero on the boundary
provided that the shock wave is initially far away from the boundary.
Moreover, the strength of shock wave could be arbitrarily large. This work
essentially improves the result of [A. Matsumura, M. Mei, Convergence to
travelling fronts of solutions of the p-system with viscosity in the presence
of a boundary, Arch. Ration. Mech. Anal., 146(1): 1-22, 1999], where the
strength of shock wave is sufficiently small.

###### keywords:

Impermeable wall problem; large amplitude shock; asymptotic stability

35Q30; 76N10;

##  1 Introduction

We consider a 1-D isentropic Navier-Stokes system for general viscous gas,
which reads in the Lagrangian coordinate as,

$\left\\{\begin{array}[]{ll}&v_{t}-u_{x}=0,\\\
&u_{t}+p_{x}=(\mu(v)\frac{u_{x}}{v})_{x},\end{array}\right.$ (1.1)

where $t>0,x\in\mathbb{R_{+}},$ and $v(x,t)=\frac{1}{\rho(x,t)}$ is the
specific volume, $u(x,t)$ the fluid velocity, $p=av^{-\gamma}$ the pressure
with constant $a>0$, $\gamma>1$ the adiabatic constant, and
$\mu(v)=\mu_{0}v^{-\alpha}$ the viscosity coefficient with $\alpha\geq 0$.
When the viscosity $\mu(v)\equiv 0$, the system (1.1) becomes the famous Euler
system

$\left\\{\begin{array}[]{ll}&v_{t}-u_{x}=0,\\\
&u_{t}+p_{x}=0,\end{array}\right.$ (1.2)

that has rich wave phenomena such as shock and rarefaction waves. When
$\mu(v)>0$, the shock wave is mollified as the so-called viscous shock wave.
Without loss of generality, we assume $\mu_{0}=1$ in what follows.

Since the system (1.1) is regular than the Euler one (LABEL:1.2), it is very
interesting and important to study the stability of the viscous version of
shock wave, i.e., the viscous shock wave, for the viscous conservation laws
such as the NS system (1.1) with the initial data:

$(v,u)(x,0)=(v_{0},u_{0})(x)\longrightarrow(v_{\pm},u_{\pm}),\quad\text{as}\quad
x\rightarrow\infty.$ (1.3)

The stability of viscous shock wave for the Cauchy problem (1.1), (1.3) has
been extensively studied in a large amount of literature since the pioneer
works of [2, 13], see the other interesting works [1, 4, 5, 6, 7, 8, 9, 11,
14, 18]. It is noted that most of above works require the strength of shock
wave is suitably small, that is, the shock is weak. The stability of large
amplitude shock (strong shock) is more interesting and challenging in both
mathematics and physics, see works [3, 13, 6, 10, 16, 19, 20].

Matsumura-Nishihara [13] showed that the viscous shock wave is stable if
$|v_{+}-v_{-}|<C(\gamma-1)^{-1}$, that is, when $\gamma\rightarrow 1$, the
strength of shock wave could be large. This condition is later relaxed in [6]
to the condition that $|v_{+}-v_{-}|<C(\gamma-1)^{-2}$. Recently, the
restriction on the strength of shock was removed in [16] by an elegant
weighted energy method as $\alpha>\frac{\gamma-1}{2}$. Vasseur-Yao [19]
removed the condition $\alpha>\frac{\gamma-1}{2}$ by introducing a beautiful
variable transformation. Moreover, He-Huang [3] extended the result of [19] to
general pressure $p(v)$ and general viscosity $\mu(v)$, where $\mu(v)$ could
be any positive smooth function.

On the other hand, it is also interesting to investigate the stability of
viscous shock wave under the effect of boundary. In 1999, Matsumura-Mei [12]
considered an impermeable wall problem of (1.1) in the half space $x\geq 0$,
i.e.,

$\left\\{\begin{array}[]{ll}(v,u)(x,0)=(v_{0},u_{0})(x)\longrightarrow(v_{+},u_{+}),~{}x\to+\infty,\\\
u(0,t)=0,~{}~{}t\in\mathbb{R_{+}},\end{array}\right.$ (1.4)

where $v_{+}>0,u_{+}<0$. The impermeable wall means that there is no flow
across the boundary so that the velocity at the boundary $x=0$ has to be zero.
It was proved in [12] that the solution of (1.1), (1.4) with $\alpha=0$ time-
asymptotically tends to an outgoing shock wave (2-shock) connecting the left
state $(v_{-},0)$ and the right one $(v_{+},u_{+})$ if
$|v_{+}-v_{-}|<C(\gamma-1)^{-2}$, and the outgoing shock is initially far away
from the boundary so that the interaction between the shock and the boundary
is weak,  where $v_{-}$ is determined by the RH condition, i.e.,

$\displaystyle\left\\{\begin{array}[]{ll}-s(v_{+}-v_{-})-(u_{+}-u_{-})=0,\\\
-s(u_{+}-u_{-})+(p(v_{+})-p(v_{-}))=0,\end{array}\right.$ (1.7)

with $u_{-}=0$. Matsumura-Nishihara [15] removed the condition that the shock
is initially far away from the boundary by extending the half space to the
whole space, with the price that the shock wave has to be weak even for
$\gamma=1$ case.

In this paper, we aim to prove that the large amplitude shock wave is still
stable for the impermeable wall problem (1.1)-(1.4). Roughly speaking, there
exists a 2-viscous shock wave (outgoing shock) $(V_{2},U_{2})$ connecting
$(v_{-},0)$ and $(v_{+},u_{+})$ with $v_{-}$ determined by the RH condition
(1.7), and $(V_{2},U_{2})$ is asymptotically stable if it is initially far
away from the boundary. The precise statement of the main result is given in
Theorem 1.

We outline the strategy as follows. Motivated by [19] and [3], we introduce a
new variable $h=u-v^{(-\alpha+1)}v_{x}$ and formulate a new equation
$\eqref{4.2}_{2}$ in which the viscous term is moved to the mass equation
$\eqref{4.2}_{1}$ so that the two nonlinear terms ${}{p}_{x}$ and
$(\frac{{}{v}_{x}}{{}{v}^{\alpha+1}})_{x}$ are decoupled and the interaction
between nonlinear terms is weaken. Since the strength of outgoing shock is
arbitrarily large, the interaction between the 2-shock and the boundary $x=0$
is strong. We have to assume that the outgoing shock is initially far away
from the boundary so that the interaction is weak. Since the boundary terms
with first order derivatives are controlled, we can obtain the low order
estimates through careful analysis. But the idea using the new system (4.27)
does not work in the higher order estimation since it is very difficult to
control the second order derivatives of boundary terms for the new variable
$h$. Note that the second derivatives of $u$ on the boundary can be
controlled, we then turn to original system (1.1) to obtain the higher order
energy estimates, and finally complete the a priori estimates.

The rest of the paper will be arranged as follows. In section 2, the outgoing
shock wave is formulated and the main result is stated. In section 3, the
problem is reformulated by the anti-derivatives of the perturbations around
the viscous shock wave. In section 4, the a priori estimates are established.
In section 5, the main theorem is proved.

Notation. The functional $\|\cdot\|_{L^{p}(\Omega)}$ is defined by
$\|f\|_{L^{p}(\Omega)}=(\int_{\Omega}|f|^{p}(\xi)\operatorname{d}\xi)^{\frac{1}{p}}$.
The symbol $\Omega$ is often omitted, when $\Omega=(0,\infty)$. As $p=2$, for
simplicity we denote,

$\|f\|=\left(\int_{0}^{\infty}f^{2}(\xi)\operatorname{d}\xi\right)^{\frac{1}{2}}.$

In addition, $H^{m}$ denotes the $m$-th order Sobolev space of functions
defined by

$\|f\|_{m}=\left(\sum_{k=0}^{m}\|\partial^{k}_{\xi}f\|^{2}\right)^{\frac{1}{2}}.$

##  2 Preliminaries and Main Theorem

###  2.1  Viscous Shock Profile and Location of the Shift.

As pointed out by [12], the solution of the impermeable wall problem
(1.1)-(1.4) is expected to tend toward the outgoing viscous shock $(V,U)(\xi)$
satisfying

$\left\\{\begin{array}[]{ll}&{-s}{V}^{\prime}-{U}^{\prime}=0,\\\
&{-s}U^{\prime}+p(V)^{\prime}=\left(\frac{U^{\prime}}{V^{\alpha+1}}\right)^{\prime},\\\
&(V,U)(-\infty)=(v_{-},0),\quad(V,U)(+\infty)=(v_{+},u_{+}),\end{array}\right.$
(2.8)

where ${}^{\prime}=\operatorname{d}/\operatorname{d\xi}$, $\xi=x-st$, $s$ is
the shock speed determined by the RH condition (1.7) and $v_{\pm}>0,u_{+}<0$
are given constants. From $(\ref{2.1})_{1}$ and $(\ref{2.1})_{2}$, one gets

$\displaystyle\begin{split}&s^{2}V^{\prime}+p(V)^{\prime}=-\left(\frac{sV^{\prime}}{V^{\alpha+1}}\right)^{\prime}.\end{split}$
(2.9)

Integrating (2.9) over $(\pm\infty,\xi)$ gives

$\displaystyle\begin{split}&\frac{sV^{\prime}}{V^{\alpha+1}}=-s^{2}V-p(V)-b=:h(V),V(\pm\infty)=v_{\pm},\end{split}$
(2.10)
$\displaystyle\begin{split}&U=-s(V-v_{-})=-s(V-v_{+})+u_{+},\end{split}$
(2.11)

where $b=-s^{2}v_{\pm}-p(v_{\pm})$.

######  Proposition 2.1 ([12]).

There exists a unique viscous shock profile $(V,U)(\xi)$ up to a shift
satisfying

$\displaystyle 0<v_{-}<V(\xi)<v_{+},\quad h(V)>0,\quad U^{\prime}<0,$ (2.12)
$\displaystyle\left|V(\xi)-v_{\pm}\right|=O(1)\left|v_{+}-v_{-}\right|e^{-C_{\pm}|\xi|},$
(2.13)

as $\xi\rightarrow\pm\infty,$ where
$C_{\pm}=\frac{v_{\pm}^{\alpha+1}}{s}|p^{\prime}(v_{\pm})+s^{2}|$,
$s=\frac{-u_{+}}{v_{+}-v_{-}}.$

We expect
$\int_{0}^{\infty}[v(x,t)-V(x-st+\beta_{0}-\beta)]\operatorname{d}x\rightarrow
0$ as $t\rightarrow\infty$. As in [12], the shift of viscous shock profile is
given by

$\displaystyle\beta_{0}=\frac{1}{v_{+}-v_{-}}\left\\{\int_{0}^{\infty}[{v}_{0}(x)-V(x-\beta)]\operatorname{d}x+\int_{0}^{\infty}U(-st-\beta)\operatorname{d}t\right\\}.$
(2.14)

###  2.2 Main Theorem.

We assume that for $\beta>0$, the initial data satisfies

$\displaystyle\begin{split}&v_{0}(x)-V(x-\beta)\in H^{1}\cap L^{1}\quad
u_{0}(x)-U(x-\beta)\in H^{1}\cap L^{1},\end{split}$ (2.15)

and

$u_{0}(0)=0$ (2.16)

as the compatibility condition. Set

$\displaystyle(A_{0},B_{0})(x):=-\int_{x}^{\infty}(v_{0}(y)-V(y-\beta),u_{0}(y)-U(y-\beta))\operatorname{d}y.$

We further assume that

$\displaystyle(A_{0},B_{0})\in L^{2}.$ (2.17)

The shift $\beta_{0}$ has the following properties.

######  Lemma 2.2 ([12]).

Under the assumptions (2.15)-(2.17), the shift $\beta_{0}$ defined by (2.14)
satisfies

$\beta_{0}\rightarrow 0\quad\text{as}\quad\|A_{0},B_{0}\|_{2}\rightarrow
0\quad\text{and}\quad\beta\rightarrow+\infty.$

The main theorem is stated as follows.

######  Theorem 1.

For any $u_{+}<0$ and $v_{+}>0$, suppose that (2.15)-(2.17) hold. Then there
exists a positive constant $\delta_{0}$ such that if

$\|(A_{0},B_{0})\|_{2}+\beta^{-1}\leq\delta_{0},$

then the initial-boundary value problem (1.1), (1.4) has a unique global
solution $(v,u)(x,t)$, satisfying

$\displaystyle v$ $\displaystyle(x,t)-V(x-st+\beta_{0}-\beta)\in
C^{0}([0,+\infty);H^{1})\cap L^{2}([0,+\infty);H^{1}),$ $\displaystyle u$
$\displaystyle(x,t)-U(x-st+\beta_{0}-\beta)\in C^{0}([0,+\infty);H^{1})\cap
L^{2}([0,+\infty);H^{2}),$ (2.18)

where $s>0$ is defined by (1.7), and

$\displaystyle\begin{split}&\sup_{x\in\mathbb{R}_{+}}|{(v,u)(x,t)}-(V,U)(x-st+\beta_{0}-\beta)|\rightarrow
0,\text{ as }t\rightarrow+\infty.\end{split}$ (2.19)

######  Remark 2.3.

The condition $v_{+}-v_{-}<C(\gamma-1)^{-2}$ in [12] is removed.

##  3 Reformulation of the Original Problem

Set

$\displaystyle\phi$
$\displaystyle(x,t)=-\int^{\infty}_{x}{}{v}(y,t)-V(y-st+\beta_{0}-\beta)\operatorname{d}y,$
$\displaystyle\psi$
$\displaystyle(x,t)=-\int^{\infty}_{x}{}{u}(y,t)-U(y-st+\beta_{0}-\beta)\operatorname{d}y,$
(3.20)

which means that we look for the solution $(v,u)(x,t)$ in the form

$\displaystyle v$ $\displaystyle(x,t)=\phi_{x}(x,t)+V(x-st+\beta_{0}-\beta),$
$\displaystyle u$ $\displaystyle(x,t)=\psi_{x}(x,t)+U(x-st+\beta_{0}-\beta).$
(3.21)

The initial perturbations $\phi$ and $\psi$ satisfy

######  Lemma 3.1 ([12]).

Under the assumptions (2.15)-(2.17), the initial perturbation
$(\phi,\psi)(x,0):=\left(\phi_{0},\psi_{0}\right)(x)\in H^{2}$ and satisfies

$\left\|\left(\phi_{0},\psi_{0}\right)\right\|_{2}\rightarrow 0\quad\text{ as
}\quad\left\|\left(A_{0},B_{0}\right)\right\|_{2}\rightarrow 0\text{ and
}\beta\rightarrow+\infty.$

Motivated by [12], substitute (3) into (1.1) and integrate the resulting
system with respect to $x$, we have

$\left\\{\begin{array}[]{ll}&\phi_{t}-\psi_{x}=0,\\\
&\psi_{t}-f(V)\phi_{x}-\frac{{\psi_{xx}}}{V^{\alpha+1}}=F,\end{array}\right.$
(3.22)

with the initial conditions and Neumann boundary condition:

$\displaystyle\left(\phi_{0},\psi_{0}\right)$ $\displaystyle(x)\in H^{2},\quad
x\geq 0,$ $\displaystyle\left.\psi_{x}\right|_{x=0}$
$\displaystyle=\left.\phi_{t}\right|_{x=0}=-U(st+\beta_{0}-\beta),\quad t\geq
0,$ (3.23)

where

$\displaystyle
f(V)=-p^{\prime}(V)+(\alpha+1)\frac{{}sV_{x}}{V^{\alpha+2}}=-p^{\prime}(V)+(\alpha+1)\frac{h(V)}{V}>0,$
(3.24) $\displaystyle F$
$\displaystyle=\frac{{}{u}_{x}}{{}{v}^{\alpha+1}}-\frac{U_{x}}{V^{\alpha+1}}-\frac{\psi_{xx}}{V^{\alpha+1}}+(\alpha+1)\frac{U_{x}\phi_{x}}{V^{\alpha+2}}-\left[p({}{v})-p(V)-p^{\prime}(V)\phi_{x}\right]$
$\displaystyle=O(1)(|\phi_{x}|^{2}+|\phi_{x}\psi_{xx}|).$ (3.25)

We will seek the solution in the functional space $X_{\delta}(0,T)$ for any
$0\leq T<+\infty$,

$\displaystyle\begin{split}X_{\delta}(0,T):=&\left\\{(\phi,\psi)\in
C([0,T];H^{2})|\phi_{x}\in L^{2}(0,T;H^{1}),\psi_{x}\in
L^{2}(0,T;H^{2})\right.\\\ &\sup_{0\leq t\leq
T}\|(\phi,\psi)(t)\|_{2}\leq\delta\\},\end{split}$

where ${\delta}\ll 1$ is small.

######  Proposition 3.2.

(A priori estimate) Suppose that $(\phi,\psi)\in X_{\delta}(0,T)$ is the
solution of (3.22), (3) for some time $T>0$. There exists a positive constant
$\delta_{0}$ independent of $T$, such that if

$\sup_{0\leq t\leq T}\|(\phi,\psi)(t)\|_{{2}}\leq\delta\leq\delta_{0},$

for $t\in[0,T]$, then

$\displaystyle\|(\phi,\psi)(t)\|_{{2}}^{2}+\int_{0}^{t}(\|\phi_{x}(t)\|^{2}_{1}+\|\psi_{x}(t)\|_{2}^{2})\operatorname{d}t\leq
C_{0}(\|(\phi_{0},\psi_{0})\|_{{2}}^{2}+e^{-C_{-}\beta}),$

where $C_{0}>1$ and $C_{-}$ are positive constants independent of $T$.

As long as Proposition 3.2 is proved, the local solution $(\phi,\psi)$ can be
extend to $T=+\infty.$ We have the following Lemma.

######  Lemma 3.3.

If $(\phi_{0},\psi_{0})\in H^{2}$, there exists a positive constant
$\delta_{1}=\frac{\delta_{0}}{\sqrt{C_{0}}}$, such that if

$\|(\phi_{0},\psi_{0})\|_{{2}}^{2}+e^{-C_{-}\beta}\leq\delta_{1}^{2},$

then the initial-boundary problem (3.22), (3) has a unique global solution
$(\phi,\psi)\in X_{\delta_{0}}(0,\infty)$ satisfying

$\displaystyle\sup_{t\geq
0}\|(\phi,\psi)(t)\|_{{2}}^{2}+\int_{0}^{\infty}(\|\phi_{x}(t)\|^{2}_{1}+\|\psi_{x}(t)\|_{2}^{2})\operatorname{d}t\leq
C_{0}(\|(\phi_{0},\psi_{0})\|_{{2}}^{2}+e^{-C_{-}\beta}).$

##  4 A Priori Estimate

Throughout this section, we assume that the problem $(\ref{3.3}),(\ref{3.4})$
has a solution $(\phi,\psi)\in X_{\delta}(0,T),$ for some $T>0$,

$\displaystyle\sup_{0\leq t\leq T}\|(\phi,\psi)(t)\|_{2}\leq\delta.$ (4.26)

It follows from the Sobolev inequality that $\frac{1}{2}v_{+}\leq
v\leq\frac{3}{2}v_{-}$, and

$\displaystyle\sup_{0\leq t\leq
T}\\{\|(\phi,\psi)(t)\|_{L^{\infty}}+\|(\phi_{x},\psi_{x})(t)\|_{L^{\infty}}\\}\leq{\delta}.$

###  4.1  Low Order Estimate.

In order to remove the condition $v_{+}-v_{-}<C(\gamma-1)^{-2}$ in [12], we
introduce a new perturbation $(\phi,\Psi)$ instead of $(\phi,\psi)$, where
$\Psi$ will be defined below.

Inspired by [19] and [3], we introduce a new variable $h$ which depends on $v$
and $u$, i.e., ${}{h}={}{u}-{}{v}^{-(\alpha+1)}{}{v}_{x}$. Through a direct
calculation, $v$ and $h$ satisfy the following system

$\left\\{\begin{array}[]{ll}&{}{v}_{t}-{}{h}_{x}=(\frac{{}{v}_{x}}{{}{v}^{\alpha+1}})_{x},\\\
&{}{h}_{t}+{}{p}_{x}=0.\end{array}\right.$ (4.27)

Then the initial-boundary conditions given in (1.4) are changed into

$\left\\{\begin{array}[]{ll}(v,h)(x,0)=(v_{0},u_{0}-{}{v_{0}}^{-(\alpha+1)}{}{v}_{0x})(x)\longrightarrow(v_{+},u_{+}),~{}x\to+\infty,\\\
h(0,t)={u}(0,t)-{}{v(0,t)}^{-(\alpha+1)}{}{v_{x}(0,t)}=-{}{v(0,t)}^{-(\alpha+1)}{}{v_{x}(0,t)},t\in\mathbb{R_{+}}.\end{array}\right.$

Let $H=U-V^{-(\alpha+1)}V_{x}$. Then (2.8) is equivalent to

$\left\\{\begin{array}[]{rl}V_{t}-H_{x}=&\left(\frac{V_{x}}{V^{\alpha+1}}\right)_{x},\\\
H_{t}+p(V)_{x}=&0,\\\
(V,H)(-\infty)=&(v_{-},0),\quad(V,H)(+\infty)=(v_{+},u_{+}).\end{array}\right.$
(4.28)

We define

$\displaystyle-\int_{x}^{\infty}({}{h}-H)\operatorname{d}x=\Psi.$ (4.29)

Substituting (4.28) from (4.27) and integrating the resulting system with
respect to $x$, we have from (4.29), $(\ref{3.1})_{1}$ that

$\left\\{\begin{array}[]{ll}&\phi_{t}-\Psi_{x}-\frac{\phi_{xx}}{V^{\alpha+1}}+(\alpha+1)\frac{V_{x}\phi_{x}}{V^{\alpha+2}}=G,\\\
&\Psi_{t}+p^{\prime}(V)\phi_{x}=-p({v}|V),\end{array}\right.$ (4.30)

where

$\displaystyle
G=\frac{{}{v}_{x}}{{}{v}^{\alpha+1}}-\frac{V_{x}}{V^{\alpha+1}}-\frac{\phi_{xx}}{V^{\alpha+1}}+(\alpha+1)\frac{V_{x}\phi_{x}}{V^{\alpha+2}},$
$\displaystyle p({v}|V)=\left(p({}{v})-p(V)\right)-p^{\prime}(V)\phi_{x},$

with the initial data

$\displaystyle\phi(x,0)\in H^{2},\quad\Psi(x,0)\in H^{1},$

and boundary data

$\displaystyle\Phi(0,t)=-\int_{x}^{\infty}[{u}(y,0)-U(y+\beta_{0}-\beta)]\operatorname{d}y+\left(V^{-(\alpha+1)}-{}{v}^{-(\alpha+1)}\right)(x,0).$

######  Lemma 4.1.

([3]) Under the assumption of (4.26), it holds that

$\displaystyle\begin{split}&p({}{v}|V)\leq C\phi_{x}^{2},\\\
&|p({}{v}|V)_{x}|\leq C(|\phi_{xx}\phi_{x}|+|V_{x}|\phi_{x}^{2}),\\\ &|G|\leq
C(|\phi_{xx}\phi_{x}|+|V_{x}|\phi^{2}_{x}).\end{split}$

In addition, some boundary estimates are given as follows.

######  Lemma 4.2.

Under the same assumptions of Proposition 3.2, for $0\leq t\leq T$, it holds
that:

$\displaystyle\left|\int_{0}^{t}(\phi\Psi)|_{x=0}\operatorname{d}t\right|\leq
Ce^{-C_{-}\beta},\quad\,\,\,\,\,\,\,\left|\int_{0}^{t}\left(\phi\phi_{x}\right)|_{x=0}\operatorname{d}t\right|\leq
Ce^{-C_{-}\beta},$ (4.31)
$\displaystyle\left|\int_{0}^{t}(\phi_{x}\phi_{t})|_{x=0}\operatorname{d}t\right|\leq
Ce^{-C_{-}\beta},\quad\,\,\,\,\left|\int_{0}^{t}\left(\psi_{x}\psi_{t}\right)|_{x=0}\operatorname{d}t\right|\leq
Ce^{-C_{-}\beta},$ (4.32)
$\displaystyle\left|\int_{0}^{t}(\psi_{x}\psi_{xx})|_{x=0}\operatorname{d}t\right|\leq
Ce^{-C_{-}\beta},\quad\left|\int_{0}^{t}\left(\psi_{xt}\psi_{xx}\right)|_{x=0}\operatorname{d}t\right|\leq
Ce^{-C_{-}\beta},$ (4.33)

and

$\displaystyle\|{\Psi}_{0}\|_{1}^{2}$ $\displaystyle\leq$
$\displaystyle\|{\psi}_{0}\|_{1}^{2}+C\|{\phi}_{0}\|_{2}^{2},\quad\|\psi\|^{2}\leq\|\Psi\|^{2}+C\|\phi\|_{1}^{2},$
$\displaystyle\|\psi_{x}\|^{2}$ $\displaystyle\leq$
$\displaystyle\|\Psi_{x}\|^{2}+C\|\phi_{x}\|_{1}^{2},$ (4.34)

where $C_{-}=\frac{v_{-}^{\alpha+1}}{s}|p^{\prime}(v_{-})+s^{2}|>0$.

######  Proof 4.3.

Note that

$\displaystyle\Psi(x,t)$ $\displaystyle=$
$\displaystyle-\int_{x}^{\infty}[{u}(y,t)-U(y-st+\beta_{0}-\beta)]\operatorname{d}y$
$\displaystyle+\left(V^{-(\alpha+1)}-{}{v}^{-(\alpha+1)}\right)(x,t)$
$\displaystyle=:$ $\displaystyle\psi(x,t)+p(x,t)\leq\psi(x,t)+C\phi_{x}(x,t),$
$\displaystyle\psi(x,t)$ $\displaystyle=$
$\displaystyle\Psi(x,t)-p(x,t)\leq\psi(x,t)+C\phi_{x}(x,t),$ (4.35)

one have (4.2) from (4.3) immediately. Motivated by [12], we have

$\displaystyle|\psi(0,t)|\leq C,\quad|\phi_{x}(0,t)|\leq
C,\quad|\phi(0,t)|\leq Ce^{-C_{-}\beta}e^{-C_{-}st}.$ (4.36)

Combining (4.3) and (4.36), we have (4.31). The estimates (4.32) and (4.33)
can be found in [12]. Thus the proof is completed.

######  Lemma 4.4.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\begin{split}&\|(\phi,\Psi)\|^{2}(t)+\int_{0}^{t}\int_{0}^{\infty}\left(\frac{1}{p^{\prime}(V)}\right)_{t}\Psi^{2}\operatorname{d}x\operatorname{d}t+\int_{0}^{t}\|\phi_{x}\|^{2}\operatorname{d}t\\\
\leq&~{}C\|(\phi_{0},\Psi_{0})\|^{2}+C{\delta}\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+Ce^{-C_{-}\beta}.\end{split}$

######  Proof 4.5.

Multiply $(\ref{4.5})_{1}$ and $(\ref{4.5})_{2}$ by $\phi$ and
$\frac{\Psi}{-p^{\prime}(V)}$ respectively, sum them up, and integrate the
result with respect to $t$ and $x$ over $[0,t]\times[0,\infty)$. We have

$\displaystyle\frac{1}{2}\int_{0}^{\infty}\left(\phi^{2}-\frac{\Psi^{2}}{p^{\prime}(V)}\right)\operatorname{d}x+\int_{0}^{t}\int_{0}^{\infty}\left\\{\frac{1}{2}\left(\frac{1}{p^{\prime}(V)}\right)_{t}\Psi^{2}+\frac{\phi_{x}^{2}}{V^{\alpha+1}}\right\\}\operatorname{d}x\operatorname{d}t$
$\displaystyle=$
$\displaystyle\int_{0}^{t}\int_{0}^{\infty}G\phi\operatorname{d}x\operatorname{d}t+\int_{0}^{t}\int_{0}^{\infty}\frac{p(v|V)\Psi}{p^{\prime}(V)}\operatorname{d}x\operatorname{d}t$
$\displaystyle-\int_{0}^{t}(\phi\Psi+(V^{-(\alpha+1)})\phi\phi_{x})|_{x=0}\operatorname{d}t+\frac{1}{2}\int_{0}^{\infty}\left(\phi^{2}-\frac{\Psi^{2}}{p^{\prime}(V)}\right)\Big{|}_{t=0}\operatorname{d}x$
$\displaystyle=:$ $\displaystyle\sum_{i=1}^{4}A_{i}.$ (4.37)

Utilize Lemma 4.1, we can get

$\displaystyle|A_{1}+A_{2}|$ $\displaystyle\leq$ $\displaystyle
C\left(\int_{0}^{t}\int_{0}^{\infty}\left|\phi_{x}\phi_{xx}\phi\right|+\left|V_{x}\phi_{x}^{2}\phi\right|+\left|\Psi\phi_{x}^{2}\right|\operatorname{d}x\operatorname{d}t\right)$
$\displaystyle\leq$ $\displaystyle
C\int_{0}^{t}\|\phi\|_{L^{\infty}}\int_{0}^{\infty}\left|\phi_{x}\phi_{xx}\right|\operatorname{d}x\operatorname{d}t+C\int_{0}^{t}(\|\phi\|_{L^{\infty}}+\|\Psi\|_{L^{\infty}})\int_{0}^{\infty}\phi_{x}^{2}\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq$ $\displaystyle
C(\|\phi\|_{2}+\|\psi\|_{1})\int_{0}^{t}\|\phi_{x}\|^{2}+\|\phi_{xx}\|^{2}\operatorname{d}t$
$\displaystyle\leq$ $\displaystyle
C\delta\int_{0}^{t}\|\phi_{x}\|^{2}+\|\phi_{xx}\|^{2}\operatorname{d}t.$
(4.38)

With the help of Lemma 4.27, one has

$\displaystyle\begin{split}|A_{3}|\leq&Ce^{-C_{-}\beta}.\end{split}$ (4.39)

Taking $\delta$ sufficiently small, using (4.5)–(4.39), we get Lemma 4.4.

######  Lemma 4.6.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\|(\phi,\Psi)(t)\|_{1}^{2}+\int_{0}^{t}\|\phi_{x}\|_{1}^{2}\operatorname{d}t\leq
C\|(\phi_{0},\Psi_{0})\|_{1}^{2}+Ce^{-C_{-}\beta}.$

######  Proof 4.7.

Multiply $(\ref{4.5})_{1}$ and $(\ref{4.5})_{2}$ by $-\phi_{xx}$ and
$\frac{\Psi_{xx}}{p^{\prime}(V)}$ respectively, sum over the result, integrate
the result with respect to $t$ and $x$ over $[0,t]\times[0,\infty)$. We have

$\displaystyle\frac{1}{2}\int_{0}^{\infty}\left(\phi_{x}^{2}-\frac{\Psi_{x}^{2}}{p^{\prime}(V)}\right)\operatorname{d}x+\int_{0}^{t}\int_{0}^{\infty}\left\\{\frac{1}{2}\left(\frac{1}{p^{\prime}(V)}\right)_{t}\Psi_{x}^{2}+\frac{\phi^{2}_{xx}}{V^{\alpha+1}}\right\\}\operatorname{d}x\operatorname{d}t$
$\displaystyle=$
$\displaystyle\frac{1}{2}\int_{0}^{\infty}\left(\phi_{x}^{2}-\frac{\Psi_{x}^{2}}{p^{\prime}(V)}\right)\Big{|}_{t=0}\operatorname{d}x$
$\displaystyle-\int_{0}^{t}\int_{0}^{\infty}\left[G-(\alpha+1)\frac{V_{x}}{V^{\alpha+2}}\phi_{x}\right]\phi_{xx}\operatorname{d}x\operatorname{d}t$
$\displaystyle-\int_{0}^{t}\int_{0}^{\infty}\left(\frac{1}{p^{\prime}(V)}\right)_{x}p^{\prime}(V)\Psi_{x}\phi_{x}\operatorname{d}x\operatorname{d}t$
$\displaystyle-\int_{0}^{t}\left(\phi_{t}\phi_{x}-\phi_{x}\Psi_{x}-\frac{\Psi_{t}\Psi_{x}}{p^{\prime}(V)}-\frac{p(v|V)}{p^{\prime}(V)}\Psi_{x}\right)\Big{|}_{x=0}\operatorname{d}t$
$\displaystyle+\int_{0}^{t}\int_{0}^{\infty}\frac{1}{p^{\prime}(V)}p(v|V)_{x}\Psi_{x}\operatorname{d}x\operatorname{d}t$
$\displaystyle=:$
$\displaystyle\frac{1}{2}\int_{0}^{\infty}\left(\phi_{x}^{2}-\frac{\Psi_{x}^{2}}{p^{\prime}(V)}\right)\Big{|}_{t=0}\operatorname{d}x+\sum_{i=1}^{4}B_{i}.$
(4.40)

Now we estimate $B_{i}$ term by term. The Cauchy inequality indicates that

$\displaystyle|B_{1}|$ $\displaystyle\leq
C\int_{0}^{t}\int_{0}^{\infty}(|\phi_{xx}\phi_{x}|+|V_{x}\phi^{2}_{x}|)|{\phi_{xx}}|+|\phi_{x}\phi_{xx}|\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq(C\delta+\varepsilon)\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}\|\phi_{x}\|^{2}\operatorname{d}t,$
(4.41)

and

$\displaystyle|B_{2}|\leq$
$\displaystyle\int_{0}^{t}\int_{0}^{\infty}\left|p^{\prime}(V)\Psi_{x}\phi_{x}\left(\frac{1}{p^{\prime}(V)}\right)_{x}\right|\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq$
$\displaystyle\frac{1}{4}\int_{0}^{t}\int_{0}^{\infty}\left(\frac{1}{p^{\prime}(V)}\right)_{t}\Psi_{x}^{2}\operatorname{d}x\operatorname{d}t+C\int_{0}^{t}\|\phi_{x}\|^{2}\operatorname{d}t.$
(4.42)

Making use of the estimate (4.32) for the boundary, we have

$\displaystyle B_{3}$
$\displaystyle=-\int_{0}^{t}\left(\phi_{t}\phi_{x}-\phi_{x}\Psi_{x}-\frac{\Psi_{t}\Psi_{x}}{p^{\prime}(V)}-\frac{p(v|V)}{p^{\prime}(V)}\Psi_{x}\right)\Big{|}_{x=0}\operatorname{d}t$
$\displaystyle=-\int_{0}^{t}(\phi_{t}\phi_{x})|_{x=0}\leq Ce^{-C_{-}\beta}.$
(4.43)

By (4.2) and the Sobolev inequality, we obtain

$\displaystyle|B_{4}|\leq$
$\displaystyle\int_{0}^{t}\int_{0}^{\infty}\left|\frac{1}{p^{\prime}(V)}p({v}|V)_{x}\Psi_{x}\right|\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq$ $\displaystyle
C\int_{0}^{t}\int_{0}^{\infty}\left|(\phi_{x}\phi_{xx}+V_{x}\phi_{x}^{2})\Psi_{x}\right|\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq$ $\displaystyle
C\int_{0}^{t}\int_{0}^{\infty}\left\\{\left|(\phi_{x}\phi_{xx}+V_{x}\phi_{x}^{2})\psi_{x}\right|+\left|(\phi_{xx}\phi_{xx}+V_{x}\phi_{x}\phi_{xx})\phi_{x}\right|\right\\}\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq$ $\displaystyle
C(\|\phi\|_{2}+\|\psi\|_{2})\int_{0}^{t}\|\phi_{x}\|^{2}+\|\phi_{xx}\|^{2}\operatorname{d}t$
$\displaystyle\leq$ $\displaystyle
C\delta\int_{0}^{t}(\|\phi_{xx}\|^{2}+\|\phi_{x}\|^{2})\operatorname{d}t.$
(4.44)

From (4.40)–(4.44), we get

$\displaystyle\begin{split}&\frac{1}{2}\int_{0}^{\infty}\left(\phi_{x}^{2}-\frac{\Psi_{x}^{2}}{p^{\prime}(V)}\right)\operatorname{d}x+\frac{1}{4}\int_{0}^{t}\int_{0}^{\infty}\left[\left(\frac{1}{p^{\prime}(V)}\right)_{t}\Psi_{x}^{2}+\frac{\phi^{2}_{xx}}{V^{\alpha+1}}\right]\operatorname{d}x\operatorname{d}t\\\
\leq&(C+C\delta+C_{\varepsilon})\int_{0}^{t}\|\phi_{x}\|^{2}\operatorname{d}t+(C\delta+\varepsilon)\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t\\\
&+Ce^{-C_{-}\beta}+C\left(\|\phi_{0x}\|^{2}+\|\Psi_{0x}\|^{2}\right).\end{split}$

Choosing $\varepsilon$ sufficiently small, together with Lemma 4.4, we
complete the proof of Lemma 4.6.

######  Lemma 4.8.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\int_{0}^{t}\|\Psi_{x}(t)\|^{2}\operatorname{d}t\leq
C\|(\phi_{0},\Psi_{0})\|_{1}^{2}+Ce^{-C_{-}\beta}.$

######  Proof 4.9.

Multiply $(\ref{4.5})_{1}$ by $\Psi_{x}$ and make use of $(\ref{4.5})_{2}$. We
get

$\displaystyle\Psi_{x}^{2}=$
$\displaystyle(\phi\Psi_{x})_{t}+[\phi(p(v)-p({V}{}))]_{x}-\phi_{x}(p(v)-p({V}{}))$
$\displaystyle-\frac{\Psi_{x}\phi_{xx}}{V^{\alpha+1}}-\Psi_{x}\left[G-(\alpha+1)\frac{V_{x}\phi_{x}}{V_{\alpha+2}}\right].$
(4.45)

Integrate $(\ref{4.20})$ with respect to $t$ and $x$ over
$[0,t]\times[0,\infty)$. We have

$\displaystyle\int_{0}^{t}\|\Psi_{x}\|^{2}\operatorname{d}t$ $\displaystyle=$
$\displaystyle-\int_{0}^{\infty}\phi\Psi_{x}|_{t=0}\operatorname{d}x+\int_{0}^{t}\int_{0}^{\infty}-\Psi_{x}\left[G-(\alpha+1)\frac{V_{x}\phi_{x}}{V_{\alpha+2}}\right]\operatorname{d}x\operatorname{d}t$
$\displaystyle+\int_{0}^{\infty}\phi\Psi_{x}\operatorname{d}x-\int_{0}^{t}\int_{0}^{\infty}\frac{\Psi_{x}\phi_{xx}}{V^{\alpha+1}}\operatorname{d}x\operatorname{d}t$
$\displaystyle-\int_{0}^{t}\int_{0}^{\infty}\phi_{x}\left(p({v})-p(V)\right)\operatorname{d}x\operatorname{d}t-\int_{0}^{t}\phi(p(v)-p({V}{}))|_{x=0}\operatorname{d}t$
$\displaystyle=:$
$\displaystyle-\int_{0}^{\infty}\phi\Psi_{x}|_{t=0}\operatorname{d}x+\sum_{i=1}^{5}H_{i}.$
(4.46)

We estimate $H_{i}$ term by term. By the Cauchy inequality, it holds that

$\displaystyle H_{1}$ $\displaystyle\leq
C\int_{0}^{t}\int_{0}^{\infty}\Psi_{x}(|\phi_{x}\phi_{xx}|+|V_{x}\phi_{x}|)\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq\varepsilon\int_{0}^{t}\|\Psi_{x}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}(\|\phi_{xx}\|^{2}+\|\phi_{x}\|^{2})\operatorname{d}t.$
(4.47)

In addition, it is straightforward to imply that

$\displaystyle H_{2}+H_{3}+H_{4}$ $\displaystyle\leq$
$\displaystyle\|(\phi,\Psi_{x})\|^{2}+\varepsilon\int_{0}^{t}\|\Psi_{x}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+C\int_{0}^{t}\|\phi_{x}\|^{2}\operatorname{d}t.$
(4.48)

Making use of the estimate (4.31) for the boundary, we have

$\displaystyle\begin{split}H_{5}&=-\int_{0}^{t}\phi(p(v)-p({V}{}))|_{x=0}\operatorname{d}t\leq
C\int_{0}^{t}\phi\phi_{x}|_{x=0}\operatorname{d}t\leq Ce^{-C_{-}\beta}.\\\
\end{split}$ (4.49)

Collecting (4.9)-(4.49) and using Lemma 4.6, we complete the proof of Lemma
4.8.

Combining Lemma 4.4-Lemma 4.8, we obtain the following low order estimates

$\displaystyle\|(\phi,\Psi)\|_{1}^{2}(t)+\int_{0}^{t}\|\Psi_{x}\|^{2}\operatorname{d}t+\int_{0}^{t}\|\phi_{x}\|_{1}^{2}\operatorname{d}t\leq
C\|(\phi_{0},\Psi_{0})\|_{1}^{2}+Ce^{-C_{-}\beta},$

which can be rewritten by the variables $\phi$ and $\psi$ as

######  Lemma 4.10.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\begin{split}&(\|\phi\|_{1}^{2}+\|\psi\|^{2})(t)+\int_{0}^{t}\|\psi_{x}\|^{2}\operatorname{d}t+\int_{0}^{t}\|\phi_{x}\|_{1}^{2}\operatorname{d}t\leq
C\|\phi_{0}\|_{2}^{2}+C\|\psi_{0}\|_{1}^{2}+Ce^{-C_{-}\beta}.\end{split}$

###  4.2 High Order Estimate.

Since the second derivative of $\Psi$ on the boundary is unknown, we turn to
the original equation (3.22) to study the higher order estimates.

######  Lemma 4.11.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\begin{split}&\|\psi_{x}\|^{2}(t)+\int_{0}^{t}\|\psi_{xx}\|^{2}\operatorname{d}t\leq
C\|\phi_{0}\|_{2}^{2}+C\|\psi_{0}\|_{1}^{2}+Ce^{-C_{-}\beta}.\end{split}$
(4.50)

######  Proof 4.12.

Multiplying $(\ref{3.3})_{2}$ by $-\psi_{xx}$, integrating the result with
respect to $t$ and $x$ over $[0,t]\times[0,\infty)$ gives

$\displaystyle\frac{1}{2}\|\psi_{x}\|^{2}(t)+\int_{0}^{t}\int_{0}^{\infty}\frac{{\psi_{xx}^{2}}}{V^{\alpha+1}}\operatorname{d}x\operatorname{d}t$
$\displaystyle=$
$\displaystyle\frac{1}{2}\|\psi_{0x}\|^{2}-\int_{0}^{t}\left\\{\psi_{x}\psi_{t}\right\\}|_{x=0}\operatorname{d}t-\int_{0}^{t}\int_{0}^{\infty}f(V)\phi_{x}\psi_{xx}\operatorname{d}x\operatorname{d}t$
$\displaystyle-\int_{0}^{t}\int_{0}^{\infty}F\psi_{xx}\operatorname{d}x\operatorname{d}t$
$\displaystyle=:$
$\displaystyle\frac{1}{2}\|\psi_{0x}\|^{2}+\sum_{i=1}^{3}M_{i}.$ (4.51)

Making use of the estimate (4.32) for the boundary, we have

$\displaystyle M_{1}\leq Ce^{-C_{-}\beta}.$ (4.52)

The Cauchy inequality implies that

$\displaystyle
M_{2}\leq\varepsilon\int_{0}^{t}\|\psi_{xx}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}\|\phi_{x}\|^{2}\operatorname{d}t.$
(4.53)

By (3) and the Sobolev inequality, yields

$\displaystyle M_{3}$ $\displaystyle\leq
C\int_{0}^{t}\int_{0}^{\infty}\left(\left|\phi_{x}\right|^{2}+\left|\phi_{x}\right|\left|\psi_{xx}\right|\right)\left|\psi_{xx}\right|\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq
C\int_{0}^{t}\int_{0}^{\infty}\left|\phi_{x}\right|\left(\left|\phi_{x}\right|^{2}+\left|\psi_{xx}\right|^{2}\right)\operatorname{d}x\operatorname{d}t$
$\displaystyle\leq
C\delta\int_{0}^{t}\left(\left\|\phi_{x}\right\|^{2}+\left\|\psi_{xx}\right\|^{2}\right)\operatorname{d}t.$
(4.54)

Substituting (4.52)-(4.12) into $(\ref{4.26})$ and using Lemma 4.10, we obtain
(4.50).

######  Lemma 4.13.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\begin{split}&\|\phi_{xx}\|^{2}+\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t\leq
C\|\phi_{0}\|_{2}^{2}+C\|\psi_{0}\|_{1}^{2}+Ce^{-C_{-}\beta}+C\delta\int_{0}^{t}\left\|\psi_{xxx}\right\|^{2}\operatorname{d}t.\end{split}$
(4.55)

######  Proof 4.14.

Differentiating $(\ref{3.3})_{1}$ with respect to $x$, using
$(\ref{3.3})_{2},$ we have

$\displaystyle\begin{split}\frac{\phi_{xt}}{V^{\alpha+1}}+f(V)\phi_{x}=\psi_{t}-F.\end{split}$
(4.56)

Differentiating $(\ref{4.31})$ in respect of $x$ and multiplying the
derivative by $\phi_{xx}$, integrating the result in respect of $t$ and $x$
over $[0,t]\times[0,\infty)$, using (2.10), one has

$\displaystyle\frac{1}{2}\int_{0}^{\infty}\frac{\phi_{xx}^{2}}{V^{\alpha+1}}\operatorname{d}x+\int_{0}^{t}\int_{0}^{\infty}\left(f(V)-\frac{(\alpha+1)h(V)}{2V}\right)\phi_{xx}^{2}\operatorname{d}x\operatorname{d}t$
$\displaystyle=$
$\displaystyle\frac{1}{2}\int_{0}^{\infty}\frac{\phi_{xx}^{2}}{V^{\alpha+1}}\Big{|}_{t=0}\operatorname{d}x-\int_{0}^{\infty}\psi_{x}\phi_{xx}\Big{|}_{t=0}\operatorname{d}x+\int_{0}^{\infty}\psi_{x}\phi_{xx}\operatorname{d}x$
$\displaystyle+\int_{0}^{t}\psi_{x}\psi_{xx}\Big{|}_{x=0}\operatorname{d}t+\int_{0}^{t}\|\psi_{xx}\|^{2}\operatorname{d}t-\int_{0}^{t}\int_{0}^{\infty}F_{x}\phi_{xx}\operatorname{d}x\operatorname{d}t$
$\displaystyle+(\alpha+1)\int_{0}^{t}\int_{0}^{\infty}\frac{{}V_{x}}{V^{\alpha+2}}\phi_{xt}\phi_{xx}\operatorname{d}x\operatorname{d}t-\int_{0}^{t}\int_{0}^{\infty}f(V)_{x}\phi_{x}\phi_{xx}\operatorname{d}x\operatorname{d}t$
$\displaystyle=:$
$\displaystyle\frac{1}{2}\int_{0}^{\infty}\frac{\phi_{xx}^{2}}{V^{\alpha+1}}\Big{|}_{t=0}\operatorname{d}x-\int_{0}^{\infty}\psi_{x}\phi_{xx}\Big{|}_{t=0}\operatorname{d}x+\sum_{i=1}^{6}N_{i}.$
(4.57)

By (2.12) and (3.24), one has

$\displaystyle f(V)-\frac{(\alpha+1)h(V)}{2V}\geq-p^{\prime}(v_{+})>0.$ (4.58)

The Cauchy inequality yields

$\displaystyle
N_{1}\leq\varepsilon\|\phi_{xx}\|^{2}+C_{\varepsilon}\|\psi_{x}\|^{2}.$ (4.59)

Making use of the estimate (4.33) for the boundary, it follows that

$\displaystyle N_{2}\leq Ce^{-C_{-}\beta}.$ (4.60)

$N_{3}$ can be controlled by (4.50). By the Cauchy inequality, we have

$\displaystyle\begin{split}|N_{4}|\leq&\varepsilon\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}\left\|F_{x}\right\|^{2}\operatorname{d}t.\\\
\end{split}$

Using

$\displaystyle\begin{split}\left\|F_{x}\right\|^{2}&\leq
C\int_{0}^{\infty}\left(\phi_{x}^{4}+\phi_{x}^{2}\phi_{xx}^{2}+\psi_{xx}^{2}\phi_{xx}^{2}+\psi_{xxx}^{2}\phi_{x}^{2}+\phi_{x}^{2}\psi_{xx}^{2}\right)\operatorname{d}x\\\
&\leq
C\delta\left(\left\|\phi_{x}\right\|_{1}^{2}+\left\|\psi_{x}\right\|_{2}^{2}\right),\end{split}$

we have the estimate of $N_{4}$

$\displaystyle\begin{split}|N_{4}|\leq&\varepsilon\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+C_{\varepsilon}\delta\int_{0}^{t}\left(\left\|\phi_{x}\right\|_{1}^{2}+\left\|\psi_{x}\right\|_{2}^{2}\right)\operatorname{d}t.\\\
\end{split}$ (4.61)

The Cauchy inequality yields

$\displaystyle\begin{split}|N_{5}|\leq
C&\int_{0}^{t}\int_{0}^{\infty}\left|\frac{{}V_{x}}{V^{\alpha+2}}\psi_{xx}\phi_{xx}\right|\operatorname{d}x\operatorname{d}t\leq\varepsilon\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}\left\|\psi_{xx}\right\|^{2}\operatorname{d}t,\\\
\end{split}$ (4.62)
$\displaystyle\begin{split}|N_{6}|\leq&\varepsilon\int_{0}^{t}\|\phi_{xx}\|^{2}\operatorname{d}t+C_{\varepsilon}\int_{0}^{t}\left\|\phi_{x}\right\|^{2}\operatorname{d}t.\\\
\end{split}$ (4.63)

Choosing $\varepsilon$ small, substituting (4.58)-(4.63) into (4.14) and using
Lemma 4.10, Lemma 4.11, we have (4.55).

On the other hand, differentiating the second equation of (3.22) with respect
to $x$, multiplying the derivative by $-\psi_{xxx}$, integrating the resulting
equality over $[0,\infty)\times[0,t]$, using Lemma 4.10-Lemma 4.13, we can get
the highest order estimate in the same way, which is listed as follows and the
proof is omitted.

######  Lemma 4.15.

Under the same assumptions of Proposition 3.2, it holds that

$\displaystyle\begin{split}&\|\psi_{xx}(t)\|^{2}+\int_{0}^{t}\|\psi_{xxx}\|^{2}\operatorname{d}t\leq
C\|(\phi_{0},\psi_{0})\|_{2}^{2}+Ce^{-C_{-}\beta}.\end{split}$ (4.64)

Finally, Proposition 3.2 is obtained by Lemma 4.8-Lemma 4.15.

##  5 Proof of Theorem 1

Now we turn to the proof of main theorem, i.e., Theorem 1. It is
straightforward to imply (1) from Lemma 3.3. It remains to show (2.19). We
will use the following useful lemma.

######  Lemma 5.1.

([13]) Assume that the function $f(t)\geq 0\in L^{1}(0,+\infty)\cap
BV(0,+\infty)$. Then it holds that $f(t)\rightarrow 0$ as
$t\rightarrow\infty$.

######  Proof 5.2.

(Proof of Theorem 1.) Differentiating the first equation of (3.22) with
respect to $x$, multiplying the resulting equation by $\phi_{x}$, and
integrating on $(0,\infty)$, we have

$\left|\frac{\operatorname{d}}{\operatorname{d}t}\left(\|\phi_{x}\|^{2}\right)\right|\leq
C(\|\phi_{x}\|^{2}+\|\psi_{xx}\|^{2}).$

Using Lemma 3.3, we have

$\int_{0}^{\infty}\left|\frac{\operatorname{d}}{\operatorname{d}t}\left(\|\phi_{x}\|^{2}\right)\right|\operatorname{dt}\leq
C\left\\{\left\|\left(\phi_{0},\psi_{0}\right)\right\|_{2}^{2}+e^{-c_{-}\beta}\right\\}\leq
C,$

which implies $\|\phi_{x}\|^{2}\in L^{1}(0,+\infty)\cap BV(0,+\infty)$. By
Lemma 5.1, we have

$\|\phi_{x}\|\rightarrow 0\quad\text{as}\quad t\rightarrow+\infty.$

Since $\|\phi_{xx}\|$ is bounded, the Sobolev inequality implies that

$\displaystyle\|{}{v}-V\|_{\infty}^{2}=\|\phi_{x}\|_{\infty}^{2}\leq
2\|\phi_{x}(t)\|\|\phi_{xx}(t)\|\rightarrow 0.$

Similarly, we have

$\displaystyle\|{}{u}-U\|_{\infty}^{2}=\|\psi_{x}\|_{\infty}^{2}\leq
2\|\psi_{x}(t)\|\|\psi_{xx}(t)\|\rightarrow 0.$

Therefore, the proof of Theorem 1 is completed.

## References

  * [1] H. Freistuhler, D. Serre, $L^{1}$ stability of shock waves in scalar viscous conservation laws, Comm. Pure Appl. Math., 51(3): 291–301,1998.
  * [2] J. Goodman, Nonlinear asymptotic stability of viscous shock profiles for conservation laws, Arch. Rational Mech. Anal., 95(4): 325–344,1986.
  * [3] L. He, F. Huang, Nonlinear stability of large amplitude viscous shock wave for general viscous gas, J. Differential Equations, 269(2):1226–1242,2020.
  * [4] F. Huang, A. Matsumura, Stability of a composite wave of two viscous shock waves for full compressible Navier-Stokes equation, Comm. Math. Phys., 289(3): 841–861,2009.
  * [5] J. Humpherys, G. Lyng, K. Zumbrun, Multidimensional stability of large-amplitude Navier-Stokes shocks, Arch. Ration. Mech. Anal., 226(3): 923–973,2017.
  * [6] S. Kawashima, A. Matsumura, Asymptotic stability of traveling wave solutions of systems for one-dimensional gas motion, Commun. Math. Phys., 101(1): 97–127,1985.
  * [7] T. Liu, Pointwise convergence to shock waves for viscous conservation laws, Comm. Pure Appl. Math., 50(11): 1113–1182,1997.
  * [8] T. Liu, Y. Zeng, Time-asymptotic behavior of wave propagation around a viscous shock profile, Comm. Math. Phys., 290(1): 23–82,2009.
  * [9] T. Liu, Y. Zeng, Shock waves in conservation laws with physical viscosity, Mem. Amer. Math. Soc., 234(1105): 2015.
  * [10] C. Mascia, K. Zumbrun, Stability of large-amplitude viscous shock profiles of hyperbolic-parabolic system, Arch. Ration. Mech. Anal., 172(1): 93–131,2004.
  * [11] A. Matsumura, Waves in compressible fluids: viscous shock, rarefaction, and contact waves, Handbook of mathematical analysis in mechanics of viscous fluids, Springer, Cham, 2495–2548, 2018.
  * [12] A. Matsumura, M. Mei, Convergence to travelling fronts of solutions of the $p$-system with viscosity in the presence of a boundary, Arch. Ration. Mech. Anal., 146(1), 1–22,1999.
  * [13] A. Matsumura, K. Nishihara, On the stability of travelling wave solutions of a one-dimensional model system for compressible viscous gas, Japan. J. Appl. Math., 2(1): 17–25,1985.
  * [14] A. Matsumura, K. Nishihara, Asymptotic stability of traveling waves for scalar viscous conservation laws with non-convex nonlinearity, Comm. Math. Phys., 165(1): 83–96,1994.
  * [15] A. Matsumura, K. Nishihara, Global Solutions for Nonlinear Differential Equations-Mathematical Analysis on Compressible Viscous Fluids (In Japanese), Nippon Hyoronsha, 2004.
  * [16] A. Matsumura, Y. Wang, Asymptotic stability of viscous shock wave for a one-dimensional isentropic model of viscous gas with density dependent viscosity, Methods Appl. Anal., 17(3): 279–290,2010.
  * [17] J. Smoller, Shock Waves and Reaction-Diffusion Equations, Springer-Verlag, New York, Berlin, 1983.
  * [18] A. Szepessy, Z. Xin,Nonlinear stability of viscous shock waves, Arch. Ration. Mech. Anal., 122(1): 53–103,1993.
  * [19] A. Vasseur, L. Yao, Nonlinear stability of viscous shock wave to one-dimensional compressible isentropic Navier-Stokes equations with density dependent viscous coefficient, Commun. Math. Sci., 14(8): 2215–2228,2016.
  * [20] K. Zumbrun, Stability of large-amplitude shock waves of compressible Navier-Stokes equations, with an appendix by Helge Kristian Jenssen and Gregory Lyng Handbook of Mathematical Fluid Dynamics, vol. III, North-Holland, Amsterdam, 311–533, 2004.


11institutetext: University College London, London, UK
11email: {david.griffiths.16, j.boehm<EMAIL_ADDRESS>
# Finding Your (3D) Center:
3D Object Detection Using a Learned Loss

David Griffiths*  0000-0002-8582-138X Jan Boehm  0000-0003-2190-0449 Tobias
Ritschel

###### Abstract

Massive semantically labeled datasets are readily available for 2D images,
however, are much harder to achieve for 3D scenes. Objects in 3D repositories
like ShapeNet are labeled, but regrettably only in isolation, so without
context. 3D scenes can be acquired by range scanners on city-level scale, but
much fewer with semantic labels. Addressing this disparity, we introduce a new
optimization procedure, which allows training for 3D detection with raw 3D
scans while using as little as 5 % of the object labels and still achieve
comparable performance. Our optimization uses two networks. A _scene network_
maps an entire 3D scene to a set of 3D object centers. As we assume the scene
not to be labeled by centers, no classic loss, such as Chamfer can be used to
train it. Instead, we use another network to emulate the loss. This _loss
network_ is trained on a small labeled subset and maps a non-centered 3D
object in the presence of distractions to its own center. This function is
very similar – and hence can be used instead of – the gradient the supervised
loss would provide. Our evaluation documents competitive fidelity at a much
lower level of supervision, respectively higher quality at comparable
supervision. Supplementary material can be found at: dgriffiths3.github.io

###### Keywords:

3D learning; 3D point clouds; 3D object detection; Unsupervised

##  1 Introduction

††* Corresponding author.

We can reason about one 3D chair as we do about a 2D chair image, however, we
cannot yet machine-understand a point cloud of a 3D room as we would do for a
2D room image. For 2D images, massive amounts of manual human labeling have
enabled amazing state-of-the-art object detectors [26, 10, 19, 34]. We also
have massive repositories of clean 3D objects [3] which we can classify thanks
to deep 3D point processing [25]. But we do not have, despite commendable
efforts [28, 6], and probably might never have, 3D scene labeling at the
extent of 2D images. We hence argue that progress in 3D understanding even
more critically depends on reducing the amount of supervision required.

While general unsupervised detection is an elusive goal, we suggest taking a
shortcut: while we do not have labeled 3D scenes, we do have labeled 3D
objects. The key idea in this work is to first teach a loss network everything
that can be learned from seeing snippets of labeled objects. Next, we use this
network to learn a scene network that explores the relation of objects within
scenes, but without any scene labels, i. e., on raw scans.

After reviewing previous work, we will show how this cascade of networks is
possible when choosing a slightly more primitive loss than the popular Chamfer
loss and we propose two network architectures to implement it. Results show
how a state-of-the-art, simple, fast and feed-forward 3D detection network can
achieve similar Chamfer distance and<EMAIL_ADDRESS>scores to a supervised approach,
but with only 5 % of the labels.

##  2 Previous Work

2D object detection has been addressed by deep-learning based approaches like
Fast R-CNN [10], YOLO [26], SSD [19] or the stacked hourglass architecture
[21] with great success.

In early work Song and Xiao [29] have extended sliding window-detection to a
3D representation using voxels with templates of Hough features and SVM
classifiers. This approach was later extended to deep templates [30]. Both
approaches use fully-supervised training on object locations given by bounding
boxes. We compare to such a sliding window approach using a point-based deep
template. Hou et al. [14] complement a voxel-based approach with color 2D
image information which more easily represents finer details.

Karpathy et al. [15] detect objects by over-segmenting the scene and
classifying segments as objects based on geometric properties such as
compactness, smoothness, etc. Similarly, Chen et al. [4] minimize other
features to 3D-detect objects in street scans.

While modern software libraries make voxels simple to work with, they are
limited in the spatial extent of scenes they can process, and the detail of
the scene they can represent. Qi et al. [24] were first to suggest an approach
to work on raw point clouds. 3D object detection in point clouds is
investigated by Qi et al. [23] and Engelcke et al. [7] map the scene to votes,
then those votes are clustered and each cluster becomes a proposal. The
vectors pointing from a seed to a vote are similar to the loss network
gradients proposed in our method, but for VoteNet, this is part of the
architecture during training and testing while for us these vectors are only
part of the training. Finally, VoteNet is trained fully supervised with object
positions. The idea of merging 2D images and 3D processing is applicable to
point clouds as well, as shown by Ku et al. [16] and Qi et al. [22].

Zhou and Tuzel [35] question the usefulness of points for 3D detection and
have suggested to re-sample point clouds to voxels again. Also Chen et al. [5]
show how combining point inputs, volume convolutions and point proposals can
lead to good results. For a survey on 3D detection, also discussing trade-offs
of points and voxels, see the survey by Griffiths and Boehm [11].

Our architecture is inspired by Fast R-CNN [10, 27], which regresses object
proposals in one branch, warps them into a canonical frame and classifies them
in the other branch. Recently, Yang et al. [33] have shown how direct mapping
of a point cloud to bounding boxes is feasible. Feng et al. [8] train a
network with supervision that makes multiple proposals individually and later
reasons about their relation. Also, Zhou et al. [34] first work on center
points for object representation alone and later regress the 2D bounding box
and all other object features from image content around the points. All these
works tend to first extract proposals in a learned fashion and then reason
about their properties or relations in a second, learned step. We follow this
design for the scene network, but drive its learning in an entirely different,
unsupervised, way. Finally, all of these works require only one feed-forward
point cloud network pass, a strategy we will follow as well.

Unrelated to 2D or 3D detection, Adler and Öktem [1] have proposed to replace
the gradient computation in an optimization problem by a neural network. In
computer vision, this idea has been used to drive light field [9] or
appearance synthesis [20]. We take this a step further and use a network to
emulate the gradients in a very particular optimization: the training of
another network.

##  3 Our Approach

We learn two networks: a scene network and a loss network (Fig. 1). The first
(Fig. 1, bottom) is deployed, while the second (Fig. 1, top) is only used in
training.

Figure 1: Our approach proceeds in two steps of training (row) with different
training data (column one and two), networks (column three), outputs (column
four), gradients (column five) and supervision (column six). Object level
training (first row) data comprises of 3D scene patches with known objects
that are not centered. The loss network maps off-center scenes to their center
(big black arrow). Its learning follows the gradient of a quadratic potential
(orange field) that has the minimum at the offset that would center the
object. This offset is the object-level supervision, as seen in the last
column. The scene network (second row) is trained to map a scene to all object
centers, here for three chairs. The gradient to train the scene network is
computed by running the loss network from the previous step once for each
object (here three times: blue, pink, green). Note, that there is no scene-
level supervision (cross).

The _scene network_ maps 3D scenes to sets of 3D object centers. The input
data is a 3D point cloud. The output is a fixed sized list of 3D object
centers. We assume a feed-forward approach, that does not consider any
proposals [14, 21, 29, 30] or voting [23, 22], but directly regresses centers
from input data [34, 33].

The _loss network_ emulates the loss used to train the scene network. The
input data is again a 3D point cloud, but this time of a single object,
displaced by a random amount and subject to some other distortions. Output is
not the scalar loss, but the gradient of a Mean Squared Error loss function.

In the following, we will first describe the training (Sec. 3.1) before
looking into the details of both the scene and loss network implementation
(Sec. 3.2).

###  3.1 Training

The key contribution of our approach is a new way of training. We will first
look into a classic baseline with scene-level supervision, then introduce a
hypothetical oracle that solves almost the same problem and finally show how
this problem can be solved without scene-level supervision by our approach.

Figure 2:  a) A 2D scene with three chair objects, supervised by centers
(orange) and their predictions (blue). b) The same scene, with the vector
field of the oracle ${\nabla}$ shown as arrows. c) A 2D Slice through a 6D
cost function. d) A 2D Slice through an alternative cost function, truncated
at the Voronoi cell edges. The oracle is the gradient of this. e) The simple
task of the loss network: given a chair not in the center (top), regress an
offset such that it becomes centered.

Supervised Consider learning the parameters $\theta$ of a scene network
${\mathcal{S}}_{\theta}$ which regresses object centers
${\mathcal{S}}_{\theta}(\mathbf{x}_{i})=\mathbf{\hat{c}}$ from a scene
$\mathbf{x}_{i}$. The scene is labeled by a set of 3D object centers
$\mathbf{c}_{i}$ (Fig. 2, a). This is achieved by minimizing the expectation

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}{\mathbb{E}}_{i}[H({\mathcal{S}}_{\theta}(\mathbf{x}_{i})-\mathbf{c}_{i})],$
(1)

using a two-sided Chamfer loss between the label point set $\mathbf{c}_{i}$
and a prediction $\mathbf{\hat{c}_{i}}$

$\displaystyle
H(\mathbf{\hat{c}},\mathbf{c})={\mathbb{E}}_{i}[\min_{j}||\mathbf{\hat{c}}_{i}-\mathbf{c}_{j}||_{2}^{2}]+{\mathbb{E}}_{i}[\min_{j}||\mathbf{c}_{i}-\mathbf{\hat{c}}_{j}||_{2}^{2}].$
(2)

Figure 3: Chamfer loss.

Under $H$, the network is free to report centers in any order, and ensures all
network predictions are close to a supervised center (precision) and all
supervised centers are close to at least one network prediction (recall) (Fig.
3).

In this work, we assume the box center supervision $\mathbf{c}_{i}$ to not be
accessible. Tackling this, we will first introduce an oracle solving a similar
problem.

Oracle Consider, instead of supervision, an _oracle_ function
${\nabla}(\mathbf{x})$ which returns for a 3D scene $\mathbf{p}$ _the smallest
offset by which we need to move the scene so that the world center falls onto
an object center_ (Fig. 2, b). Then, learning means to

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}{\mathbb{E}}_{i,j}[||{\nabla}(\underbrace{\mathbf{x}_{i}\ominus{\mathcal{S}}_{\theta}(\mathbf{x}_{i})_{j}}_{\mathbf{y}_{\theta,i,j}})||_{2}^{2}],$
(3)

where $\mathbf{x}\ominus\mathbf{d}$ denotes shifting a scene $\mathbf{x}$ by
an offset $\mathbf{d}$. The relation between Eq. 1 and Eq. 3 is intuitive:
knowing the centers is very similar to pointing to the nearest center from
every location. It is, however, not quite the same. It assures every network
prediction would map to a center, but does not assure, that there is a
prediction for every center. We will need to deal with this concern later, by
assuring space is well covered, so that there are enough predictions such that
at least one maps to every center. We will denote a scene $i$ shifted to be
centered around object $j$ by a _scene network_ with parameters $\theta$ as
$\mathbf{y}_{\theta,i,j}$.

Every location that maps to itself, i. e., a _fixed point_ [31] of ${\nabla}$,
is an object center. Hence, we try to get a scene network that returns the
roots of the gradient field of the distance function around each object center
(Fig. 2, c):

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}{\mathbb{E}}_{i,j}[||{\nabla}(\mathbf{y}_{\theta,i,j})||_{2}^{2}].$ (4)

Learned loss The key idea is to emulate this oracle with a _loss_ network
${\mathcal{L}}_{\phi}$ having parameters $\phi$ as in

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}{\mathbb{E}}_{i,j}[||{\mathcal{L}}_{\phi}(\mathbf{y}_{\theta,i,j})||_{2}^{2}].$
(5)

The loss network does not need to understand any global scene structure, it
only locally needs to _center_ the scene around the nearest object (Fig. 2,
d). This task can be learned by working on local 3D object _patches_ , without
scene-level supervision. So we can train the loss network on any set of
objects ${\mathbf{o}}_{k}$, translated by a known offset $\mathbf{d}_{k}$
using

$\displaystyle\underset{\phi}{\operatorname{arg\,min}\
}{\mathbb{E}}_{k}[||\mathbf{d}_{k}-{\mathcal{L}}_{\phi}({\mathbf{o}}_{k}\ominus\mathbf{d}_{k})||_{2}].$
(6)

As the loss network is local, it is also only ever trained on 3D patches.
These can be produced in several different ways: sampling of CAD models, CAD
models with simulated noise, by pasting simulated results on random scene
pieces, etc. In our experiments, we use a small labeled scene subset to
extract objects as follows: we pick a random object center and a 3D box of 1
meter size such that at least point representing an object surface is present
in the box. Hence the center of the object is offset by a random, but known
$\mathbf{d}_{k}$ we regress and subject to natural clutter. Note, that the box
does not, and does not have to, strictly cover the entire object – which are
of different sizes – but has to be just large enough to guess the center. Alg.
1 demonstrates how the _loss network_ output can be used to provide _scene
network_ supervision.

$\mathcal{L}_{\phi}:\mathbb{R}^{n\times 3}\rightarrow\mathbb{R}^{3}$;

$\mathcal{S}_{\theta}:\mathbb{R}^{m\times 3}\rightarrow\mathbb{R}^{k\times
3}$;

$\mathtt{crop}:\mathbb{R}^{m\times 3}\rightarrow\mathbb{R}^{n\times 3}$;

while  _loss training_ do

$x=\mathtt{sampleScene()}$;

$o=\mathtt{randObjectCenter()}$;

$d=\mathtt{randOffset()}$;

$p=\mathtt{crop}(x\ominus(o+d))$;

$\nabla=\frac{\partial}{\partial_{\phi}}||\mathcal{L}_{\phi}(p)-d||_{2}^{2}$;

$\phi=\mathtt{optimizer}(\phi,\nabla)$;

end while

while  _scene training_ do

$x=\mathtt{sampleScene()}$;

$c=\mathcal{S}_{\theta}(x)$;

for  _$i=1\ldots k$_ do

$p=\mathtt{crop}(x\ominus c_{i})$;

$\nabla_{i}=\mathcal{L}_{\phi}(p)$;

end for

$\theta=\mathtt{optimizer}(\theta,\nabla)$;

end while

Algorithm 1 $\mathcal{L}$: loss network, $\mathcal{S}$: scene network, $k$:
proposal count, $n$ 3D patch point count, $m$ scene point count.

Varying object count The above was assuming the number of objects
${n_{\mathrm{c}}}$ to be known. It did so when assuming a vector of a known
dimension as supervision in Eq. 1 and did so, when assuming the oracle Eq. 3
and its derivations were returning gradient vectors of a fixed size. In our
setting this number is unknown. We address this by bounding the number of
objects and handling occupancy i. e., a weight indicting if an object is
present or not, at two levels.

First, we train an occupancy branch ${\mathcal{O}}_{\phi}$ of the loss network
that classifies occupancy of a single patch, much like the loss network
regresses the center. We define space to be _occupied_ , if the 3D patch
contains any points belonging to the given objects surface. This branch is
trained on the same patches as the loss network plus an equal number of
additional 3D patches that do not contain any objects i.e. occupancy is zero.

Second, the occupancy branch is used to support the training of the scene
network which has to deal with the fact that the number of actual centers is
lower than the maximal number of centers. This is achieved by ignoring the
gradients to the scene networks parameters $\theta$ if the occupancy network
reports the 3D patch about a center to not contain an object of interest. So
instead of Eq. 5, we learn

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}{\mathbb{E}}_{i,j}[{\mathcal{O}}_{\phi}(\mathbf{y}_{\theta,i,j}){\mathcal{L}}_{\phi}(\mathbf{y}_{\theta,i,j})].$
(7)

The product in the sum is zero for centers of 3D patches that the loss network
thinks, are not occupied and hence should not affect the learning.

#### Overlap

When neither object centers nor their count are known, there is nothing to
prevent two network outputs to map to the same center. While such duplicates
can to some level be addressed by non-maximum suppression as a (non-
differentiable) post-process to testing, we have found it essential to already
prevent them (differentiable) from occurring when training the scene network.
Without doing so, our training degenerates to a single proposal.

To this end, we avoid _overlap_. Let $v(q_{1},q_{2})$ be a function that is
zero if the bounding boxes of the object in the scene centers do not overlap,
one if they are identical and otherwise be the ratio of intersection. We then
optimize

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}c_{1}(\theta)={\mathbb{E}}_{i,j,k}\left[{\mathcal{O}}_{\phi}(\mathbf{y}_{\theta,i,j}){\mathcal{L}}_{\phi}(\mathbf{y}_{\theta,i,j})+v(\mathbf{y}_{\theta,i,j},\mathbf{y}_{\theta,i,k})\right].$
(8)

We found that in case of a collision instead of mutually repelling all
colliding objects, it can be more effective if out of multiple colliding
objects, the collision acts on all but one winner object (winner-takes-all).
To decide the winner, we again use the gradient magnitude: if multiple objects
collide, the one that is already closest to the target i. e., the one with the
smallest gradient, remains unaffected ($v=0$) and takes possession of the
target, while all others adapt.

Additional features For other object properties such as size, orientation,
class of object, etc. we can proceed in two similar steps. First, we know the
object-level property vector $\mathbf{q}$, so we can train a _property branch_
denoted ${\mathcal{P}}_{\theta}$ that shares parameters $\theta$ with the loss
network to regresses the property vector from the same displaced 3D patches as
in Eq. 6

$\displaystyle\underset{\phi}{\operatorname{arg\,min}\
}{\mathbb{E}}_{k}[||\mathbf{q}_{k}-{\mathcal{P}}_{\phi}({\mathbf{o}}_{k}\ominus\mathbf{d}_{k})||_{1}].$
(9)

For scene-level learning we extend the scene network by a branch
${\mathcal{T}}_{\theta}$ to emulate what the property network had said about
the 3D patch at each center, but now with global context and on a scene-level

$\displaystyle\underset{\theta}{\operatorname{arg\,min}\
}c_{1}(\theta)+\alpha\cdot{\mathbb{E}}_{i,j}[|{\mathcal{T}}_{\theta}(\mathbf{y}_{\theta,i,j})-{\mathcal{P}}\phi(\mathbf{y}_{\theta,i,j})|_{1}].$
(10)

For simplicity, we will denote occupancy just as any other object property and
assume it to be produced by ${\mathcal{T}}$ just, that it has a special
meaning in training as defined in Eq. 7. We will next detail the architecture
of all networks.

Figure 4: The object (left) and scene (right) network. Input denoted orange,
output blue, trainable parts yellow, hard-coded parts in italics. Please see
Sec. 3.2 for a details.

###  3.2 Network

Both networks are implemented using PointNet++ [25] optimized using ADAM. We
choose particularly simple designs and rather focus on the analysis of changes
from the levels of supervision we enable.

#### Loss and occupancy network

The loss network branches ${\mathcal{L}}$ and ${\mathcal{O}}$ share parameters
$\phi$ and both map 4,096 3D points to a 3D displacement vector, occupancy and
other scalar features (left in Fig. 4).

#### Scene network

The scene network branches ${\mathcal{S}}$ and ${\mathcal{T}}$ jointly map a
point cloud to a vector of 3D object centers and property vectors (including
occupancy), sharing parameters $\theta$. The box branch ${\mathcal{S}}$ first
generates positions, next the scene is cropped around these positions and each
3D patch respectively fed into a small PointNet++ encoder ${\mathcal{M}}$ to
produce crop specific local feature encodings. Finally, we concatenate the
global scene latent code ${\mathcal{S}}_{z}$ with the respective local latent
code ${\mathcal{M}}_{z}$ and pass it through the scene property branch
${\mathcal{T}}$ MLP.

The scene property branch is trained sharing all weights across all instances
for all objects. This is intuitive, as deciding that e. g., a chair’s
orientation is the same for all chairs (the back rest is facing backwards),
can at the same time be related to global scene properties (alignment towards
a table).

Instead of learning the centers, we learn the residual relative to a uniform
coverage of 3D space such that no object is missed during training. The
Hammersley pattern [32] assures that, no part of 3D space is left uncovered.

We assume a fixed number of 32,768 input points for one scene. Note, that we
do not use color as input, a trivial extension. Each MLP sub-layer is an MLP
consisting of 3 fully-connected layers where layer 1 has 512 hidden states and
the final layer contains the branch specific output nodes.

#### Post-process

Our scene network returns a set of oriented bounding boxes with occupancy. To
reduce this soft answer to a set of detected objects, e. g., to compute mAP
metrics, we remove all bounding boxes with occupancy below a threshold
$\tau_{\mathrm{o}}$, which we set to 0.9 in all our results.

In the evaluation, the same will be done for our ablations Sliding and
Supervised, just that these also require additional non-maximum suppression
(NMS) as they frequently propose boxes that overlap. To construct a final list
of detections, we pick the proposal with maximal occupancy and remove any
overlapping proposal with IoU $>.25$ and repeat until no proposals remain.

##  4 Evaluation

We compare to different variants of our approach under different metrics and
with different forms of supervision as well as to other methods.

###  4.1 Protocol

Data sets

We consider two large-scale sets of 3D scanned scenes: Stanford 2D-3D-S
dataset (S3D) [2] and ScanNet [6]. From both we extract, for each scene, the
list of object centers and object features for all objects of one class.

We split the dataset in three parts (Fig. 5): First, the test dataset is the
official test dataset (pink in Fig. 5). The remaining training data is split
into two parts: a labeled, and and unlabeled part. The labeled part (orange in
Fig. 5) has all 3D scenes with complete annotations on them. The unlabeled
part (blue in Fig. 5) contains only raw 3D point cloud without annotation.
Note, that the labeled data is a subset of the unlabeled data, not a different
set.

Figure 5: Label ratio.

We call the ratio of labeled over unlabeled data the _label ratio_. To more
strictly evaluate transfer across data sets, we consider ScanNet completely
unlabeled. All single-class results are reported for the class chair.

Metrics Effectiveness is measured using the Chamfer distance (less is better)
also used as a loss in Eq. 1 and the established mean Average Precision
<EMAIL_ADDRESS>(more is better) of a x % bounding box overlap test. X is chosen at
25 %.

Methods We consider the following _three_ methods: Supervised is the
supervised approach define by Eq. 1. This method can be trained only on the
labeled part of the training set. Sliding window is an approach that applies
our loss network, trained on the labeled data, to a dense regular grid of 3D
location in every 3D scene to produce a heat map from which final results are
generated by NMS. Ours is our method. The loss network is trained on the
labeled data (orange in Fig. 5). The scene network is trained on the unlabeled
data (blue in Fig. 5), which includes the labeled data (but without accessing
the labels) as a subset.

###  4.2 Results

Effect of supervision The main effect to study is the change of 3D detection
quality in respect to the level of supervision. In Tbl. 1, different rows show
different label ratios. The columns show Chamfer error and<EMAIL_ADDRESS>for the
class chair trained and tested on S3D.

Table 1: Chamfer error (less is better) and<EMAIL_ADDRESS>(more is better)
(columns), as a function of supervision (rows) in units of label ratio on the
S3D class chair. Right, the supervision-quality-relation plotted as a graph
for every method (color).

| Chamfer error | mAP
---|---|---
Ratio |  Sup |  Sli |  Our |  Sup |  Sli |  Our
1 % | 1.265 | .850 | .554 | .159 | .473 | .366
5 % | .789 | .577 | .346 | .352 | .562 | .642
25 % | .772 | .579 | .274 | .568 | .573 | .735
50 % | .644 | .538 | .232 | .577 | .589 | .773
75 % | .616 | .437 | .203 | .656 | .592 | .785
100 % | .557 | .434 | .178 | .756 | .598 | .803

We notice that across all levels of supervision, Our approach performs better
in Chamfer error and mAP than Sliding window using the same object training or
Supervised training of the same network. It can further be seen, how all
methods improve with more labels. Looking at a condition with only 5 %
supervision, Our method can perform similar to a Supervised method that had
20$\times$ the labeling effort invested. At this condition, our detection is
an acceptable $.642$, which Supervised will only beat when at least 75 % of
the dataset is labeled. It could be conjectured, that the scene network does
no more than emulating to slide a neural-network object detector across the
scene. If this was true, Sliding would be expected to perform similar or
better than Ours, which is not the case. This indicates, that the scene
network has indeed learned something not known at object level, something
about the relation of the global scene without ever having labels on this
level.

Figure 6: Error distribution.

Fig. 6 plots the rank distribution (horizontal axis) of Chamfer distances
(vertical axis) for different methods (colors) at different levels of
supervision (lightness). We see that Our method performs well across the
board, Supervised has a more steep distribution compared to Sliding,
indicating it produces good as well as bad results, while the former is more
uniform. In terms of supervision scalability, additional labeling invested
into our method (brighter shades of yellow) result in more improvements to the
right side of the curve, indicating, additional supervision is reducing high
error-responses while already-good answers remain.

Transfer across data sets So far, we have only considered training and testing
on S3D. In Tbl. 2, we look into how much supervision scaling would transfer to
another data set, ScanNet. Remember, that we treat ScanNet as unlabeled, and
hence, the loss network will be strictly only trained on objects from S3D. The
three first rows in Tbl. 2 define the conditions compared here: a loss network
always trained on S3D, a scene network trained on either S3D or ScanNet and
testing all combinations on both data sets.

Table 2: Transfer across data sets: Different rows show different levels of
supervision, different columns indicate different methods and metrics. The
plot on the right visualizes all methods in all conditions quantified by two
metrics. Training either on S3D or on ScanNet. The metrics again are Chamfer
error (also the loss) and<EMAIL_ADDRESS>Colors in the plot correspond to different
training, dotted/solid to different test data.

Loss: | S3D | S3D
---|---|---
Scene: |  S3D |  ScanNet
Test: | S3D | ScanNet | S3D | ScanNet
Ratio | Err. | mAP | Err. | mAP | Err. | mAP | Err. | mAP
1% | 0.554 | .366 | 1.753 | .112 | 0.579 | .296 | 0.337 | .548
5% | 0.346 | .642 | 0.727 | .138 | 0.466 | .463 | 0.703 | .599
50% | 0.232 | .773 | 0.588 | .380 | 0.447 | .497 | 0.258 | .645
100% | 0.178 | .803 | 0.789 | .384 | 0.336 | .555 | 0.356 | .661

Column two and three in Tbl. 2 and the dotted violet line in the plot, iterate
the scaling of available label data we already see in Tbl. 1 when training and
testing on S3D. Columns four and five, show a method trained on S3D but tested
on ScanNet. We find performance to be reduced, probably, as the domain of
ScanNet is different from the one of S3D. If we include the unlabeled scenes
of ScanNet in the training, as seen in columns six to nine, the quality
increases again, to competitive levels, using only S3D labels and 0 % of the
labels available for ScanNet.

Table 3: Performance of the loss network for different label ratio (rows) on
different test data and according to different metrics (columns). 1Class not
present in ScanNet.

|  |  |  |  S3D |  ScanNet
---|---|---|---|---|---
Ratio | Class | #Sce | #Obj | Err. | Acc. | Err. | Acc.
1 % | chair | 11 | 2400 | .0085 | .853 | .0099 | .843
5 % | chair | 54 | 16,000 | .0052 | .936 | .0075 | .899
25 % | chair | 271 | 47,200 | .0049 | .949 | .0071 | .907
50 % | chair | 542 | 121,191 | .0046 | .953 | .0069 | .902
75 % | chair | 813 | 162,000 | .0045 | .955 | .0065 | .920
100 % | chair | 1084 | 223,980 | .0043 | .960 | .0068 | .911
5 % | table | 54 | 5060 | .0078 | .921 | —1 | —1
5 % | bcase | 54 | 4780 | .0093 | .819 | —1 | —1
5 % | column | 54 | 2780 | .0100 | .855 | —1 | —1

Tbl. 3 further illustrate the loss network: how good are we at finding vectors
that point to an object center? We see that the gradient error and the
confidence error, both go down moderately with more labels when training and
testing on S3D (violet). The fact that not much is decreasing in the loss
network, while the scene network keeps improving, indicates the object task
can be learned from little data, and less object-level supervision is required
than what can be learned on a scene level, still. We further see, that the
loss network generalizes between data sets from the fact that it is trained on
S3D (violet curve) but when tested on ScanNet (green curve) goes down, too.

Table 4: Chamfer error and<EMAIL_ADDRESS>reported for varying the number of scenes.

|  Our
---|---
#Sce | Err. | mAP
66 | .643 | .079
330 | .509 | .242
1648 | .506 | .360
3295 | .457 | .412
4943 | .435 | .479
6590 | .407 | .599

Besides seeing how quality scales with the amount of labeled supervision for
training the loss network, it is also relevant to ask what happens when the
amount of unlabeled training data for the scene network is increased while
holding the labeled data fixed. This is analyzed in Tbl. 4. Here we took our
loss network and trained it at 5 % label ratio on S3D and tested on ScanNet.
Next, the scene network was trained, but on various number of scenes from
ScanNet, which, as we said, is considered unlabeled. The number of scenes
changes over columns, resp. along the horizontal axis in the plot. We see that
without investing any additional labeling effort, the scene network keeps
increasing substantially, indicting what was learned on a few labeled S3D
objects can enable understanding the structure of ScanNet.

Different classes Tbl. 1 was analyzing the main axis of contribution:
different levels of supervision but for a single class. This has shown that at
around a label ratio of 5 % Our method performs similar to a Supervised one.
Holding the label ration of 5 % fixed and repeating the experiment for other
classes, is summarized in Tbl. 5. We see, that the relation between
Supervised, Sliding and Ours is retained across classes.

Table 5: Chamfer error (less is better) and<EMAIL_ADDRESS>precision (more is better)
(columns), per class (rows) at a supervision of 5 % labeling ratio.

| Chamfer error | mAP
---|---|---
Class |  Sup |  Sli |  Our |  Sup |  Sli |  Our
chair | 0.789 | 0.577 | .346 | .352 | .562 | .642
table | 1.144 | 1.304 | .740 | .282 | .528 | .615
bookcase | 1.121 | 1.427 | .979 | .370 | .298 | .640
column | 0.900 | 2.640 | .838 | .490 | .353 | .654

Comparison to other work In Tbl. 6 we compare our approach to other methods.
Here, we use 20 % of ScanNet V2 for testing and the rest for training. Out of
the training data, we train our approach once with 100 % labeled and once with
only 5 % labeled. Other methods were trained at 100 % label ratio.

Table 6: Performance (mAP(%) with IoU threshold .25) of different methods (rows) on all classes (columns) of ScanNet V2. 15 images. 2Only xyz. 3Their ablation; similar to our backbone.  Method |  |  cabinet |  bed |  chair |  sofa |  table |  door |  window |  bookshelf |  picture |  counter |  desk |  curtain |  fridge |  curtain |  toilet |  sink |  bathtub |  other | mAP
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
3DSIS1 | [14] | 19.8 | 69.7 | 66.2 | 71.8 | 36.1 | 30.6 | 10.9 | 27.3 | 0.0 | 10.0 | 46.9 | 14.1 | 53.8 | 36.0 | 87.6 | 43.0 | 84.3 | 16.2 | 40.2
3DSIS2 | [14] | 12.8 | 63.1 | 66.0 | 46.3 | 26.9 | 8.0 | 2.8 | 2.3 | 0.0 | 6.9 | 33.3 | 2.5 | 10.4 | 12.2 | 74.5 | 22.9 | 58.7 | 7.1 | 25.4
MTML | [17] | 32.7 | 80.7 | 64.7 | 68.8 | 57.1 | 41.8 | 39.6 | 58.8 | 18.2 | 0.4 | 18.0 | 81.5 | 44.5 | 100.0 | 100.0 | 44.2 | 100.0 | 36.4 | 54.9
VoteNet | [23] | 36.3 | 87.9 | 88.7 | 89.6 | 58.8 | 47.3 | 38.1 | 44.6 | 7.8 | 56.1 | 71.7 | 47.2 | 45.4 | 57.1 | 94.9 | 54.7 | 92.1 | 37.2 | 58.7
BoxNet3 | [17] | No per-class information available | 45.4
3D-BoNet | [33] | 58.7 | 88.7 | 64.3 | 80.7 | 66.1 | 52.2 | 61.2 | 83.6 | 24.3 | 55.0 | 72.4 | 62.0 | 51.2 | 100.0 | 90.9 | 75.1 | 100.0 | 50.1 | 68.7
Ours 100 % |  | 43.0 | 70.8 | 58.3 | 16.0 | 44.6 | 28.0 | 13.4 | 58.2 | 4.9 | 69.9 | 74.0 | 75.0 | 36.0 | 58.9 | 79.0 | 47.0 | 77.9 | 48.2 | 50.2
Ours 005 % |  | 38.1 | 68.9 | 58.9 | 88.8 | 42.5 | 21.1 | 9.0 | 53.2 | 6.8 | 53.9 | 68.0 | 62.3 | 26.5 | 45.6 | 69.9 | 40.4 | 66.9 | 48.0 | 48.3

We see that our approach provides competitive performance, both at 100 % of
the labels, as well as there is only a small drop when reducing supervision by
factor 20$\times$. Our mAP at 100 % of the labels is better than both variants
(with and without color) of 3DSIS [14] from 2018 and similar to MTML [17] from
2019. VoteNet [23] and 3D-BoNet [33] are highly specialized architectures from
2019 that have a higher mAP. We have included BoxNet from Qi et al. [23], an
ablation they include as a vanilla 3D detection approach that is similar to
what we work with. We achieve similar even slightly better performance, yet at
5 % of the supervision. In some categories, our approach wins over all
approaches. We conclude that a simple backbone architecture we use is no
contribution and cannot win over specialized ones, but that it also is
competitive to the state-of-the-art. We should note here, as we do not carry
out Semantic instance segmentation in our network, we did not test on the
official test ScanNet benchmark test set. Instead, we reserve 20% of the
labeled training scenes for testing.

Qualitative results Fig. 7 shows qualitative example results of our approach.

Figure 7: Qualitative results of our approach and the ground truth for chair
on S3D.

Computational efficiency Despite the additional complexity in training, at
deployment, out network is a direct and fast forward architecture, mapping a
point cloud to bounding boxes. Finding 20 proposals in 32,768 points takes 189
ms, while the supervised takes the same amount of time, with the small
overhead of a NMS (190 ms) on a Nvidia RTX 2080Ti. Our CPU implementation of
sliding window requires 4.7 s for the same task on a i7-6850K CPU @ 3.60GHz.
All results are computed with those settings.

##  5 Discussion

#### How can Ours be better than the Supervised?

It is not obvious why at 100 % label ratio in Tbl. 1, the Supervised
architecture performs at an mAP of .756 while Ours remains slightly higher at
an mAP of .803. This is not just variance of the mAP estimation (computed
across many identical objects and scenes).

A possible explanation for this difference is, that our training is no drop-in
replacement for supervised training. Instead, it optimizes a different loss
(truncation to the nearest object and collision avoidance) that might turn out
to be better suited for 3D detection than what it was emulating in the
beginning. We, for example, do not require NMS. As our training does not
converge without those changes to the architecture, some of the effects
observed might be due to differences in architecture and not due to the
training. We conjecture future work might consider exploring different losses,
involving truncation and collision, even when labels are present.

#### Why Hammersley?

Other work has reasoned about what intermediate points to use when processing
point clouds. When voting [23], the argument is, that the centers of bounding
boxes are not part of the point set, and hence using a point set that is any
subset of the input is not a good solution. While we do not vote, we also have
chosen not to use points of the scene as the initial points. We also refrain
from using any improved sampling of the surface, such as Poisson disk [13]
sampling as we do not seek to cover any particular instance but space in
general, covered by scenes uniformly.

#### How can the scene network be “better” than the loss network?

As the loss network is only an approximation to the true loss, one might ask,
how a scene network, trained with this loss network, can perform better than
the loss network alone, e. g., how can it, consistently (Tbl. 1, 2, 4 and 5),
outperform SlidingWindow?

Let us assume, that a scene network trained by a clean supervision signal can
use global scene structure to solve the task. If now the supervision signal
would start to be corrupted by noise, recent work has shown for images [18] or
point clouds [12], that a neural network trained under noise will converge to
a result that is very similar to the clean result: under $\mathcal{L}_{2}$ it
will converge to the mean of the noise, under $\mathcal{L}_{1}$ to its median,
etc. The amount of variance of that noise does not influence the result, what
matters is that the noise is unbiased. In our case, this means if we were to
have supervision by noisy bounding boxes, that would not change anything,
except that the scene network training would converge slower but still to the
mean or median of that noise distribution, which is, the correct result. So
what was done in our training, by using a network to approximate the loss,
means to just introduce another form of noise into the training.

##  6 Conclusion

We have suggested a novel training procedure to reduce the 3D labeling effort
required to solve a 3D detection task. The key is to first learn a loss
function on a small labeled local view of the data (objects), which is then
used to drive a second learning procedure to capture global relations
(scenes). The way to enlightenment here is to “find your center”: the simple
task of taking any piece of 3D scene and shifting it so it becomes centered
around the closest object. Our analysis indicates that the scene network
actually understands global scene structure not accessible to a sliding
window. Our network achieves state of the art results, executes in a fraction
of a second on large point clouds with typically only 5 % of the labeling
effort. We have deduced what it means exactly to learn the loss function, the
new challenges associated with this problem and proposed several solutions to
overcome these challenges.

In future work, other tasks might benefit from similar decoupling of
supervision labels and a learned loss, probably across other domains or
modalities.

## References

  * Adler and Öktem [2017] Adler, J., Öktem, O.: Solving ill-posed inverse problems using iterative deep neural networks. Inverse Problems 33(12) (2017)
  * Armeni et al. [2017] Armeni, I., Sax, S., Zamir, A.R., Savarese, S.: Joint 2D-3D-semantic data for indoor scene understanding. arXiv:1702.01105 (2017)
  * Chang et al. [2015] Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al.: Shapenet: An information-rich 3D model repository. arXiv:1512.03012 (2015)
  * Chen et al. [2015] Chen, X., Kundu, K., Zhu, Y., Berneshawi, A.G., Ma, H., Fidler, S., Urtasun, R.: 3D object proposals for accurate object class detection. In: NIPS (2015)
  * Chen et al. [2019] Chen, Y., Liu, S., Shen, X., Jia, J.: Fast point R-CNN. In: ICCV (2019)
  * Dai et al. [2017] Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T., Nießner, M.: Scannet: Richly-annotated 3D reconstructions of indoor scenes. In: CVPR (2017)
  * Engelcke et al. [2017] Engelcke, M., Rao, D., Wang, D.Z., Tong, C.H., Posner, I.: Vote3deep: Fast object detection in 3d point clouds using efficient convolutional neural networks. In: ICRA (2017)
  * Feng et al. [2019] Feng, M., Gilani, S.Z., Wang, Y., Zhang, L., Mian, A.: Relation graph network for 3D object detection in point clouds. arXiv:1912.00202 (2019)
  * Flynn et al. [2019] Flynn, J., Broxton, M., Debevec, P., DuVall, M., Fyffe, G., Overbeck, R., Snavely, N., Tucker, R.: Deepview: View synthesis with learned gradient descent. In: CVPR (2019)
  * Girshick [2015] Girshick, R.: Fast R-CNN. In: ICCV (2015)
  * Griffiths and Boehm [2019] Griffiths, D., Boehm, J.: A review on deep learning techniques for 3d sensed data classification. Remote Sensing 11(12) (2019)
  * Hermosilla et al. [2019] Hermosilla, P., Ritschel, T., Ropinski, T.: Total denoising: Unsupervised learning of 3d point cloud cleaning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 52–60 (2019)
  * Hermosilla et al. [2018] Hermosilla, P., Ritschel, T., Vázquez, P.P., Vinacua, À., Ropinski, T.: Monte carlo convolution for learning on non-uniformly sampled point clouds. ACM Trans. Graph. (proc. SIGGRAPH Asia) 37(6), 1–12 (2018)
  * Hou et al. [2019] Hou, J., Dai, A., Nießner, M.: 3D-SIS: 3D semantic instance segmentation of RGB-D scans. In: CVPR (2019)
  * Karpathy et al. [2013] Karpathy, A., Miller, S., Fei-Fei, L.: Object discovery in 3d scenes via shape analysis. In: ICRA (2013)
  * Ku et al. [2018] Ku, J., Mozifian, M., Lee, J., Harakeh, A., Waslander, S.L.: Joint 3D proposal generation and object detection from view aggregation. In: IROS (2018)
  * Lahoud et al. [2019] Lahoud, J., Ghanem, B., Pollefeys, M., Oswald, M.R.: 3d instance segmentation via multi-task metric learning. In: ICCV (2019)
  * Lehtinen et al. [2018] Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., Aila, T.: Noise2noise: Learning image restoration without clean data. arXiv:1803.04189 (2018)
  * Liu et al. [2016] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: SSD: Single shot multibox detector. In: ECCV (2016)
  * Maximov et al. [2019] Maximov, M., Leal-Taixe, L., Fritz, M., Ritschel, T.: Deep appearance maps. In: ICCV (2019)
  * Newell et al. [2016] Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: ECCV (2016)
  * Qi et al. [2020] Qi, C.R., Chen, X., Litany, O., Guibas, L.J.: Imvotenet: Boosting 3d object detection in point clouds with image votes. In: arXiv preprint arXiv:2001.10692 (2020)
  * Qi et al. [2019] Qi, C.R., Litany, O., He, K., Guibas, L.J.: Deep hough voting for 3D object detection in point clouds. In: ICCV (2019)
  * Qi et al. [2017a] Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660 (2017a)
  * Qi et al. [2017b] Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv:1706.02413 (2017b)
  * Redmon et al. [2016] Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: CVPR (2016)
  * Shi et al. [2019] Shi, S., Wang, X., Li, H.: PointRCNN: 3D object proposal generation and detection from point cloud. In: CVPR (2019)
  * Song et al. [2015] Song, S., Lichtenberg, S.P., Xiao, J.: Sun RGB-D: A RGB-D scene understanding benchmark suite. In: CVPR (2015)
  * Song and Xiao [2014] Song, S., Xiao, J.: Sliding shapes for 3D object detection in depth images. In: ECCV (2014)
  * Song and Xiao [2016] Song, S., Xiao, J.: Deep sliding shapes for amodal 3d object detection in RGB-D images. In: CVPR (2016)
  * Weisstein [2020a] Weisstein, E.: Fixed point (2020a), URL http://mathworld.wolfram.com/FixedPoint.html
  * Weisstein [2020b] Weisstein, E.: Hammersley point set (2020b), URL http://mathworld.wolfram.com/HammersleyPointSet.html
  * Yang et al. [2019] Yang, B., Wang, J., Clark, R., Hu, Q., Wang, S., Markham, A., Trigoni, N.: Learning object bounding boxes for 3D instance segmentation on point clouds. arXiv:1906.01140 (2019)
  * Zhou et al. [2019] Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv:1904.07850 (2019)
  * Zhou and Tuzel [2018] Zhou, Y., Tuzel, O.: Voxelnet: End-to-end learning for point cloud based 3d object detection. In: CVPR (2018)

## Supplementary

We will here give further details of our architecture. For simplicity we
ignore batch dimensions. All branches for both loss and scene networks have a
shared PointNet++ encoder shown in Tbl. 7 with input 4,096 and 32,768
respectively. We use the original “Single Scale Grouping” architecture from
PointNet++. Unlike the original implementation no batch normalization, weight
regularization or dropout layers are used. We define the implemented
architectures in Tbl. 7 and Tbl. 8.

Table 7: PointNet++ encoder consisting of four “Set Abstraction” (SA) layers. Input is either 32,768 or 4,096 points for training scene or loss networks respectively. Points indicates the number of points selected from the fartest point sampling algorithm. Samples is the maximum points per neighborhood defined using a ball point query with radius $r$. Layer | MLP | Points | Samples | Radius | Activation | Output Shape
---|---|---|---|---|---|---
Input |  |  |  |  |  |  |  | $(32768/4096)$ | $\times$ | $3$
SA | [64, | 64, | 128] | 1024 | 32 | 0.1 | ReLU | $1024$ | $\times$ | $128$
SA | [64, | 64, | 128] | 256 | 32 | 0.2 | ReLU | $256$ | $\times$ | $128$
SA | [128, | 128, | 256] | 64 | 64 | 0.4 | ReLU | $64$ | $\times$ | $256$
SA | [256, | 512, | 128] | None | None | None | ReLU | $1$ | $\times$ | $128$

Our local encoder is a 3 layer SSG network, which typically consists of half
the number of units for each respective MLP layer.

Table 8: PointNet++ local encoder consisting of 3 SA layers. See Tbl. 7 caption for further details regarding columns. Layer | MLP | Points | Samples | Radius | Activation | Output Shape
---|---|---|---|---|---|---
Input |  |  |  |  |  |  |  | $4096$ | $\times$ | $3$
SA | [32, | 32, | 64] | 1024 | 32 | 0.1 | ReLU | $256$ | $\times$ | $128$
SA | [64, | 64, | 128] | 256 | 32 | 0.2 | ReLU | $64$ | $\times$ | $256$
SA | [128, | 128, | 128] | None | None | None | ReLU | $1$ | $\times$ | $128$

Our loss network architecture has a shared PointNet++ encoder with three
branches; gradient, property and objectness. Each branch consists of two
fully-connected layers with a ReLU activation for the hidden layer. We use
softmax for both single and multi-class examples. For single class where $k=2$
the first class represents empty and the second class represents present. In a
multi-class setting $k$ is the number of classes. We do not have a empty class
for multi-class. Both gradient and property branches do not have a final layer
activation function.

Table 9: Loss network architecture. PNet++ is defined in Tbl. 7. Branch | Layer | Kernel | Activation | Output Shape
---|---|---|---|---
Trunk | PNet++ |  | ReLU | $1$ | $\times$ | $128$
Gradient | FC | $1\times 1$ | ReLU | $1$ | $\times$ | $512$
FC | $1\times 1$ | None | $1$ | $\times$ | $3$
Property | FC | $1\times 1$ | ReLU | $1$ | $\times$ | $512$
FC | $1\times 1$ | None | $1$ | $\times$ | $5$
Objectness | FC | $1\times 1$ | ReLU | $1$ | $\times$ | $512$
FC | $1\times 1$ | Softmax | $1$ | $\times$ | $k$

The scene network architecture is similar to the loss network but is designed
for multiple proposals $n$ as apposed to the loss network where $n=1$. The
property and objectness branches take patch codes generated by the local
encoder. The patches are local crops from the center branch proposals. Note
that the output of the PointNet++ encoder and local encoder are both $1\times
128$. These latent codes are concatenated for input into the scene network
property and objectness branches.

Table 10: Scene network architecture. PNet++ and PNet++ local are defined in Tbl. 7 and 8 respectively. Branch | Layer | Kernel | Activation | Output Shape
---|---|---|---|---
Trunk | PNet++ encoder |  | ReLU | $1$ | $\times$ | $128$
Center | FC | $1\times 1$ | ReLU | $n$ | $\times$ | $512$
FC | $1\times 1$ | None | $n$ | $\times$ | $3$
Property | PNet++ local | - | ReLU | $n$ | $\times$ | $128$
FC | $1\times 1$ | ReLU | $n$ | $\times$ | $512$
FC | $1\times 1$ | None | $n$ | $\times$ | $5$
Objectness | PNet++ local | - | ReLU | $n$ | $\times$ | $128$
FC | $1\times 1$ | ReLU | $n$ | $\times$ | $512$
FC | $1\times 1$ | Softmax | $n$ | $\times$ | $k$

All network weights were initialized using the Glorot Normal initialization.
We do not employ regularization or batch normalization in any of the networks.

Figure 8: Visualizations for the _scene network_ during training. a) A scene
consisting of a single proposal and single object. Colored points indicate
those passed seen by the _loss network_ for gradient estimation (red arrow).
b) A scene consisting of 5 proposals and 5 objects. c) Here we show 10
proposals with 1 object. Our overlap penalty (blue arrow) is applied to all
but the closest proposal to an object. d) 20 proposals with 5 objects. e) 10
proposals with 1 object. Here we visualize the final scene network output
which consists of location, bounding box and orientation. Proposals with an
objectness score below a threshold are shown as grey dots. For full video
sequences visit the project page at: dgriffiths3.github.io.