$\Gamma=\sigma(T/N)^{3}$, so that we may restrict $0<c\leq\sigma\leq C$, using
$\Gamma_{c}\sim(T/N)^{3}$ and discarding an $O(h^{\infty})$ contribution from
$\sigma>C$. Therefore,

(3.267) $V_{N,1,\varepsilon_{1},\varepsilon_{2}}=a^{1/2}\int
e^{\frac{i}{h}\Psi_{N,\varepsilon_{1},\varepsilon_{2}}}\mu_{\varepsilon_{1},\varepsilon_{2},N}\,d\sigma
d\theta+O_{C^{\infty}}(h^{\infty})$

where the new symbol $\mu_{\varepsilon_{1},\varepsilon_{2},N}$ is such that,
for all $k$ and $\kappa$,

(3.268)
$|\partial_{\theta}^{\kappa}\partial_{\sigma}^{k}\mu_{\varepsilon_{1},\varepsilon_{2},N}|\leq
C_{\kappa,k}\frac{T}{N}\sigma^{-2/3-k}\,.$

As we have

(3.269) $|\partial^{2}_{\sigma}\Psi_{N,\varepsilon_{1},\varepsilon_{2}}|\geq
c_{0}(\frac{T}{N})^{2}Ta^{3/2}\,,$

we now get by stationnary phase

(3.270)
$|V_{N,1,\varepsilon_{1},\varepsilon_{2}}|\leq\frac{C_{0}}{h^{d}}a^{1/2}\inf(1,(\frac{|N|}{\lambda\Gamma_{c}})^{-1/2})\frac{T}{N}(\frac{h}{t})^{\frac{d-2}{2}}\frac{N}{T}\frac{1}{(\lambda
T)^{1/2}}$

and one remarks that $\lambda\Gamma_{c}N^{-1}\sim\lambda T^{3}N^{-4}$; writing
$T/N^{2}=(T/N)^{2}/T$ and choosing $A^{\prime}$, $N^{\prime}$ as above, we get

(3.271)
$|V_{N,1,\varepsilon_{1},\varepsilon_{2}}|\leq\frac{C_{0}}{h^{d}}a^{1/2}(\frac{h}{t})^{\frac{d-2}{2}}\frac{A^{\prime}}{T}$

Notice how our estimate of each $V_{N,1,\varepsilon_{1},\varepsilon_{2}}$ is
worst than before, but we do have a better estimate on
$|\mathcal{N}_{1}(x,y,t)|$ to compensate for it.

We then sum over $N\in\mathcal{N}_{1}$: either $\Delta N\geq 2C$, and from
$\Delta N\leq 2CT\lambda^{-2}(A^{\prime})^{-3-1/2}$, we get

(3.272) $|\sum_{|N|\geq
2}V_{N,1,\varepsilon_{1},\varepsilon_{2}}(t,x,y)|\leq\frac{C_{0}}{h^{d}}a^{1/2}(\frac{h}{t})^{\frac{d-2}{2}}\frac{A^{\prime}}{T}T\lambda^{-2}(A^{\prime})^{-3-1/2}$

As $A^{\prime}$ is large and $a^{1/2}\lambda^{-2}\leq h^{1/3}$, we are done.

If on the other hand $\Delta N\leq 2C$, we directly get

(3.273) $|\sum_{|N|\geq
2}V_{N,1,\varepsilon_{1},\varepsilon_{2}}(t,x,y)|\leq\frac{C_{0}}{h^{d}}a^{1/2}(\frac{h}{t})^{\frac{d-2}{2}}\frac{1}{T^{1/2}\lambda^{1/2}}$

and from $T\geq T_{0}>0$ and $a^{1/2}\lambda^{-1/2}\leq h^{1/3}$ we are done.
$\Box$

We now proceed with the special case $|N|=1$ and only highlight the
differences with the previous case $|N|\geq 2$. Recall that here we have
$A>A_{1}$ where $A_{1}$ is a (large) constant. In order for the phase
$\Psi_{1,\varepsilon_{1},\varepsilon_{2}}$ to have a critical point at
$A=A_{c}$, $T$ will have to be small, actually $T\sim A_{c}^{-1/2}$ and one
may compute
$(\partial^{2}_{A}\Psi_{1,\varepsilon_{1},\varepsilon_{2}})(A_{c})=\sim
T^{3}$. Changing variable $A=T^{-2}\sigma$, one easily checks that the
rescaled symbols in the definition (3.226) of $V_{1,1}$ are classical symbols
of degree zero in $\sigma$, uniformly in $T$, provided $\sigma>\sigma_{0}>0$.
By non degenerate stationnary phase, we get

(3.274)
$|V_{1,1}|\leq\frac{C}{h^{d}}\left(\frac{h}{t}\right)^{\frac{d-2}{2}}\frac{1}{\lambda^{1/2}T^{1/2}}\,.$

Again, we have $(\lambda T)^{-1/2}\lesssim h^{1/3}$ and we are done.  $\Box$

######  Remark 3.13.

One should notice that $h^{1/3}\leq a^{1/8}h^{1/4}$ as $a>h^{2/3}$. As such,
at least for for $|t|\leq C\sqrt{a}$, (3.274) is at least as good as the bound
where $h^{1/3}$ gets replaced by $a^{1/4}(h/t)^{1/4}$, as stated for $V_{1}$
in [8].

##  4\. Construction of whispering gallery modes

We recall the formula of the parametrix $\mathcal{P}_{h,a}$ obtained by the
distributional bracket

(4.1)
$\mathcal{P}_{h,a}(t,x,y)=\langle\sum_{N\in{\mathbb{Z}}}e^{-iNL(\omega)},K_{\omega}(g_{h,a})(t,x,y)\rangle_{\omega}$

and the Airy-Poisson formula (2.25) which yields

(4.2)
$\mathcal{P}_{h,a}(t,x,y)=2\pi\sum_{k\in{\mathbb{N}}^{*}}\frac{1}{L^{\prime}(\omega_{k})}K_{\omega_{k}}(g_{h,a})(t,x,y)\,.$

We introduce (2.39) (the formula of $g_{h,a}$ before the stationary phase with
respect to $s,\alpha$) in the integral defining $K_{\omega_{k}}$: the integral
with respect to $y^{\prime}$ yields a factor $h^{d-1}$ (and
$\theta^{\prime}=\theta$), while applying the stationary phase with respect to
$(\alpha,{\varrho})$ provides a factor $h$ since the critical points
$(\alpha=h^{2/3}\omega_{k},{\varrho}=\partial_{\alpha}\Phi(a,0,\theta,h^{2/3}\omega_{k},\sigma)$
are non-degenerate. We obtain

(4.3) $K_{\omega_{k}}(g_{h,a})(t,x,y)=\frac{h^{-d-1/3}}{2\pi}\int
e^{\frac{i}{h}(t\rho(h^{2/3}\omega_{k},\theta)+\Phi(x,y,\theta,h^{2/3}\omega_{k},s)-\Phi(a,0,\theta,h^{2/3}\omega_{k},s^{\prime}))}p_{0,h}dsds^{\prime}d\theta,$

where $\Phi$ has been defined in (2.15) and were the symbol is of the form

$p_{0,h}=p_{0,h}(x,a,y,\theta/h,\omega_{k},s/h^{1/3},s^{\prime}/h^{1/3},h),$

with $p_{0,h}(x,a,y,\eta,\omega_{k},\sigma,\sigma^{\prime},h)$ of order $0$.
Using (4.2) and (4.3), there exits a symbols $\breve{p}_{0,h}$ of order $0$
such that the parametrix takes the following form

(4.4)
$\mathcal{P}_{h,a}(t,x,y)=\sum_{k\in{\mathbb{N}}^{*}}\frac{h^{-d+1/3}}{L^{\prime}(\omega_{k})}\int
e^{\frac{i}{h}t\rho(h^{2/3}\omega_{k},\theta)}\breve{p}_{0,h}G(x,y,\theta/h,\omega_{k})\overline{G(a,0,\theta/h,\omega_{k})}d\theta$

where $G$ is the one introduced in (2.7) (there is a factor $h^{-\frac{1}{3}}$
inside each $G$ \- which appears from the change of variables with respect to
$s$, ($s^{\prime}$, respectively) therefore the exponent of $h$ had changed
into $(-d+1/3)$).

###  4.1. Pseudo-differential calculus; construction of gallery modes.

######  Definition 4.1.

Let $G$ be defined in (2.7), satisfying (2.8) and let like before
$\tau=\tau(\omega,\eta)=\sqrt{|\eta|^{2}+\omega q(\eta)^{2/3}}$ and
$\rho(\alpha,\theta)=\sqrt{|\theta|^{2}+\alpha q(\theta)^{2/3}}$, where, for
$\eta=\theta/h$ and $\alpha=h^{2/3}\omega$ we have
$\tau(\omega,\eta)=\rho(\alpha,\theta)/h$. We set

(4.5) $e(x,y,\eta,\omega)=\frac{1}{2\pi\sqrt{L^{\prime}(\omega)}}\int
q(\eta)^{1/6}e^{i(\sigma^{3}/3+\sigma(xq^{1/3}(\eta)-\omega)+\rho\Gamma(x,y,\sigma
q^{1/3}(\eta)/\rho,\eta/\rho))}pd\sigma.$

Then
$e(x,y,\eta,\omega)=\frac{q(\eta)^{1/6}}{\sqrt{L^{\prime}(\omega_{k})}}e^{-iy\eta}G(x,y,\eta,\omega)$
from (2.13); we now define, for $g\in L^{2}(\mathbb{R}^{d-1})$,

$F_{k}(g)(x,y):=\frac{1}{(2\pi)^{d-1}}\int
e^{i(y-z)\eta}e(x,y,\eta,\omega_{k})g(z)d\eta dz=\int
G(x,y,\eta,\omega_{k})\hat{g}(\eta)d\eta.$

For a given $N$ and $1\leq k\leq N$ we consider the operator

$j_{N}:(L^{2}(\mathbb{R}^{d-1}))^{N}\rightarrow
L^{2}(\mathbb{R}^{d-1}(L^{2}(\mathbb{R}_{+}))$

defined as follows

$j_{N}(g_{1},...,g_{N})=\sum_{k=1}^{N}F_{k}(g_{k}).$

######  Proposition 4.2.

For $N\lesssim h^{-\theta}$, $\theta<\frac{1}{10}$, the following holds

(4.6)
$\|j_{N}(g_{1},...,g_{N})\|_{L^{2}(\Omega)}\simeq\|(g_{1},...,g_{N})\|_{(L^{2}(\mathbb{R}^{d-1}))^{N}}.$

Before the proof of the proposition we shall slightly modify $j_{N}$ by
replacing it by $j_{N}\circ\mathfrak{L}$, where $\mathfrak{L}$ is an isometry
on $L^{2}$ suitably chosen. For $g\in L^{2}(\mathbb{R}^{d-1})$ set

(4.7) $\mathfrak{L}(g)(z):=\int
e^{i(z-y^{\prime})\eta-i|\eta|B_{0}(y^{\prime},\eta/|\eta|)}g(y^{\prime})dy^{\prime}d\eta,$

where $B_{0}$ is the first term in the development of $B$ in (5.2) near the
glancing region $\mathcal{G}$, homogeneous of degree $0$, such that the phase
function $\Gamma=B_{0}+(|\eta|-1)B_{2}+xA_{1}+O(...)$.

######  Lemma 4.3.

$\mathfrak{L}$ is an isometry on $L^{2}(\mathbb{R}^{d-1})$.

The lemma follows using the form of $B_{0}$ from Lemma 5.2 and (5.9).

######  Remark 4.4.

Our goal here is to get rid of the term homogeneous of degree zero of the
phase function of $e(x,y,\eta,\omega_{k})$. In fact, in order to prove
Proposition 4.2, we shall be reduced to estimate the derivatives of
$e(.,\omega_{k})$ with respect to $(y,\eta)$: while in the Friedlander case
there is no dependence on $y$ in the formula of $e_{k}$, in the general
situation the derivative with respect to $y$ yields a factor which is very bad
for our estimates and which comes from the homogeneous of degree zero part of
the phase $\tau\Gamma(x,y,\sigma q^{1/3}(\eta)/\tau,\eta/\tau)$. We have:

$\displaystyle\tau\Gamma(x,y,\sigma q^{1/3}(\eta)/\tau,\eta/\tau)$
$\displaystyle=\tau(\omega,\eta)B_{0}(y,\eta/|\eta|)+\tau(|\eta|/\tau-1)B_{2}+...$
$\displaystyle=\tau(\omega,\eta)B_{0}(y,\eta/|\eta|)+\frac{\omega}{2}|\eta|^{1/3}B_{2}+....$

######  Remark 4.5.

Since $\mathfrak{L}$ is an isometry, it will be enough to prove Proposition
4.2 where $j_{N}$ is replaced by $\tilde{j}_{N}$, where we have set

$\tilde{j}_{N}(g_{1},...,g_{N}):=j_{N}\Big{(}\mathfrak{L}(g_{1}),...,\mathfrak{L}(g_{N})\Big{)}=\sum_{k=1}^{N}F_{k}\circ\mathfrak{L}(g_{k}).$

We set
$\tilde{e_{k}}(x,y,\eta)=e^{-i|\eta|B_{0}(y,\eta/|\eta|)}e(x,y,\eta,\omega_{k})$
and we compute

$\displaystyle F_{k}\circ\mathfrak{L}(g)(x,y)$
$\displaystyle=\frac{1}{(2\pi)^{d-1}}\int_{z}\int_{\eta}e^{i(y-z)\eta}e(x,y,\eta,\omega_{k})\mathfrak{L}(g)(z)d\eta
dz$
$\displaystyle=\frac{1}{(2\pi)^{d-1}}\int_{y^{\prime}}\int_{\eta}e^{i(y-y^{\prime})\eta-i|\eta|B_{0}(y^{\prime},\eta/|\eta|)}e(x,y,\eta,\omega_{k})g(y^{\prime})d\eta
dy^{\prime}$
$\displaystyle=\frac{1}{(2\pi)^{d-1}}\int_{y^{\prime}}\int_{\eta}e^{i(y-y^{\prime})\eta+i|\eta|(B_{0}(y,\eta/|\eta|)-B_{0}(y^{\prime},\eta/|\eta|))}\tilde{e_{k}}(x,y,\eta)g(y^{\prime})d\eta
dy^{\prime}$
$\displaystyle=\frac{1}{(2\pi)^{d-1}}\int_{y^{\prime}}\int_{\eta}e^{i(y-y^{\prime})\Theta}\tilde{e_{k}}(x,y,\Theta)\Big{|}\frac{d\Theta}{d\eta}\Big{|}g(y^{\prime})d\Theta
dy^{\prime},$

where in the last line we have set
$(y-y^{\prime})\eta+|\eta|(B_{0}(y,\eta/|\eta|)-B_{0}(y^{\prime},\eta/|\eta|))=(y-y^{\prime})\Theta(y,y^{\prime},\eta)$
whose Jacobian satisfies
$\Big{|}\frac{d\Theta}{d\eta}\Big{|}=1+O(|\nabla_{y}B_{0}|)$.

###### Proof.

(of Proposition 4.2) In order to show that for some $\theta\in[0,1]$ and
$N\leq h^{-\theta}$ we do have (4.6) it will be enough to prove that
$\tilde{j}_{N}^{*}\tilde{j}_{N}$ is bounded and invertible on
$(L^{2}(\mathbb{R}^{d-1}))^{N}$. In the following we compute this operator
explicitly. Let $f\in L^{2}(\Omega)$, then for $(g_{1},...,g_{N})\in
L^{2}(\mathbb{R}^{d-1})$ we have

$\displaystyle<(g_{1},...,g_{N}),\tilde{j}_{N}^{*}(f)>_{(L^{2}(\mathbb{R}^{d-1}))^{N}}$
$\displaystyle=<\tilde{j}_{N}(g_{1},...,g_{N}),f>_{L^{2}(\mathbb{R}^{d-1}(L^{2}(\mathbb{R}_{+}))}$
$\displaystyle=\int_{\Omega}\sum_{k=1}^{N}F_{k}(\mathfrak{L}(g_{k}))(x,y)\overline{f}(x,y)dxdy$
$\displaystyle=\sum_{k=1}^{N}\int_{\mathbb{R}^{d-1}}g_{k}(y)\overline{(F_{k}\circ\mathfrak{L})^{*}(f)}(y)dy.$

Setting $\tilde{F_{k}}=F_{k}\circ\mathfrak{L}$, we obtain
$\tilde{j}_{N}(f)=(\overline{\tilde{F_{1}}^{*}(f)},...,\overline{\tilde{F_{N}}^{*}(f)})$,
hence

$\tilde{j}_{N}^{*}(\tilde{j}_{N}(g_{1},...,g_{N}))=\Big{(}\tilde{F_{1}}^{*}\Big{(}\sum\tilde{F_{k}}(g_{k})\Big{)},...,\tilde{F_{N}}^{*}\Big{(}\sum\tilde{F_{k}}(g_{k})\Big{)}\Big{)}.$

The matrix $M_{N}$ associated to the operator
$\tilde{j_{N}}^{*}\circ\tilde{j_{N}}:(L^{2}(\mathbb{R}^{d-1}))^{N}\rightarrow(L^{2}(\mathbb{R}^{d-1}))^{N}$
is given by
$M_{N}:=(\tilde{F_{j}}^{*}\circ\tilde{F_{k}})_{j,k\in\\{1,...,N\\}}$. We
compute the operators $\tilde{F_{j}}^{*}\tilde{F_{k}}$ explicitly. Our goal is
to show that $M_{N}$ is a matrix of pseudo-differential operators in $y$,
bounded and invertible, uniformly with respect to $N\lesssim h^{-\theta}$ if
$\theta<\frac{1}{10}$. For $g\in L^{2}(\mathbb{R}^{d-1})$ and $f\in
L^{2}(\Omega)$ we have
$<\tilde{F_{k}}(g),f>_{L^{2}(\Omega)}=<g,\tilde{F_{k}}^{*}(f)>_{L^{2}(\mathbb{R}^{d-1})}$,
therefore

$\displaystyle<\tilde{F_{k}}(g),f>_{L^{2}(\Omega)}$
$\displaystyle=\int_{\Omega}\Big{(}\frac{1}{(2\pi)^{d-1}}\int_{z}\int_{\eta}e^{i(y-z)\eta}e(x,y,\eta,\omega_{k})\mathfrak{L}(g)(z)d\eta
dz\Big{)}\overline{f}(x,y)dxdy$
$\displaystyle=\int_{\Omega}\Big{(}\frac{1}{(2\pi)^{d-1}}\int_{y^{\prime}}\int_{\eta}e^{i(y-y^{\prime})\eta-i|\eta|B_{0}(y^{\prime},\eta/|\eta|)}e(x,y,\eta,\omega_{k})g(y^{\prime})d\eta
dy^{\prime}\Big{)}\overline{f}(x,y)dxdy$
$\displaystyle=\int_{y^{\prime}}g(y^{\prime})\Big{(}\overline{\frac{1}{(2\pi)^{d-1}}\int_{\Omega}\int_{\eta}e^{i(y^{\prime}-y)\eta+i|\eta|B_{0}(y^{\prime},\eta/|\eta|)}\overline{e(x,y,\eta,\omega_{k})}f(x,y)d\eta
dxdy}\Big{)}dy^{\prime}$
$\displaystyle=\int_{z}g(z)\overline{\tilde{F_{k}}^{*}(f)}(z)dz=<g,\tilde{F_{k}}^{*}(f)>_{L^{2}(\mathbb{R}^{d-1})},$

which yields the formula of $\tilde{F_{k}}^{*}(f)$. We compute
$\tilde{F_{j}}^{*}\circ\tilde{F_{k}}(g)(z)$ for $g\in
L^{2}(\mathbb{R}^{d-1})$, $z\in\mathbb{R}^{d-1}$:

$\displaystyle\tilde{F_{j}}^{*}\circ\tilde{F_{k}}(g)(z)$
$\displaystyle=\frac{1}{(2\pi)^{d-1}}\int_{\Omega}\int_{\eta}e^{i(z-y)\eta+i|\eta|B_{0}(z,\eta/|\eta|)}\overline{e(x,y,\eta,\omega_{j})}d\eta\tilde{F_{k}}(g)(x,y)dxdy$
(4.8)
$\displaystyle=\int_{z^{\prime}}M_{j,k}(z,z^{\prime})g(z^{\prime})dz^{\prime},$

where we have

$M_{j,k}(z,z^{\prime})=\frac{1}{(2\pi)^{d-1}}\int_{y,\eta,\eta^{\prime}}e^{i\Big{[}(z-y)\eta+|\eta|(B_{0}(z,\eta/|\eta|)-B_{0}(y,\eta/|\eta|))\Big{]}-i\Big{[}(z^{\prime}-y)\eta^{\prime}+|\eta^{\prime}|(B_{0}(z^{\prime},\eta^{\prime}/|\eta^{\prime}|)-B_{0}(y,\eta^{\prime}/|\eta^{\prime}|))\Big{]}}\\\
\times\int_{0}^{\infty}\overline{\tilde{e_{j}}(x,y,\eta)}\tilde{e_{k}}(x,y,\eta^{\prime})dxd\eta^{\prime}d\eta
dy.$

Let like before $\Theta=\Theta(z,y,\eta)$ (respectively
$\Theta^{\prime}=\Theta^{\prime}(z^{\prime},y,\eta^{\prime})$) such that

$(z-y)\Theta=(z-y)\eta+|\eta|(B_{0}(z,\eta/|\eta|)-B_{0}(y,\eta/|\eta|),$

then $|\nabla_{\eta}\Theta|\simeq 1+O(|\nabla_{y}B_{0}|)$ is close to $1$ and
hence $|\nabla_{\Theta}\eta|\simeq 1+O(|\nabla_{y}B_{0}|)$ also, hence

(4.9)
$M_{j,k}(z,z^{\prime})=\frac{1}{(2\pi)^{d-1}}\int_{y,\Theta,\Theta^{\prime}}e^{i(z-y)\Theta-i(z^{\prime}-y)\Theta^{\prime}}|\nabla_{\Theta}\eta||\nabla_{\Theta^{\prime}}\eta^{\prime}|\int_{0}^{\infty}\overline{\tilde{e_{j}}(x,y,\Theta)}\tilde{e_{k}}(x,y,\Theta^{\prime})dxd\Theta^{\prime}d\Theta
dy.$

######  Remark 4.6.

Notice that in the case of the Friedlander operator we have $B_{0}=0$
everywhere and we do not need to introduce the operator $\mathfrak{L}$ (which
is the identity in this case); moreover the corresponding gallery modes
$e_{j}$, $e_{k}$ being orthogonal, the integral over the normal variable $x$
equals $\delta_{j,k}$ and hence $F_{j}^{*}\circ F_{k}=0$ for $j\neq k$, while
$F_{k}^{*}\circ F_{k}(g)=g$ . The corresponding matrix satisfies
$M_{N}=I_{N}$.

Taking in (4.9) $y=z^{\prime}+w$, $\Theta^{\prime}=\Theta-\zeta$ yields

(4.10)
$M_{j,k}(z,z^{\prime})=\int_{\Theta}e^{i(z-z^{\prime})\Theta}m_{j,k}(z,z^{\prime},\Theta)d\Theta,$
$m_{j,k}(z,z^{\prime},\Theta)=\frac{1}{(2\pi)^{d-1}}\int_{\zeta}\int_{w}e^{-iw\zeta}\sigma\int_{0}^{\infty}\overline{\tilde{e_{j}}(x,z^{\prime}+w,\Theta)}\tilde{e_{k}}(x,z^{\prime}+w,\Theta-\zeta)dxdwd\zeta,$

where we set

(4.11)
$\sigma=\sigma(z,z^{\prime},\Theta;w,\zeta)=|\nabla_{\Theta}\eta|(z,z^{\prime}+w,\Theta)|\nabla_{\Theta^{\prime}}\eta^{\prime}|(z^{\prime},z^{\prime}+w,\Theta-\zeta).$

Notice that in (4.10) we can replace $m_{j,k}(z,z^{\prime},\Theta)$ by
$m_{j,k}(z^{\prime},z^{\prime},\Theta)$ without changing the integral modulo
$O(\Theta^{-\infty})$. Since $\Theta=\frac{\theta}{h}$, for some $\theta\simeq
1$, $h\in(0,1]$, taking $\zeta=\varrho/h$,
$\tilde{\sigma}(z^{\prime},.)=\sigma(z^{\prime},z^{\prime},.)$, we replace
$m_{j,k}(z^{\prime},z^{\prime},\Theta)$ by
$\tilde{m}_{j,k}(z^{\prime},\theta,h)$ given by

$\tilde{m}_{j,k}(z^{\prime},\theta,h)=\frac{1}{(2\pi
h)^{d-1}}\int_{\varrho}\int_{w}e^{-\frac{i}{h}w\varrho}\tilde{\sigma}\int_{0}^{\infty}\overline{\tilde{e_{j}}(x,z^{\prime}+w,\theta/h)}\tilde{e_{k}}(x,z^{\prime}+w,(\theta-\varrho)/h)dxdwd\varrho.$

######  Proposition 4.7.

For $N\leq h^{-\theta}$ for $\theta\in[0,\frac{1}{10})$, the matrix $M_{N}$
can be written under the form $M_{N}=D_{N}+R_{N}$, where

  1. (1)

$D_{N}=\text{diag}(\tilde{F_{k}}^{*}\circ\tilde{F_{k}})_{k\in\\{1,...,N\\}}$
is a matrix of pseudo-differential operators;

  2. (2)

$R_{N}$ satisfies $\|D_{N}^{-1}R_{N}\|\leq\frac{1}{2}$.

######  Remark 4.8.

Notice that it will be easier to estimate the Hilbert-Schmidt norm of the
elements of $M_{N}$ instead of their $L^{2}$ norm. Recall that if
$k:\mathbb{R}^{2n}\rightarrow\mathbb{C}$ is such that
$\int_{\mathbb{R}^{2n}}|k(x,y)|^{2}dxdy<\infty$, then the operator
$K:L^{2}(\mathbb{R}^{n},\mathbb{C})\rightarrow
L^{2}(\mathbb{R}^{n},\mathbb{C})$ by

$Kg(z)=\int k(z,z^{\prime})g(z^{\prime})dz^{\prime}$

is Hilbert-Schmidt, with Hilbert-Schmidt norm
$\|K\|_{HS}=\|k\|_{L^{2}}\geq\|K\|_{L^{2}\rightarrow L^{2}}$. From the formula
(4.1), it follows that in order to estimate the Hilbert-Schmidt norms of
$\tilde{F_{j}}^{*}\circ\tilde{F_{k}}$ it is enough to estimate the $L^{2}$
norms of their kernels $M_{j,k}$ defined in (4.10). Since
$\|M_{j,k}\|_{L^{2}_{z,z^{\prime}}}=\|\tilde{m}_{j,k}(z^{\prime},\theta,h)\|_{L^{2}_{z^{\prime},\theta}}$,
we estimate the last norms first for $j=k$, then for $j\neq k$.

######  Definition 4.9.

We recall the definition of the class of symbols $\mathcal{S}_{\gamma}$ :

$\mathcal{S}_{\gamma}:=\\{a\in C^{\infty}||\partial^{\beta}a(w,\varrho,h)|\leq
C_{\beta}h^{-\gamma|\beta|}\\}.$

######  Proposition 4.10.

Let $a(w,\varrho,h):\mathbb{R}^{2n}\rightarrow\mathbb{C}$ be smooth,
$a\in\mathcal{S}_{\frac{1}{2}-\epsilon}$, for some small $\epsilon>0$. Then we
have

$\frac{1}{(2\pi
h)^{n}}\int_{\varrho}\int_{w}e^{-\frac{i}{h}w\varrho}a(w,\varrho,h)dwd\varrho\simeq\sum\frac{h^{|\beta|}}{|\beta|!}\Big{(}\frac{<D_{w},D_{\varrho}>}{i}\Big{)}^{|\beta|}a(w,\varrho,h)\Big{|}_{(w,\varrho)=(0,0)}.$

We define

(4.12)
$a_{j,k}(z^{\prime},\theta;w,\varrho;h):=\tilde{\sigma}(z^{\prime},\theta/h;w,\varrho/h)\int_{0}^{\infty}\overline{\tilde{e_{j}}(x,z^{\prime}+w,\theta/h)}\tilde{e_{k}}(x,z^{\prime}+w,(\theta-\varrho)/h)dx.$

Using Proposition 4.10, we intend to write
$\tilde{m}_{j,k}(z^{\prime},\theta,h)$ under the form

(4.13)
$\tilde{m}_{j,k}(z^{\prime},\theta,h)\simeq\sum_{\beta}\frac{h^{|\beta|}}{i^{|\beta|}|\beta|!}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{j,k}(z^{\prime},\theta;w,\varrho;h)|_{w=0,\varrho=0}.$

We now prove Proposition 4.7), starting with proving that $D_{N}$ is a matrix
of pseudo-differential operators. For, we have to show that $\exists C>0$
independing on $N\lesssim h^{-\theta}$, such that the following holds :

(4.14) $\|\tilde{F_{k}}^{*}\circ\tilde{F_{k}}\|_{HS}\leq C\text{ and
}\|(\tilde{F_{k}}^{*}\circ\tilde{F_{k}})^{-1}\|_{HS}\leq C.$

Notice that this is equivalent to saying that
$\tilde{F_{k}}^{*}\circ\tilde{F_{k}}:L^{2}(\mathbb{R}^{d-1})\rightarrow
L^{2}(\mathbb{R}^{d-1})$ is a pseudo-differential operator, uniformly elliptic
w.r.t. $1\leq k\leq N\lesssim h^{-\theta}$. From (4.1), (4.10) and Remark 4.8
it follows that
$\|\tilde{F_{k}}^{*}\circ\tilde{F_{k}}\|_{HS}=\|M_{k,k}\|_{L^{2}_{z,z^{\prime}}}=\|\tilde{m}_{k,k}(z^{\prime},\theta,h)\|_{L^{2}_{z^{\prime},\theta}}$
and using (4.13) (for values $k\leq N\lesssim h^{-\theta}$ for which we are
allowed to apply Proposition 4.10, that we shall determine below) we get

(4.15)
$\|a_{k,k}(z^{\prime},\theta;0,0;h)\|_{L^{2}_{z^{\prime},\theta}}-\Big{\|}\sum_{|\beta|\geq
1}\frac{h^{|\beta|}}{|\beta|!}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{k,k}(z^{\prime},\theta;w,\varrho;h)|_{w=0,\varrho=0}\Big{\|}_{L^{2}_{z^{\prime},\theta}}\leq\|\tilde{m}_{k,k}(z^{\prime},\theta,h)\|_{L^{2}_{z^{\prime},\theta}}\\\
\leq\|a_{k,k}(z^{\prime},\theta;0,0;h)\|_{L^{2}_{z^{\prime},\theta}}+\Big{\|}\sum_{|\beta|\geq
1}\frac{h^{|\beta|}}{|\beta|!}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{k,k}(z^{\prime},\theta;w,\varrho;h)|_{w=0,\varrho=0}\Big{\|}_{L^{2}_{z^{\prime},\theta}}.$

We start with the $L^{2}$ norm of $a_{k,k}(z^{\prime},\theta;0,0;h)$, which
equals

$a_{k,k}(z^{\prime},\theta;0,0;h)=\tilde{\sigma}(z^{\prime},z^{\prime},\theta/h;0,0)\|\tilde{e_{k}}(.,z^{\prime},\theta/h)\|^{2}_{L^{2}_{x}}.$

Since $e_{k}$ are $L^{2}$ normalised in the normal variable, the same holds
for $\tilde{e}_{k}$, as the phase function $B_{0}$ does not depend on $x$. On
the other hand, $\tilde{\sigma}=\sigma|_{z=z^{\prime}}$ is an elliptic symbol
as defined in (4.11). Therefore $a_{k,k}$ is elliptic and
$\|a_{k,k}(z^{\prime},\theta;0,0;h)\|_{L^{2}_{z^{\prime},\theta}}\simeq 1$. In
order Proposition 4.10 to apply, it remains to check that for $|\beta|\geq 1$
we have

$h^{|\beta|}\Big{\|}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{k,k}(z^{\prime},\theta;w,\varrho;h)|_{w=0,\varrho=0}\Big{\|}_{L^{2}_{z^{\prime},\theta}}\leq
h^{\epsilon}.$

for some $\epsilon>0$ as small as we want. We need the following lemma:

######  Lemma 4.11.

Let $\rho=(y,\theta)$, then the following holds, uniformly for
$h^{2/3}\omega_{k}\ll 1$:

(4.16) $\|\partial^{\beta}_{\rho}\tilde{e_{k}}(.,y,\theta/h)\|_{L^{2}(x\geq
0)}\lesssim\omega_{k}^{\frac{3|\beta|}{2}}+\Big{(}\omega_{k}/h^{1/3}\Big{)}^{|\beta|}.$

Using (4.12) for $j=k$, the properties of $\tilde{\sigma}$ and (4.16), we get

$h^{|\beta|}\Big{\|}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{k,k}(z^{\prime},\theta;w,\varrho;h)|_{w=0,\varrho=0}\Big{\|}_{L^{2}_{z^{\prime},\theta}}\lesssim
C_{\beta}h^{\frac{|\beta|}{3}}k^{4|\beta|/3},\quad\forall|\beta|\geq 1,$

and if we ask this to be smaller than a small, positive power of $h$ for all
$k\lesssim h^{-\theta}$ it follows that we must have $\theta<\frac{1}{4}$. It
is clear that the proof will follow the same path for the inverse operator
$(\tilde{F}_{k}^{*}\circ\tilde{F}_{k})^{-1}$, with the same condition on
$\theta$.

For the second part of the proof it will be enough to show that
$\|R_{N}\|_{(L^{2}(\mathbb{R}^{d-1}))^{N}}\leq h^{\epsilon}$ for some small
$\epsilon>0$. Let $j\neq k$ and write

$\|\tilde{m}_{j,k}(z^{\prime},\theta,h)\|_{L^{2}_{z^{\prime},\theta}}\leq\|a_{j,k}(z^{\prime},\theta;0,0;h)\|_{L^{2}_{z^{\prime},\theta}}+\Big{\|}\sum_{|\beta|\geq
1}\frac{h^{|\beta|}}{|\beta|!}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{j,k}(z^{\prime},\theta;w,\varrho;h)|_{w=0,\varrho=0}\Big{\|}_{L^{2}_{z^{\prime},\theta}}.$

To estimate $\|a_{j,k}\|_{L^{2}_{z^{\prime},\theta}}$ we use the Cauchy-
Schwartz estimate and the following lemma :

######  Lemma 4.12.

For $j\neq k$ such that $h^{2/3}\omega_{k}\ll 1$ and $h^{2/3}\omega_{j}\ll 1$,
the following holds:

(4.17) $<\tilde{e_{k}},\tilde{e_{j}}>_{L^{2}(x>0)}\lesssim
h^{1/3}\Big{(}1+\frac{1}{|\omega_{k}-\omega_{j}|}\Big{)}.$

Since $z^{\prime}$ remains bounded, $\theta$ is close to one and
$\tilde{\sigma}$ is a smooth symbol, this gives

$\|a_{j,k}\|_{L^{2}_{z^{\prime},\theta}}\lesssim<\tilde{e_{k}},\tilde{e_{j}}>_{L^{2}(x>0)}\lesssim
h^{1/3}\Big{(}1+\frac{1}{|\omega_{k}-\omega_{j}|}\Big{)},$ $\sum_{j\neq
k,j,k\leq N}\|a_{j,k}\|_{L^{2}_{z^{\prime},\theta}}\lesssim
2h^{1/3}\sum_{j<k\leq
N}\Big{(}1+\frac{1}{|\omega_{k}-\omega_{j}|}\Big{)}\lesssim h^{1/3}N^{2}.$

Taking the derivatives w.r.t. $(w,\varrho)$ and using (4.16) and (4.17)
yields:

(4.18)
$\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{j,k}(z^{\prime},\theta,;w,\varrho;h)|_{(w,\varrho)=(0,0)}=\sum_{l_{1}+l_{2}=0}^{|\beta|}\sum_{q=0}^{|\beta|}C_{|\beta|}^{q}\partial^{(|\beta|-l_{1}-l_{2},|\beta|-q)}_{(w,\varrho)}\tilde{\sigma}(z^{\prime},\theta/h;w,\varrho/h)\\\
\times\int_{0}^{\infty}\partial^{l_{1}}_{w}\overline{\tilde{e}_{j}(x,z^{\prime}+w,\theta/h)}\partial^{(l_{2},q)}_{(w,\varrho)}\tilde{e}_{k}(x,z^{\prime}+w,(\theta-\varrho)/h)dx\Big{|}_{(w,\varrho)=(0,0)}.$

To estimate (4.18) we now apply Cauchy Schwartz and use (4.16) which give
bounds for the integral over $x$ (and we also use the properties of $B_{0}$ to
deduce that the symbol $\tilde{\sigma}$ is bounded together with all its
derivatives). For $1\leq j,k\leq N$ such that $h^{2/3}\omega_{j}\ll 1$ and
$h^{2/3}\omega_{k}\ll 1$ we write

$\|R_{N}\|_{HS}\leq\sum_{j,k\leq N,j\neq
k}\|\tilde{m}_{j,k}(z^{\prime},\theta,h)\|_{L^{2}_{z^{\prime},\theta}}\leq
h^{1/3}N^{2}+\sum_{|\beta|\geq 1}h^{|\beta|}\|\sum_{j\neq k,j,k\leq
N}\partial^{\beta}_{w}\partial^{\beta}_{\varrho}a_{j,k}(z^{\prime},\theta,.;h)|_{(0,0)}\|_{L^{2}}\\\
\leq h^{1/3}N^{2}+\sum_{|\beta|\geq 1}C_{\beta}h^{|\beta|}\sum_{j\neq
k,j,k\leq
N}\Big{(}\frac{(jk)^{2|\beta|/3}}{h^{2|\beta|/3}}+\frac{k^{4|\beta|/3}}{h^{2|\beta|/3}}\Big{)}\\\
\leq h^{1/3}N^{2}+\sum_{|\beta|\geq 1}h^{\frac{|\beta|}{3}}\Big{(}\sum_{1\leq
j\leq N}j^{2/3}\Big{)}^{2}\simeq h^{1/3}N^{2}+\sum_{|\beta|\geq
1}h^{\frac{|\beta|}{3}}N^{2+4|\beta|/3}.$

If we impose the Hilbert-Schmidt norm of $R_{N}$ to satisfy
$\|R_{N}\|_{HS}\leq h^{\epsilon}$ for some small $\epsilon>0$, we obtain a
restriction on $\theta$, where $N\leq h^{-\theta}$: we must have

$\frac{|\beta|}{3}-\theta(2+\frac{4|\beta|}{3})>0\quad\forall|\beta|\geq 1,$

which gives $\theta<\frac{|\beta|}{6+4|\beta|}$ and for $|\beta|=1$ this gives
$\theta<\frac{1}{10}$. For values of $N\ll h^{-1/10}$ we also have
$h^{1/3}N^{2}\ll h^{\frac{1}{3}-\frac{1}{5}}=h^{\frac{2}{15}}$ and therefore
this term is also very small. Since $\|R_{N}\|_{HS}\lesssim h^{\epsilon}$ for
$\frac{1}{3}-\theta(2+\frac{4}{3})\geq\epsilon>0$, the Hilbert-Schmidt norm of
$D_{N}^{-1}\circ R_{N}$ will be also as small. ∎

###  4.2. Dispersion estimates for $a$ small

Until now we have provided sharp uniform dispersive bounds for the Green
function $\mathcal{P}_{h,a}$ for all $h^{2/3}\ll a\leq 1$, using formula (4.1)
(where the sum is indexed over the number of reflections on the boundary). In
this part we provide dispersive estimates for $\mathcal{P}_{h,a}$ for $a$ very
small, $a\in(0,h^{2/3(1-\epsilon)}]$ for some $\epsilon>0$; in order to do
this, we use the formula (4.2) (in terms of gallery modes, constructed above).

From formula (4.4) and the definition (4.5) (where the factor $q(\eta)^{1/6}$
gives a factor $h^{-1/3}$, which appears twice) it follows that

(4.19) $\displaystyle\mathcal{P}_{h,a}(t,x,y)$
$\displaystyle=\sum_{k\in{\mathbb{N}}^{*}}h^{-(d-1)}\int
e^{\frac{i}{h}(y\theta+t\rho(h^{2/3}\omega_{k},\theta))}\breve{p}_{0,h}e(x,y,\theta/h,\omega_{k})\overline{{e}(a,0,\theta/h,\omega_{k})}d\theta$
$\displaystyle=\sum_{k\in{\mathbb{N}}^{*}}h^{-(d-1)}\int
e^{\frac{i}{h}(y\theta+|\theta|B_{0}(y,\theta/|\theta|)+t\rho(h^{2/3}\omega_{k},\theta))}\breve{p}_{0,h}\tilde{e}(x,y,\theta/h,\omega_{k})\overline{\tilde{e}(a,0,\theta/h,\omega_{k})}d\theta$

since
$e(a,0,\theta/h,\omega)=\frac{e^{-i|\eta|B_{0}(0,\eta/|\eta|)}}{\sqrt{L^{\prime}(\omega_{k})}}q(\eta)^{1/6}{G}(a,0,\eta,\omega)=\tilde{e}(a,0,\theta/h,\omega)$
as $B_{0}(0,\eta/|\eta|)=0$.

######  Remark 4.13.

Notice that formula (4.19) coincides with the parametrix $\mathcal{P}_{F,h,a}$
if $\Delta$ is the Friedlander model operator
$\Delta_{F}=\partial^{2}_{xx}+(1+x)\Delta_{y}$: indeed, in this case the
eigenfunctions $\\{e_{k}(x,\eta)\\}_{k\geq 1}$ of the operator
$-\partial^{2}_{xx}+(1+x)|\eta|^{2}$ associated to the eigenvalues
$\lambda_{k}(\eta)=|\eta|^{2}+\omega_{k}|\eta|^{4/3}$ are given by

$e_{k}(x,\eta)=\frac{f_{k}}{k^{1/6}}|\eta|^{1/3}Ai(|\eta|^{2/3}x-\omega_{k}),$

and setting $\eta=\theta/h$ yields

(4.21)
$\mathcal{P}_{F,h,a}(t,x,y)=\sum_{k\in{\mathbb{N}}^{*}}\frac{1}{h^{d-1}}\int_{\mathbb{R}^{d-1}}e^{it\sqrt{\lambda_{k}(\theta/h)}}e^{iy\theta/h}\\\
\psi(\theta)e_{k}(x,\theta/h)\overline{e_{k}(a,\theta/h)}d\theta.$

To estimate $\mathcal{P}_{h,a}$ for $a\in(0,h^{2/3(1-\epsilon)}]$,
$\epsilon>0$ small, we distinguish the three cases. The main contribution
corresponds to values $k\lesssim\lambda=\frac{a^{3/2}}{h}\lesssim
h^{-\epsilon}$ and we need to deal separately with two situations, according
to whether $t$ is small or not.

  * •

When the time is very small, $t\lesssim h^{\epsilon}$, the stationary phase
does not apply: in this case we use Lemma [8, Lemma 3.5] and its the
corresponding version involving the derivatives of the Airy function, that we
summarise here:

######  Lemma 4.14.

There exists $C_{0}$ such that for $L\geq 1$ the following holds true

(4.22) $\sup_{b\in{\mathbb{R}}}\Big{(}\sum_{1\leq k\leq
L}k^{-1/3}Ai^{2}(b-\omega_{k})\Big{)}\leq C_{0}L^{1/3}.$

Moreover, we have

(4.23) $\sup_{b\in{\mathbb{R}}_{+}}\Big{(}\sum_{1\leq k\leq
L}k^{-1/3}h^{2/3}Ai^{\prime 2}(b-\omega_{k})\Big{)}\leq C_{0}h^{2/3}L,$ (4.24)
$\sup_{b\in{\mathbb{R}}_{-}}\Big{(}\sum_{1\leq k\leq
L}k^{-1/3}h^{2/3}Ai^{\prime 2}(b-\omega_{k})\Big{)}\leq C_{0}h^{1/3}L^{2/3}.$

Using Theorem 2.1 we can rewrite each mode
$e(x,y,\eta,\omega_{k})=\frac{q(\eta)^{1/6}e^{-iy\eta}}{\sqrt{L^{\prime}(\omega_{k})}}G(x,y,\eta,\omega_{k})$
(and hence each $\tilde{e}$) defined in (4.5) as an integral with phase
functions $\psi$ and $\zeta$; introducing this in (4.19) and applying the
Cauchy-Schwartz inequality, the dispersive inequalities for $t$ small reduce
to estimates like (4.22) and (4.23), (4.24): using the above Lemma yields

(4.25) $\|\mathcal{P}_{h,a}(t,x,y)\|_{L^{\infty}}\leq
h^{-d}h^{1/3}\Big{(}L^{1/3}+h^{1/3}L^{2/3}+h^{2/3}L\Big{)},$

where we have first obtained a factor $h^{-2/3}=h^{-1}h^{1/3}$ from
$q(\theta/h)^{1/3}$ and where the terms in the bracket correspond to (4.22)
and (4.23), (4.24). Let now $L=h^{-\epsilon}$ and $t\leq\frac{1}{L}$, then we
obtain the bound

$\|\mathcal{P}_{h,a}(t,x,y)\|_{L^{\infty}}\leq
h^{-d}\Big{(}\Big{(}\frac{h}{t}\Big{)}^{1/3}+\Big{(}\frac{h}{t}\Big{)}^{2/3}+\Big{(}\frac{h}{t}\Big{)}\Big{)}\lesssim
h^{-d}\Big{(}\frac{h}{t}\Big{)}^{1/3}.$

  * •

When the time is larger, $h^{1/3}\ll h^{\epsilon}\lesssim t$, for values of
$k\in[1,h^{-\epsilon}]$ we can apply the stationary phase method for the phase
function

$\psi(x,y,\theta,\omega_{k})-\psi(a,0,\theta,\omega_{k})+t\rho(h^{2/3}\omega_{k},\theta)$

and where the symbol is given by

(4.26)
$\breve{p}_{0,h}\Big{(}p_{0}Ai+xp_{1}h^{1/3}|\theta|^{-1/3}Ai^{\prime}\Big{)}(\zeta(x,y,\theta/h,\omega_{k}))\Big{(}p_{0}Ai\\\
{}+ap_{1}h^{1/3}|\theta|^{-1/3}Ai^{\prime}\Big{)}(\zeta(a,0,\theta/h,\omega_{k})).$

Indeed, using Theorem 2.1 we can write $G$ in terms of $\psi$ and $\zeta$,
where $\psi$ is homogeneous of degree $1$ in $\theta$; from Proposition 5.3
and the homogeneity property of $\psi$, it follows that it has the following
form:

$\psi(x,y,\theta,\omega)=y\theta+|\theta|\Big{(}B_{0}(y,\theta/|\theta)+(|\theta|-1)B_{2}(y,\theta/|\theta|)+\mathcal{H}_{j\geq
3}\Big{)}.$

Since $B_{0}(0,\omega)=0$ and $B_{2}(0,\omega)=0$, we can write

(4.27)
$t\rho(h^{2/3}\omega_{k},\theta)+\psi(x,y,\theta,\omega_{k})-\psi(a,0,\theta,\omega_{k})=t\rho(h^{2/3}\omega_{k},\theta)+y\theta\\\
+|\theta|(B_{0}(y,\theta/|\theta|)+(|\theta|-1)B_{2}(y,\theta/|\theta|)+\mathcal{H}_{j\geq
3}).$

Since
$\rho(h^{2/3}\omega_{k},\theta)=\sqrt{|\theta|^{2}+h^{2/3}\omega_{k}q^{2/3}(\theta)}\simeq|\theta|+\frac{1}{2}h^{2/3}\omega_{k}q(\theta/|\theta|)^{2/3}|\theta|^{1/3}+O(h^{4/3}\omega^{2}_{k})$,
$B_{0}(y,\omega)=O(y^{3})$ and $B_{2}(y,\omega)=O(y^{2})$, the critical point
satisfies in the two dimensional case (when $\theta=|\theta|$) satisfy

$y(1+O(y))+t(1+\frac{1}{6}h^{2/3}\omega_{k}\theta^{-2/3}+O((h^{2/3}\omega_{k}\theta^{-1/3})^{2}))=0.$

The second derivative of the phase function (with respect to $\theta$) equals
$-\frac{t}{9}h^{2/3}\omega_{k}\theta^{-5/3}(1+O((h^{2/3}\omega_{k})^{1/2}))$:
indeed, the terms belonging to $\mathcal{H}_{j\geq 3}$ contain small factors
of size $(h^{2/3}\omega_{k})^{3/2}\simeq hk$, since if $x>Ch^{2/3}\omega_{k}$
or $a>Ch^{2/3}\omega_{k}$ for some constant $C>2$ then the terms involving
Airy functions decrease exponentially. It remains to show that we can apply
the stationary phase with large parameter $\frac{t\omega_{k}}{h^{1/3}}$ for
$t>h^{\epsilon}$, $\epsilon>0$ like before.

If we consider the second derivative with respect to $\theta$ of the symbol
(4.26), the main contribution is given by $\omega_{k}^{11/4}$ and occurs when
$\zeta(x,y,\theta/\theta,\omega_{k})$ is large and
$\zeta(a,0,\theta/h,\omega_{k})$ is bounded and when in the product
$Ai(\zeta(x,y,\theta/\theta,\omega_{k}))Ai(\zeta(a,0,\theta/h,\omega_{k}))$ we
take the second derivative of the first Airy factor only. Therefore, it
suffices to check that there exists $\epsilon>0$ such that

$\frac{h^{1/3}}{t\omega_{k}}\omega^{11/4}\ll 1\quad\text{for}\quad
t>h^{\epsilon}\text{ and }k\lesssim h^{-\epsilon}.$

This does hold as long as $\epsilon<\frac{2}{13}$. We then estimate, for
$\epsilon<2/13$ and $t>h^{\epsilon}$ :

$\|\mathcal{P}_{h,a}(t,x,y)\|_{L^{\infty}}\leq h^{-d+1/3}\sum_{1\leq k\lesssim
h^{-\epsilon}}\frac{1}{k^{1/3}}\Big{(}\frac{h^{1/3}}{t\omega_{k}}\Big{)}^{1/2}\simeq
h^{-d}\Big{(}\frac{h}{t}\Big{)}^{1/2}h^{-\epsilon/3}\lesssim
h^{-d}\Big{(}\frac{h}{t}\Big{)}^{1/3}.$

The last case corresponds to values $2h^{-\epsilon}\leq k\leq 1/h$ and
$t\in(0,1]$, $0\leq x,a\lesssim h^{\frac{2}{3}(1-\epsilon)}$. For these values
it is clear that $|\zeta(x,y,\eta,\omega_{k})|\geq\frac{\omega_{k}}{2}$ and
$|\zeta(a,y,\eta,\omega_{k})|\geq\frac{\omega_{k}}{2}$, therefore we can apply
the stationary phase method in (4.3) in both $s,s^{\prime}$: for small values
of $x,a\lesssim h^{\frac{2}{3}(1-\epsilon)}$, we obtain two factors $\simeq
h^{1/2}/|\omega_{k}|^{1/4}$ and since the integration with respect to $\theta$
provides a constant independent of $h,a$, we obtain in this case

$\|\mathcal{P}_{h,a}(t,x,y)\|_{L^{\infty}}\leq
h^{-d-1/3}\sum_{2h^{-\epsilon}\leq k\leq
1/h}\frac{1}{k^{1/3}}\frac{h}{\omega_{k}^{1/2}}\lesssim
h^{-(d-1)-1/3}\sum_{k\leq 1/h}k^{-2/3}\simeq h^{-d+1/3}\,,$

which ends the proof of (3.3) in Theorem 3.1, and therefore the proof of (1.1)
in Theorem 1.3 is complete. $\Box$

##  5\. Appendix

###  5.1. Some properties of the generating function of $\chi_{M}$ introduced
in Lemma 2.2

We present in this appendix the construction of the phase function $\Gamma$
that generates the canonical transformation $\chi_{M}$.

Let $\eta=\theta/h$ with $\theta=\varrho\omega$, $\omega=\eta/|\eta|$ for
$\varrho$ near $1$ and let the glancing set be given by
$\mathcal{GL}=\\{X=0,\xi=0,\varrho-1=0\\}$. We will work with formal taylor
expansions near $\mathcal{GL}$ of the form

$F=\sum_{a,b,c}f_{a,b,c}(Y,\omega)X^{a}(\varrho-1)^{b}\xi^{c}$

We attribute a weight to each factor $X,\varrho-1,\xi$: a monomial of the form
$X^{a}(\varrho-1)^{b}\xi^{c}$ is homogeneous of degree $k$ if and only if
$c+2(a+b)=k$. Given such a formal serie $F(X,Y,\xi,\eta)$ defined near the
glancing set $\mathcal{GL}$, we write $F\simeq\sum_{k\geq 0}F_{k}$, where
$F_{k}$ is homogeneous of degree $k$; we also write $F\in\mathcal{H}_{\geq j}$
if and only if $F\simeq\sum_{k\geq j}F_{k}$, where, like before, $F_{k}$ is
homogeneous of degree $k$. Therefore,

$F_{0}=f_{0}(Y,\omega);\quad F_{1}=\xi f_{1}(Y,\omega);\quad
F_{2}=Xf_{2}^{0}(Y,\omega)+(\varrho-1)f_{2}^{1}(Y,\omega)+\xi^{2}f_{2}^{2}(Y,\omega);\quad\text{etc}.$

Replacing $x,\Xi,\Theta$ by their formulas (2.11) (as functions of
$(X,Y,\xi,\eta)$) and using that from (2.9)

(5.1) $\Xi^{2}+R(X,Y,\Theta)=1\quad\text{ if and only if
}\quad\xi^{2}+|\eta|^{2}+xq(\eta)=1,$

(where notice that $y$ doesn’t appear at all) yields

(5.2) $\left\\{\begin{array}[]{l}B\simeq\sum_{j\geq
0}(\varrho-1)^{j}B_{2j}(Y,\omega),\\\ A\simeq\sum_{k\geq
1}A_{k}\end{array}\right.$

Indeed, since $\Xi|_{\mathcal{G}}=0$ we must have $A_{0}=0$. We also have
$\Xi(X,Y,\xi,\eta)\in\mathcal{H}_{\geq 1}$ and
$x(X,Y,\xi,\eta)\in\mathcal{H}_{\geq 2}$. Moreover, from the proof of
Melrose’s theorem it follows that if formal series of the form (5.2) satisfy
(2.11) and (5.1), then there exist $C^{\infty}$ functions $B,A$ with the same
Taylor development near $\mathcal{GL}$ satisfying (2.11) and (5.1).

We first consider in (5.1) the terms homogeneous of order $\leq 3$. Let
$\Xi=\Xi_{1}+\Xi_{2}+\mathcal{H}_{j\geq 3}$, then
$\Xi^{2}=\Xi_{1}^{2}+2\Xi_{1}\Xi_{2}+\mathcal{H}_{j\geq 4}$ and therefore it
will be enough to have information on $\Xi_{1}$ and $\Xi_{2}$ homogeneous of
order one and two, respectively. From the third equation in (2.11) this means
that is is enough to know $A$ up to second order homogeneity terms. The first
equation in (2.11) will allow to obtain $x$ up to $\mathcal{H}_{j\geq 4}$
terms. Let

(5.3)
$\left\\{\begin{array}[]{l}A=\xi\ell(Y,\omega)+[\alpha(Y,\omega)X+\beta(y,\omega)(\varrho-1)+\gamma(Y,\omega)\xi^{2}]+\mathcal{H}_{j\geq
3},\\\ \frac{\partial
A}{\partial\xi}=\ell(Y,\omega)+2\xi\gamma(Y,\omega)+\mathcal{H}_{j\geq 2},\\\
x=X(1+\ell(Y,\omega))+2\xi\gamma(Y,\omega)+X\mathcal{H}_{j\geq
2}.\end{array}\right.$

Using $xq(\theta)=1-\xi^{2}-|\theta|^{2}$ and that
$q(\theta)=\varrho^{2}q(\omega)$ yields

$X(1+\ell+2\xi\gamma+\mathcal{H}_{j\geq
2})\varrho^{2}q(\omega)=1-\xi^{2}-\varrho^{2}$

which gives the form of $X$ in terms of $(Y,\omega)$, $\xi$ and $\varrho-1$
(of homogeneity $0,1,2$, respectively) as follows

$X=\frac{(1-\xi^{2}-\varrho^{2})}{\varrho^{2}q(\omega)(1+\ell)}\Big{(}1-\frac{2\xi\gamma}{1+\ell}+\mathcal{H}_{j\geq
2}\Big{)}.$

Eventually we get

(5.4)
$X=\frac{-2(\varrho-1)-\xi^{2}}{q(\omega)(1+\ell)}\Big{(}1-\frac{2\xi\gamma}{1+\ell}\Big{)}+\mathcal{H}_{j\geq
4}.$

We replace (5.4) in $\Xi^{2}+R(X,Y,\Theta)=1$ which gives, using also the
third line of (2.11),

$\Xi=\xi+(\xi\ell+\alpha
X+\beta(\varrho-1)+\gamma\xi^{2})+X\alpha=\xi(1+\ell)+(2\alpha
X+\beta(\varrho-1)+\gamma\xi^{2}),$

therefore $\Xi_{1}=\xi(1+\ell)$ and $\Xi_{2}=(2\alpha
X+\beta(\varrho-1)+\gamma\xi^{2})$ and
$\Xi^{2}=\xi^{2}(1+\ell)^{2}+2(1+\ell)\xi(2\alpha
X+\beta(\varrho-1)+\gamma\xi^{2})+\mathcal{H}_{j\geq 4}$ and using
$\Xi^{2}+R(X,Y,\Theta)=1$ yields, modulo $\mathcal{H}_{j\geq 4}$

(5.5) $\xi^{2}(1+\ell)^{2}+2(1+\ell)\xi(2\alpha
X+\beta(\varrho-1)+\gamma\xi^{2})+R_{0}(Y,\Theta)+XR_{1}(Y,\Theta)=1.$

Here we recall from (2.2) and (2.3) that we had set
$R_{0}=R|_{x=0}=\Theta^{2}+O(Y^{2})$ and $R_{1}=\frac{\partial R}{\partial
x}|_{x=0}=R_{1}(Y,\Theta)$ (notice that we have $q(\eta)=R_{1}(0,\eta)$). It
remains to develop $\Theta$ modulo $\mathcal{H}_{j\geq 4}$. From the forth
line of (2.11) we have (see the notations in (5.2)):

$\Theta=\theta+\frac{\partial B}{\partial Y}+X\frac{\partial A}{\partial
Y}=\omega+\frac{\partial B_{0}}{\partial
Y}(Y,\omega)+(\varrho-1)(\omega+\frac{\partial B_{2}}{\partial
Y})+X\xi\frac{\partial\ell}{\partial Y}+\mathcal{H}_{j\geq 4},$

which allows to identify

$\Theta_{0}=\omega+\frac{\partial B_{0}}{\partial
Y}(Y,\omega),\quad\Theta_{1}=0,\quad\Theta_{2}=(\varrho-1)(\omega+\frac{\partial
B_{2}}{\partial Y}),\quad\Theta_{3}=X\xi\frac{\partial\ell}{\partial Y}.$

Now (5.5) becomes, modulo $\mathcal{H}_{j\geq 4}$

(5.6) $\xi^{2}(1+\ell)^{2}+2(1+\ell)\xi(2\alpha
X+\beta(\varrho-1)+\gamma\xi^{2})+R_{0}(Y,\Theta_{0})+\nabla_{\Theta}R_{0}(Y,\Theta_{0})(\Theta_{2}+\Theta_{3})+XR_{1}(Y,\Theta_{0})=1.$

which reads also as follows

(5.7)
$X\Big{[}R_{1}(Y,\Theta_{0})+4\alpha\xi(1+\ell)+\xi\nabla_{\Theta}R_{0}(Y,\Theta_{0})\frac{\partial\ell}{\partial
Y}\Big{]}\\\
=1-R_{0}(Y,\Theta_{0})-\xi^{2}(1+\ell)^{2}-2(1+\ell)\xi(\beta(\varrho-1)+\gamma\xi^{2})-\nabla_{\Theta}R_{0}(Y,\Theta_{0})(\varrho-1)(\omega+\frac{\partial
B_{2}}{\partial Y}).$

It remains to introduce (5.4) in the last equation (5.7) and identify the
terms homogeneous of degree $0$, $1$, $2$ and $3$:

(5.8) $\left\\{\begin{array}[]{l}R_{0}(Y,\omega+\frac{\partial B_{0}}{\partial
Y}(Y,\omega))=1\text{ (order 0)};\\\
\frac{1}{q(\omega)(1+\ell)}=\frac{(1+\ell)^{2}}{R_{1}}\text{ (order 2, terms
in }\xi^{2});\\\ \frac{\nabla_{\Theta}R_{0}}{R_{1}}(\omega+\frac{\partial
B_{2}}{\partial Y})=\frac{2}{(1+\ell)q(\omega)}\text{ (order 2, terms in
}(\varrho-1));\\\
\Big{[}4\alpha(1+\ell)+\nabla_{\Theta}R_{0}\frac{\partial\ell}{\partial
Y}\Big{]}\frac{(1+\ell)^{2}}{R_{1}^{2}}-\frac{2\gamma(1+\ell)}{R_{1}}=\frac{2\gamma}{q(\omega)(1+\ell)^{2}}\text{
(order 3, terms in }\xi^{3});\\\
\Big{[}4\alpha(1+\ell)+\nabla_{\Theta}R_{0}\frac{\partial\ell}{\partial
Y}\Big{]}\frac{\nabla_{\Theta}R_{0}}{R^{2}_{1}}(\omega+\frac{\partial
B_{2}}{\partial
Y})-\frac{2\beta(1+\ell)}{R_{1}}=\frac{4\gamma}{q(\omega)(1+\ell)^{2}}\text{
(order 3, terms in }\xi(\varrho-1)).\end{array}\right.$

######  Remark 5.1.

The system (5.8) consists of five equations, while the unknowns are the
following six functions $B_{0}(Y,\omega)$, $B_{2}(Y,\omega)$,
$\ell(Y,\omega)$, $\alpha(Y,\omega)$, $\beta(Y,\omega)$ and
$\gamma(Y,\omega)$. However, the first three equations in (5.8) involve only
three unknown functions : $B_{0}$, $B_{2}$ and $\ell$ and we start by
determining them.

  * •

Consider the first equation in (5.8):

######  Lemma 5.2.

For $\theta\in\mathbb{R}^{d-1}\setminus 0$, there exists an unique function
$\phi(Y,\theta)$, homogeneous of degree $1$ in $\theta$, solving the eikonal
equation

$R_{0}(Y,\partial_{Y}\phi)=\theta^{2},\quad\phi|_{Y\theta=0}=0.$

Since $R_{0}(Y,\Theta)=\Theta^{2}+O(Y^{2})$, we must have
$\phi(Y,\theta)=Y\theta(1+O(Y^{2}))$. It suffices to take

(5.9) $B_{0}(Y,\omega)=\phi(Y,\omega)-Y\omega.$

As a consequence we have $\nabla^{2}_{Y}B_{0}(0,\omega)=0$,
$\nabla_{Y}B_{0}(0,\omega)=0$ and $B_{0}(0,\omega)=0$.

  * •

Consider the next two equations in (5.8), corresponding to terms homogeneous
of order $2$: the second equation defines the function $\ell$ since we have:

$(1+\ell)^{3}=\frac{R_{1}(Y,\omega+\frac{\partial B_{0}}{\partial
Y})}{q(\omega)},$

and using the form of $B_{0}$ and that $q(\omega)=R_{1}(0,\omega)$ we also
find $\ell(0,\omega)=0$. It then follows that the third equation in (5.8) is a
linear transport equation for $B_{2}(Y,\omega)$ and we can take
$B_{2}|_{Y\omega=0}=0$ (at $Y=0$, the transport field is $2\omega\nabla_{Y}$).

  * •

Consider now the last two equations in (5.8), which form a linear system in
$\alpha,\beta,\gamma$.

(5.10)
$\left\\{\begin{array}[]{l}\frac{\alpha}{q(1+\ell)}-\gamma=-\nabla_{\Theta}R_{0}\frac{\partial\ell}{\partial
Y}\frac{(1+\ell)}{4R_{1}},\\\
\frac{\alpha}{q(1+\ell)}-\frac{\beta}{4}-\frac{\gamma}{2}=-\nabla_{\Theta}R_{0}\frac{\partial\ell}{\partial
Y}\frac{(1+\ell)}{4R_{1}},\end{array}\right.$

which gives $\gamma=2\beta$ and we can take $\alpha=0$. Moreover, at $Y=0$, we
have

$3\nabla_{Y}\ell(0,\omega)=\frac{1}{q(\omega)}\nabla_{Y}R_{1}(0,\omega),$

therefore it follows that

$\nabla_{\Theta}R_{0}\frac{\partial\ell}{\partial
Y}\frac{(1+\ell)}{R_{1}}|_{Y=0}=\frac{1}{3R_{1}^{2}(0,\omega)}\\{R_{0},R_{1}\\}(0,\omega)=\beta(0,\omega).$

We have obtained the following:

######  Proposition 5.3.

The phase function $\Gamma$ has the following form near the glancing set
$\mathcal{G}$:

(5.11)
$\left\\{\begin{array}[]{l}\Gamma(X,Y,\xi,\theta)=B(Y,\omega)+X\Big{[}\xi\ell(Y,\omega)+\gamma(Y,\omega)(\xi^{2}+\theta^{2}-1)+\mathcal{H}_{j\geq
3}\Big{]},\\\
B(Y,\omega)=B_{0}(Y,\omega)+(\varrho-1)B_{2}(Y,\omega)+\mathcal{H}_{j\geq
3},\end{array}\right.$

where $\omega=\theta/|\theta|$, $\varrho=|\theta|$. Also, $B_{0}$ is defined
in (5.9), $B_{2}$ and $\ell$ are described above, while $\gamma$ is given by

$\gamma(Y,\omega)=\nabla_{\Theta}R_{0}\frac{\partial\ell}{\partial
Y}\frac{(1+\ell)}{4R_{1}}.$

Moreover, we have

$\chi_{M}\Big{(}\mathcal{GL}_{M}=\\{x=0,\xi=0,\theta^{2}=1\\}\Big{)}=\mathcal{GL}=\\{X=0,\Xi=0,R_{0}(Y,\Theta)=1\\}.$

The restriction to the glancing regime is given by

$y=Y+\frac{\partial
B}{\partial\theta}(Y,\theta)|_{\varrho=1},\quad\Theta=\theta+\frac{\partial
B_{0}}{\partial Y}(Y,\theta).$

######  Remark 5.4.

The restriction of the canonical transformation $\chi_{M}$ to the glancing set
$\mathcal{GL}_{M}$ preserves the canonical foliation:
$\chi_{M}\Big{(}\\{y=y_{0}+s\theta_{0},\theta=\omega_{0},\omega_{0}^{2}=1\\}\Big{)}$
is an integral curve of $H_{R_{0}}$ on $R_{0}=1$. Be aware that the natural
parametrization given by the symbols of the operators is not preserved.

To end the proof of lemma 2.2, it remains to find in (5.1) the terms
homogeneous of order $>3$. It turns out that this gives a cascade of linear
equations (of the same nature than (5.10) for the determination of
$\alpha,\beta,\gamma$) which can be easily solved by induction. Observe that
the fact the formal expansion is not uniquely defined reflect the fact that
the group of canonical transformation which preserves the model
$\\{x=0,\xi^{2}+|\eta|^{2}+xq(\eta)=1\\}$ is not trivial.

## References

  * [1] K. G. Andersson and R. B. Melrose.  The propagation of singularities along gliding rays.  Invent. Math., 41(3):197–232, 1977.
  * [2] Matthew D. Blair, Hart F. Smith, and Christopher D. Sogge.  Strichartz estimates for the wave equation on manifolds with boundary.  Ann. Inst. H. Poincaré Anal. Non Linéaire, 26(5):1817–1829, 2009.
  * [3] Gregory Eskin.  Parametrix and propagation of singularities for the interior mixed hyperbolic problem.  J. Analyse Math., 32:17–62, 1977.
  * [4] Lars Hörmander.  The analysis of linear partial differential operators. I.  Classics in Mathematics. Springer-Verlag, Berlin, 2003.  Distribution theory and Fourier analysis, Reprint of the second (1990) edition [Springer, Berlin; MR1065993 (91m:35001a)].
  * [5] Oana Ivanovici.  Counterexamples to Strichartz estimates for the wave equation in domains.  Math. Ann., 347(3):627–673, 2010.
  * [6] Oana Ivanovici.  Counterexamples to the Strichartz inequalities for the wave equation in general domains with boundary.  J. Eur. Math. Soc. (JEMS), 14(5):1357–1388, 2012.
  * [7] Oana Ivanovici and Gilles Lebeau.  Dispersion for the wave and Schrödinger equation outside a strictly convex obstacle.  preprint, 2016.
  * [8] Oana Ivanovici, Gilles Lebeau, and Fabrice Planchon.  Dispersion for the wave equation inside strictly convex domains I: the Friedlander model case.  Ann. of Math. (2), 180(1):323–380, 2014.
  * [9] Oana Ivanovici, Gilles Lebeau, and Fabrice Planchon.  Estimations de strichartz pour les ondes dans le modèle de friedlander en dimension 3.  In Séminaire Laurent Schwartz – EDP et applications, 2013–2014, Sémin. Équ. Dériv. Partielles, pages Exp. No. III, 12. École Polytech., Palaiseau, 2014.
  * [10] Oana Ivanovici, Gilles Lebeau, and Fabrice Planchon.  Strichartz estimates for the wave equation on a 2d model convex domain.  preprint, 2016.
  * [11] V. Ja. Ivriĭ.  Propagation of the singularities of a solution of the wave equation near the boundary.  Dokl. Akad. Nauk SSSR, 239(4):772–774, 1978.
  * [12] R. B. Melrose.  Equivalence of glancing hypersurfaces.  Invent. Math., 37(3):165–191, 1976.
  * [13] R. B. Melrose and J. Sjöstrand.  Singularities of boundary value problems. I.  Comm. Pure Appl. Math., 31(5):593–617, 1978.
  * [14] R. B. Melrose and J. Sjöstrand.  Singularities of boundary value problems. II.  Comm. Pure Appl. Math., 35(2):129–168, 1982.
  * [15] Richard B. Melrose and Michael E. Taylor.  Boundary Problems for Wave Equations With Grazing and Gliding Rays.  Available at http://www.unc.edu/math/Faculty/met/wavep.html.
  * [16] Richard B. Melrose and Michael E. Taylor.  Near peak scattering and the corrected Kirchhoff approximation for a convex obstacle.  Adv. in Math., 55(3):242–315, 1985.
  * [17] Richard B. Melrose and Michael E. Taylor.  The radiation pattern of a diffracted wave near the shadow boundary.  Comm. Partial Differential Equations, 11(6):599–672, 1986.
  * [18] Hart F. Smith and Christopher D. Sogge.  On the critical semilinear wave equation outside convex obstacles.  J. Amer. Math. Soc., 8(4):879–916, 1995.
  * [19] Hart F. Smith and Christopher D. Sogge.  On the $L^{p}$ norm of spectral clusters for compact manifolds with boundary.  Acta Math., 198(1):107–153, 2007.
  * [20] Daniel Tataru.  Strichartz estimates for second order hyperbolic operators with nonsmooth coefficients. III.  J. Amer. Math. Soc., 15(2):419–442 (electronic), 2002.
  * [21] Maciej Zworski.  High frequency scattering by a convex obstacle.  Duke Math. J., 61(2):545–634, 1990.


# Sequential Short-Text Classification with
Recurrent and Convolutional Neural Networks

Ji Young Lee
MIT
<EMAIL_ADDRESS>&Franck Dernoncourt11footnotemark: 1
MIT
<EMAIL_ADDRESS>These authors contributed equally to this work.

(Dec 14, 2015)

###### Abstract

Recent approaches based on artificial neural networks (ANNs) have shown
promising results for short-text classification. However, many short texts
occur in sequences (e.g., sentences in a document or utterances in a dialog),
and most existing ANN-based systems do not leverage the preceding short texts
when classifying a subsequent one. In this work, we present a model based on
recurrent neural networks and convolutional neural networks that incorporates
the preceding short texts. Our model achieves state-of-the-art results on
three different datasets for dialog act prediction.

Accepted as a conference paper at NAACL 2016

##  1 Introduction

Short-text classification is an important task in many areas of natural
language processing, including sentiment analysis, question answering, or
dialog management. Many different approaches have been developed for short-
text classification, such as using Support Vector Machines (SVMs) with rule-
based features [Silva et al., 2011], combining SVMs with naive Bayes [Wang and
Manning, 2012], and building dependency trees with Conditional Random Fields
[Nakagawa et al., 2010]. Several recent studies using ANNs have shown
promising results, including convolutional neural networks (CNNs) [Kim, 2014,
Blunsom et al., 2014, Kalchbrenner et al., 2014] and recursive neural networks
[Socher et al., 2012].

Most ANN systems classify short texts in isolation, i.e., without considering
preceding short texts. However, short texts usually appear in sequence (e.g.,
sentences in a document or utterances in a dialog), therefore using
information from preceding short texts may improve the classification
accuracy. Previous works on sequential short-text classification are mostly
based on non-ANN approaches, such as Hidden Markov Models (HMMs) [Reithinger
and Klesen, 1997], [Stolcke et al., 2000], maximum entropy [Ang et al., 2005],
and naive Bayes [Lendvai and Geertzen, 2007].

Figure 1: RNN (left) and CNN (right) architectures for generating the vector
representation $\mathbf{s}$ of a short text $\mathbf{x}_{1:\ell}$. For CNN,
Conv refers to convolution operations, and the filter height $h=3$ is used in
this figure.

Inspired by the performance of ANN-based systems for non-sequential short-text
classification, we introduce a model based on recurrent neural networks (RNNs)
and CNNs for sequential short-text classification, and evaluate it on the
dialog act classification task. A dialog act characterizes an utterance in a
dialog based on a combination of pragmatic, semantic, and syntactic criteria.
Its accurate detection is useful for a range of applications, from speech
recognition to automatic summarization [Stolcke et al., 2000]. Our model
achieves state-of-the-art results on three different datasets.

##  2 Model

Our model comprises two parts. The first part generates a vector
representation for each short text using either the RNN or CNN architecture,
as discussed in Section 2.1 and Figure 1. The second part classifies the
current short text based on the vector representations of the current as well
as a few preceding short texts, as presented in Section 2.2 and Figure 2.

Figure 2: Four instances of the two-layer feedforward ANN used for predicting
the probability distribution over the classes $\mathbf{z}_{i}$ for the
$i^{th}$ short-text $\mathbf{X}_{i}$. S2V stands for short text to vector,
which is the RNN/CNN architecture that generates $\mathbf{s}_{i}$ from
$\mathbf{X}_{i}$. From left to right, the history sizes $(d_{1},d_{2})$ are
$(0,0),(2,0),(0,2)$ and $(1,1)$. $(0,0)$ corresponds to the non-sequential
classification case.

We denote scalars with italic lowercases (e.g., $k,\,b_{f}$), vectors with
bold lowercases (e.g., $\mathbf{s},\,\mathbf{x}_{i}$), and matrices with
italic uppercases (e.g., $W_{f}$). We use the colon notation
$\mathbf{v}_{i:j}$ to denote the sequence of vectors
$(\mathbf{v}_{i},\mathbf{v}_{i+1},\dotsc,\mathbf{v}_{j})$.

###  2.1 Short-text representation

A given short text of length $\ell$ is represented as the sequence of
$m$-dimensional word vectors $\mathbf{x}_{1:\ell},$ which is used by the RNN
or CNN model to produce the $n$-dimensional short-text representation
$\mathbf{s}.$

####  2.1.1 RNN-based short-text representation

We use a variant of RNN called Long Short Term Memory (LSTM) [Hochreiter and
Schmidhuber, 1997]. For the $t^{th}$ word in the short-text, an LSTM takes as
input $\mathbf{x}_{t},\mathbf{h}_{t-1},\mathbf{c}_{t-1}$ and produces
$\mathbf{h}_{t},\mathbf{c}_{t}$ based on the following formulas:

$\displaystyle\mathbf{i}_{t}$
$\displaystyle=\sigma(W_{i}\mathbf{x}_{t}+U_{i}\mathbf{h}_{t-1}+\mathbf{b}_{i})$
$\displaystyle\mathbf{f}_{t}$
$\displaystyle=\sigma(W_{f}\mathbf{x}_{t}+U_{f}\mathbf{h}_{t-1}+\mathbf{b}_{f})$
$\displaystyle\tilde{\mathbf{c}}_{t}$
$\displaystyle=\text{tanh}(W_{c}\mathbf{x}_{t}+U_{c}\mathbf{h}_{t-1}+\mathbf{b}_{c})$
$\displaystyle\mathbf{c}_{t}$
$\displaystyle=\mathbf{f}_{t}\odot\mathbf{c}_{t-1}+\mathbf{i}_{t}\odot\tilde{\mathbf{c}}_{t}$
$\displaystyle\mathbf{o}_{t}$
$\displaystyle=\sigma(W_{o}\mathbf{x}_{t}+U_{o}\mathbf{h}_{t-1}+\mathbf{b}_{o})$
$\displaystyle\mathbf{h}_{t}$
$\displaystyle=\mathbf{o}_{t}\odot\text{tanh}(\mathbf{c}_{t})$

where $W_{j}\in\mathbb{R}^{n\times m},\;U_{j}\in\mathbb{R}^{n\times n}$ are
weight matrices and $\mathbf{b}_{j}\in\mathbb{R}^{n}$ are bias vectors, for
$j\in\\{i,f,c,o\\}$. The symbols $\sigma(\cdot)$ and tanh$(\cdot)$ refer to
the element-wise sigmoid and hyperbolic tangent functions, and $\odot$ is the
element-wise multiplication. $\mathbf{h}_{0}=\mathbf{c}_{0}=\mathbf{0}$.

In the pooling layer, the sequence of vectors $\mathbf{h}_{1:\ell}$ output
from the RNN layer are combined into a single vector
$\mathbf{s}\in\mathbb{R}^{n}$ that represents the short-text, using one of the
following mechanisms: last, mean, and max pooling. Last pooling takes the last
vector, i.e., $\mathbf{s}=\mathbf{h}_{\ell},$ mean pooling averages all
vectors, i.e., $\mathbf{s}=\tfrac{1}{\ell}\sum_{t=1}^{\ell}\mathbf{h}_{t},$
and max pooling takes the element-wise maximum of $\mathbf{h}_{1:\ell}.$

####  2.1.2 CNN-based short-text representation

Using a _filter_ $W_{f}\in\mathbb{R}^{h\times m}$ of height $h,$ a convolution
operation on $h$ consecutive word vectors starting from $t^{th}$ word outputs
the scalar _feature_

$\displaystyle c_{t}=\text{ReLU}(W_{f}\tiny{\,\bullet\,}X_{t:t+h-1}+b_{f})$

where $X_{t:t+h-1}\in\mathbb{R}^{h\times m}$ is the matrix whose $i^{th}$ row
is $\mathbf{x}_{i}\in\mathbb{R}^{m},$ and $b_{f}\in\mathbb{R}$ is a bias. The
symbol $\bullet$ refers to the dot product and ReLU$(\cdot)$ is the element-
wise rectified linear unit function.

We perform convolution operations with $n$ different filters, and denote the
resulting features as $\mathbf{c}_{t}\in\mathbb{R}^{n},$ each of whose
dimensions comes from a distinct filter. Repeating the convolution operations
for each window of $h$ consecutive words in the short-text, we obtain
$\mathbf{c}_{1:\ell-h+1}.$ The short-text representation
$\mathbf{s}\in\mathbb{R}^{n}$ is computed in the max pooling layer, as the
element-wise maximum of $\mathbf{c}_{1:\ell-h+1}.$

###  2.2 Sequential short-text classification

Let $\mathbf{s}_{i}$ be the $n$-dimensional short-text representation given by
the RNN or CNN architecture for the $i^{th}$ short text in the sequence. The
sequence $\mathbf{s}_{i-d_{1}-d_{2}\,:\,i}$ is fed into a two-layer
feedforward ANN that predicts the class for the $i^{th}$ short text. The
hyperparameters $d_{1},d_{2}$ are the history sizes used in the first and
second layers, respectively.

The first layer takes as input $\mathbf{s}_{i-d_{1}-d_{2}\,:\,i}$ and outputs
the sequence $\mathbf{y}_{i-d_{2}\,:\,i}$ defined as

$\displaystyle\mathbf{y}_{j}=\text{tanh}\left(\sum_{d=0}^{d_{1}}W_{-d}\,\mathbf{s}_{j-d}+\mathbf{b}_{1}\right),\;\forall
j\in[i-d_{2},i]$

where $W_{0},W_{-1},W_{-2}\in\mathbb{R}^{k\times n}$ are the weight matrices,
$\mathbf{b}_{1}\in\mathbb{R}^{k}$ is the bias vector,
$\mathbf{y}_{j}\in\mathbb{R}^{k}$ is the class representation, and $k$ is the
number of classes for the classification task.

Similarly, the second layer takes as input the sequence of class
representations $\mathbf{y}_{i-d_{2}:i}$ and outputs
$\mathbf{z}_{i}\in\mathbb{R}^{k}$:

$\displaystyle\mathbf{z}_{i}=\text{softmax}\left(\sum_{j=0}^{d_{2}}W_{-j}\,\mathbf{y}_{i-j}+\mathbf{b}_{2}\right)$

where $U_{0},U_{-1},U_{-2}\in\mathbb{R}^{k\times k}$ and
$\mathbf{b}_{2}\in\mathbb{R}^{k}$ are the weight matrices and bias vector.

The final output $\mathbf{z}_{i}$ represents the probability distribution over
the set of $k$ classes for the $i^{th}$ short-text: the $j^{th}$ element of
$\mathbf{z}_{i}$ corresponds to the probability that the $i^{th}$ short-text
belongs to the $j^{th}$ class.

##  3 Datasets and Experimental Setup

###  3.1 Datasets

We evaluate our model on the dialog act classification task using the
following datasets:

  * •

DSTC 4: Dialog State Tracking Challenge 4 [Kim et al., 2015, Kim et al.,
2016].

  * •

MRDA: ICSI Meeting Recorder Dialog Act Corpus [Janin et al., 2003, Shriberg et
al., 2004]. The 5 classes are introduced in [Ang et al., 2005].

  * •

SwDA: Switchboard Dialog Act Corpus [Jurafsky et al., 1997].

For MRDA, we use the train/validation/test splits provided with the datasets.
For DSTC 4 and SwDA, only the train/test splits are provided.111All
train/validation/test splits can be found at https://github.com/Franck-
Dernoncourt/naacl2016 Table 1 presents statistics on the datasets.

Dataset | $|C|$ | $|V|$ | Train | Validation | Test
---|---|---|---|---|---
DSTC 4 | 89 | 6k | 24 (21k) | 5 (5k) | 6 (6k)
MRDA | 5 | 12k | 51 (78k) | 11 (16k) | 11 (15k)
SwDA | 43 | 20k | 1003 (193k) | 112 (23k) | 19 (5k)
Table 1: Dataset overview. $|C|$ is the number of classes, $|V|$ the
vocabulary size. For the train, validation and test sets, we indicate the
number of dialogs (i.e., sequences) followed by the number of utterances
(i.e., short texts) in parenthesis.

###  3.2 Training

The model is trained to minimize the negative log-likelihood of predicting the
correct dialog acts of the utterances in the train set, using stochastic
gradient descent with the Adadelta update rule [Zeiler, 2012]. At each
gradient descent step, weight matrices, bias vectors, and word vectors are
updated. For regularization, dropout is applied after the pooling layer, and
early stopping is used on the validation set with a patience of 10 epochs.

##  4 Results and Discussion

To find effective hyperparameters, we varied one hyperparameter at a time
while keeping the other ones fixed. Table 2 presents our hyperparameter
choices.

Hyperparameter | Choice | Experiment Range
---|---|---
LSTM output dim. $(n)$ | 100 | 50 – 1000
LSTM pooling | max | max, mean, last
LSTM direction | unidir. | unidir., bidir.
CNN num. of filters $(n)$ | 500 | 50 – 1000
CNN filter height $(h)$ | 3 | 1 – 10
Dropout rate | 0.5 | 0 – 1
Word vector dim. $(m)$ | 200, 300 | 25 – 300
Table 2: Experiments ranges and choices of hyperparameters. Unidir refers to the regular RNNs presented in Section 2.1.1, and bidir refers to bidirectional RNNs introduced in [Schuster and Paliwal, 1997]. $d_{1}$ $d_{2}$ | LSTM | CNN
---|---|---
| 0 | 1 | 2 | 0 | 1 | 2
DSTC4 |  0 | 63.1 (62.4, 63.6) | 65.7 (65.6, 65.7) | 64.7 (63.9, 65.3) | 64.1 (63.5, 65.2) | 65.4 (64.7, 66.6) | 65.1 (63.2, 65.9)
|  1 |  65.8 (65.5, 66.1) | 65.7 (65.3, 66.1) | 64.8 (64.6, 65.1) | 65.3 (64.1, 65.9) | 65.1 (62.1, 66.2) | 64.9 (64.4, 65.6)
|  2 | 65.7 (65.0, 66.2) | 65.5 (64.4, 66.1) | 64.9 (64.6, 65.2) | 65.7 (64.9, 66.3) |  65.8 (65.2, 66.1) | 65.4 (64.5, 66.0)
MRDA |  0 | 82.8 (82.4, 83.1) | 83.2 (82.9, 83.4) | 82.9 (82.4, 83.4) | 83.2 (83.0, 83.4) | 83.5 (82.9, 84.0) | 83.8 (83.4, 84.2)
|  1 | 83.2 (82.6, 83.7) | 83.8 (83.5, 84.4) | 83.6 (83.2, 83.8) |  84.6 (84.5, 84.9) |  84.6 (84.4, 84.8) | 84.1 (83.8, 84.4)
|  2 |  84.1 (83.5, 84.4) | 83.9 (83.4, 84.7) | 83.3 (82.6, 84.2) | 84.4 (84.1, 84.8) |  84.6 (84.5, 84.7) | 84.4 (84.2, 84.7)
SwDA |  0 | 66.3 (65.1, 68.0) | 67.9 (66.3, 68.6) | 67.8 (66.7, 69.0) | 67.0 (65.3, 68.7) | 69.1 (68.5, 70.0) | 69.7 (69.2, 70.9)
|  1 | 68.4 (67.8, 68.8) | 67.8 (65.5, 68.9) | 67.3 (65.5, 69.5) | 69.9 (69.1, 70.9) | 69.8 (69.3, 70.6) | 69.9 (68.8, 70.6)
|  2 |  69.5 (68.9, 70.2) | 67.9 (66.5, 69.4) | 67.7 (66.9, 68.9) |  71.4 (70.4, 73.1) | 71.1 (70.2, 72.1) | 70.9 (69.7, 71.7)
Table 3: Accuracy (%) on different architectures and history sizes
$d_{1},d_{2}$. For each setting, we report average (minimum, maximum) computed
on 5 runs. Sequential classification ($d_{1}+d_{2}>0$) outperforms non-
sequential classification ($d_{1}=d_{2}=0$). Overall, the CNN model
outperformed the LSTM model for all datasets, albeit by a small margin except
for SwDA. We also tried a variant of the LSTM model, gated recurrent units
[Cho et al., 2014], but the results were generally lower than LSTM.

We initialized the word vectors with the 300-dimensional word vectors
pretrained with word2vec on Google News [Mikolov et al., 2013a, Mikolov et
al., 2013b] for DSTC 4, and the 200-dimensional word vectors pretrained with
GloVe on Twitter [Pennington et al., 2014] for MRDA and SwDA, as these choices
yielded the best results among all publicly available word2vec, GloVe, SENNA
[Collobert, 2011, Collobert et al., 2011] and RNNLM [Mikolov et al., 2011]
word vectors.

The effects of the history sizes $d_{1}$ and $d_{2}$ for the short-text and
the class representations, respectively, are presented in Table 3 for both the
LSTM and CNN models. In both models, increasing $d_{1}$ while keeping
$d_{2}=0$ improved the performances by 1.3-4.2 percentage points. Conversely,
increasing $d_{2}$ while keeping $d_{1}=0$ yielded better results, but the
performance increase was less pronounced: incorporating sequential information
at the short-text representation level was more effective than at the class
representation level.

Using sequential information at both the short-text representation level and
the class representation level does not help in most cases and may even lower
the performances. We hypothesize that short-text representations contain
richer and more general information than class representations due to their
larger dimension. Class representations may not convey any additional
information over short-text representations, and are more likely to propagate
errors from previous misclassifications.

Table 4 compares our results with the state-of-the-art. Overall, our model
shows competitive results, while requiring no human-engineered features.
Rigorous comparisons are challenging to draw, as many important details such
as text preprocessing and train/valid/test split may vary, and many studies
fail to perform several runs despite the randomness in some parts of the
training process, such as weight initialization.

Model | DSTC 4 | MRDA | SwDA
---|---|---|---
CNN | 65.5 | 84.6 | 73.1
LSTM | 66.2 | 84.3 | 69.6
Majority class | 25.8 | 59.1 | 33.7
SVM | 57.0 | – | –
Graphical model | – | 81.3 | –
Naive Bayes | – | 82.0 | –
HMM | – | – | 71.0
Memory-based Learning | – | – | 72.3
Interlabeler agreement | – | – | 84.0
Table 4: Accuracy (%) of our models and other methods from the literature. The
majority class model predicts the most frequent class. SVM: [Dernoncourt et
al., 2016]. Graphical model: [Ji and Bilmes, 2006]. Naive Bayes: [Lendvai and
Geertzen, 2007]. HMM: [Stolcke et al., 2000]. Memory-based Learning: [Rotaru,
2002]. All five models use features derived from transcribed words, as well as
previous predicted dialog acts except for Naive Bayes. The interlabeler
agreement could be obtained only for SwDA. For the CNN and LSTM models, the
presented results are the test set accuracy of the run with the highest
accuracy on the validation set.

##  5 Conclusion

In this article we have presented an ANN-based approach to sequential short-
text classification. We demonstrate that adding sequential information
improves the quality of the predictions, and the performance depends on what
sequential information is used in the model. Our model achieves state-of-the-
art results on three different datasets for dialog act prediction.

## References

  * [Ang et al., 2005]  Jeremy Ang, Yang Liu, and Elizabeth Shriberg.  2005\.  Automatic dialog act segmentation and classification in multiparty meetings.  In ICASSP (1), pages 1061–1064.
  * [Blunsom et al., 2014]  Phil Blunsom, Edward Grefenstette, Nal Kalchbrenner, et al.  2014\.  A convolutional neural network for modelling sentences.  In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.
  * [Cho et al., 2014]  Kyunghyun Cho, Bart van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio.  2014\.  On the properties of neural machine translation: Encoder-decoder approaches.  arXiv preprint arXiv:1409.1259.
  * [Collobert et al., 2011]  Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa.  2011\.  Natural language processing (almost) from scratch.  The Journal of Machine Learning Research, 12:2493–2537.
  * [Collobert, 2011]  Ronan Collobert.  2011\.  Deep learning for efficient discriminative parsing.  In International Conference on Artificial Intelligence and Statistics, number EPFL-CONF-192374.
  * [Dernoncourt et al., 2016]  Franck Dernoncourt, Ji Young Lee, Trung H. Bui, and Hung H. Bui.  2016\.  Adobe-MIT submission to the DSTC 4 Spoken Language Understanding pilot task.  In 7th International Workshop on Spoken Dialogue Systems (IWSDS).
  * [Hochreiter and Schmidhuber, 1997]  Sepp Hochreiter and Jürgen Schmidhuber.  1997\.  Long short-term memory.  Neural computation, 9(8):1735–1780.
  * [Janin et al., 2003]  Adam Janin, Don Baron, Jane Edwards, Dan Ellis, David Gelbart, Nelson Morgan, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, et al.  2003\.  The ICSI meeting corpus.  In Acoustics, Speech, and Signal Processing, 2003. Proceedings.(ICASSP’03). 2003 IEEE International Conference on, volume 1, pages I–364. IEEE.
  * [Ji and Bilmes, 2006]  Gang Ji and Jeff Bilmes.  2006\.  Backoff model training using partially observed data: application to dialog act tagging.  In Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pages 280–287. Association for Computational Linguistics.
  * [Jurafsky et al., 1997]  Dan Jurafsky, Elizabeth Shriberg, and Debra Biasca.  1997\.  Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual.  Institute of Cognitive Science Technical Report, pages 97–102.
  * [Kalchbrenner et al., 2014]  Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom.  2014\.  A convolutional neural network for modelling sentences.  arXiv preprint arXiv:1404.2188.
  * [Kim et al., 2015]  Seokhwan Kim, Luis Fernando D’Haro, Rafael E. Banchs, Jason Williams, and Matthew Henderson.  2015\.  Dialog State Tracking Challenge 4: Handbook.
  * [Kim et al., 2016]  Seokhwan Kim, Luis Fernando D’Haro, Rafael E. Banchs, Jason Williams, and Matthew Henderson.  2016\.  The Fourth Dialog State Tracking Challenge.  In Proceedings of the 7th International Workshop on Spoken Dialogue Systems (IWSDS).
  * [Kim, 2014]  Yoon Kim.  2014\.  Convolutional neural networks for sentence classification.  In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1746–1751. Association for Computational Linguistics.
  * [Lendvai and Geertzen, 2007]  Piroska Lendvai and Jeroen Geertzen.  2007\.  Token-based chunking of turn-internal dialogue act sequences.  In Proceedings of the 8th SIGDIAL Workshop on Discourse and Dialogue, pages 174–181.
  * [Mikolov et al., 2011]  Tomas Mikolov, Stefan Kombrink, Anoop Deoras, Lukar Burget, and Jan Cernocky.  2011\.  Rnnlm-recurrent neural network language modeling toolkit.  In Proc. of the 2011 ASRU Workshop, pages 196–201.
  * [Mikolov et al., 2013a]  Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  2013a.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781.
  * [Mikolov et al., 2013b]  Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean.  2013b.  Distributed representations of words and phrases and their compositionality.  In Advances in neural information processing systems, pages 3111–3119.
  * [Nakagawa et al., 2010]  Tetsuji Nakagawa, Kentaro Inui, and Sadao Kurohashi.  2010\.  Dependency tree-based sentiment classification using CRFs with hidden variables.  In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 786–794. Association for Computational Linguistics.
  * [Pennington et al., 2014]  Jeffrey Pennington, Richard Socher, and Christopher D Manning.  2014\.  GloVe: global vectors for word representation.  Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), 12:1532–1543.
  * [Reithinger and Klesen, 1997]  Norbert Reithinger and Martin Klesen.  1997\.  Dialogue act classification using language models.  In EuroSpeech. Citeseer.
  * [Rotaru, 2002]  Mihai Rotaru.  2002\.  Dialog act tagging using memory-based learning.  Term project, University of Pittsburgh, pages 255–276.
  * [Schuster and Paliwal, 1997]  Mike Schuster and Kuldip K Paliwal.  1997\.  Bidirectional recurrent neural networks.  Signal Processing, IEEE Transactions on, 45(11):2673–2681.
  * [Shriberg et al., 2004]  Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey.  2004\.  The ICSI meeting recorder dialog act (MRDA) corpus.  Technical report, DTIC Document.
  * [Silva et al., 2011]  Joao Silva, Luísa Coheur, Ana Cristina Mendes, and Andreas Wichert.  2011\.  From symbolic to sub-symbolic information in question classification.  Artificial Intelligence Review, 35(2):137–154.
  * [Socher et al., 2012]  Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng.  2012\.  Semantic compositionality through recursive matrix-vector spaces.  In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1201–1211. Association for Computational Linguistics.
  * [Stolcke et al., 2000]  Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer.  2000\.  Dialogue act modeling for automatic tagging and recognition of conversational speech.  Computational linguistics, 26(3):339–373.
  * [Wang and Manning, 2012]  Sida Wang and Christopher D Manning.  2012\.  Baselines and bigrams: Simple, good sentiment and topic classification.  In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers-Volume 2, pages 90–94. Association for Computational Linguistics.
  * [Zeiler, 2012]  Matthew D Zeiler.  2012\.  Adadelta: An adaptive learning rate method.  arXiv preprint arXiv:1212.5701.