$\displaystyle=\frac{1}{C}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))\left(\int_{\mathbb{R}^{m}}\mu_{k-t}(-\bm{\mathbf{w}})\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}-\int_{\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})}\mu_{k-t}(-\bm{\mathbf{w}})\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}\right).$
(33)

For $\bm{\mathbf{w}}\in\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})$, we have $\lVert\bm{\mathbf{w}}\rVert\geq
k/\delta^{\prime}$. This implies (as a weak bound)

$\displaystyle\mu_{k-t}(-\bm{\mathbf{w}})$
$\displaystyle=\frac{1}{\left(\sqrt{2\pi(k-t)}\right)^{m}}\exp\left(-\lVert\bm{\mathbf{w}}\rVert_{2}^{2}/(2(k-t))\right)$
$\displaystyle\leq\exp(-k/(2(\delta^{\prime})^{2})=\exp(-m^{1+\Omega(1)})=o(p).$

Since $\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))<1$ for
$\delta^{\prime}=o(1)$, it follows that

$\displaystyle\frac{1}{C}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))\int_{\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})}\mu_{k-t}(-\bm{\mathbf{w}})\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}\leq
o(p)\int_{\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})}\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}\leq
o(p).$

Therefore, by adding to (33) and dividing by $1+o(1)$,

$\displaystyle p$
$\displaystyle\geq\frac{1}{2C}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))\int_{\mathbb{R}^{m}}\mu_{k-t}(-\bm{\mathbf{w}})\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}$
$\displaystyle=\frac{1}{2C}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))\frac{1}{\left(\sqrt{2\pi(k-t)}\right)^{m}\left(\sqrt{2\pi
t}\right)^{m}}\int_{\mathbb{R}^{m}}\exp\left(\frac{-\lVert\bm{\mathbf{w}}\rVert^{2}}{2(k-t)}-\frac{\lVert\bm{\mathbf{w}}\rVert^{2}}{2t}\right)d\bm{\mathbf{w}}$
$\displaystyle=\frac{1}{2C}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))\frac{1}{\left(\sqrt{2\pi(k-t)}\right)^{m}\left(\sqrt{2\pi
t}\right)^{m}}\int_{\mathbb{R}^{m}}\exp\left(-\frac{k}{2t(k-t)}\lVert\bm{\mathbf{w}}\rVert^{2}\right)d\bm{\mathbf{w}}$
$\displaystyle=\frac{1}{2C}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))(2\pi
k)^{-m/2}.$

Using this lower bound on $p$, we can use (28) to see that

$Y_{0}\leq
C^{2}\operatorname*{Vol}(B_{\delta^{\prime}}(\mathbf{0}))^{2}\left((2\pi)^{2}k^{2}\right)^{-m/2}\exp\left(t^{2}m/k^{2}\right)\leq
C_{1}p^{2}\exp\left(t^{2}m/k^{2}\right),$

for some constant $C_{1}=O(1)$.

(Subcase 2) Bounding $Y_{1}$: For $\bm{\mathbf{w}}\in\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})$, we have
$\lVert\bm{\mathbf{w}}\rVert_{2}\geq k/\delta^{\prime}$. For any
$\bm{\mathbf{x}}\in B_{\delta^{\prime}}(-\bm{\mathbf{w}})$, it follows by the
triangle inequality that

$\lVert\bm{\mathbf{x}}\rVert\geq\lVert-\bm{\mathbf{w}}\rVert-\delta^{\prime}\geq
k/\delta^{\prime}-\delta^{\prime}\geq k/(2\delta^{\prime}),$

since $\delta^{\prime}=o(1)$. Therefore, since $k-t\leq k$, for any
$\bm{\mathbf{x}}\in B_{\delta^{\prime}}(-\bm{\mathbf{w}})$, we have

$\displaystyle\mu_{k-t}(\bm{\mathbf{x}})$
$\displaystyle=\frac{1}{\left(\sqrt{2\pi(k-t)}\right)^{m}}\exp\left(-\lVert\bm{\mathbf{x}}\rVert_{2}^{2}/(2(k-t))\right)$
$\displaystyle\leq\frac{1}{\left(\sqrt{2\pi(k-t)}\right)^{m}}\exp\left(-k/(8(\delta^{\prime})^{2})\right)$
$\displaystyle\leq\exp\left(-m^{1+\Omega(1)}\right).$

As a weak bound, since
$\operatorname*{Vol}(B_{\delta^{\prime}}(-\bm{\mathbf{w}}))\leq 1$ (as
$\delta^{\prime}=o(1)$), this implies for all $\bm{\mathbf{x}}\in
B_{\delta^{\prime}}(-\bm{\mathbf{w}})$,

$\left(\int_{B_{\delta^{\prime}}(-\bm{\mathbf{w}})}\mu_{k-t}(\bm{\mathbf{x}})d\bm{\mathbf{x}}\right)^{2}\leq\exp\left(-m^{1+\Omega(1)}\right),$

and therefore,

$\displaystyle Y_{1}$ $\displaystyle=\int_{\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})}\left(\int_{B_{\delta^{\prime}}(-\bm{\mathbf{w}})}\mu_{k-t}(\bm{\mathbf{x}})d\bm{\mathbf{x}}\right)^{2}\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}$
$\displaystyle\leq\exp\left(-m^{1+\Omega(1)}\right)\int_{\mathbb{R}^{m}\setminus
B_{k/\delta^{\prime}}(\mathbf{0})}\mu_{t}(\bm{\mathbf{w}})d\bm{\mathbf{w}}\leq\exp\left(-m^{1+\Omega(1)}\right).$

Since $p^{2}=\exp\left(-\Theta(m\log
m)\right)=\omega\left(\exp\left(-m^{1+\Omega(1)}\right)\right)$, it follows
that

$Y_{1}=o(p^{2}).$

Combining the bounds on $Y_{0}$ and $Y_{1}$ via Eq. (21), there is a constant
$C_{2}=O(1)$ for which

$\operatorname*{Cov}(X_{\bm{\mathbf{u}}},X_{\bm{\mathbf{v}}})\leq
Y_{0}+Y_{1}\leq C_{1}p^{2}\exp\left(t^{2}m/k^{2}\right)+o(p^{2})\leq
C_{2}p^{2}\exp\left(t^{2}m/k^{2}\right).$

Using the definition of $Z_{1}$, (18), and $t\leq k/4$, we have

$\displaystyle
Z_{1}=\sum_{t=1}^{k/4}\sum_{\begin{subarray}{c}\bm{\mathbf{u}},\bm{\mathbf{v}}\in\mathcal{S}_{n,k}\\\
I(\bm{\mathbf{u}},\bm{\mathbf{v}})=t\end{subarray}}\operatorname*{Cov}(X_{\bm{\mathbf{u}}},X_{\bm{\mathbf{v}}})$
$\displaystyle\leq\sum_{t=1}^{k/4}\sum_{\begin{subarray}{c}\bm{\mathbf{u}},\bm{\mathbf{v}}\in\mathcal{S}_{n,k}\\\
I(\bm{\mathbf{u}},\bm{\mathbf{v}})=t\end{subarray}}C_{2}p^{2}\exp\left(t^{2}m/k^{2}\right)$
$\displaystyle\leq
C_{2}p^{2}\sum_{t=1}^{k/4}\binom{k}{t}\left(\frac{n}{k}\right)^{2k-t}\exp\left(t^{2}m/k^{2}\right)$
$\displaystyle\leq
C_{2}p^{2}\left(\frac{n}{k}\right)^{2k}\sum_{t=1}^{k/4}\binom{k}{t}\left(\frac{n}{k}\right)^{-t}\exp\left(tm/(4k)\right)$
$\displaystyle\leq
C_{2}p^{2}\left(\frac{n}{k}\right)^{2k}\sum_{t=1}^{k/4}\left(\frac{k^{2}\exp(m/(4k))}{n}\right)^{t}$
$\displaystyle=C_{2}\left(\frac{n}{k}\right)^{1.8k}\sum_{t=1}^{k/4}\left(\frac{k^{2}\exp(m/(4k))}{n}\right)^{t}.$

Since this geometric sum starts at index $t=1$, to show
$Z_{1}=o((n/k)^{1.8k})$, it suffices to show that the base,
$\frac{k^{2}\exp(m/(4k))}{n}$, is $o(1)$. To see this, recall that

$n=k\cdot\exp\left(\Theta(m/k\cdot\log m)\right),$

and $k=m^{\epsilon}$ for $\epsilon\in(0,1)$, making

$\displaystyle\frac{k^{2}\exp(m/(4k))}{n}$
$\displaystyle=\frac{k\exp(m/(4k))}{\exp\left(\Theta(m/k\cdot\log m)\right)}$
$\displaystyle\leq m\exp\left(-\Theta(m/k\cdot\log m)\right)$
$\displaystyle\leq m\exp\left(-m^{\Omega(1)}\log m\right)$
$\displaystyle=o(1),$

as desired.

Bounding $Z_{2}$: For this case when $t>k/4$, we can use the (very) weak bound
that

$\operatorname*{Cov}(X_{\bm{\mathbf{u}}},X_{\bm{\mathbf{v}}})=\operatorname*{\mathbb{E}}[X_{\bm{\mathbf{u}}}X_{\bm{\mathbf{v}}}]-\operatorname*{\mathbb{E}}[X_{\bm{\mathbf{u}}}]\operatorname*{\mathbb{E}}[X_{\bm{\mathbf{v}}}]\leq\operatorname*{\mathbb{E}}[X_{\bm{\mathbf{u}}}X_{\bm{\mathbf{v}}}]\leq
1.$

Using (18), we get

$\displaystyle
Z_{2}=\sum_{t>k/4}\sum_{\begin{subarray}{c}\bm{\mathbf{u}},\bm{\mathbf{v}}\in\mathcal{S}_{n,k}\\\
I(\bm{\mathbf{u}},\bm{\mathbf{v}})=t\end{subarray}}\operatorname*{Cov}(X_{\bm{\mathbf{u}}},X_{\bm{\mathbf{v}}})$
$\displaystyle\leq\sum_{t>k/4}\binom{k}{t}\left(\frac{n}{k}\right)^{2k-t}$
$\displaystyle\leq\sum_{t>k/4}\binom{k}{t}\left(\frac{n}{k}\right)^{1.75k}$
$\displaystyle\leq\left(\frac{n}{k}\right)^{1.75k}\cdot 2^{k}$
$\displaystyle=o\left(\left(\frac{n}{k}\right)^{1.8k}\right),$

where the last line uses the fact that $n/k=\omega(1)$.

##  Appendix C Reduction to $k$-$\mathsf{SLR}$ with Independent Noise

The noise
$\bm{\mathbf{w}}=\bm{\mathbf{y}}-\bm{\mathbf{X}}\bm{\mathbf{\theta}}^{*}$ in
our instance as described in Theorem 4 depends on
$\bm{\mathbf{R}}\bm{\mathbf{e}}$, whose entries are marginally Gaussian with
variance $\lVert\bm{\mathbf{e}}\rVert_{2}^{2}$, but not independent from the
samples in $\bm{\mathbf{X}}$. By adding sufficient flooding noise
$\bm{\mathbf{\xi}}$ to the vector $\bm{\mathbf{y}}$, and by starting with
$\mathsf{BinaryBDD}_{d,\alpha}$ instances for which
$\alpha=1/\operatorname{poly}(m)$, we can obtain a distribution over the noise
that is at most $1/2$-away in total variation distance from being independent
Gaussian.

######  Lemma 12.

Let $d,m,n,k,m_{1}$ be integers such that $k$ divides $d$, $m\geq\Omega(d)$
and $n=k\cdot 2^{d/k}$ and $m_{1}=m-k$. Let $\alpha=o(1/(m\sqrt{\log m}))$ be
a real number. Then, there is a $\operatorname{poly}(n,m)$-time reduction from
$\mathsf{BinaryBDD}_{d,\alpha}$ to $k$-$\mathsf{SLR}$ in dimension $n$ with
$m$ samples that succeeds with probability at least $9/10$. If the
$\mathsf{BinaryBDD}_{d,\alpha}$ instance has lattice basis $\mathbf{B}$, the
resulting $k$-$\mathsf{SLR}$ instance $(\mathbf{X},\mathbf{y})$ has parameter
$\delta=\Theta(\lambda_{1,\mathsf{bin}}(\mathbf{B}))$ and design matrix
$\bm{\mathbf{X}}$ as structured in Theorem 4. Further, there is a $k$-sparse
vector $\bm{\mathbf{\theta}}^{*}$ such that

$\displaystyle\mathbf{X}\bm{\mathbf{\theta}}^{*}+\bm{\mathbf{w}}=\bm{\mathbf{y}}$

such that noise $\bm{\mathbf{w}}$ is drawn from a distribution that is at most
$1/2$-away from $\mathcal{N}(0,\sigma^{2})^{\otimes m}$ in total variation
distance, where

$\displaystyle\sigma=\Theta\left(\alpha\cdot\lambda_{1,\mathsf{bin}}(\mathbf{B})\cdot
m\sqrt{\log m}\right).$

###### Sketch..

Let
$(\bm{\mathbf{B}},\bm{\mathbf{t}}:=\bm{\mathbf{B}}\bm{\mathbf{z}}+\bm{\mathbf{e}})$
be our $\mathsf{BinaryBDD}_{d,\alpha}$ instance, where
$\bm{\mathbf{B}}\in\mathbb{R}^{d\times d}$, $\bm{\mathbf{z}}\in\\{-1,1\\}^{d}$
and
$\|\bm{\mathbf{e}}\|_{2}\leq\alpha\cdot\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})$.
Let $\bm{\mathbf{R}}\in\mathbb{R}^{m_{1}\times d}$ be a random matrix where
each entry is drawn i.i.d. from $\mathcal{N}(0,1)$. We define the design
matrix $\bm{\mathbf{X}}$ and target vector $\bm{\mathbf{y}}$ for
$k$-$\mathsf{SLR}$ similar to the definition in Theorem 4. The main difference
is that to achieve Gaussian noise that is independent of the samples
$\bm{\mathbf{X}}$, we add noise vector
$\bm{\mathbf{\xi}}=\begin{pmatrix}\bm{\mathbf{\xi}}_{1}\\\
\bm{\mathbf{\xi}}_{2}\end{pmatrix}$ to the vector $\bm{\mathbf{y}}$, to get
the instance $(\bm{\mathbf{X}},\bm{\mathbf{y}})$ where

$\displaystyle\bm{\mathbf{X}}=\begin{pmatrix}\bm{\mathbf{X}}_{1}\\\
\bm{\mathbf{X}}_{2}\end{pmatrix}=\begin{pmatrix}\bm{\mathbf{R}}\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}\\\
\gamma\bm{\mathbf{G}}_{\mathsf{partite}}\end{pmatrix},\quad\text{ and
}\quad\bm{\mathbf{y}}=\begin{pmatrix}\bm{\mathbf{y}}_{1}\\\
\bm{\mathbf{y}}_{2}\end{pmatrix}=\begin{pmatrix}\bm{\mathbf{R}}\bm{\mathbf{t}}+\bm{\mathbf{\xi}}_{1}\\\
\gamma\bm{\mathbf{1}}+\bm{\mathbf{\xi}}_{2}\end{pmatrix},$

where each entry in $\bm{\mathbf{\xi}}_{1},\bm{\mathbf{\xi}}_{2}$ is sampled
i.i.d from $\mathcal{N}(0,\sigma^{2})$. As before in Theorem 4, we get an
approximation $\widehat{\lambda}_{1}$ for
$\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})$. (Iterating over different guesses
will take at most polynomial time in the size of the problem description.) In
the remainder of the proof, we will assume we know a value
$\widehat{\lambda}_{1}$ such that
$\widehat{\lambda}_{1}\in[\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}}),2\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}}))$.

We set the parameters $\delta,\sigma,\gamma$ such that

$\displaystyle\delta^{2}$
$\displaystyle=\frac{3m_{1}\cdot\widehat{\lambda}_{1}^{2}}{100m}=\Theta(\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})^{2}),$
$\displaystyle\sigma$
$\displaystyle=\Theta(\alpha\cdot\widehat{\lambda}_{1}\cdot
m_{1}\sqrt{\log(m_{1})}),\text{ and}$ $\displaystyle\gamma$
$\displaystyle=\Omega\left(\frac{\sigma_{\max}(\bm{\mathbf{B}})}{\widehat{\lambda}_{1}}\cdot\sqrt{\frac{d}{k}}\cdot(\delta\sqrt{m}+\sigma\sqrt{k})\right).$

Suppose there is an algorithm that returns a $k$-sparse
$\widehat{\bm{\mathbf{\theta}}}$ when run on instance
$(\bm{\mathbf{X}},\bm{\mathbf{y}})$ with parameter $\delta$ such that

$\displaystyle\frac{\|\bm{\mathbf{X}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{y}}\|_{2}^{2}}{m}\leq\delta^{2}.$
(34)

Then, we round $\widehat{\bm{\mathbf{\theta}}}$ to the nearest integer
entrywise to get
$\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})\in\mathbb{Z}^{n}$, and we
output
$\bm{\mathbf{G}}_{\mathsf{sparse}}\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})\in\mathbb{Z}^{d}$
as the $\mathsf{BinaryBDD}_{d,\alpha}$ solution.

First, we show that this reduction is complete. For completeness to hold, it
suffices to choose $\delta,\sigma$ such that the following holds.

$\displaystyle\frac{1}{m}\lVert\bm{\mathbf{X}}\bm{\mathbf{\theta}}^{*}-\bm{\mathbf{y}}\rVert_{2}^{2}$
$\displaystyle=\frac{1}{m}\left(\lVert\bm{\mathbf{R}}\bm{\mathbf{e}}+\bm{\mathbf{\xi}}_{1}\rVert_{2}^{2}+\lVert\bm{\mathbf{\xi}}_{2}\rVert_{2}^{2}\right)\leq
O\left(\frac{\lVert\bm{\mathbf{R}}\bm{\mathbf{e}}\rVert_{2}^{2}+\lVert\bm{\mathbf{\xi}}_{1}\rVert_{2}^{2}+\lVert\bm{\mathbf{\xi}}_{2}\rVert_{2}^{2}}{m}\right)\leq
O\left(\frac{m_{1}\lVert\bm{\mathbf{e}}\rVert_{2}^{2}+m\sigma^{2}}{m}\right)$
$\displaystyle\leq
O(\lVert\bm{\mathbf{e}}\rVert_{2}^{2}+\sigma^{2})\leq\delta^{2}.$

Now, we proceed to prove that the reduction is sound. For the sake of
contradiction, suppose that
$\bm{\mathbf{G}}_{\mathsf{sparse}}\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})=\bm{\mathbf{z}}^{\prime}$
for some $\bm{\mathbf{z}}^{\prime}\neq\bm{\mathbf{z}}$ such that Equation (34)
holds. Since Equation (34) holes, in particular, we know that
$\|\bm{\mathbf{X}}_{2}-\bm{\mathbf{y}}_{2}\|_{2}^{2}=\lVert\gamma\bm{\mathbf{G}}_{\mathsf{partite}}\widehat{\bm{\mathbf{\theta}}}-\gamma\bm{\mathbf{1}}-\bm{\mathbf{\xi}}_{2}\rVert_{2}^{2}\leq
m\delta^{2}$. Since $\|\bm{\mathbf{\xi}}_{2}\|_{2}\leq O(\sigma\sqrt{k})$ with
high probability, we set $\gamma$ such that

$\displaystyle\lVert\bm{\mathbf{G}}_{\mathsf{partite}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{1}}\rVert_{2}\leq\frac{\delta\sqrt{m}+\|\bm{\mathbf{\xi}}_{2}\|_{2}}{\gamma}<1/2.$

Since $\widehat{\bm{\mathbf{\theta}}}$ is $k$-sparse, this means that it
cannot have any part with all zero entries. So each part of
$\widehat{\bm{\mathbf{\theta}}}$ must have exactly one non-zero entry. By the
choice of $\gamma$, each of these non-zero entries must be in the interval
$(1/2,3/2)$, and must round to $1$. Therefore
$\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})\in\mathcal{S}_{n,k}$ and
$\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})$ and
$\widehat{\bm{\mathbf{\theta}}}$ have the same support. This implies that
$\bm{\mathbf{z}}^{\prime}=\bm{\mathbf{G}}_{\mathsf{sparse}}\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})\in\\{-1,+1\\}^{d}$
and
$\bm{\mathbf{G}}_{\mathsf{partite}}\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})=\mathbf{1}$
by Lemma 6. Since we can restrict to the support of
$\widehat{\bm{\mathbf{\theta}}}$, we have

$\displaystyle\lVert\widehat{\bm{\mathbf{\theta}}}\rVert-\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})_{2}=\lVert\bm{\mathbf{G}}_{\mathsf{partite}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{G}}_{\mathsf{partite}}\mathsf{round}(\bm{\mathbf{G}})\rVert=\lVert\mathbf{G}_{\mathsf{partite}}\widehat{\bm{\mathbf{\theta}}}-\mathbf{1}\rVert_{2}\leq\frac{\delta\sqrt{m}+\lVert\bm{\mathbf{\xi}}_{2}\rVert_{2}}{\gamma}.$

Since
$\widehat{\bm{\mathbf{\theta}}}-\mathsf{round}(\widehat{\bm{\mathbf{\theta}}})$
is also $k$-sparse and partite, it follows that

$\displaystyle\lVert\mathbf{G}_{\mathsf{sparse}}(\widehat{\bm{\mathbf{\theta}}}-\mathsf{round}(\widehat{\bm{\mathbf{\theta}}}))\rVert_{2}\leq\sqrt{\frac{d}{k}}\cdot\frac{\delta\sqrt{m}+\lVert\bm{\mathbf{\xi}}_{2}\rVert_{2}}{\gamma},$

as each column of any block of $\bm{\mathbf{G}}_{\mathsf{sparse}}$ has
$\ell_{2}$ norm exactly $\sqrt{d/k}$. Multiplying on the left by $\mathbf{B}$,
we have

$\displaystyle\lVert\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}(\widehat{\bm{\mathbf{\theta}}}-\mathsf{round}(\widehat{\bm{\mathbf{\theta}}}))\rVert_{2}$
$\displaystyle=\lVert\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}-\bm{\mathbf{B}}\bm{\mathbf{z}}^{\prime}\rVert_{2}$
$\displaystyle\leq\sigma_{\max}(\bm{\mathbf{B}})\cdot\sqrt{\frac{d}{k}}\cdot\frac{\delta\sqrt{m}+\lVert\bm{\mathbf{\xi}}_{2}\rVert_{2}}{\gamma}.$

On the other hand, by definition of
$\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})$, since
$\bm{\mathbf{z}}\neq\bm{\mathbf{z}}^{\prime}\in\\{-1,1\\}^{d}$, we know that
$\lVert\bm{\mathbf{B}}\bm{\mathbf{z}}-\bm{\mathbf{B}}\bm{\mathbf{z}}^{\prime}\rVert_{2}\geq\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})$.
Therefore, by triangle inequality, and by our choice of $\gamma$, we have

$\displaystyle\lVert\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{B}}\bm{\mathbf{z}}\rVert_{2}$
$\displaystyle\geq\lVert\bm{\mathbf{B}}\bm{\mathbf{z}}-\bm{\mathbf{B}}\bm{\mathbf{z}}^{\prime}\rVert_{2}-\lVert\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{B}}\bm{\mathbf{z}}^{\prime}\rVert_{2}\geq\frac{49}{50}\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}}).$

By incorporating the error $\bm{\mathbf{e}}$ and multipling on the left by
$\bm{\mathbf{R}}$, we have

$\displaystyle\lVert\bm{\mathbf{X}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{y}}\rVert_{2}$
$\displaystyle\geq\lVert\bm{\mathbf{R}}\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{R}}(\bm{\mathbf{B}}\bm{\mathbf{z}}+\bm{\mathbf{e}})-\bm{\mathbf{\xi}}_{1}\rVert_{2}$
$\displaystyle\geq\sigma_{\min}(\bm{\mathbf{R}})\cdot\left(\lVert\bm{\mathbf{B}}\bm{\mathbf{G}}_{\mathsf{sparse}}\widehat{\bm{\mathbf{\theta}}}-\bm{\mathbf{B}}\bm{\mathbf{z}}\rVert_{2}-\lVert\bm{\mathbf{e}}\rVert_{2}-\lVert\bm{\mathbf{\xi}}_{1}\rVert_{2}\right)$
$\displaystyle\geq\frac{\sqrt{m_{1}}}{2}\cdot\left(\frac{49}{50}\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})-\alpha\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})-c_{1}\sigma\sqrt{m_{1}}\right).$

The last inequality follows because $\lVert\bm{\mathbf{\xi}}_{1}\rVert_{2}\leq
c_{1}\sigma\sqrt{m_{1}}$ for some constant $c_{1}>0$ with probability at least
$1-e^{-\Omega(m_{1})}$. Also, by Lemma 2, we know that
$\sigma_{\min}(\bm{\mathbf{R}})\geq\sqrt{m_{1}}/2$ with probability at least
$1-e^{-\Omega(m_{1})}$. By Equation (34), we get a contradiction as long as

$\displaystyle\delta\sqrt{m}<\sqrt{m_{1}}\cdot\left(\frac{49}{50}\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})-\alpha\cdot\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})-c_{1}\sigma\sqrt{m_{1}}\right).$

On the other hand, we know

$\displaystyle\delta\sqrt{m}=\frac{\widehat{\lambda}_{1}\sqrt{3m_{1}}}{10\sqrt{m}}\cdot\sqrt{m}\leq\frac{1}{5}\cdot\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})\cdot\sqrt{3m_{1}}.$

Combining these inequalities we get a contradiction if $\alpha<c_{2}$ and
$\sigma<c_{3}\cdot\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})/\sqrt{m_{1}}$, for
some constants $c_{2},c_{3}>0$. Since $\alpha=o(1/(m\sqrt{\log m}))$, and by
the choice of $\sigma$, these conditions are met.

Finally, we want the distributions of
$\bm{\mathbf{R}}\bm{\mathbf{e}}+\bm{\mathbf{\xi}}_{1}$ and that of
$\bm{\mathbf{\xi}}_{1}$ to have total variation distance at most $1/2$. With
probability at least $99/100$,
$\lVert\bm{\mathbf{R}}\bm{\mathbf{e}}\rVert_{\infty}\leq
O\left(\lVert\bm{\mathbf{e}}\rVert_{2}\cdot\sqrt{\log(m_{1})}\right)$ by
standard Gaussian tail bounds and a union bound. By Lemma 13, and applying a
union bound, as long as
$\sigma\geq\Omega\left(\lVert\bm{\mathbf{e}}\rVert_{2}\cdot
m_{1}\sqrt{\log(m_{1})}\right)$, the distributions of
$\bm{\mathbf{R}}\bm{\mathbf{e}}+\bm{\mathbf{\xi}}_{1}$ and of
$\bm{\mathbf{\xi}}_{1}$ are at most $1/2$-apart in total variation distance.
It suffices that
$\sigma\geq\Omega(\alpha\cdot\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})\cdot
m_{1}\sqrt{\log(m_{1})})$.

Since $m_{1}=\Theta(m)$, choosing
$\sigma=\Theta\left(\alpha\cdot\lambda_{1,\mathsf{bin}}(\bm{\mathbf{B}})\cdot
m\sqrt{\log(m)}\right)$, $\delta=\Theta(\lambda_{1,\mathsf{bin}})$ and
$\alpha=o(1/(m\sqrt{\log m}))$ satisfies all of these constraints. This
completes the proof. ∎

######  Lemma 13.

Let $|x|\leq b$ be a real number and let $Y\sim\mathcal{N}(0,\sigma^{2})$ be a
random variable. If $\sigma\geq\frac{b}{2\epsilon}$, then the distributions of
$x+Y$ and $Y$ have total variation distance at most $\epsilon$.

###### Proof.

The random variable $x+Y$ is distributed as $\mathcal{N}(x,\sigma^{2})$, so
the KL divergence between the distributions of $x+Y$ and $Y$ is
$\mathsf{KL}(x+Y\|Y)=\frac{x^{2}}{2\sigma^{2}}\leq\frac{b^{2}}{2\sigma^{2}}$.
By Pinsker’s inequality, the total variation distance between the two
distributions is
$\mathsf{TV}(x+Y,Y)\leq\sqrt{\frac{1}{2}\mathsf{KL}(x+Y\|Y)}\leq\frac{b}{2\sigma}\leq\epsilon$.
∎


# Anti-Hawking Phenomena around a Rotating BTZ Black Hole

Matthew P. G. Robbins1,2,3 and Robert B. Mann1,2,3,4 Department of Physics and
Astronomy, University of Waterloo, Waterloo ON, Canada, N2L 3G  Perimeter
Institute for Theoretical Physics, 31 Caroline Street North, Waterloo ON,
Canada, N2L 2Y5  Waterloo Centre for Astrophysics, University of Waterloo,
Waterloo ON, Canada, N2L 3G1  Institute for Quantum Computing, University of
Waterloo, Waterloo ON, Canada, N2L 3G

###### Abstract

In both flat and curved spacetimes, there are weak and strong versions of the
anti-Unruh/anti-Hawking effects, in which the KMS field temperature is anti-
correlated with the response of a detector and its inferred temperature. We
investigate for the first time the effects on the weak and strong anti-Hawking
effects for an Unruh-DeWitt detector orbiting a BTZ black hole in the co-
rotating frame. We find that rotation can significantly amplify the strength
of the weak anti-Hawking effect, whereas it can either amplify or reduce the
strength of the strong anti-Hawking effect depending on boundary conditions.
For the strong anti-Hawking effect, we find a non-monotonic relationship
between the angular momentum and detector temperature for each boundary
condition. In addition, we note that the weak anti-Hawking effect is
independent of a changing AdS length, while a longer AdS length increases the
temperature range of the strong anti-Hawking effect.

##  I Introduction

The nature of the quantum vacuum is fundamental to the study of quantum
information. By coupling the vacuum field to particle detectors, it is
possible to gain understanding of the quantum vacuum and its response to the
structure of spacetime. This is particularly interesting if topological
features or horizons are present. These detectors are often taken to be simple
two-level quantum systems (known as Unruh-DeWitt detectors) that interact with
an underlying scalar field, a model that captures the essential features of
the light-matter interaction MartinMartinez:2012th ; Funai:2021jpc . Such
detectors have been employed to study the structure of spacetime Smith2016 ;
Ng2017 , black holes Henderson2018 ; Tjoa2020 , and the thermality of de
Sitter spacetime Steeg2009 ; Huang2017 .

Consider a detector with uniform acceleration in flat spacetime. The detector
will experience the Unruh effect, in which the temperature increases in
proportion to acceleration Fulling1973 ; Davies1975 ; Unruh1976 , and heat up.
This effect arises because the temperature of the vacuum of one set of modes
is different than the vacuum temperature of a second set of modes. Though
highly idealized in its original assumptions (such as that of an eternally
uniformly accelerating detector), a model-independent derivation of the Unruh
effect has been given in the context of axiomatic quantum field theory
sewell1982quantum , with the field temperature (given by the Kubo-Martin-
Schwinger (KMS) condition Kubo1957 ; Martin1959 ; Haag1967 ) and the
temperature measured by the detector being the same. There have since been
many demonstrations that detectors undergoing other forms of acceleration
(non-uniform, circular) get hot Bell:1986ir ; Costa:1987vv ; Percoco:1991iv ;
Villalba:1999uj ; Ostapchuk:2011ud ; Hu:2012jr ; Doukas2013 . In these more
general situations the field temperature is positively correlated with the
detector temperature, with the latter a monotonically increasing function of
the former.

In the last few years it has been shown that some physical situations exhibit
the so-called anti-Unruh effect instead, in which the temperature of the field
is no longer positively correlated with that measured by the detector
Brenna2016 ; Garay2016 . This anti-Unruh effect can be split into two cases: a
weak Anti-Unruh effect (as the temperature of the field increases, the
detector clicks less often) and a strong Anti-Unruh effect (the field
temperature and detector temperature are inversely related) Garay2016 .

When considering black holes, the Hawking effect is the analogue of the Unruh
effect Birrell1982 . While in general detector temperatures are positively
correlated with the field temperature outside a black hole Hodgkinson2012 ;
Smith2014 ; Hodgkinson:2014iua ; Ng:2014kha , recently an anti-Hawking effect
was shown to also exist, in which a static Unruh-DeWitt (UdW) detector
exhibited both strong and weak versions of the phenomenon Henderson:2019uqo .
This was explicitly demonstrated for the (2+1) dimensional static Banados-
Teitelboim-Zanelli (BTZ) black hole. For sufficiently small black holes, the
temperature measured by the detector would decrease as the Kubo-Martin
Schwinger (KMS) field temperature of the Hawking radiation increased. The
anti-Hawking effect has since been observed for a broader range of boundary
conditions Campos:2020twd , though its weak version is not observed for
massless topological black holes in four spacetime dimensions Campos:2020lpt .

The effects of spacetime dragging due to rotation on these phenomena are much
less understood. The quantum vacuum around a rotating black hole is known to
exhibit features significantly different from its non-rotating counterpart
Winstanley_2001 ; Casals_2008 ; Ottewill:2000yr . Several investigations of
the behaviour of quantum scalar fields in the background of a rotating BTZ
black hole have been carried out Mann:1996ze ; Singh:2011gd ; Bussola:2017wki
; Meitei:2018mgo , and studies investigating the response of Unruh-DeWitt
detectors in such spacetimes have also been undertaken Hodgkinson2012 .
Recently it was shown that rotation has very significant effects on the
entanglement harvesting abilities of UdW detectors, with the harvested
entanglement being considerably amplified at intermediate distances (about
20-50 horizon radii) from the black hole Robbins2020 .

Motivated by this, we study here the implications of rotation for the anti-
Hawking effect. We find that rotation increases the intensity of the weak
anti-Hawking effect, but has a negligible influence on its threshold critical
temperature. However for the strong anti-Hawking effect, we find that there is
a strong dependence on the angular momenta, with the effect becoming stronger
or weaker depending on the boundary conditions. The influence of AdS length on
the strong and weak versions of the effect is likewise distinct: the weak
anti-Hawking effect is independent of AdS length whereas the strong version
sees an increased temperature range.

##  II Unruh-DeWitt Detectors

To model the interaction between the detectors and the field, we take the
detectors to be two-level quantum systems with ground state $\ket{0}_{D}$ and
excited state $\ket{1}_{D}$, separated by an energy gap $\Omega_{D}$. We shall
assume that these detectors have a spacetime trajectory $x_{D}(\tau)$. The
interaction Hamiltonian is

$\displaystyle
H_{D}=\lambda\chi_{D}(\tau)\left(e^{i\Omega_{D}\tau}\sigma^{+}+e^{-i\Omega_{D}\tau}\sigma^{-}\right)\otimes\phi[x_{D}(\tau)]\,,$
(1)

where the switching function dictating the duration of the interaction between
the detector and field is $\chi_{D}(\tau)$. $\lambda\ll 1$ is the field-
detector coupling constant, and the ladder operators that raise and lower the
energy levels of the detectors are $\sigma^{+}=\ket{1}_{D}\bra{0}_{D}$,
$\sigma^{-}=\ket{0}_{D}\bra{1}_{D}$, respectively.

Let the initial state of the detector-field system be
$\ket{\Psi_{i}}=\ket{0}_{D}\ket{0}$. The final state after a time $t$ is then
$\ket{\Psi_{f}}=U(t,0)\ket{\psi_{i}}$, where $U(t,0)=\mathcal{T}e^{-i\int
dt\left[\frac{d\tau_{D}}{dt}H_{D}(\tau_{D}))\right]}$, with $\mathcal{T}$
being the time-ordering operator. With the reduced density operator
$\rho=\text{Tr}_{\phi}\ket{\Psi_{f}}\bra{\Psi_{f}}$ being the state of the
system after integrating over the field’s degrees of freedom, we have
Smith2016 ; Smith2017

$\displaystyle\rho_{AB}=\begin{pmatrix}1-P_{D}&0\\\
0&P_{D}\end{pmatrix}+\mathcal{O}(\lambda^{4})\,,$ (2)

where

$\displaystyle P_{D}=\lambda^{2}\int
d\tau_{D}d\tau_{D}^{\prime}\chi_{D}(\tau_{D})\chi_{D}(\tau_{D}^{\prime})e^{-i\Omega_{D}(\tau_{D}-\tau_{D}^{\prime})}W(x_{D}(\tau_{D}),x_{D}(\tau_{D}^{\prime}))$
(3)

is the detector’s transition probability. We note that this quantity depends
on the two-point correlation function,
$W(x,x^{\prime})=\braket{0}{\phi(x)\phi(x^{\prime})}{0}$ (also called the
Wightman function) of the vacuum. From the transition probability, we can then
define the response function,

$\displaystyle\mathcal{F}=\frac{P_{D}}{\lambda^{2}\sigma}\,,$ (4)

where $\sigma$ describes the timescale of interaction between the field and
the detector. In this paper, we shall focus on a Gaussian switching function,
$\chi_{D}(\tau)=e^{-\frac{\tau^{2}}{2\sigma^{2}}}$.

We will consider fields whose Wightman functions obey the relation

$W(\tau-i/T_{KMS},\tau^{\prime})=W(\tau^{\prime},\tau)$ (5)

known as the Kubo-Martin-Schwinger (KMS) condition Kubo1957 ; Martin1959 ;
Haag1967 . The quantity $T_{KMS}$ in (5) can be regarded as the temperature of
the quantum field in the spacetime. It depends only on the nature of the
quantum field and the spacetime background.

Correspondingly, we can also define the detector’s temperature from its
excitation to de-excitation ratio (EDR). Let

$\displaystyle\mathcal{R}=\frac{\mathcal{F}(\Omega)}{\mathcal{F}(-\Omega)}\,,$
(6)

such that there exists a temperature, $T$, that obeys the same form of the KMS
condition Fewster2016 ,

$\displaystyle\mathcal{R}=e^{-\Omega/T}\,.$ (7)

Labelling the temperature that obeys this condition by $T_{EDR}$, we have

$\displaystyle T_{EDR}=-\frac{\Omega}{\log\mathcal{R}}$ (8)

The quantity $T_{EDR}$ can be regarded as the temperature that the UdW
detector registers in the spacetime.

Normally we expect $T_{EDR}$ and $T_{KMS}$ to be positively correlated: as the
black hole gets hotter, the field temperature increases and the temperature
registered by the UdW detector likewise increases. This is indeed the case for
most situations in black hole physics. As noted above, it was recently shown
that this is not always the case Henderson:2019uqo , and that sometimes the
contrary situation, known as the anti-Hawking effect, occurs. As with the
anti-Unruh effect Brenna2016 ; Garay2016 , we define

$\displaystyle\frac{d\mathcal{F}(\Omega)}{dT_{KMS}}<0\quad\textrm{weak}$ (9)
$\displaystyle\frac{\partial T_{EDR}}{\partial T_{KMS}}<0\quad\textrm{strong}$
(10)

for the weak and strong anti-Hawking effects respectively.

##  III Rotating BTZ Black holes

We can write the action of our system as $S=S_{EH}+S_{\phi}$, where

$\displaystyle S_{EH}=\frac{1}{16\pi}\int R\sqrt{-g}d^{3}x$ (11)

is the Einstein-Hilbert action ($R$ is the Ricci scalar and $g$ is the
determinant of the metric tensor $g_{\mu\nu}$) and

$\displaystyle
S_{\phi}=-\int\left(\frac{1}{2}g^{\mu\nu}\partial_{\mu}\phi\partial_{\nu}\phi+\frac{1}{16}R\phi^{2}\right)\sqrt{-g}d^{3}x$
(12)

is the action for a conformally-coupled scalar field $\phi$. We are interested
in analyzing both the KMS temperature of the field and the EDR temperature of
a detector near a rotating BTZ black hole, whose line element is Banados1992

$ds^{2}=-(N^{\perp})^{2}dt^{2}+f^{-2}dr^{2}+r^{2}(d\phi+N^{\phi}dt)^{2}$ (13)

where, $N^{\perp}=f=\sqrt{-M+\frac{r^{2}}{\ell^{2}}+\frac{J^{2}}{4r^{2}}}$ and
$N^{\phi}=-\frac{J}{2r^{2}}$ with $M=\frac{r_{+}^{2}+r_{-}^{2}}{\ell^{2}}$ the
mass of the black hole and $J=\frac{2r_{+}r_{-}}{\ell}$ its angular momentum.
The Hawking temperature is

$\displaystyle
T_{H}=\frac{1}{2\pi\ell}\left(\frac{r_{+}^{2}-r_{-}^{2}}{r_{+}}\right)\,,$
(14)

where the inner and outer horizon radii are denoted by $r_{-}$ and $r_{+}$,
and $\ell$ is the AdS length. Note that $|J|\leq M\ell$, with extremality
occurring when $r_{+}=r_{-}$ (i.e. $J=M\ell$).

(a) $\zeta=1$

(b) $\zeta=0$

(c) $\zeta=-1$

Figure 1: Response functions for a black hole of mass $M=1/10$ for Dirichlet,
transparent, and Neumann boundary conditions and an energy gap of
$\Omega\ell=1/10$. The inset plots correspond to $M=100$. As expected, the
rotation of the black hole has a smaller effect for larger masses. As the mass
of the black hole increases, the weak anti-Hawking effect goes away for
$\zeta=1$ and $\zeta=0$. Note that for $\zeta=-1$, the weak anti-Hawking
effect is still present even for large mass black holes, with the distinctions
between the different rotation parameters so tiny that the curves effectively
all overlap.

(a) $\Omega\ell=1$

(b) $\Omega\ell=1/10$

(c) $\Omega\ell=1/100$

Figure 2: Response of a rotating BTZ black hole with mass $M=1/1000$ and
Dirichlet boundary conditions ($\zeta=1$). We note that for transparent and
Neumann boundary conditions, qualitatively similar results are obtained.

In the Hartle-Hawking vacuum, a conformally coupled-scalar field has a
Wightman function that can be written as the image sum over the Wightman
functions for AdS3 Lifschytz1994 ; Carlip1998 ,

$\displaystyle
W_{BTZ}(x,x^{\prime})=\sum_{n=-\infty}^{\infty}\eta^{n}W_{AdS_{3}}(x,\Gamma^{n}x^{\prime})$
(15)

where $\Gamma x^{\prime}$ takes $(t,r,\phi)\to(t,r,\phi+2\pi)$, $\eta=1$ is an
untwisted scalar field and $\eta=-1$ is a twisted scalar field. This yields
Hodgkinson2012 ; Smith2014

$\displaystyle W_{BTZ}$
$\displaystyle=\frac{1}{4\pi}\frac{1}{2\sqrt{\ell}}\sum_{n=-\infty}^{n=\infty}\eta^{n}\left(\frac{1}{\sqrt{\sigma_{\epsilon}(x,\Gamma^{n}x^{\prime})}}-\frac{\zeta}{\sqrt{\sigma_{\epsilon}(x,\Gamma^{n}x^{\prime})+2}}\right)$
(16)

where

$\displaystyle\sigma_{\epsilon}(x,\Gamma^{n}x^{\prime})^{2}=$
$\displaystyle-1+\sqrt{\alpha(r)\alpha(r^{\prime})}\cosh\left[\frac{r_{+}}{\ell}(\Delta\phi-2\pi
n)-\frac{r_{-}}{\ell^{2}}(t-t^{\prime})\right]$ (17)
$\displaystyle-\sqrt{(\alpha(r)-1)(\alpha(r^{\prime})-1)}\cosh\left[\frac{r_{+}}{\ell^{2}}(t-t^{\prime})-\frac{r_{-}}{\ell}(\Delta\phi-2\pi
n)\right]$

and

$\displaystyle\alpha(r)$
$\displaystyle=\frac{r^{2}-r_{-}^{2}}{r_{+}^{2}-r_{-}^{2}}\qquad\Delta\phi=\phi-\phi^{\prime}$
(18)

with the respective boundary conditions as $\zeta=1$ (Dirichlet), $\zeta=0$
(transparent), and $\zeta=-1$ (Neumann). We shall take the detector to have
switching function $\chi_{D}(\tau_{D})=e^{-\tau_{D}^{2}/2\sigma^{2}}$ and only
consider untwisted scalar fields with $\eta=1$.

To calculate the transition probabilities, we will work in the co-rotating
frame of the detectors Hodgkinson2012 :

$\displaystyle t$ $\displaystyle=\frac{\ell
r_{+}\tau}{\sqrt{r^{2}-r_{+}^{2}}\sqrt{r_{+}^{2}-r_{-}^{2}}}$ (19)
$\displaystyle\phi$
$\displaystyle=\frac{r_{-}\tau}{\sqrt{r^{2}-r_{+}^{2}}\sqrt{r_{+}^{2}-r_{-}^{2}}}\
.$ (20)

in which case Vagenas2002

$\displaystyle T_{KMS}=T_{H}/\gamma\,,$ (21)

where

$\displaystyle\gamma=\frac{\sqrt{r^{2}-r_{+}^{2}}\sqrt{r_{+}^{2}-r_{-}^{2}}}{r_{+}}$
(22)

is the Lorentz factor. Straightforward calculations show that we can rewrite
equation (3) as
$P_{D}=\sum_{n=-\infty}^{\infty}\eta^{n}\left\\{I_{n}^{-}-\zeta
I_{n}^{+}\right\\}$, where

$\displaystyle
I_{n}^{\pm}=K_{P}\int_{-\infty}^{\infty}dz\frac{e^{-a\left(z-\frac{2\pi
nr_{-}}{\ell}\right)^{2}}e^{-i\beta\left(z-\frac{2\pi
nr_{-}}{\ell}\right)}}{\sqrt{\left(\cosh(\alpha_{n}^{\pm})-\cosh\left(z\right)\right)}}$
(23)

and

$\displaystyle K_{P}$
$\displaystyle=\frac{\lambda^{2}\sigma_{D}}{4\sqrt{2\pi}}$ (24) $\displaystyle
a$ $\displaystyle=\frac{1}{(4\pi
T_{KMS}\sigma_{D})^{2}}\qquad\beta=\frac{\Omega_{D}}{2\pi T_{KMS}}$ (25)
$\displaystyle\cosh(\alpha_{n}^{\pm})$ $\displaystyle=\pm
4\ell^{2}\pi^{2}T_{KMS}^{2}+(1+4\ell^{2}\pi^{2}T_{KMS}^{2})\cosh\frac{2\pi
nr_{+}}{\ell}$ (26)

In the limit of an infinite interaction time (i.e. $\sigma\to\infty$), we note
that we can write the $n=0$ term (corresponding to AdS spacetime) analytically
as

$\displaystyle\lim_{\sigma\to\infty}P_{D,n=0}=\lim_{\sigma\to\infty}I_{0}=\frac{\sqrt{\pi}}{4}\left[1-\tanh\frac{\Omega_{D}}{2T_{KMS}}\right]\left[1-\zeta
P_{-1/2+i\beta}\left(\cosh\alpha_{0}^{+}\right)\right]$ (27)

To investigate the influence of rotation (and AdS length) on the weak and
strong anti-Hawking effects, we must determine the dependence of the response
function and EDR temperature on the KMS temperature. To vary the latter, we
locate the detector at $r=R_{D}$ in the co-rotating frame and solve (21) and
(22) for $R_{D}$ in terms of $T_{KMS}$ and the other parameters. The response
$P_{D}$ in (3) and the EDR temperature are then functions of $T_{KMS}$.

##  IV Weak Anti-Hawking Effect for Rotating BTZ Black Holes

To set the context for our investigation, we first compare the situation for
large and intermediate mass black holes with a detector energy gap of
$\Omega_{D}\ell=1/10$ (with other energy gaps yielding qualitatively similar
results). This is shown in Figure 1, where we plot detector response as a
function of $T_{KMS}$. The general trend is that for both masses and all
boundary conditions the response is suppressed at small KMS temperatures and
asymptotes to a constant value at a large ones. Apart from these two general
features, there is notably distinct behaviour as these parameters are varied.
In the main figure, we depict the situation for an intermediate mass $M=1/10$,
where we see that rotation has marginal impact at small and large KMS
temperatures, but significantly amplifies detector response at intermediate
KMS temperatures. An even larger influence is due to boundary conditions,
where we observe that the weak anti-Hawking effect is absent for Dirichlet
boundary conditions, but present for the other two. The negative slope to the
right of the peak is steeper for Neumann boundary conditions, indicative of
increasing strength of the weak effect as $\zeta$ decreases. In the inset of
each subfigure, we consider the large mass $M=100$ case, where we see that the
weak anti-Hawking effect disappears for $\zeta=1,0$ boundary conditions, yet
remains for $\zeta=-1$, recovering earlier results Henderson:2019uqo  for
large-mass black holes. We see that there is negligible dependence of the
response on angular momentum in this large mass case for all KMS temperatures.

(a) $\Omega\ell=1$

(b) $\Omega\ell=1/10$

(c) $\Omega\ell=1/100$

Figure 3:  Derivative of the response with respect to the KMS temperature (9)
of a rotating BTZ black hole with mass $M=1/1000$, and Dirichlet boundary
conditions ($\zeta=1$). We note that for transparent and Neumann boundary
conditions, qualitatively similar results are obtained.

It is clear from this that rotational effects are more pronounced for smaller
mass black holes, and so in Figure 2, we plot the dependence of the response
function on KMS temperature for $M=1/1000$. The weak effect is now clearly
evident for Dirichlet boundary conditions, with the peak response enhanced as
much as sevenfold, compared to the left diagram in Figure 1. Similar results
hold for the other boundary conditions. As before, as rotation increases, the
response is amplified for all values of $T_{KMS}$. For all values of rotation
and all gaps, the response asymptotes to a value of
$P_{D}=\frac{\sqrt{\pi}}{4}$, in accord with equation (27), noting from
equation (23) that $I_{n}\to 0$ for $n\neq 0$.

The strength of the weak anti-Hawking effect depends on the magnitude of the
negative value of the slope of equation (9) after the peak. We see that this
increases with decreasing gap, showing that smaller gap enhances the weak
anti-Hawking effect, which we illustrate for Dirichlet boundary conditions in
Figure 3. The slope peaks at $\frac{d\mathcal{F}}{dT_{KMS}\ell}\approx-0.15$
for the large energy gap and $\frac{d\mathcal{F}}{dT_{KMS}\ell}\approx-0.6$
for the small gap. For each gap, we also see that the weak anti-Hawking effect
is amplified with increasing rotation, by as much as 50% for near-extremal
black holes, for all gaps in the figure.

Furthermore, we find that the weak anti-Hawking effect occurs after a critical
value of $T_{KMS}$ that depends on the detector’s energy gap, but not on the
rotation of the black hole, again evident from Figure 3. Though we have only
illustrated results for Dirichlet boundary conditions $\zeta=1$, we emphasize
that this critical value depends on $\zeta$, with the critical temperature
becoming smaller as $\zeta\to\\-1$.

Finally, we note that changing the AdS length will not change the strength of
the weak effect. Physically, this is because the AdS length is the only length
scale present (as $\sigma\to\infty$), and everything is calibrated against
this length.

##  V Strong Anti-Hawking Effect for Rotating BTZ Black Holes

Let us now turn our attention to the strong anti-Hawking effect. In Figures 4
and 5, we plot the relationship between the EDR and KMS temperatures for
$M=1/1000$ and different boundary conditions, with $\Omega\sigma=1$ and
$\Omega\sigma=1/10$, respectively. We note several interesting features.

(a) $\zeta=1$

(b) $\zeta=0$

(c) $\zeta=-1$

Figure 4: EDR temperature for a black hole of mass $M=1/1000$ and energy gap
of $\Omega\sigma=1$. We plot KMS temperature down to $T_{KMS}\ell=10^{-5}$.
The insets show the relation between the EDR temperature and KMS temperature
for larger values of $T_{KMS}$.

(a) $\Omega\sigma=1$

(a) $\Omega\sigma=1/100$

Figure 5: EDR temperature for a black hole of mass $M=1/1000$. The results are
similar for $\Omega\sigma=1/100$. The insets show the EDR temperature for our
first three values of the angular momentum, plotted on a linear scale (rather
than a log scale, as is the case for the main plots).

First, it is evident from Figure 4 that a strong anti-Hawking effect is
present for all three boundary conditions. This is clear at low values of
$T_{KMS}$, where we see a negative slope indicative of the strong effect.
Eventually a minimum is reached and $T_{EDR}$ begins to increase with
$T_{KMS}$. The insets indicate the behaviour at large $T_{KMS}$, where we see
that this quantity is indeed positively correlated with $T_{EDR}$, and there
is minimal dependence on angular momentum, even for small $M$. Unlike the weak
effect, we see that the maximum decreases with increasing values of angular
momentum for all boundary conditions. Consequently the strength of the strong
effect (the slope in equation (10)) likewise diminishes. As extremality is
approached, the strong effect essentially vanishes.

The exception is for Dirichlet boundary conditions. In Figure 5, we plot the
EDR temperature for Dirichlet boundary conditions for $\Omega\sigma=1$ and
$\Omega\sigma=1/100$. In the insets, we see that there is a small strong anti-
Hawking effect for non-rotating black holes, similar to what was found in
Henderson:2019uqo . As the angular momentum increases, the peak in Figure 5
moves rightward, and the threshold value of $T_{KMS}$ at which the strong
effect appears increases. This threshold value will reach a maximum for some
$J/M\ell$, after which it decreases with increasing $J/M\ell$ as extremality
is approached, as is clear from Figure 6. We also qualitatively see that the
magnitude of the slope becomes larger for larger angular momenta, indicating
that the strong anti-Hawking effect becomes stronger. For smaller values of
the energy gap, we see that the strong anti-Hawking effect increases in
strength, with the $\Omega\sigma=1/10$ case being similar to
$\Omega\sigma=1/100$. This can be seen by noting that $J/M\ell=0.9$ does not
exhibit the strong anti-Hawking effect for $\Omega\sigma=1$, but there is a
small effect present at this value of the angular momentum for
$\Omega\sigma=1/100$.

For transparent and Neumann boundary conditions, we find that the strong anti-
Hawking effect is much more pronounced, as can be seen from Figure 4.
Furthermore, as $\zeta$ increases from $-1$ to $1$ in Figure 4, the range of
$T_{KMS}\ell$ over which the strong effect is present also decreases, and is
very small for Dirichlet boundary conditions. This range decreases with
increasing angular momentum for transparent and Neumann boundary conditions.
There is a minimal value of $T_{EDR}\ell$ as function of $T_{KMS}\ell$, and
this minimal value decreases as the angular momentum of the black hole
increases.

For Dirichlet boundary conditions and sufficiently small angular momentum,
larger angular momenta also yields a decreasing range of $T_{KMS}$
temperatures for which the strong effect holds, up to a critical value of
$J/M\ell$. Beyond this value, increasing angular momenta results in a greater
range of $T_{KMS}$ temperatures for which the effect is present. As a result,
the anti-Hawking effect appears to nearly disappear for near-extremal black
holes $J\geq 0.9999M\ell$ for transparent and Neumann boundary conditions, yet
is still present for Dirichlet boundary conditions. In Figure 5, we see that
there is a strong anti-Hawking effect for a non-rotating black hole, but the
effect disappears (or almost disappears, depending on the energy gap) as the
angular momentum increases to $J/M\ell=0.9$.

Beyond this, however, as we continue to approach extremality in the Dirichlet
case, the strong effect emerges at lower KMS temperatures, as shown in Figure
6. Indeed, its range and strength both get larger as $J/M\ell$ gets very close
to unity, as evidenced by the curve for $J/M\ell=0.9999$. It is quite
remarkable that there is such a strong dependence on boundary conditions.

Finally, inspection of Figures 4 and 5 indicate that the strong effect does
not monotonically depend on $J/M\ell$. Indeed, we observe a ‘crossover’ effect
at small $T_{KMS}$, in which the values of $T_{EDR}$ decrease with increasing
angular momentum as $T_{KMS}\to 0$, whereas at sufficiently large $T_{KMS}$
the rate of change of $T_{EDR}$ with respect to $T_{KMS}$ increases with
increasing angular momentum such that the higher-$J$ curves cross over the
lower-$J$ curves. At large $T_{KMS}$, we see that $T_{EDR}\sim T_{KMS}$, with
the largest $T_{EDR}$ corresponding to the largest $J$ for fixed $T_{KMS}$,
and the smallest $T_{EDR}$ corresponding to the smallest $J$. This is clearly
evident in Figure 5. We have also verified that this effect is also present
for Neumann and transparent boundary conditions, though the ‘crossover’ occurs
at larger KMS temperatures than the Dirichlet case.

By comparing Figures 2-5, we see that there is no range of $T_{KMS}$ for which
the strong anti-Hawking effect overlaps with the weak anti-Hawking effect,
with the weak anti-Hawking effect appearing for $T_{KMS}\ell\gtrsim 1$, while
the strong anti-Hawking effect appears for $T_{KMS}\ell\lesssim 0.1-0.5$. The
exact temperature range is dependent on the boundary conditions, energy gap,
and in the case of the strong anti-Hawking effect, the angular momentum.
Furthermore, the critical KMS temperature at which the strong anti-Hawking
effect disappears becomes smaller for larger angular momentum, in contrast to
the weak anti-Hawking effect where the critical temperature at which this
effect appears has minimal dependence on angular momentum. In addition, we
again note that the location of this critical temperature for the strong
effect is highly dependent on the boundary conditions.

(a) $\Omega\sigma=1$

(a) $\Omega\sigma=1/100$

Figure 6:  Strong anti-Hawking effect for a near-extremal black hole of mass
$M=1/1000$ and Dirchlet boundary conditions. The results are similar for
$\Omega\sigma=1/10$.

Our last consideration is that of the impact of changing the AdS length on the
strong effect. Here the situation differs from the weak anti-Hawking effect as
now there is a second length scale present ($\sigma$, the width of the
switching function). In Figure 7, we consider the effect of changing the AdS
length for a non-rotating BTZ black hole, compared to a near-extremal BTZ
black hole. In the non-rotating case, increasing AdS length increases the
range of $T_{KMS}$ temperatures where the strong anti-Hawking effect holds.
However, the marginal effect of increasing $\ell$ is reduced for larger and
larger values of the AdS length. We also see that for small $T_{KMS}\ell$, a
larger AdS length will broaden the initial peak. In the case of a near-
extremal black hole, the situation is similar. As we saw in Figure 4a, there
was a tiny strong anti-Hawking effect present for near-extremal black holes.
For larger AdS lengths, we similarly see that the temperature range of the
strong anti-Hawking effect increases in size. However, we note that this
effect is still relatively weak and only becomes noticeable for larger values
of $\ell$.

(a) $J/M\ell=0$

(b) $J/M\ell=0.9999$

Figure 7: Changing AdS lengths for the strong anti-Hawking effect for a black
hole of mass $M=1/1000$, Dirichlet boundary conditions, and energy gap of
$\Omega\sigma=1/10$. We plot KMS temperature down to $T_{KMS}\ell=10^{-5}$.
The insets show the effect of changing AdS length on the EDR temperature for
larger values of $T_{KMS}$.

##  VI Conclusion

As with entanglement harvesting Robbins2020 , rotation can have a significant
impact on the anti-Hawking effect. For large-mass black holes with Dirichlet
and transparent boundary conditions, the weak anti-Hawking effect vanishes, as
expected; it is present for Neumann boundary conditions Henderson:2019uqo . In
all cases the effects of rotation are negligible. But as the mass of the black
hole decreases, rotation significantly amplifies the weak version of the
effect.

The impact of rotation on the strong effect is somewhat inverted. We find that
rotation tends to weaken the strength of the strong anti-Hawking effect for
transparent and Neumann boundary conditions, with it nearly vanishing for
near-extremal black holes. In contrast, for Dirichlet boundary conditions,
larger angular momenta causes the strong anti-Hawking effect to be reduced
before being amplified again.

Furthermore, for the strong anti-Hawking effect, the relationship between
angular momentum and detector temperature is non-monotonic for each boundary
condition, leading to a ‘crossover’ phenomenon that is most prominent for
Dirichlet boundary conditions. For small $T_{KMS}L$, larger angular momenta
yield smaller $T_{EDR}L$, whereas for larger values of $T_{KMS}L$, larger
angular momenta yield larger $T_{EDR}$. More work is needed to better
understand how this cross-over effect comes about and its dependence on the
boundary conditions. It would also be interesting to consider whether $3+1$
dimensional rotating black holes also exhibit these same findings as the
rotating BTZ black hole.

While the weak anti-Hawking effect is independent of the AdS length $\ell$,
increasing AdS length increases the range of $T_{KMS}$ temperatures where the
strong anti-Hawking effect holds. A larger AdS length will also broaden the
initial peak for small $T_{KMS}\ell$. However as $\ell$ continues to increase,
its impact on the strong effect becomes increasingly marginal.

In summary, our results indicate that the effects of spacetime dragging on the
quantum vacuum can significantly modify detector response as small field KMS
field temperatures, as exemplified by the anti-Hawking effect(s). The role of
boundary conditions is very important; indeed, it is surprising that there is
such a strong dependence of the strong anti-Hawking effect on boundary
conditions for small-mass rotating black holes. The origin of this effect
merits further study.

Acknowledgements MR was funded by an Ontario Graduate Scholarship. This
research was supported in part by the Natural Sciences and Engineering
Research Council of Canada, Asian Office of Aerospace Research & Development
Grant FA2386-19-1-4077, and the Perimeter Institute for Theoretical Physics.
Research at Perimeter Institute is supported in part by the Government of
Canada through the Department of Innovation, Science and Economic Development
Canada and by the Province of Ontario through the Ministry of Colleges and
Universities.

## References

  * [1] Eduardo Martin-Martinez, Miguel Montero, and Marco del Rey.  Wavepacket detection with the Unruh-DeWitt model.  Phys. Rev. D, 87(6):064038, 2013.
  * [2] Nicholas Funai.  Investigations into quantum light-matter interactions, their approximations and applications.  PhD thesis, U. Waterloo (main), 2021.
  * [3] Eduardo Martín-Martínez, Alexander R. H. Smith, and Daniel R. Terno.  Spacetime structure and vacuum entanglement.  Phys. Rev. D, 93:044001, Feb 2016.
  * [4] Keith K. Ng, Robert B. Mann, and Eduardo Martín-Martínez.  Over the horizon: Distinguishing the schwarzschild spacetime and the $\mathbb{R}{\mathbb{p}}^{3}$ spacetime using an unruh-dewitt detector.  Phys. Rev. D, 96:085004, Oct 2017.
  * [5] Laura J Henderson, Robie A Hennigar, Robert B Mann, Alexander R H Smith, and Jialin Zhang.  Harvesting entanglement from the black hole vacuum.  Classical and Quantum Gravity, 35(21):21LT02, oct 2018.
  * [6] Erickson Tjoa and Robert B. Mann.  Harvesting correlations in schwarzschild and collapsing shell spacetimes.  Journal of High Energy Physics, 2020(8):155, 2020.
  * [7] Greg Ver Steeg and Nicolas C. Menicucci.  Entangling power of an expanding universe.  Phys. Rev. D, 79(4):044027, February 2009.
  * [8] Zhiming Huang and Zehua Tian.  Dynamics of quantum entanglement in de sitter spacetime and thermal minkowski spacetime.  Nuclear Physics B, 923:458–474, 2017.
  * [9] Stephen A. Fulling.  Nonuniqueness of Canonical Field Quantization in Riemannian Space-Time.  Phys. Rev. D, 7(10):2850–2862, May 1973.
  * [10] P. C. W. Davies.  Scalar production in Schwarzschild and Rindler metrics.  Journal of Physics A Mathematical General, 8(4):609–616, April 1975\.
  * [11] W. G. Unruh.  Notes on black-hole evaporation.  Phys. Rev. D, 14(4):870–892, August 1976.
  * [12] Geoffrey L Sewell.  Quantum fields on manifolds: Pct and gravitationally induced thermal states.  Annals of Physics, 141(2):201–224, 1982.
  * [13] Ryogo Kubo.  Statistical-mechanical theory of irreversible processes. i. general theory and simple applications to magnetic and conduction problems.  Journal of the Physical Society of Japan, 12(6):570–586, 1957.
  * [14] Paul C. Martin and Julian Schwinger.  Theory of many-particle systems. i.  Phys. Rev., 115:1342–1373, Sep 1959.
  * [15] R. Haag, N. M. Hugenholtz, and M. Winnink.  On the equilibrium states in quantum statistical mechanics.  Communications in Mathematical Physics, 5(3):215–236, 1967.
  * [16] J. S. Bell and J. M. Leinaas.  The Unruh Effect and Quantum Fluctuations of Electrons in Storage Rings.  Nucl. Phys. B, 284:488, 1987.
  * [17] I. Costa.  SEPARABLE COORDINATES AND PARTICLE CREATION. 1. THE KLEIN-GORDON EQUATION.  Rev. Bras. Fis., 17:585–600, 1987.
  * [18] Umberto Percoco and Victor M. Villalba.  Particle creation in an asymptotically uniformly accelerated frame.  Class. Quant. Grav., 9:307–316, 1992.
  * [19] Victor M. Villalba and Juan Mateu.  Vacuum effects in an asymptotically uniformly accelerated frame with a constant magnetic field.  Phys. Rev. D, 61:025007, 2000.
  * [20] David C. M. Ostapchuk, Shih-Yuin Lin, Robert B. Mann, and B. L. Hu.  Entanglement Dynamics between Inertial and Non-uniformly Accelerated Detectors.  JHEP, 07:072, 2012.
  * [21] B. L. Hu, Shih-Yuin Lin, and Jorma Louko.  Relativistic Quantum Information in Detectors-Field Interactions.  Class. Quant. Grav., 29:224005, 2012.
  * [22] Jason Doukas, Shih-Yuin Lin, B. L. Hu, and Robert B. Mann.  Unruh effect under non-equilibrium conditions: oscillatory motion of an unruh-dewitt detector.  Journal of High Energy Physics, 2013(11):119, 2013.
  * [23] W.G. Brenna, Robert B. Mann, and Eduardo Martín-Martínez.  Anti-unruh phenomena.  Physics Letters B, 757:307–311, 2016.
  * [24] Luis J. Garay, Eduardo Martín-Martínez, and José de Ramón.  Thermalization of particle detectors: The unruh effect and its reverse.  Phys. Rev. D, 94:104048, Nov 2016.
  * [25] N. D. Birrell and P. C. W. Davies.  Quantum Fields in Curved Space.  Cambridge Monographs on Mathematical Physics. Cambridge University Press, 1982.
  * [26] Lee Hodgkinson and Jorma Louko.  Static, stationary, and inertial unruh-dewitt detectors on the btz black hole.  Phys. Rev. D, 86:064031, Sep 2012.
  * [27] Alexander R. H. Smith and Robert B. Mann.  Looking inside a black hole.  Classical and Quantum Gravity, 31(8):082001, April 2014.
  * [28] Lee Hodgkinson, Jorma Louko, and Adrian C. Ottewill.  Static detectors and circular-geodesic detectors on the Schwarzschild black hole.  Phys. Rev. D, 89(10):104002, 2014.
  * [29] Keith K. Ng, Lee Hodgkinson, Jorma Louko, Robert B. Mann, and Eduardo Martin-Martinez.  Unruh-DeWitt detector response along static and circular geodesic trajectories for Schwarzschild-AdS black holes.  Phys. Rev. D, 90(6):064003, 2014.
  * [30] Laura J. Henderson, Robie A. Hennigar, Robert B. Mann, Alexander R. H. Smith, and Jialin Zhang.  Anti-Hawking phenomena.  Phys. Lett. B, 809:135732, 2020.
  * [31] Lissa De Souza Campos and Claudio Dappiaggi.  The anti-Hawking effect on a BTZ black hole with Robin boundary conditions.  Phys. Lett. B, 816:136198, 2021.
  * [32] Lissa de Souza Campos and Claudio Dappiaggi.  Ground and thermal states for the Klein-Gordon field on a massless hyperbolic black hole with applications to the anti-Hawking effect.  Phys. Rev. D, 103(2):025021, 2021.
  * [33] Elizabeth Winstanley.  Classical super-radiance in kerr–newman–anti-de sitter black holes.  Physical Review D, 64(10), Oct 2001.
  * [34] Marc Casals, Sam Dolan, Panagiota Kanti, and Elizabeth Winstanley.  Bulk emission of scalars by a rotating black hole.  Journal of High Energy Physics, 2008(06):071–071, Jun 2008.
  * [35] Adrian C. Ottewill and Elizabeth Winstanley.  Divergence of a quantum thermal state on Kerr space-time.  Phys. Lett. A, 273:149–152, 2000.
  * [36] Robert B. Mann and Sergei N. Solodukhin.  Quantum scalar field on three-dimensional (BTZ) black hole instanton: Heat kernel, effective action and thermodynamics.  Phys. Rev. D, 55:3622–3632, 1997.
  * [37] Dharm Veer Singh and Sanjay Siwach.  Scalar Fields in BTZ Black Hole Spacetime and Entanglement Entropy.  Class. Quant. Grav., 30:235034, 2013.
  * [38] Francesco Bussola, Claudio Dappiaggi, Hugo R. C. Ferreira, and Igor Khavkine.  Ground state for a massive scalar field in the BTZ spacetime with Robin boundary conditions.  Phys. Rev. D, 96(10):105016, 2017.
  * [39] I. Ablu Meitei, T. Ibungochouba Singh, S. Gayatri Devi, N. Premeshwari Devi, and K. Yugindro Singh.  Quantum gravity effects on scalar particle tunneling from rotating BTZ black hole.  Int. J. Mod. Phys. A, 33(12):1850070, 2018.
  * [40] Matthew P. G. Robbins, Laura J. Henderson, and Robert B. Mann.  Entanglement Amplification from Rotating Black Holes.  arXiv e-prints, page arXiv:2010.14517, October 2020.
  * [41] Smith, Alexander R. H.  Detectors, reference frames, and time, 2017.
  * [42] Christopher J. Fewster, Benito A. Juárez-Aubry, and Jorma Louko.  Waiting for Unruh.  Classical and Quantum Gravity, 33(16):165003, August 2016.
  * [43] Maximo Banados, Claudio Teitelboim, and Jorge Zanelli.  Black hole in three-dimensional spacetime.  Physical Review Letters, 69(13):1849–1851, September 1992.
  * [44] Gilad Lifschytz and Miguel Ortiz.  Scalar field quantization on the (2+1)-dimensional black hole background.  Phys. Rev. D, 49:1929–1943, Feb 1994.
  * [45] Steven Carlip.  Quantum Gravity in 2 1 Dimensions.  Cambridge Monographs on Mathematical Physics. Cambridge University Press, 1998.
  * [46] Elias C. Vagenas.  Two-Dimensional Dilatonic Black Holes and Hawking Radiation.  Modern Physics Letters A, 17(10):609–618, January 2002.


# Applications of fixed point theorems in the theory of invariant subspaces

Rafa Espínola and Miguel Lacruz  Rafa Espínola -<EMAIL_ADDRESS>Miguel Lacruz-
<EMAIL_ADDRESS>Departamento de Análisis Matemático, Facultad de Matemáticas,
Universidad de Sevilla, Avenida Reina Mercedes, 41012 Seville (SPAIN)

###### Abstract

We survey several applications of fixed point theorems in the theory of
invariant subspaces. The general idea is that a fixed point theorem applied to
a suitable map yields the existence of invariant subspaces for an operator on
a Banach space.

###### Key words and phrases:

Invariant subspace; Fixed point

###### 1991 Mathematics Subject Classification:

47A15, 47H10

##  1\. Introduction

One of the most recalcitrant unsolved problems in operator theory is the
invariant subspace problem. The question has an easy formulation. Does every
operator on an infinite dimensional, separable complex Hilbert space have a
non trivial invariant subspace?

Despite the simplicity of its statement, this is a very difficult problem and
it has generated a very large amount of literature. We refer the reader to the
expository paper of Yadav [1] for a detailed account of results related to the
invariant subspace problem.

In this survey we discuss some applications of fixed point theorems in the
theory of invariant subspaces. The general idea is that a fixed point theorem
applied to a suitable map yields the existence of invariant subspaces for an
operator on a Banach space.

In Section 2 we consider the striking theorem of Lomonosov [2] about the
existence of invariant subspaces for algebras containing compact operators.
The proof of this theorem is based on the Schauder fixed point theorem.

In Section 3 we present a recent result of Lomonosov, Radjavi, and Troitsky
[3] about the existence of invariant subspaces for localizing algebras. The
proof of this result is based on the Ky Fan fixed point theorem for
multivalued maps. The idea of using fixed point theorems for multivalued maps
in the search for invariant subspaces was first introduced by Androulakis [4].

In Section 4 we consider an extension of Burnside’s theorem to infinite
dimensional Banach spaces. This result is originally due to Lomonosov [5]. We
present a proof of it in a special case that was obtained independently by
Scott Brown [6] and that once again is based on the Schauder fixed point
theorem.

In Section 5 we address the existence of invariant subspaces for operators on
Krein space of indefinite product, and we present a result of Albeverio,
Makarov, and Motovilov [7] whose proof uses the Banach fixed point theorem.

The rest of this section contains some notation, a precise statement of the
invariant subspace problem, and a few historical remarks.

Let $E$ be an infinite dimensional, complex Banach space and let
${\mathcal{B}}(E)$ denote the algebra of all bounded linear operators on $E.$
A subspace of $E$ is by definition a closed linear manifold in $E.$

A subspace $M\subseteq E$ is said to be invariant under an operator
$T\in{\mathcal{B}}(E)$ provided that $TM\subseteq M,$ and a subspace
$M\subseteq E$ is said to be invariant under a subalgebra
${\mathcal{R}}\subseteq{\mathcal{B}}(H)$ provided that $M$ is invariant under
every $R\in{\mathcal{R}}.$ A subalgebra
${\mathcal{R}}\subseteq{\mathcal{B}}(H)$ is said to be transitive provided
that the only subspaces invariant under ${\mathcal{R}}$ are the trivial ones,
$M=\\{0\\}$ and $M=E.$ This is equivalent to say that the subspace
$\\{Rx:R\in{\mathcal{R}}\\}$ is dense in $E$ for each $x\in
E\backslash\\{0\\}.$

The commutant of a set of operators ${\mathcal{S}}\subseteq{\mathcal{B}}(E)$
is the subalgebra ${\mathcal{S}}^{\prime}$ of all operators
$R\in{\mathcal{B}}(E)$ such that $SR=RS$ for all $S\in{\mathcal{S}}.$ A
subspace $M\subseteq E$ is said to be hyperinvariant under an operator
$T\in{\mathcal{B}}(E)$ provided that $M$ is invariant under
$\\{T\\}^{\prime}.$

The invariant subspace problem is the question of whether every operator in
${\mathcal{B}}(E)$ has a non trivial invariant subspace. This is one of the
most important open problems in operator theory.

The origin of this question goes back to 1935, when von Neumann proved the
unpublished result that any compact operator on a Hilbert space has a non
trivial invariant subspace. Aronszajn and Smith [8] extended this result in
1954 to general Banach spaces. Bernstein and Robinson [9] used non standard
analysis to prove in 1966 that every polynomially compact operator on a
Hilbert space has a non trivial invariant subspace. Halmos [10] obtained a
proof of the same result using classical methods.

Lomonosov [2] proved in 1973 that any non scalar operator on a Banach space
that commutes with a non zero compact operator has a non trivial
hyperinvariant subspace. The result of Lomonosov came into the scene like a
lightning bolt in a clear sky, generalizing all the previously known results,
and introducing the use of the Schauder fixed point theorem as a new technique
to produce invariant subspaces.

Enflo [11] constructed in 1976 the first example of an operator on a Banach
space without non trivial invariant subspaces. The example circulated in
preprint form and it did not appear published until 1987, when it was
recognized as correct work [12]. In the meantime, Beauzamy [13] simplified the
technique, and further examples were given by Read [14, 15].

Very recently, Argyros and Haydon [16] constructed an example of an infinite
dimensional, separable Banach space such that every continuous operator is the
sum of a compact operator and a scalar operator, so that every operator on it
has a non trivial invariant subspace.

However, after so many decades, the question about the existence of invariant
subspaces for operators on Hilbert space is still an open problem.

##  2\. Invariant subspaces for algebras containing compact operators

We start with a fixed point theorem that is the key for the main result in
this section. The use of this result is one of the main ideas in the technique
of Lomonosov. We shall denote by $\overline{{\rm conv}}(S)$ the closed convex
hull of a subset $S\subseteq E.$

######  Proposition 2.1.

[17, Proposition 1] Let $E$ be a Banach space, let $C\subseteq E$ be a closed
convex set, and let $\Phi:C\to E$ be a continuous mapping such that $\Phi(C)$
is a relatively compact subset of $C.$ Then there is a point $x_{0}\in C$ such
that $\Phi(x_{0})=x_{0}.$

###### Proof.

Let $Q$ denote the closure of $\Phi(C).$ It follows from a theorem of Mazur
that $\overline{{\rm conv}}(Q)$ is a compact, convex subset of $E,$ and since
$C$ is closed and convex, we have $\overline{{\rm conv}}(Q)\subseteq C.$ Since
$\Phi(C)\subseteq Q,$ we have $\Phi(\overline{{\rm conv}}(Q))\subseteq
Q\subseteq\overline{{\rm conv}}(Q),$ and now the result follows from the
Schauder fixed point theorem. ∎

######  Theorem 2.2.

[17, Theorem 2] Let ${\mathcal{R}}\subseteq{\mathcal{B}}(E)$ be a transitive
algebra and let $K\in{\mathcal{B}}(E)$ be a non zero compact operator. Then
there is a operator $R\in{\mathcal{R}}$ and there is a vector $x_{1}\in E$
such that $RKx_{1}=x_{1}.$

###### Proof.

We may assume without loss of generality that $\|K\|=1.$ Choose an $x_{0}\in
E$ such that $\|Kx_{0}\|>1,$ so that $\|x_{0}\|>1.$ Consider the closed ball
$B=\\{x\in E:\|x-x_{0}\|\leq 1\\}.$ Then, for each $R\in{\mathcal{R}},$
consider the open set $G_{R}=\\{y\in E:\|Ry-x_{0}\|<1\\}.$ Since
${\mathcal{R}}$ is a transitive algebra, we have

$\bigcup_{R\in{\mathcal{R}}}G_{R}=E\backslash\\{0\\}.$

Since $K$ is a compact operator, $\overline{KB}$ is a compact subset of $E,$
and since $\|K\|=1$ and $\|Kx_{0}\|>1,$ we have $0\notin\overline{KB}.$ Thus,
the family $\\{G_{R}:R\in{\mathcal{R}}\\}$ is an open cover of
$\overline{KB}.$ Hence, there exist finitely many operators
$R_{1},\ldots,R_{n}\in{\mathcal{R}}$ such that

$\overline{KB}\subseteq\bigcup_{i=1}^{n}G_{R_{i}}.$

Next, for each $y\in\overline{KB}$ and $i=1,\ldots,n$ we define
$\alpha_{i}(y)=\max\\{0,1-\|R_{i}y-x_{0}\|\\}.$ Then $0\leq\alpha_{i}(y)\leq
1,$ and for each $y\in\overline{KB}$ there is an $i=1,\ldots,n$ such that
$y\in G_{R_{i}},$ so that $\alpha_{i}(y)>0.$ Thus
$\displaystyle{\sum_{i=1}^{n}\alpha_{i}(y)>0}$ for each $y\in\overline{KB},$
and we may define

$\beta_{i}(y)=\frac{\alpha_{i}(y)}{\displaystyle{\sum_{j=1}^{n}\alpha_{j}(y)}}$

for $i=1,\ldots,n$ and $y\in\overline{KB}.$ Now, each $\beta_{i}$ is a
continuous function from $\overline{KB}$ into ${\mathbb{R}}.$ Hence, we may
define a continuous mapping $\Phi:B\to E$ by the expression

$\Phi(x)=\sum_{i=1}^{n}\beta_{i}(Kx)R_{i}Kx.$

We claim that $\Phi(B)\subseteq B.$ Indeed, for each $x\in B$ we have
$\displaystyle{\sum_{i=1}^{n}\beta_{i}(Kx)=1,}$ so that

$\|\Phi(x)-x_{0}\|=\left\|\sum_{i=1}^{n}\beta_{i}(Kx)(R_{i}Kx-
x_{0})\right\|\leq\sum_{i=1}^{n}\beta_{i}(Kx)\|R_{i}Kx-x_{0}\|.$

If $\|R_{i}Kx-x_{0}\|>1$ then $\alpha_{i}(Kx)=0$ and therefore
$\beta_{i}(Kx)=0.$ Hence

$\|\Phi(x)-x_{0}\|\leq\sum_{i=1}^{n}\beta_{i}(Kx)=1,$

and this completes the proof of our claim. Finally, each operator $R_{i}K$ is
compact, so that each $R_{i}KB$ is relatively compact, and it follows from an
earlier mentioned theorem of Mazur that $\displaystyle{Q=\overline{{\rm
conv}}\bigcup_{i=1}^{n}R_{i}KB}$ is compact. Since $\Phi(B)\subseteq Q,$ the
set $\Phi(B)$ is a relatively compact subset of $B.$ Now we apply Proposition
2.1 to find a vector $x_{1}\in B$ such that $\Phi(x_{1})=x_{1}.$ Since
$0\notin B,$ we have $x_{1}\neq 0.$ Then we consider the operator defined by

$Rx=\sum_{i=1}^{n}\beta_{i}(Kx_{1})R_{i}x,$

and we conclude that $R\in{\mathcal{R}}$ and $RKx_{1}=x_{1},$ as we wanted. ∎

######  Corollary 2.3.

[2], [17, Theorem 3] Every non scalar operator that commutes with a non zero
compact operator has a non trivial, hyperinvariant subspace.

###### Proof.

Let $T\in{\mathcal{B}}(E)$ be a non scalar operator and suppose that $T$
commutes with a nonzero compact operator $K.$ We must show that the commutant
$\\{T\\}^{\prime}$ is non transitive. Suppose, on the contrary, that
$\\{T\\}^{\prime}$ is transitive. We can apply Theorem 2.2 to find an operator
$R\in\\{T\\}^{\prime}$ such that $\lambda=1$ is an eigenvalue of the compact
operator $RK$ with associated finite dimensional eigenspace $F=\ker(RK-I).$
Since $T$ commutes with $RK,$ we observe that $T$ maps $F$ into itself and
therefore $T$ must have an eigenvalue. Since $T$ is non scalar, the
corresponding eigenspace $M$ cannot be the whole $E$, and it is invariant
under $\\{T\\}^{\prime}.$ The contradiction has arrived. ∎

##  3\. Invariant subspaces for localizing algebras

In this section we use the following fixed point theorem of Ky Fan [18].
Recall that if $\Omega$ is a topological space and
$\Phi:\Omega\to{\mathcal{P}}(\Omega)$ is a point to set map from $\Omega$ to
the power set of $\Omega,$ then $\Phi$ is said to be upper semicontinuous if
for every $x_{0}\in\Omega$ and every open set $U\subseteq\Omega$ such that
$\Phi(x_{0})\subseteq U$ there is a neighborhood $V$ of $x_{0}$ such that
$\Phi(x)\subseteq U$ for every $x\in V.$ In terms of convergence of nets, this
definition is equivalent to say that for every $x\in\Omega,$ for every net
$(x_{\alpha})$ with $x_{\alpha}\to x,$ and for every
$y_{\alpha}\in\Phi(x_{\alpha})$ such that the net $(y_{\alpha})$ converges to
some $y\in\Omega,$ we have $y\in\Phi(x).$

######  Theorem 3.1 (Ky Fan fixed point theorem [18]).

Let $C$ be a compact convex subset of a locally convex space and let
$\Phi\colon C\to{\mathcal{P}}(C)$ be an upper semicontinuous mapping such that
$\Phi(x)$ is a non empty, closed convex set for every $x\in C.$ Then there is
an $x_{0}\in C$ such that $x_{0}\in\Phi(x_{0}).$

A subalgebra ${\mathcal{R}}\subseteq{\mathcal{B}}(H)$ is said to be strongly
compact if its unit ball is precompact in the strong operator topology. An
important example of a strongly compact algebra is the commutant of a compact
operator with dense range. We shall denote by ${\rm ball}(\mathcal{R})$ the
unit ball of $\mathcal{R}.$

This notion was introduced by Lomonosov [19] as a means to prove the existence
of invariant subspaces for essentially normal operators on Hilbert spaces.
Recall that an operator $T$ on a Hilbert space is said to be essentially
normal if $T^{\ast}T-TT^{\ast}$ is a compact operator. Lomonosov showed that
if an essentially normal operator $T$ has the property that both its commutant
$\\{T\\}^{\prime}$ and the commutant of its adjoint $\\{T^{\ast}\\}^{\prime}$
fail to be strongly compact, then $T$ has a nontrivial invariant subspace.

Thus, in order to solve the invariant subspace problem for essentially normal
operators, it suffices to consider only operators with a strongly compact
commutant.

Lomonosov, Radjavi and Troitsky [3] obtained a result about the existence of
invariant subspaces for an operator with a strongly compact commutant under
the additional assumption that the commutant of the adjoint is a localizing
algebra.

A subalgebra ${\mathcal{R}}\subseteq{\mathcal{B}}(E)$ is said to be localizing
provided that there is a closed ball $B\subseteq E$ such that $0\notin B$ and
such that for every sequence $(x_{n})$ in $B$ there is a subsequence
$(x_{n_{j}})$ and a sequence of operators $(R_{j})$ in ${\mathcal{R}}$ such
that $\|R_{j}\|\leq 1$ and $(R_{j}x_{n_{j}})$ converges in norm to some non
zero vector. An important example of a localizing algebra is any algebra
containing a non zero compact operator.

######  Proposition 3.2.

[3, Proof of Theorem 2.3] Let ${\mathcal{R}}\subseteq{\mathcal{B}}(E)$ be a
transitive localizing algebra, let $B\subseteq E$ be a closed ball as above,
and let $T\in{\mathcal{R}}^{\prime}$ be a nonzero operator. Then there exists
an $r>0$ such that for every $x\in B$ we have $r\,{\rm
ball}({\mathcal{R}})(Tx)\cap B\neq\emptyset.$

###### Proof.

First, $T$ is one to one, because ${\mathcal{R}}$ is transitive and $\ker T$
is invariant under ${\mathcal{R}}.$ If this is not so, then for every $n\geq
1$ there is a vector $x_{n}\in B$ such that $\|R\|\geq n$, whenever
$R\in{\mathcal{R}}$ and $RTx_{n}\in B$. Since ${\mathcal{R}}$ is localizing,
there is a subsequence $(x_{n_{j}})$ and a sequence $(R_{j})$ in
${\mathcal{R}}$ such that $\|R_{j}\|\leq 1$ and $(R_{j}x_{n_{j}})$ converges
in norm to some nonzero vector $x\in X$. We have $TR_{j}=R_{j}T$ for all
$j\geq 1$, so that $(R_{j}Tx_{n_{j}})$ converges to $Tx$ in norm. Now $Tx\neq
0$ because $T$ is injective and $x\neq 0$. Since ${\mathcal{R}}$ is
transitive, there is an operator $R\in{\mathcal{R}}$ such that $RTx\in{\rm
int}\,B$. It follows that there is a $j_{0}\geq 1$ such that
$RR_{j}Tx_{n_{j}}\in{\rm int}\,B$ for every $j\geq j_{0}$. Since
$RR_{j}\in{\mathcal{R}}$, the choice of the sequence $(x_{n})$ implies that
$\|RR_{j}\|\geq n_{j}$ for every $j\geq j_{0}$, and this is a contradiction
because $\|RR_{j}\|\leq\|R\|$ for every $j\geq 1$. ∎

If $E$ is a Banach space then $E^{\ast}$ denotes its dual space. If
${\mathcal{R}}\subseteq{\mathcal{B}}(E)$ is a subalgebra, then
${\mathcal{R}}^{\ast}$ denotes the subalgebra of ${\mathcal{B}}(E^{\ast})$ of
the adjoints of the elements of ${\mathcal{R}},$ that is
${\mathcal{R}}^{\ast}=\\{R^{\ast}:R\in{\mathcal{R}}\\}.$

######  Theorem 3.3.

[3, Theorem 2.3] Let $E$ be a complex Banach space, let
${\mathcal{R}}\subseteq{\mathcal{B}}(E)$ be a strongly compact subalgebra such
that ${\mathcal{R}}^{\ast}$ is a transitive localizing algebra and it is
closed in the weak-$\ast$ operator topology. If $T\in{\mathcal{R}}^{\prime}$
is a non zero operator then there is an operator $R\in{\mathcal{R}}$ and there
is a non zero vector $x^{\ast}\in E^{\ast}$ such that
$R^{\ast}T^{\ast}x^{\ast}=x^{\ast}.$ Moreover, the operator $T^{\ast}$ has a
non trivial invariant subspace.

###### Proof.

We shall apply Proposition 3.2 to the algebra ${\mathcal{R}}^{\ast}.$ Let
$B^{\ast}\subseteq E^{\ast}$ be a closed ball as in the definition of a
localizing algebra, let $r>0$ be a positive number as in Proposition 3.2, and
define a multivalued map $\Phi:B^{\ast}\to{\mathcal{P}}(B^{\ast})$ by the
expression

$\Phi(x^{\ast})=r\,{\rm ball}({\mathcal{R}}^{\ast})(T^{\ast}x^{\ast})\cap
B^{\ast}.$

Then, $\Phi(x^{\ast})$ is a non empty, convex subset of $B^{\ast}.$ Also,
$\Phi(x^{\ast})$ is weak-$\ast$ closed because ${\rm
ball}({\mathcal{R}}^{\ast})(T^{\ast}x^{\ast})$ is weak-$\ast$ compact as the
image of ${\rm ball}({\mathcal{R}}^{\ast})$ under the map $R^{\ast}\to
R^{\ast}T^{\ast}x^{\ast},$ which is continuous from ${\mathcal{B}}(E^{\ast})$
with the weak-$\ast$ operator topology into $E^{\ast}$ with the weak-$\ast$
topology, and ${\rm ball}({\mathcal{R}}^{\ast})$ is compact in the weak-$\ast$
operator topology.

We claim that $\Phi$ is upper semicontinuous for the weak-$\ast$ topology.
Indeed, let $x^{\ast},y^{\ast}\in B^{\ast},$ and let $(x^{\ast}_{\alpha})$ and
$(y^{\ast}_{\alpha})$ be two nets in $B^{\ast}$ with $x^{\ast}_{\alpha}\to
x^{\ast},$ $y^{\ast}_{\alpha}\to y^{\ast}$ in the weak-$\ast$ topology, and
such that $y^{\ast}_{\alpha}\in\Phi(x^{\ast}_{\alpha}).$ We must show that
$y^{\ast}\in\Phi(x^{\ast}).$ Since
$y^{\ast}_{\alpha}\in\Phi(x^{\ast}_{\alpha}),$ there is an
$R^{\ast}_{\alpha}\in\,{\rm ball}({\mathcal{R}}^{\ast})$ such that
$y^{\ast}_{\alpha}=rR^{\ast}_{\alpha}T^{\ast}x^{\ast}_{\alpha}.$ Since ${\rm
ball}({\mathcal{R}})$ is precompact in the strong operator topology, there
exists a subnet $(R_{\alpha_{\beta}})$ that converges in the strong operator
topology to some $R\in{\mathcal{B}}(E).$ Thus, $R^{\ast}_{\alpha_{\beta}}\to
R^{\ast}$ in the weak-$\ast$ operator topology. Notice that ${\rm
ball}({\mathcal{R}}^{\ast})$ is compact in this topology because ${\rm
ball}({\mathcal{B}}(E^{\ast}))$ is compact in this topology and
${\mathcal{R}}^{\ast}$ is closed in this topology. It follows that
$R^{\ast}\in{\rm ball}({\mathcal{R}}^{\ast}).$ Let $x\in E$ and notice that
$\|TR_{\alpha_{\beta}}x-TRx\|\to 0.$ Then

$\displaystyle\langle x,y^{\ast}_{\alpha_{\beta}}\rangle$ $\displaystyle=$
$\displaystyle\langle
x,rR^{\ast}_{\alpha_{\beta}}T^{\ast}x^{\ast}_{\alpha_{\beta}}\rangle$
$\displaystyle=$ $\displaystyle r\langle
TR_{\alpha_{\beta}}x,x^{\ast}_{\alpha_{\beta}}\rangle$ $\displaystyle=$
$\displaystyle r\langle
TR_{\alpha_{\beta}}x-TRx,x^{\ast}_{\alpha_{\beta}}\rangle+r\langle
TRx,x^{\ast}_{\alpha_{\beta}}\rangle.$

We have $\langle TR_{\alpha_{\beta}}x-TRx,x^{\ast}_{\alpha_{\beta}}\rangle\to
0$ and $\langle TRx,x^{\ast}_{\alpha_{\beta}}\rangle\to\langle
TRx,x^{\ast}\rangle=\langle x,R^{\ast}T^{\ast}x^{\ast}\rangle,$ so that

$\langle x,y^{\ast}_{\alpha_{\beta}}\rangle\to\langle
x,rR^{\ast}T^{\ast}x^{\ast}\rangle.$

Since $x\in E$ is arbitrary, $y^{\ast}_{\alpha_{\beta}}\to
rR^{\ast}T^{\ast}x^{\ast}$ in the weak-$\ast$ topology, and it follows that
$y^{\ast}=rR^{\ast}T^{\ast}x^{\ast}.$ This shows that
$y^{\ast}\in\Phi(x^{\ast}),$ and the proof of our claim is complete.

Since the map $\Phi$ is upper semicontinuous and $B^{\ast}$ is compact in the
weak-$\ast$ topology, it follows from the Ky Fan fixed point theorem that
there is a vector $x^{\ast}\in B^{\ast}$ such that
$x^{\ast}\in\Phi(x^{\ast}),$ that is, there is an operator $R\in{\rm
ball}({\mathcal{R}})$ such that $x^{\ast}=rR^{\ast}T^{\ast}x^{\ast}.$ ∎

######  Corollary 3.4.

[3, Corollary 2.4] Let $T\in{\mathcal{B}}(E)$ be an operator such that
$\\{T\\}^{\prime}$ is a strongly compact algebra and $\\{T^{\ast}\\}^{\prime}$
is a localizing algebra. Then $T^{\ast}$ has a nontrivial invariant subspace.

###### Proof.

If $T^{\ast}$ has a hyperinvariant subspace then there is nothing to prove,
and otherwise $\\{T^{\ast}\\}^{\prime}$ is a transitive algebra, so that
Theorem 3.3 applies. ∎

Notice that the assumptions of Corollary 3.4 are met whenever $T$ is a compact
operator with dense range.

##  4\. An infinite dimensional version of Burnside’s theorem

Burnside’s classical theorem is the assertion that for a finite dimensional
linear space $F,$ the only transitive subalgebra of ${\mathcal{B}}(F)$ is the
whole algebra ${\mathcal{B}}(F).$ Lomonosov [5] obtained a generalization of
Burnside’s theorem to infinite dimensional Banach spaces. Scott Brown [6]
proved the same result independently for the special case of a Hilbert space
and a commutative algebra. Lindström and Schlüchtermann [20] provided a
relatively short proof of the Lomonosov result in full generality. In this
section we present a proof of the Scott Brown result that is based on Schauder
fixed point theorem.

Let $H$ be a complex, infinite dimensional, separable Hilbert space. Let
$T\in{\mathcal{B}}(H)$ and let $\|T\|_{e}$ denote the essential norm of $T,$
that is, the distance from $T$ to the space of compact operators.

######  Theorem 4.1.

[6, Theorem 1.1] Let ${\mathcal{R}}$ be a commutative subalgebra of
${\mathcal{B}}(H)$. Then there exist nonzero vectors $x,y\in H$ such that for
any $R\in{\mathcal{R}}$ we have $|\langle Rx,y\rangle|\leq\|R\|_{e}.$

###### Proof.

Consider the set ${\mathcal{E}}=\\{R\in{\mathcal{R}}:\|R\|_{e}\leq 1/16\\}.$
We claim that there is some $x\in H\backslash\\{0\\}$ such that the set
${\mathcal{E}}x$ is not dense in $H.$ The result then follows easily because
in that case there is some $y\in H\backslash\\{0\\}$ such that $|\langle
Rx,y\rangle|\leq 1$ for all $R\in{\mathcal{E}}.$ Now, for the proof of our
claim, we proceed by contradiction. Suppose that the set ${\mathcal{E}}x$ is
dense in $H$ for every $x\in H\backslash\\{0\\}.$ Choose $x_{0}\in H$ with
$\|x_{0}\|=2$ and consider the closed ball $B=\\{x\in H:\|x-x_{0}\|\leq 1\\}.$
Then, for every vector $x\in B$ there is an operator $R_{x}\in{\mathcal{E}}$
such that $\|R_{x}x-x_{0}\|<1/2.$ Next, there is a bounded operator $T_{x}$
and a compact operator $K_{x}$ such that $R_{x}=T_{x}+K_{x}$ and
$\|T_{x}\|\leq 1/8.$ Since $K_{x}$ is a compact operator, it is weak to norm
continuous on bounded sets, so that there exists an open neighborhood of $x$
in the weak topology, say $V_{x}\subseteq H,$ such that
$\|K_{x}y-K_{x}x\|<1/4$ for all $y\in V_{x}\ \cap B.$ Then consider the set
$U_{x}=V_{x}\cap B$ and notice that $U_{x}$ is an open neighborhood of $x$ in
the weak topology relative to $B.$ Moreover, for $y\in U_{x}$ we have

$\|R_{x}y-R_{x}x\|\leq\|T_{x}y-T_{x}x\|+\|K_{x}y-K_{x}x\|<2\cdot\frac{1}{8}+\frac{1}{4}=\frac{1}{2},$

and therefore $\|R_{x}y-x_{0}\|<1.$ Hence, $R_{x}U_{x}\subseteq B.$ Since $B$
is compact in the weak topology, there exist finitely many vectors
$x_{1},\ldots x_{n}\in B$ such that

$B\subseteq\bigcup_{j=1}^{n}U_{x_{j}}.$

Choose some weakly continuous functions $f_{1},\ldots,f_{n}$ on $B$ such that
${\rm supp}(f_{j})\subseteq U_{j},$ $0\leq f_{j}(x)\leq 1,$ and

$\sum_{j=1}^{n}f_{j}(x)=1\qquad\text{for all }x\in B.$

Define a weakly continuous mapping $\Phi:B\to B$ by the expression

$\Phi(x)=\sum_{j=1}^{n}f_{j}(x)R_{x_{j}}x\qquad\text{for all }x\in B,$

and apply the Schauder fixed point theorem to find a vector $y_{0}\in B$ such
that $\Phi(y_{0})=y_{0}.$ Finally, consider the operator $R\in{\mathcal{R}}$
defined by the expression

$R=\sum_{j=1}^{n}f_{j}(y_{0})R_{x_{j}}.$

Hence, $Ry_{0}=y_{0}.$ Notice that $R\neq I$ because $\|R\|_{e}\leq 1/8.$
Then, the eigenspace $M=\\{x\in H:Rx=x\\}$ is a closed nontrivial invariant
subspace for the algebra ${\mathcal{R}}.$ Thus, any vector $x\in M$ has the
property that the set ${\mathcal{E}}x$ is not dense in $H.$ The contradiction
has arrived. ∎

##  5\. Invariant subspaces for operators on Krein space

Let $H_{1},H_{2}$ be two Hilbert spaces and consider the orthogonal direct sum
$H=H_{1}\oplus H_{2}.$ Let $P_{1},P_{2}$ denote the orthogonal projections
from $H$ onto $H_{1},H_{2},$ respectively. Consider the operator
$J:=P_{1}-P_{2}.$ The Krein space is the space $H$ provided with the
indefinite product

$[x,y]:=\langle Jx,y\rangle,\qquad x,y\in H.$

Notice that $J$ is a selfadjoint involution, that is, $J^{\ast}=J$ and
$J^{2}=I.$ The operator $J$ is sometimes called the fundamental symmetry of
the Krein space.

A vector $x\in H$ is said to be non negative provided that $[x,x]\geq 0,$ and
a subspace $M\subseteq H$ is said to be non negative provided that $[x,x]\geq
0$ for all $x\in M.$

Every operator $T\in{\mathcal{B}}(H)$ has a matrix representation

$T=\left[\begin{array}[]{cc}T_{11}&T_{12}\\\ T_{21}&T_{22}\end{array}\right]$

with respect to the decomposition $H=H_{1}\oplus H_{2}.$

There is a natural, one to one and onto correspondence between the maximal non
negative invariant subspaces $M$ of an operator $T\in{\mathcal{B}}(H)$ and the
contractive solutions $X\in{\mathcal{B}}(H_{1},H_{2})$ of the so called
operator Riccati equation

$XT_{12}X+XT_{11}-T_{22}X-T_{21}=0.$

The correspondence $X\leftrightarrow M$ is given by $M=\\{x_{1}\oplus
Xx_{1}:x_{1}\in H_{1}\\},$ where $\|X\|\leq 1.$ The operator $T$ is usually
called the hamiltonian operator of the operator Ricatti equation.

An operator $T\in{\mathcal{B}}(H)$ is said to be $J$-selfadjoint provided that
$[Tx,y]=[x,Ty]$ for every $x,y\in H.$ This is equivalent to say that
$JT=T^{\ast}J,$ or in other words, $T_{11}^{\ast}=T_{11},$
$T_{22}^{\ast}=T_{22},$ and $T_{12}^{\ast}=-T_{21}.$

A classical theorem of Krein is the assertion that, if the hamiltonian
operator $T$ is $J$-selfadjoint and the corner operator $T_{12}$ is compact,
then there exists a maximal non negative invariant subspace for $T.$

Albeverio, Makarov, and Motovilov [7] addressed the question of the existence
and uniqueness of contractive solutions to the operator Riccati equation under
the condition that the diagonal entries in the hamiltonian operator have
disjoint spectra, that is, $\sigma(T_{11})\cap\sigma(T_{22})=\emptyset.$ They
proved the following

######  Theorem 5.1.

[7, Theorem 3.6 and Lemma 3.11] There is some universal constant $c>0$ such
that whenever the corner operator $T_{12}$ satisfies the condition

$\|T_{12}\|<c\cdot{\rm dist}[\sigma(T_{11}),\sigma(T_{22})],$

there is a unique solution $X$ to the operator Riccati equation with
$\|X\|\leq 1.$

An earlier result in this direction was given by Motovilov [21, Corollary 1]
with the stronger assumption that the corner operator $T_{12}$ is Hilbert-
Schmidt. Adamjan, Langer, and Tretter [22] extended the technique to the case
that the hamiltonian operator is not $J$-selfadjoint. Kostrykin, Makarov, and
Motovilov [23] adopted the assumption that $\sigma(T_{11})$ lies in a gap of
$\sigma(T_{22})$ and they showed that the best constant, in that context, is
$c=\sqrt{2}.$

We present a proof of Theorem 5.2 that is based on Banach fixed point theorem.
This method can be found in the paper of Albeverio, Motovilov and Shkalikov
[24, Theorem 4.1]. A basic tool is the bounded linear operator $R$ defined for
$X\in{\mathcal{B}}(H_{1},H_{2})$ by the expression

$R(X):=T_{22}X-XT_{11}.$

It follows from the Rosenblum theorem that the map $R$ is invertible. The main
result is the following

######  Theorem 5.2.

[24, Theorem 4.1] If the operators $T_{11},T_{22}$ have disjoint spectra and
the corner operator $T_{12}$ satisfies the estimate

$\|T_{12}\|<\frac{1}{2\|R^{-1}\|}$

then there is a unique solution $X$ to the operator Riccati equation with
$\|X\|\leq 1.$

The following upper bound on the norm of the inverse $R^{-1}$ can be found in
the work of Albeverio, Makarov, and Motovilov [7, Theorem 2.7]. See also the
paper by Bhatia and Rosenthal [4, p.15] for this interesting result and other
related issues.

######  Theorem 5.3.

[7, Theorem 2.7] If the operators $T_{11},T_{22}$ have disjoint spectra then

$\|R^{-1}\|\leq\frac{\pi}{2}\cdot\frac{1}{{\rm
dist}[\sigma(T_{11}),\sigma(T_{22})]}$

Notice that Theorem 5.1 becomes a corollary of Theorem 5.2 and Theorem 5.3
with the constant $c=1/\pi.$

###### Proof of Theorem 5.2..

Consider the quadratic map $Q$ defined for $X\in{\mathcal{B}}(H_{1},H_{2})$ by
the expression

$Q(X):=XT_{12}X-T_{21}.$

It is clear that the operator Riccati equation can be expressed as

$Q(X)-R(X)=0,$

or equivalently, $X=R^{-1}(Q(X)).$ Thus, the solutions of the operator Riccati
equation are the fixed points of the map $S:=R^{-1}\circ Q.$ Now, let us check
that the map $S$ takes the unit ball of ${\mathcal{B}}(H_{1},H_{2})$ into
itself. Indeed, if $\|X\|\leq 1$ then

$\displaystyle\|S(X)\|$ $\displaystyle=$
$\displaystyle\|R^{-1}(Q(X))\|\leq\|R^{-1}\|\cdot\|Q(X)\|$ $\displaystyle\leq$
$\displaystyle\|R^{-1}\|\cdot(\|T_{12}\|\cdot\|X\|^{2}+\|T_{21}\|)$
$\displaystyle\leq$
$\displaystyle\|R^{-1}\|\cdot(\|T_{12}\|+\|T_{21}\|)=2\|R^{-1}\|\cdot\|T_{12}\|<1.$

Also, the map $S$ is contractive, for if $\|X\|,\|Y\|\leq 1$ then

$\displaystyle\|Q(X)-Q(Y)\|$ $\displaystyle=$
$\displaystyle\|XT_{12}X-YT_{12}Y\|$ $\displaystyle\leq$
$\displaystyle\|XT_{12}X-XT_{12}Y\|+\|XT_{12}Y-YT_{12}Y\|$ $\displaystyle\leq$
$\displaystyle(\|X\|+\|Y\|)\cdot\|T_{12}\|\cdot\|X-Y\|\leq
2\|T_{12}\|\cdot\|X-Y\|,$

and from this inequality it follows that

$\displaystyle\|S(X)-S(Y)\|$ $\displaystyle=$
$\displaystyle\|R^{-1}(Q(X)-Q(Y))\|$ $\displaystyle\leq$
$\displaystyle\|R^{-1}\|\cdot\|Q(X)-Q(Y)\|\leq
2\|R^{-1}\|\cdot\|T_{12}\|\cdot\|X-Y\|,$

so that the map $S$ satisfies a Lipschitz condition with a Lipschitz constant
$2\|R^{-1}\|\cdot\|T_{12}\|<1.$ The result now follows at once as a
consequence of the Banach fixed point theorem. ∎

## Competing interests

The authors declare that they have no competing interests.

## Author’s contributions

Both authors contributed equally and significantly in writing this article.
Both authors read and approved the final manuscript.

## Acknowledgements

This research was partially supported by Junta de Andalucía under projects
FQM-127 and FQM-3737, and by Ministerio de Educación, Cultura y Deporte under
projects MTM2012-34847C02-01 and MTM2009-08934.

## References

  * [1] Yadav BS: The present state and heritages of the invariant subspace problem. _Milan J. Math._ 2005, 73:289–316.
  * [2] Lomonosov VI: Invariant subspaces of the family of operators that commute with a completely continuous operator. _Funkcional. Anal. i Priložen._ 1973, 7(3):55–56.
  * [3] Lomonosov VI, Radjavi H, Troitsky VG: Sesquitransitive and localizing operator algebras. _Integral Equations Operator Theory_ 2008, 60(3):405–418.
  * [4] Androulakis G: A new method for constructing invariant subspaces. _J. Math. Anal. Appl._ 2007, 333(2):1254–1263.
  * [5] Lomonosov VI: An extension of Burnside’s theorem to infinite-dimensional spaces. _Israel J. Math._ 1991, 75(2-3):329–339.
  * [6] Brown SW: Lomonosov’s theorem and essentially normal operators. _New Zealand J. Math._ 1994, 23:11–18.
  * [7] Albeverio S, Makarov KA, Motovilov AK: Graph subspaces and the spectral shift function. _Canad. J. Math._ 2003, 55(3):449–503.
  * [8] Aronszajn N, Smith KT: Invariant subspaces of completely continuous operators. _Ann. of Math. (2)_ 1954, 60:345–350.
  * [9] Bernstein AR, Robinson A: Solution of an invariant subspace problem of K. T. Smith and P. R. Halmos. _Pacific J. Math._ 1966, 16:421–431.
  * [10] Halmos PR: Invariant subspaces of polynomially compact operators. _Pacific J. Math._ 1966, 16:433–437.
  * [11] Enflo P: On the invariant subspace problem in Banach spaces. In _Séminaire Maurey–Schwartz (1975–1976) Espaces $L^{p}$, applications radonifiantes et géométrie des espaces de Banach, Exp. Nos. 14-15_, Centre Math., École Polytech., Palaiseau 1976:7.
  * [12] Enflo P: On the invariant subspace problem for Banach spaces. _Acta Math._ 1987, 158(3-4):213–313.
  * [13] Beauzamy B: Un opérateur sans sous-espace invariant: simplification de l’exemple de P. Enflo. _Integral Equations Operator Theory_ 1985, 8(3):314–384.
  * [14] Read CJ: A solution to the invariant subspace problem. _Bull. London Math. Soc._ 1984, 16(4):337–401.
  * [15] Read CJ: A solution to the invariant subspace problem on the space $l_{1}$. _Bull. London Math. Soc._ 1985, 17(4):305–317.
  * [16] Argyros SA, Haydon RG: A hereditarily indecomposable ${\mathcal{L}}_{\infty}$-space that solves the scalar-plus-compact problem. _Acta Math._ 2011, 206:1–54.
  * [17] Pearcy C, Shields AL: A survey of the Lomonosov technique in the theory of invariant subspaces. In _Topics in operator theory_ , Providence, R.I.: Amer. Math. Soc. 1974:219–229. Math. Surveys, No. 13.
  * [18] Fan K: Fixed-point and minimax theorems in locally convex topological linear spaces. _Proc. Nat. Acad. Sci. U. S. A._ 1952, 38:121–126.
  * [19] Lomonosov VI: A construction of an intertwining operator. _Funktsional. Anal. i Prilozhen._ 1980, 14:67–68.
  * [20] Lindström M, Schlüchtermann G: Lomonosov’s techniques and Burnside’s theorem. _Canad. Math. Bull._ 2000, 43:87–89.
  * [21] Motovilov AK: The removal of an energy dependence from the interaction in two-body systems. _J. Math. Phys._ 1991, 32(12):3509–3518.
  * [22] Adamjan V, Langer H, Tretter C: Existence and uniqueness of contractive solutions of some Riccati equations. _J. Funct. Anal._ 2001, 179(2):448–473.
  * [23] Kostrykin V, Makarov KA, Motovilov AK: Perturbation of spectra and spectral subspaces. _Trans. Amer. Math. Soc._ 2007, 359:77–89.
  * [24] Albeverio S, Motovilov AK, Shkalikov AA: Bounds on variation of spectral subspaces under $J$-self-adjoint perturbations. _Integral Equations Operator Theory_ 2009, 64(4):455–486.