# Graviton-photon oscillations as a probe of quantum gravity

Andrea Palessandro<EMAIL_ADDRESS>Deloitte AI Institute

###### Abstract

The Gertsenshtein effect could in principle be used to detect a single
graviton by firing it through a region filled with a constant magnetic field
that enables its conversion to a photon, which can be efficiently detected via
standard techniques. The quantization of the gravitational field could then be
inferred indirectly. We show that for currently available single-photon
detector technology, the Gertsenshtein detector is generically inefficient,
meaning that the probability of detection is $\ll 1$. The Gertsenshtein
detector can become efficient on astrophysical scales for futuristic single-
photon detectors sensitive to frequencies in the Hz to kHz range. It is not
clear whether such devices are in principle possible.

##  1 Introduction

To this day, more than a century after the birth of General Relativity,
physics has yet to produce a satisfactory theory of quantum gravity. Given the
difficulties in quantizing gravity, one might ask whether gravity is in fact
quantized and, if it is, whether there is any experimental way to detect its
quantization.

In a famous paper [1], Bohr and Rosenfeld showed that a classical
electromagnetic field interacting with a quantum-mechanical measuring
apparatus leads to inconsistencies in the theoretical description of the
system. Freeman Dyson, following Bronstein [2], pointed out that, due to its
universally attractive nature, the same is not true of gravity: it is
perfectly consistent to have a classical gravitational field interacting with
quantum matter.

There is a long tradition of (thought) experiments attempting to prove the
quantum nature of gravity in the spirit of Bohr and Rosenfeld [3, 4, 5, 6, 7],
but, on closer inspection, they all ultimately fail to compel such a
quantization [8, 9, 10]. The basic reason is that gravity introduces a minimum
length which acts as a fundamental limit to any distance measurement [11].
Trying to resolve finer distances results in black hole production, with any
excess energy further increasing the minimum region of momentum uncertainty.
The experiments that have been proposed so far, both imagined and real, all
end up relying on trans-planckian resolution, and this is why they fail to
demonstrate quantization. The cosmological argument of Krauss and Wilczek [12]
also relies on trans-planckian physics, as I will explain in §6, and is
therefore plagued by similar problems.

If the quantization of the gravitational field is not a logical necessity, it
can only be established empirically. Conceptually, the most immediate way to
do that is to detect a single graviton. Dyson was the first to seriously
investigate the feasibility of graviton detection and found it to be
impossible in practice due to gravitational collapse [13]. Rothman and Boughn
later provided detailed arguments and calculations in support of Dyson’s
conclusion [14, 15].

One of the many graviton detector architectures proposed by Dyson utilizes the
conversion of gravitons to photons, known as the Gertsenshtein effect after
its discoverer [16]. In this experiment, a single graviton is incident on a
region filled with a constant magnetic field, which couples the gravitational
and electromagnetic degrees of freedom [17, 18]. In this region, the graviton
can convert to a photon, which can then be detected with high probability
using traditional photon counting technology. The question we ask in this work
is the following:

> Using a detector architecture based on the Gertsenshtein effect, is it
> possible, in principle, to reliably detect a single graviton?

We tentatively conclude that the answer to this question is no for currently
existing single-photon detectors, if reliably is taken to mean “with
probability one”. Futuristic single-photon detector technology sensitive to
frequencies many orders of magnitude below the current near infrared limit
(roughly in the range from Hz to kHz) could in principle allow the efficient
detection of single gravitons using a Gertsenshtein architecture. However, it
is not clear whether (single) photons with such long wavelengths can ever be
efficiently detected either.

Interestingly, the inefficiency of the Gertsenshtein detector is ultimately
due to nonlinear electromagnetic vacuum polarization effects, which destroy
the coherence of graviton-photon mixing below cosmological scales. A
consistent treatment of graviton-photon oscillations in an expanding universe
shows that these always lie beyond the Hubble radius and are in principle
unobservable [19].

The paper is structured in the following way: in §2 we write down the general
coupled gravito-electromagnetic equations in a polarized vacuum. In §3 we
derive the Gertsenshtein effect in the weak field limit using the formalism of
[20] and make contact with previous results. In §4 we study the effect in the
strong field regime and explain why coherent oscillations must stop above the
Schwinger limit [21]. In §5 we discuss in details a simple experiment based on
the Gertsenshtein effect that could detect a single graviton, and conclude
that the experiment cannot realistically be performed in our universe, in the
sense specified above. In §6 we compare the Gertsenshtein detector with other
graviton detector architectures present in the literature, and find that the
reason why the Gertsenshtein detector is inefficient is qualitatively
different than in all other cases.

##  2 Graviton-photon oscillations in the non-linear regime

In this section we study graviton-photon oscillations in the non-linear
electromagnetic regime using the formalism of [20]. Specifically, we include
self-interactions of the electromagnetic field to one-loop order. We work with
natural units $\varepsilon_{0}=\hbar=c=1$, and metric signature $(-,+,+,+)$.
Here and below $T_{\mu\nu}$ is the electromagnetic stress-energy tensor,
$F_{\mu\nu}=\partial_{\mu}A_{\nu}-\partial_{\nu}A_{\mu}$ the electromagnetic
field tensor,
$\tilde{F}_{\mu\nu}=1/2\,\epsilon^{\mu\nu\alpha\beta}F_{\alpha\beta}$ its
dual, $\alpha\equiv q^{2}/4\pi$ the fine structure constant, $q$ the electron
charge, $m_{e}$ the electron mass, and $G\equiv m_{p}^{-2}$ Newton’s constant
written in terms of the Planck mass $m_{p}$.

For small metric perturbations

$g_{\mu\nu}=\eta_{\mu\nu}+h_{\mu\nu},\quad|h_{\mu\nu}|\ll 1,$ (2.1)

the Lagrangian density describing the interaction between the gravitational
and electromagnetic fields is

$\mathcal{L}=\mathcal{L}_{G}+\mathcal{L}_{\text{EM}}+\mathcal{L}_{I},$ (2.2)

where

$\mathcal{L}_{G}=-\frac{1}{16\pi
G}\left(\frac{1}{2}\partial_{\mu}h_{\alpha\beta}\partial^{\mu}h^{\alpha\beta}-\frac{1}{2}\partial_{\mu}h\partial^{\mu}h+\partial_{\mu}h^{\mu\nu}\partial_{\nu}h-\partial_{\mu}h^{\mu\nu}\partial_{\rho}h^{\rho}_{\nu}\right)$
(2.3)

is the kinetic term of the gravitational field,

$\mathcal{L}_{\text{EM}}=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}+\mathcal{L}^{(1)}_{EM}$
(2.4)

is the kinetic term of the electromagnetic field, which includes vacuum
polarization effects to one loop order, and

$\mathcal{L}_{I}=\frac{1}{2}h_{\mu\nu}T^{\mu\nu}$ (2.5)

is the linearized interaction term.

The electric and magnetic fields in vacuum111More precisely, when vacuum
polarization effects are negligible. are defined in terms of the field tensor
as $E_{i}\equiv F_{i0}$ and $B_{i}\equiv\epsilon_{ijk}F^{jk}/2$. With the aid
of the gauge and Lorentz invariants

$\displaystyle\begin{split}S&=-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}=\frac{1}{2}(E^{2}-B^{2}),\\\
P&=-\frac{1}{4}F_{\mu\nu}\tilde{F}^{\mu\nu}=\vec{E}\cdot\vec{B},\end{split}$
(2.6)

we can write the one-loop correction to the classical QED Lagrangian as [22]

$\mathcal{L}_{EM}^{(1)}(a,b)=-\frac{m_{e}^{4}}{8\pi^{2}}\int_{0}^{\infty}ds\frac{e^{-s}}{s^{3}}\left[(as)(bs)\cot(as)\coth(bs)-1+\frac{s^{2}}{3}(a^{2}-b^{2})\right],$
(2.7)

where

$a\equiv\frac{q}{m_{e}^{2}}\sqrt{\sqrt{S^{2}+P^{2}}+S}=\frac{E}{E_{c}},\quad
b\equiv\frac{q}{m_{e}^{2}}\sqrt{\sqrt{S^{2}+P^{2}}-S}=\frac{B}{B_{c}}.$ (2.8)

In natural units $E_{c}=B_{c}\equiv m_{e}^{2}/q$ is the Schwinger limit, above
which the electromagnetic field becomes nonlinear. The Lagrangian (2.7)
encodes the quantum corrections to photon propagation due to the creation of
virtual electron-positron pairs from the vacuum when background
electromagnetic fields are present.

The coupled gravito-electromagnetic equations are then given by the least
action principle:

$\displaystyle\begin{split}\frac{\partial\mathcal{L}}{\partial
h_{\mu\nu}}-\partial_{\alpha}\frac{\partial\mathcal{L}}{\partial(\partial_{\alpha}h_{\mu\nu})}&=0,\\\
\frac{\partial\mathcal{L}}{\partial
A_{\nu}}-\partial_{\mu}\frac{\partial\mathcal{L}}{\partial(\partial_{\mu}A_{\nu})}&=0.\end{split}$
(2.9)

Given the Lagrangian (2.2), the equation of motion for the metric perturbation
$h_{\mu\nu}$ is

$\frac{1}{2}\left(\partial_{\sigma}\partial_{\mu}h^{\sigma}_{\nu}+\partial_{\sigma}\partial_{\nu}h^{\sigma}_{\mu}-\partial_{\mu}\partial_{\nu}h-\Box
h_{\mu\nu}-\eta_{\mu\nu}\partial_{\rho}\partial_{\lambda}h^{\rho\lambda}+\eta_{\mu\nu}\Box
h\right)=8\pi GT_{\mu\nu}.$ (2.10)

In the transverse-traceless (TT) gauge,
$\partial_{\mu}h^{\mu\nu}=h^{\mu}_{\mu}=0$, and the equation simplifies to
222The d’Alembertian form of the gravitational wave equation is valid for
flat, static spacetimes only. In the more general case of a curved background,
a covariant approach like the one used in [23] or [24] must be employed.

$\Box h_{\mu\nu}=-16\pi GT_{\mu\nu}.$ (2.11)

Using the chain rule, the equation of motion for the electromagnetic field can
be written as

$-\frac{1}{2}\partial_{\mu}\left(\frac{\partial\mathcal{L}_{EM}}{\partial
F_{\alpha\beta}}\frac{\partial
F_{\alpha\beta}}{\partial\partial_{\mu}A_{\nu}}\right)=-\partial_{\mu}\left(\frac{\partial\mathcal{L}_{EM}}{\partial
F_{\mu\nu}}\right)=0.$ (2.12)

Defining the dielectric tensor [25]

$G^{\mu\nu}\equiv-\frac{\partial\mathcal{L}_{EM}}{\partial F_{\mu\nu}},$
(2.13)

we can write (2.12) in compact form as

$\partial_{\mu}G^{\mu\nu}=0.$ (2.14)

The Bianchi identity still holds:

$\partial_{\mu}\tilde{F}^{\mu\nu}=0.$ (2.15)

Equations (2.14) and (2.15) are the dynamical equations for the
electromagnetic field (linearly) coupled to gravity when vacuum polarization
effects cannot be neglected.

The physical fields in the presence of vacuum polarization are $\vec{D}$ and
$\vec{H}$, and can be defined in terms of the dielectric tensor $G^{\mu\nu}$
in complete analogy with the Maxwell case:

$\displaystyle\begin{split}D_{i}&\equiv
G_{i0}=-\frac{\partial\mathcal{L}_{EM}}{\partial
F^{i0}}=\frac{\partial\mathcal{L}_{EM}}{\partial E^{i}},\\\
H_{i}&\equiv\frac{1}{2}\epsilon_{ijk}G^{jk}=-\frac{1}{2}\epsilon_{ijk}\frac{\partial\mathcal{L}_{EM}}{\partial
F^{jk}}=-\frac{\partial\mathcal{L}_{EM}}{\partial B^{i}}.\end{split}$ (2.16)

Note that at low energies one can ignore the nonlinear terms, so
$\mathcal{L}_{EM}=(E^{2}-B^{2})/2$, and the equations above reduce to
$\vec{E}=\partial{\mathcal{L}_{EM}}/\partial\vec{E}$ and
$\vec{B}=-\partial{\mathcal{L}_{EM}}/\partial\vec{B}$, as they should.

We now assume that $h_{\mu\nu}$ describes a gravitational wave with two
independent polarizations $h_{11}=-h_{22}$ and $h_{12}=h_{21}$. Furthermore,
we assume that both the gravitational and electromagnetic waves are
propagating in the $z$ direction, and that in addition to the internal
electric and magnetic fields of the electromagnetic wave
$\vec{e}=e(z,t)\hat{x}$ and $\vec{b}=b(z,t)\hat{y}$, there is a background
magnetic field $\vec{B_{0}}=B_{0}\hat{x}$ such that
$\vec{B}=\vec{b}+\vec{B_{0}}$, with $B_{0}\gg b,e$.

In this setup, we can write (2.16) as $\vec{D}=\vec{E}+\vec{P}$ and
$\vec{H}=\vec{B}+\vec{M}$, where

$\displaystyle\begin{split}\vec{P}=\frac{q^{2}}{am_{e}^{4}}\frac{\partial\mathcal{L}^{(1)}_{EM}}{\partial
a}\vec{E},\\\
\vec{M}=-\frac{q^{2}}{bm_{e}^{4}}\frac{\partial\mathcal{L}^{(1)}_{EM}}{\partial
b}\vec{B},\end{split}$ (2.17)

are the vacuum polarization and magnetization vectors, respectively. Thus,
given (2.16) and (2.17) we can very generally incorporate vacuum polarization
effects in our setup by redefining the physical fields as
$\vec{D}\equiv\varepsilon\vec{E}$ and $\vec{H}\equiv\mu^{-1}\vec{B}$, with

$\displaystyle\begin{split}\varepsilon=1+\frac{q^{2}}{am_{e}^{4}}\frac{\partial\mathcal{L}^{(1)}_{EM}}{\partial
a},\\\
\mu^{-1}=1-\frac{q^{2}}{bm_{e}^{4}}\frac{\partial\mathcal{L}^{(1)}_{EM}}{\partial
b},\end{split}$ (2.18)

the electric permittivity and magnetic permeability of the vacuum,
respectively. These deviate from their classical values as a result of one
loop QED contributions to vacuum polarization.

We have kept the metric implicit in (2.14). We can make it explicit by writing
it in covariant form as

$\partial_{\mu}\left(g^{\alpha\mu}g^{\beta\nu}G_{\alpha\beta}\right)=0.$
(2.19)

For small metric perturbations $g^{\mu\nu}=\eta^{\mu\nu}-h^{\mu\nu}$,
therefore, to first order in $h_{\mu\nu}$

$\partial_{\mu}G^{\mu\nu}-G^{\mu}_{\,\,\beta}\partial_{\mu}h^{\beta\nu}=0.$
(2.20)

For $\nu=0$, (2.20) then gives $\vec{\nabla}\cdot\vec{D}=0$. For $\nu=i$, it
gives333From here on we use the comma notation for partial derivatives, e.g.
$\partial_{\mu}f_{\nu}\equiv f_{\nu,\mu}$.

$\dot{D}_{i}-\epsilon_{ijk}H_{k,j}-\dot{h}_{ij}D_{j}+\epsilon_{jkl}H_{l}h_{ij,k}=0.$
(2.21)

Note that for $h_{ij}=0$ the equation above reduces to the flat space result
$\dot{\vec{D}}=\vec{\nabla}\crossproduct\vec{H}$. Taking the curl of (2.21)
gives

$\epsilon_{ijk}\dot{D}_{k,j}-\epsilon_{ijk}\epsilon_{klm}H_{m,lj}-\epsilon_{ijk}\dot{h}_{kl}D_{l,j}+\epsilon_{ijk}\epsilon_{lmn}(H_{n}h_{kl,m})_{,j}=0.$
(2.22)

The Bianchi identity (2.15) gives the homogeneous Maxwell’s equations
$\vec{\nabla}\cdot\vec{B}=0$, and
$\dot{\vec{B}}=-\vec{\nabla}\crossproduct\vec{E}$. Given
$\vec{D}\equiv\varepsilon\vec{E}$ and $\vec{H}\equiv\mu^{-1}\vec{B}$, we can
write these in terms of the physical fields as

$\displaystyle\begin{split}&\vec{\nabla}\cdot\vec{H}=0,\\\
&\dot{\vec{H}}=-v^{2}(\vec{\nabla}\crossproduct\vec{D}),\end{split}$ (2.23)

where $v^{2}\equiv(\varepsilon\mu)^{-1}$ is the group velocity of the
electromagnetic radiation.

We can use (2.23) to write the equation of motion for the propagating field
$b$ by eliminating any dependency on $e$ in (2.22). For $B_{0}\gg b,e$, this
gives

$\ddot{b}-v^{2}b^{\prime\prime}=v^{2}h^{\prime\prime}_{12}B_{0}.$ (2.24)

To write down the corresponding equation for the graviton, we need to work out
the electromagnetic stress-energy tensor. This is defined as

$4\pi
T^{\mu\nu}=g^{\mu\nu}\mathcal{L}_{EM}-\frac{\partial\mathcal{L}_{EM}}{\partial(\partial_{\mu}A_{\alpha})}\partial^{\nu}A_{\alpha}=G^{\mu\alpha}F^{\nu}_{\,\,\alpha}+g^{\mu\nu}\mathcal{L}_{EM},$
(2.25)

up to a total derivative. We know from (2.11) that, to first approximation,
$T_{\mu\nu}$ is locally conserved, since

$\partial_{\mu}T^{\mu\nu}=-\frac{1}{16\pi G}\Box\partial_{\mu}h^{\mu\nu}=0,$
(2.26)

by virtue of the transverse-traceless condition 444This is of course only the
lowest order approximation to the true conservation law
$\nabla_{\mu}T^{\mu\nu}=0$.. One can also see this directly from (2.25):

$4\pi\partial_{\mu}T^{\mu\nu}=(\partial_{\mu}G^{\mu\alpha})F^{\nu}_{\alpha}+G^{\mu\alpha}(\partial_{\mu}F^{\nu}_{\alpha})+g^{\mu\nu}\partial_{\mu}\mathcal{L}_{EM}.$
(2.27)

The first term is zero because of the equation of motion (2.14). After
contraction with the antisymmetric tensor $G^{\mu\alpha}$, only the
antisymmetric part of $\partial_{\mu}F^{\nu}_{\alpha}$ for the indices
$(\mu,\alpha)$ survives in the second term. The third term can be worked out
straightforwardly using the chain rule and the definition (2.13). Therefore,

$4\pi\partial_{\mu}T^{\mu\nu}=\frac{1}{2}G_{\mu\alpha}\left(\partial^{\mu}F^{\nu\alpha}+\partial^{\alpha}F^{\mu\nu}+\partial^{\nu}F^{\alpha\mu}\right)=0,$
(2.28)

since the quantity in brackets is the Bianchi identity (2.15).

Given (2.16), we can write the spatial components of the tensor (2.25) in
terms of the physical fields as

$4\pi T_{ij}=-(D_{i}E_{j}+B_{i}H_{j})+\frac{1}{2}\delta_{ij}(H\cdot B+D\cdot
E),$ (2.29)

since $\mathcal{L}_{EM}=(\varepsilon E^{2}-\mu^{-1}B^{2})/2\equiv(D\cdot
E-H\cdot B)/2$. In our setup, $h_{12}$ is the only metric component that mixes
linearly with the electromagnetic wave. Given the electromagnetic stress-
energy tensor (2.29), the gravitational wave equation (2.11) for $h_{12}$ is

$\ddot{h}_{12}-h^{\prime\prime}_{12}=-4GB_{0}\varepsilon v^{2}b.$ (2.30)

Note the additional factors of $v^{2}$ and $\varepsilon$ in (2.24) and (2.30)
compared to the wave equations of [20] (see in particular equations 2.13 and
2.14 of the paper). At low energies $B_{0}\ll B_{c}$, $\mu\approx 1$,
$\varepsilon\approx 1$, and $v\approx 1$ (the speed of light), and the results
of [20] are retrieved.

The two coupled differential equations that describe graviton-photon
oscillations in a polarized medium are

$\displaystyle\begin{split}\ddot{b}-v^{2}b^{\prime\prime}&=v^{2}h^{\prime\prime}_{12}B_{0},\\\
\ddot{h}-h^{\prime\prime}&=-4GB_{0}\varepsilon v^{2}b,\end{split}$ (2.31)

where we write $h\equiv h_{12}$ to avoid clutter. The exact solution of (2.31)
can be written as

$\displaystyle\begin{split}h&=\mathcal{A}e^{i(kz-\omega_{+}t)}+\frac{4GB_{0}\varepsilon
v^{2}}{\omega_{-}^{2}-k^{2}}\mathcal{B}e^{i(kz-\omega_{-}t)},\\\
b&=\frac{\omega_{+}^{2}-k^{2}}{4GB_{0}\varepsilon
v^{2}}\mathcal{A}e^{i(kz-\omega_{+}t)}+\mathcal{B}e^{i(kz-\omega_{-}t)},\end{split}$
(2.32)

with

$\omega_{\pm}(k)=\sqrt{\frac{1+v^{2}}{2}k^{2}\pm\frac{1}{2}\sqrt{16GB_{0}^{2}\varepsilon
v^{4}k^{2}+k^{4}(1-v^{2})^{2}}}.$ (2.33)

Note first that for $B_{0}\rightarrow 0$ the solution decouples into a sum of
non-interacting fields with different group velocities, namely
$\omega_{+}(k)\rightarrow k$, $\omega_{-}(k)\rightarrow vk$, and
$h\rightarrow\mathcal{A}e^{i\omega(z-t)}$,
$b\rightarrow\mathcal{B}e^{ik(z-vt)}$, corresponding to a freely propagating
graviton and photon, respectively.

For $B_{0}\neq 0$, one can expand (2.33) around $\omega_{+}=k$ or
$\omega_{-}=vk$ to extract the mixing frequency of the corresponding
oscillation mode:

$\displaystyle\begin{split}\omega_{+}(k)&\approx
k-\frac{k(1-v^{2})}{4}+\frac{1}{4}\sqrt{16GB_{0}^{2}\varepsilon
v^{4}+k^{2}(1-v^{2})^{2}}\equiv k+2\omega_{g},\\\ \omega_{-}(k)&\approx
vk+\frac{k(1-v^{2})}{4}-\frac{1}{4}\sqrt{16GB_{0}^{2}\varepsilon
v^{4}+k^{2}(1-v^{2})^{2}}\equiv vk+2\omega_{\gamma},\end{split}$ (2.34)

where $\omega_{g}$ ($\omega_{\gamma}$) is the mixing frequency of the graviton
(photon) oscillation mode and it is assumed that $k\gg\omega_{g,\gamma}$.

The solution (2.32) is equivalent to Dyson’s wave packet decomposition [20]
given that, for example,
$\exp(ikz-i\omega_{+}t)\equiv\exp(i\omega_{r}(z-t))\exp(-i\omega_{s}(z+t))$ if
one chooses $\omega_{r}$ and $\omega_{s}$ such that $k=\omega_{r}-\omega_{s}$
and $\omega_{+}=\omega_{r}+\omega_{s}$. The decomposition has an easy physical
interpretation: the rapid component $\exp(i\omega_{r}(z-t))$ defines the shape
of the wave packet and the field (particle) energy $\omega_{r}$, while the
slow component $\exp(-i\omega_{s}(z+t))$ describes the oscillation between
photon and graviton states with frequency $\omega_{s}$. The mixing frequency
is given by $2\omega_{s}=\omega_{+}-k$, which by (2.34) is

$\omega_{s}=\frac{1}{8}\left(\sqrt{16GB_{0}^{2}\varepsilon
v^{4}+\omega_{r}^{2}(1-v^{2})^{2}}-\omega_{r}(1-v^{2})\right).$ (2.35)

Note that for $v\rightarrow 1$, one recovers the classical result
$\omega_{s}\rightarrow\sqrt{G}B_{0}/2$, while $\omega_{s}\rightarrow 0$ in the
decoupled limit $B_{0}\rightarrow 0$. It is assumed that
$\omega_{r}\gg\omega_{s}$, which is also the prerequisite for the expansion in
(2.34).

Given that $0\leq v\leq 1$ and $\sqrt{x^{2}+y^{2}}-y\leq x$ $\forall x,y\geq
0$, it is easy to see that the mixing frequency in (2.35) is bounded by

$4\omega_{s}^{2}\leq GB_{0}^{2}\varepsilon v^{4}\leq GB_{0}^{2}.$ (2.36)

This means that the mixing length is always larger than the value one would
get by disregarding vacuum polarization effects, or in a hypothetical world
where there are no photon-electron interactions, i.e. for $\alpha=0$. This is
qualitatively different from the result of [18] (see also [26] for detailed
calculations), who find a shorter mixing length when vacuum polarization
effects are included.

In §5 we will study a graviton detector based on graviton-to-photon conversion
(inverse Gertsenshtein effect). In our setup, the graviton impinges on a
magnetic region of size $D$ at $z=0$ and is slowly converted into a photon.
Using Dyson’s formalism, we can write the general wave packet solution for
both fields inside and outside the detector by requiring continuity of the
functions and their derivatives at $z=0$ [20]. Given an incident right-
traveling gravitational plane wave $\mathcal{A}e^{i\omega_{r}(z-t)}$, the wave
packet inside the detector is given by

$\displaystyle\begin{split}\tilde{h}&=\frac{-i\omega_{r}\mathcal{A}}{2\sqrt{G}}e^{i\omega_{r}(z-t)}e^{-i\omega_{s}z}\approx\frac{-i\omega_{r}\mathcal{A}}{2\sqrt{G}}e^{i\omega_{r}(z-t)}\cos\omega_{s}z,\\\
b&=\frac{\omega_{r}\omega_{s}}{GB_{0}\varepsilon
v^{2}}\mathcal{A}e^{i\omega_{r}(z-t)}\left(e^{-i\omega_{s}z}-1\right)\approx-i\frac{\omega_{r}\omega_{s}}{GB_{0}\varepsilon
v^{2}}\mathcal{A}e^{i\omega_{r}(z-t)}\sin\omega_{s}z,\\\ \end{split}$ (2.37)

where $\tilde{h}\equiv\dot{h}/2\sqrt{G}$ represents the physical gravitational
state. From (2.37), it is clear that the probability of a single graviton
emerging as a photon after a distance $D$ in the presence of an external
magnetic field $B_{0}$ is

$P=\frac{4\omega_{s}^{2}}{GB_{0}^{2}\varepsilon^{2}v^{4}}\sin^{2}(\omega_{s}D).$
(2.38)

Given the bound (2.36),

$P\leq\frac{4\omega_{s}^{2}}{GB_{0}^{2}\varepsilon^{2}v^{4}}\leq 1,$ (2.39)

so that the probability is always less than or equal to one. Whenever $v\neq
1$, $4\omega_{s}^{2}<GB_{0}^{2}\varepsilon^{2}v^{4}$, and the probability of
conversion (2.38) is always strictly less than one. This indicates that when
vacuum polarization effects are taken into account the conversion of gravity
into light (and viceversa) is only partial.

The results of this section are completely general and valid for any field
strength. In order to proceed further, we need explicit expressions for the
electric permittivity and the magnetic permeability of the vacuum in (2.18),
which we derive in the next two sections in both the weak and strong field
limit.

##  3 Graviton-photon oscillations in the weak field limit

In the weak field limit $E\ll E_{c}$ and $B\ll B_{c}$, therefore $a,b\ll 1$.
Additionally, in our setup the magnetic field dominates over the other
components, therefore $b\gg a$.

In this limit one can expand the trigonometric functions $x\cot x$ and $x\coth
x$ in (2.7) in a power series around $x=0$. Keeping only the lowest order
terms, one can write

$(as)(bs)\cot(as)\coth(bs)=1+\frac{s^{2}}{3}(b^{2}-a^{2})-s^{4}\left(\frac{b^{4}}{45}+\frac{a^{4}}{45}+\frac{a^{2}b^{2}}{9}\right).$
(3.1)

The first two terms cancel the last two of (2.7), and one is left with

$\mathcal{L}_{EM}^{(1)}(a,b)\approx\frac{m_{e}^{4}}{8\pi^{2}}\int_{0}^{\infty}ds\,s\,e^{-s}\left(\frac{b^{4}}{45}+\frac{a^{4}}{45}+\frac{a^{2}b^{2}}{9}\right)=\frac{m_{e}^{4}}{360\pi^{2}}(a^{4}+b^{4}+5a^{2}b^{2}).$
(3.2)

Given the definitions in (2.8), this is nothing but the Euler-Heisenberg
Lagrangian [27]

$\mathcal{L}_{EH}=\frac{q^{4}}{360\pi^{2}}\left(4S^{2}+7P^{2}\right)=\frac{\alpha^{2}}{90m_{e}^{4}}\left[(F_{\mu\nu}F^{\mu\nu})^{2}+\frac{7}{4}(\tilde{F}_{\mu\nu}F^{\mu\nu})^{2}\right].$
(3.3)

Having the closed form (3.2) for the electromagnetic Lagrangian as a function
of $a$ and $b$ allows us to compute the electric permittivity and magnetic
permeability in (2.18). In the weak field limit, and for $b\gg a$, these are

$\displaystyle\begin{split}\varepsilon&=1+\frac{q^{2}b^{2}}{36\pi^{2}}=1+\frac{4\alpha^{2}B_{0}^{2}}{9m_{e}^{4}},\\\
\frac{1}{\mu}&=1-\frac{q^{2}b^{2}}{90\pi^{2}}=1-\frac{8\alpha^{2}B_{0}^{2}}{45m_{e}^{4}}.\end{split}$
(3.4)

The group velocity of the electromagnetic radiation then is

$v^{2}=\frac{1}{\varepsilon\mu}=\frac{1-\frac{8\alpha^{2}B_{0}^{2}}{45m_{e}^{4}}}{1+\frac{4\alpha^{2}B_{0}^{2}}{9m_{e}^{4}}}$
(3.5)

For $B_{0}\ll B_{c}$, $v^{2}\rightarrow 1$ and the mixing frequency (2.35)
reduces to the known result $\omega_{s}=\sqrt{G}B_{0}/2$. It is also clear
from (2.35) and (3.5) that the mixing frequency increases with $B_{0}$ up to a
point, and then starts decreasing. Indeed, close to the Schwinger limit
$v^{2}\rightarrow 0$, and $\omega_{s}\rightarrow 0$. Physically, this is due
to the fact that the non-linear interactions encoded in (3.3) slow down the
electromagnetic radiation, but not gravity, which always propagates at the
speed of light in vacuum. Assuming they start in sync, after a distance
$\Delta x$ the phase difference between the two waves will be
$\Delta\phi=\omega_{r}(1-v)\Delta x$, which reaches order one at the
decoherence length [13]

$L_{c}=\frac{1}{\omega_{r}(1-v)}\approx\frac{45m_{e}^{4}}{14\alpha^{2}B_{0}^{2}\omega_{r}}.$
(3.6)

For values of the background magnetic field small compared to the critical
value $B_{c}$, $v\approx 1$ and $L_{c}>2/\sqrt{G}B_{0}$, therefore decoherence
takes place after mixing and one retrieves the classical result. On the other
hand, when $B_{0}$ reaches values close to the Schwinger limit, the
decoherence length becomes smaller than the (classical) mixing length, and
prevents oscillations above that scale. Beyond the Schwinger limit the Euler-
Heisenberg Lagrangian is no longer sufficient, and one needs to consider
higher order terms in (3.1). However, as we show in §4, it is reasonable to
expect that beyond this limit oscillations cease altogether.

##  4 Graviton-photon oscillations in the strong field limit

In our setup, the strong field regime corresponds to $B\gg B_{c}$, but $E\ll
E_{c}$, therefore the relevant limit is $a\ll 1$ and $b\gg 1$. As in the
previous section, we need to compute the derivatives of (2.7) with respect to
$a$ and $b$, for $a\ll 1$ and $b\gg 1$ 555Note that it is inconsistent to
first set $a=0$ in (2.7) and then compute the derivatives, as this procedure
always leads to $\vec{P}=0$. I would like to thank Mikhail Medvedev for
clarifying this point..

Integration of the individual terms in (2.7) leads to divergences. However, if
one regularizes the integrals, the divergent pieces cancel out, leaving only
the finite part. The calculation is involved and is presented in details in
appendix A. The integrals are [28, 29, 30, 31]

$\displaystyle\begin{split}\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
a}&=\frac{am_{e}^{4}}{12\pi^{2}}\left(b-\ln b\right),\\\
\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
b}&=\frac{bm_{e}^{4}}{12\pi^{2}}\ln b.\end{split}$ (4.1)

The electric permittivity and magnetic permeability are then given by (2.18):

$\displaystyle\begin{split}\varepsilon=1+\frac{\alpha}{3\pi}\left(b-\ln
b\right),\\\ \mu^{-1}=1-\frac{\alpha}{3\pi}\ln b.\end{split}$ (4.2)

Therefore, the group velocity is

$v^{2}=\frac{1}{\varepsilon\mu}\approx\frac{1-\frac{\alpha}{3\pi}\ln\left(\frac{B_{0}}{B_{c}}\right)}{1+\frac{\alpha}{3\pi}\frac{B_{0}}{B_{c}}}.$
(4.3)

In the strong field limit $B_{0}\gg B_{c}$, therefore $v\approx 0$, and
$\omega_{s}\approx 0$, vindicating our expectation that oscillations should
stop above the Schwinger limit.

##  5 The Gertsenshtein detector

Given what we know about the Gertsenshtein effect in the non-linear
electromagnetic regime, is it possible, in principle, to detect a single
graviton using graviton-to-photon conversion?

One can set up a basic experiment that would be able to achieve this by
filling a region of space of linear dimension $D$ with a background magnetic
field of intensity $B_{0}$. If a graviton of energy $\omega_{r}$ is incident
upon this region, it is converted to a photon with probability (2.38):

$P=\frac{4\omega_{s}^{2}}{GB_{0}^{2}\varepsilon^{2}v^{4}}\sin^{2}(\omega_{s}D),$
(5.1)

where the mixing frequency $\omega_{s}$ is given by (2.35). The photon can
then be detected with very high probability by a photodetector and the
quantization of the gravitational field inferred indirectly. The probability
of conversion above thus measures the efficiency of the detector as a whole.
We will say that the detector is efficient at detecting gravitons if, across a
distance $D$, $P\sim\mathcal{O}(1)$.

If we choose $D$ of the order of the oscillation length,
$D\sim\omega_{s}^{-1}$, to maximize the probability of detection, the
condition for an efficient detector is

$\frac{4\omega_{s}^{2}}{GB_{0}^{2}\varepsilon^{2}v^{4}}\sim\mathcal{O}(1).$
(5.2)

The crucial question is: can this be achieved in our universe?

We know from §4 that oscillations stop above the Schwinger limit
($\omega_{s}=0$), so we can restrict our analysis to the weak field regime
$B_{0}<B_{c}$. We also know from §3 that condition (5.2) is satisfied in the
limit $B_{0}\ll B_{c}$, where $4\omega_{s}^{2}\approx GB_{0}^{2}$ and
$\varepsilon,v\approx 1$. Clearly, one can take $B_{0}=0$ to satisfy the
condition on the probability, but then the mixing length would become
infinite, so what we really want to do is satisfy (5.2) while at the same time
minimizing $D$. The question then becomes: what is the maximum value of
$B_{0}$ before $\omega_{s}$ starts deviating from its unpolarized value, i.e.
before coherence is destroyed?

From (2.35) it is clear that as soon as the second term in the square root
dominates over the first, the mixing frequency quickly goes to zero.
Therefore, the maximum possible value of $B_{0}$ before vacuum polarization
effects kick in and oscillations stop is given by equating the two terms in
the square root, meaning $\bar{B}_{0}\sim\omega_{r}(1-v^{2})/4\sqrt{G}$. Since
$\bar{B}_{0}\ll B_{c}$, $1-v^{2}\approx
28\alpha^{2}\bar{B}_{0}^{2}/45m_{e}^{4}$, one gets

$\bar{B}_{0}\sim\frac{\sqrt{G}m_{e}^{4}}{\alpha^{2}\omega_{r}}.$ (5.3)

One can arrive at the same condition by equating the coherence length (3.6)
with the classical mixing length: $\omega_{r}(1-v)=\sqrt{G}\bar{B}_{0}/2$.
Note also that, since we are assuming $\bar{B}_{0}\ll B_{c}$, we need
$\omega_{r}\gg\sqrt{G}B_{c}/\alpha$.

The minimum size $\bar{D}$ for an efficient detector is the mixing length
corresponding to (5.3), namely

$\bar{D}\sim\omega_{s}^{-1}=\frac{2}{\sqrt{G}\bar{B}_{0}}\sim\frac{\alpha^{2}\omega_{r}}{Gm_{e}^{4}}.$
(5.4)

For an X-ray photodetector like the one used by the CERN Axion Solar Telescope
(CAST) [32], $\omega_{r}\sim 10^{18}\,\text{Hz}\sim 10\,\text{keV}$, and
$\bar{D}\sim 10^{23}\,\text{km}$, approximately the radius of the observable
universe.

In principle, one could consider graviton frequencies much smaller than
$10^{18}$ Hz, as the only theoretical constraint on $\omega_{r}$ is
$\omega_{r}\gg\sqrt{G}B_{c}/\alpha$, which is easily satisfied. For example,
if one takes $\omega_{r}\sim$ kHz, $\bar{D}\sim 10^{8}$ km, while the
graviton/photon wavelength is $\omega_{r}^{-1}\sim 300\,\text{km}\ll\bar{D}$.
However, in such a setup, photon detection becomes infeasible. Modern low-
frequency single-photon detectors are sensitive to near infrared wavelengths
of up to 1 $\mu$m [33, 34], many orders of magnitude below the km range needed
to (realistically) perform the experiment. Photon wavelengths in the
micrometer to nanometer range inevitably give cosmological values for the
detector size, roughly in the range from Mpc to Gpc.

At cosmological scales, however, one should take into account the effects an
expanding universe has on distances and fields [19]. Given that the magnetic
field scales as $a^{-2}$, the proper mixing length is
$D_{p}=2a^{2}/\sqrt{G}B_{0}$, while the Hubble radius is $H^{-1}=a/\dot{a}$.
Therefore, in general

$\frac{D_{p}}{H^{-1}}=\frac{2a\dot{a}}{\sqrt{G}B_{0}}.$ (5.5)

Using Friedmann’s equation $H^{2}=(8\pi G/3)\rho$, one can rewrite (5.5) as

$\frac{D_{p}}{H^{-1}}=\sqrt{\frac{32\pi\rho}{3}}\frac{a^{2}}{B_{0}}.$ (5.6)

For a perfect fluid with equation of state $p=\omega\rho$, $\rho\propto
a^{-3(1+\omega)}$, therefore

$\frac{D_{p}}{H^{-1}}=\sqrt{\frac{32\pi\rho_{0}}{3}}\frac{a^{\frac{{1-3\omega}}{2}}}{B_{0}}.$
(5.7)

For $\omega<1/3$ (which includes the matter and dark energy dominated cases),
the ratio in (5.7) increases with the expansion, exponentially in deSitter
space. In the current stage of the universe’s evolution, this means that the
proper mixing length very quickly becomes larger than the horizon.
Furthermore, assuming the intergalactic magnetic field does not dominate the
energy density of the universe, $B_{0}^{2}\lesssim\rho_{0}$, (5.7) entails

$D_{p}\gtrsim\sqrt{\frac{32\pi}{3}}H^{-1},$ (5.8)

larger than the Hubble radius. The conclusion is that when one factors in the
expansion of the universe, graviton-photon oscillations on cosmological scales
always lie outside the Hubble radius and are as a result unobservable [19].

Therefore, barring any futuristic single photon detection technology that
would allow for detection frequencies much smaller than a THz, the detection
of a single graviton using the Gertsenshtein mechanism is generically
inefficient (in any remotely realistic scenario) due to nonlinear vacuum
polarization effects. Note the crucial role played by the electromagnetic
constants in our conclusion: if the fine structure constant were much smaller,
or the electron mass much larger, than the values they take in our universe,
the Gertsenshtein experiment could in principle be performed successfully on
astrophysical or even planetary scales666Choosing different values for the
electromagnetic constants does not necessarily impact the efficiency of photon
detection. Using a simple toy model for photon detection consisting of a gas
of hydrogen atoms, their binding energy is $E_{B}=-m_{e}\alpha^{2}/2$. One
could increase the electron mass while at the same time decreasing the fine
structure constant in such a way that the binding energy stays the same, and
the photoemission process happens at the same frequency. Thus, it is possible
to imagine a universe in which (single) graviton-photon oscillations are
observable and photon detection is efficient at X-rays frequencies..

Our conclusion of course only applies to the detection of a single graviton:
the number of photons detected in the experiment is roughly equal to
$N_{\gamma}\sim PN_{G}$, where $N_{G}$ is the number of gravitons incident
upon the magnetic region. Even when $P\ll 1$, $N_{\gamma}$ can be made greater
than one if $N_{G}$ is large enough, and for relatively small detector sizes.
When $N_{G}\gg 1$, however, we are no longer probing the quantum gravitational
regime: we simply have a classical gravitational wave.

In the Gertsenshtein experiment, the nonlinearities of the electromagnetic
field seem to conspire to hide the quantization of gravity. Interestingly, the
obstruction to probing the quantum gravitational regime in the Gertsenshtein
experiment is then qualitatively different than the one encountered in other
proposed graviton detector architectures, which we explore in the next
section.

##  6 Relation to other experiments

Many different graviton detector architectures have been considered in the
literature [13, 14, 15, 12], and, so far, all of them have been shown to be
inefficient. In this section, we revisit them, trying to find the common
reason behind their inefficiency, and compare them with the Gertsenshtein
experiment of §5.

The simplest kind of graviton detector is a collection of atoms. Dyson showed
that an atom absorbs a graviton with a planckian cross section:

$\sigma_{\text{abs}}=4\pi^{2}l_{p}^{2}Q,$ (6.1)

where $l_{p}$ is the Planck length and $Q$ is a number of order one777In fact,
this result extends to generic (quantum) bound states [35].. A collection of
atoms is efficient at absorbing gravitons if the optical depth of the graviton
through the medium is greater than one:

$n_{B}\sigma_{\text{abs}}D>1,$ (6.2)

where $n_{B}$ is the density of atoms (bound states) and $D$ the size of the
detector. Assuming the mass of the detector is $M$, while the mass of a single
atom is $m_{B}$, then $n_{B}=M/(m_{B}D^{3})$, and the optical depth is

$n_{B}\sigma_{\text{abs}}D\sim\frac{GM}{D}\frac{1}{m_{B}D}\lesssim 1,$ (6.3)

given that $GM<D$ to avoid gravitational collapse and $m_{B}D>1$ for particles
whose Compton wavelength is smaller than the detector. Therefore graviton
absorption is generically inefficient. Barring any high-energy modification of
General Relativity, this also prevents the occurrence of gravitational
absorption lines in the primordial gravitational wave spectrum [35].

Architectures based on laser interferometry, such as the one used in LIGO,
fare no better. The energy density of a gravitational wave of strain $f$ and
frequency $\omega$ is

$\rho=\frac{f^{2}\omega^{2}}{32\pi G},$ (6.4)

while the energy density of a single graviton of the same frequency is
$\omega^{4}$. Therefore, in order to detect a single graviton, a LIGO-type
apparatus needs to be sensitive to a strain

$f=\sqrt{32\pi}\omega l_{p}.$ (6.5)

The measured strain is nothing but the fractional change in the distance $D$
between two masses: $f=\Delta D/D$. Thus the detector needs to be sensitive to
a distance variation of

$\Delta D=fD=\sqrt{32\pi}(\omega D)l_{p}\lesssim\sqrt{32\pi}l_{p},$ (6.6)

given that the distance $D$ between the two masses has to be smaller than the
graviton wavelength: $D<\omega^{-1}$. Consequently, a LIGO-type interferometer
needs to measure distances to Planck length accuracy in order to be able to
detect a single graviton, and is therefore technically inefficient at this
task.

A kind of cosmic detector was envisioned in [12]. Here, the authors point out
that the observation of a cosmological gravitational wave background
associated with an inflationary phase would provide evidence for the
quantization of the gravitational field. However, we know that if the
inflationary phase lasts too long, the length scales we observe nowadays
correspond to modes smaller than the Planck length at the onset of inflation.
This is the trans-Planckian problem of inflationary cosmology [36, 37]. It has
been conjectured [38] that this problem can never arise in any consistent UV-
complete theory of gravity. According to this trans-Planckian Censorship
Conjecture (TCC), no mode that exits the Hubble horizon could ever have had a
wavelength smaller than the Planck length. If $a_{i}$ and $a_{f}$ are the
values of the scale factor at the beginning and end of the inflation phase,
respectively, the TCC reads [39]

$\frac{a_{f}}{a_{i}}<\frac{m_{p}}{H_{f}},$ (6.7)

where $H_{f}$ is the Hubble rate at the end of inflation. Given that after
inflation $a_{f}/a_{i}$ is exponentially large, the condition above severely
limits the scale of inflation $H_{f}$. In particular, if inflation provides a
causal mechanism for the origin of structure, the current Hubble volume must
originate from the Hubble volume at the beginning of inflation, meaning that
roughly

$\frac{1}{H_{0}}\approx\frac{1}{H_{f}}e^{N}\frac{T_{R}}{T_{0}},$ (6.8)

where $H_{0}$ is the Hubble rate today, $N$ the number of e-foldings, and
$T_{0}$, $T_{R}$ the temperatures today and at reheating, respectively. If
reheating is fast, $T_{R}\approx\sqrt{m_{p}H_{f}}$. Furthermore, given that
the matter energy density today is larger than the radiation energy density by
a factor $T_{eq}/T_{0}$, where $T_{eq}$ is the temperature at matter-radiation
equality, $H_{0}=\sqrt{T_{0}^{3}T_{eq}}/m_{p}$, and (6.8) gives

$\frac{a_{f}}{a_{i}}=e^{N}\sim\sqrt{\frac{m_{p}H_{f}}{T_{0}T_{eq}}}.$ (6.9)

Plugging this back into (6.7) gives the condition

$H_{f}<(m_{p}T_{0}T_{eq})^{1/3},$ (6.10)

forcing the tensor-to-scalar ratio $r$ to be

$r\sim 10^{8}\left(\frac{H_{f}}{m_{p}}\right)^{2}<10^{-30},$ (6.11)

practically unobservable [39]. In short, if one makes the minimal assumption
that subplanckian modes during inflation cannot cross the horizon,
classicalize, and become observable (TCC), tensor modes are also generically
unobservable and cannot be used to establish the quantization of gravity. Any
model of inflation that predicts observable tensor modes also makes
predictions about physical quantities today that rely on unknown trans-
Planckian physics, and its validity is therefore dubious.

In the first two experiments just reviewed, the obstruction in observing the
quantization of the gravitational field seems to stem from the impossibility
to resolve scales below the Planck length. This fundamental limit in the
precision of distance measurements is itself a consequence of gravitational
collapse in General Relativity [11]: trying to resolve finer distances
requires so much energy that the system is bound to collapse to a black hole.
In a sense then, the parameter space that would enable graviton detection is
always hidden behind the event horizon of a black hole. It is important to
realize, however, that this conclusion relies on the assumption that
gravitational collapse generically results in black holes, i.e. singularities
hidden behind event horizons. This is the content of the (weak) Cosmic
Censorship Conjecture (CCC) [40, 41]. All we can say, then, is that in a
universe in which the CCC is true, graviton detection architectures based on
laser interferometry or graviton absorption are generically inefficient.
Interestingly, in the third experiment we reviewed it is the TCC that, if
true, would impede graviton detection. In this scenario, in place of a black
hole horizon, fluctuations in the trans-Planckian regime are instead hidden
behind the cosmological horizon so as to make them unable to turn classical
and affect macroscopic observations [42].

Therefore, in the graviton detector architectures discussed above the
nonlinear gravitational physics responsible for the formation of an event
horizon conspires to hide the quantization of the gravitational field. Note
then that the Gertsenshtein experiment discussed in §5 is qualitatively
different in that the electromagnetic field plays the role of the co-
conspirator: the nonlinearities of the electromagnetic field destroy the
coherence of the two waves to prevent oscillations at scales below the Hubble
radius. Vacuum polarization is instrumental in securing inefficiency: in its
absence, oscillations would occur inside the cosmological horizon and become
observable.

In fact, if one were to ignore vacuum polarization effects (equivalent to
taking $\alpha=0$ or $m_{e}=m_{p}$) the probability of conversion would simply
be

$P=\sin^{2}(\omega_{s}D),$ (6.12)

with $\omega_{s}=\sqrt{G}B_{0}/2$. Fixing $P=1$, one could then make $D$
arbitrarily small by increasing the strength of the magnetic field $B_{0}$
beyond the Schwinger limit. Graviton-to-photon conversion would happen with
probability 1 at arbitrarily small scales, independently of the frequency of
the incoming graviton. In such a universe, single graviton detection using the
Gertsenshtein effect would in principle be possible in the laboratory.

##  7 Conclusions

It is often assumed that gravity is quantized. Given that the quantization of
gravity does not follow logically from the quantization of matter [1], its
existence can only be experimentally decided. The Cosmic Censorship Conjecture
[40] entails that quantum gravity effects are always hidden behind event
horizons. A particular form of cosmic censorship comes into play in the
detection of single gravitons: Dyson, Rothman and Boughn [13, 14] proposed
several graviton detector architectures and showed they cannot work due to
gravitational collapse.

In this paper, we have studied a graviton detector architecture based on
graviton-to-photon conversion (originally proposed by Dyson) and we have shown
that, for photon frequencies $\omega_{r}$ consistent with current single-
photon detector sensitivities, this architecture is generically inefficient at
detecting gravitons. This is due to the nonlinearities of the electromagnetic
field, which destroy graviton-photon coherence above the scale
$\bar{B}_{0}\sim\sqrt{G}m_{e}^{4}/\alpha^{2}\omega_{r}$. For typical single-
photon detector frequencies, the magnetic field strength is such that
graviton-photon oscillations occur at cosmological scales, and one needs to
take into account the expansion of the universe. A consistent treatment of
graviton-photon oscillations in the cosmological setting reveals that these
always lie outside the Hubble radius and are in principle unobservable [19].

Contrary to other graviton detectors discussed in §6, where the inefficiency
is purely a consequence of the nonlinear gravitational processes that lead to
the formation of an event horizon, the inefficiency of the Gertsenshtein
detector is partly a result of the nonlinearity of the electromagnetic field
in the form of vacuum polarization effects: in a hypothetical scenario with no
vacuum polarization, oscillations would occur well within the cosmological
horizon, making single graviton detection possible. However, while
gravitational collapse constitutes a clear-cut obstruction to graviton
detection, the coherence breaking effect of vacuum polarization makes graviton
detection practically, but not theoretically, impossible. In particular, a
futuristic technology capable of detecting single photons at wavelengths much
larger than the nanometer range of currently existing detectors could in
principle be envisioned. If such frequencies could be achieved, an efficient
detector would be of astrophysical, instead of cosmological, size.

The nonlinearities of the gravitational field cloak quantum gravitational
effects behind event horizons. With the caveats discussed above, in the
Gertsenshtein experiment the nonlinearities of the electromagnetic field force
oscillations to lie beyond the cosmological horizon, so that graviton
detection is just outside the reach of empirical observation. If one assumes
that the inability to detect a single graviton has a deeper origin, for
example if the gravitational field arises as a purely classical entity in the
thermodynamic limit of some microscopic non-gravitational theory [43, 44, 45],
the different obstructions in the two cases could be suggestive of a duality
between electromagnetism and gravity, in the spirit of [46]. This remains the
subject for future investigation.

## Acknowledgement

I would like to thank Tony Rothman for providing invaluable feedback on an
early version of the draft, which greatly helped me clarify the presentation.

##  Appendix A Calculation of $\varepsilon$ and $\mu$ in the strong field
regime

In order to compute the electric permittivity and magnetic permeability in
(2.18), one has to compute the derivatives of (2.7) with respect to $a$ and
$b$. This appendix presents a detailed calculation of these derivatives in the
limit $a\ll 1$ and $b\gg 1$.

Let’s start from the derivative with respect to $a$. In the limit $a\ll 1$,
this is

$\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
a}=\frac{am_{e}^{4}}{12\pi^{2}}\int_{0}^{\infty}\frac{e^{-s}}{s}\left(bs\coth{bs}-1\right)ds.$
(A.1)

Defining $x\equiv bs$, one can write (A.1) as

$\frac{am_{e}^{4}}{12\pi^{2}}\int_{0}^{\infty}e^{-x/b}\left(\coth{x}-\frac{1}{x}\right)dx.$
(A.2)

Each of the two integrals in (A.2) diverges when evaluated separately, but if
one regularizes the integrals by introducing a convergence-enforcing parameter
$\epsilon$, the divergent parts are found to cancel out, leaving a finite
result after summing. That the divergent parts cancel out is a simple example
of renormalization in action.

In order to compute (A.2) we need the following standard integrals:

$\displaystyle\begin{split}&\int_{0}^{\infty}dxx^{m-1}e^{-nx}=n^{-m}\Gamma(m),\\\
&\int_{0}^{\infty}dxx^{m}e^{-nx}\coth{x}=2^{-m-1}\Gamma(1+m)\left[\zeta\left(1+m,\frac{n}{2}\right)+\zeta\left(1+m,\frac{2+n}{2}\right)\right],\end{split}$
(A.3)

where $\Gamma(x)$ is the gamma function, and $\zeta(x,y)$ the generalized
Riemann zeta function.

We regularize the integral (A.2) by multiplying both terms by $x^{\epsilon}$:

$\frac{am_{e}^{4}}{12\pi^{2}}\int_{0}^{\infty}\left(x^{\epsilon}e^{-x/b}\coth{x}-x^{\epsilon-1}e^{-x/b}\right)dx.$
(A.4)

Using (A.3), the integral in (A.4) evaluates to

$\begin{split}&\int_{0}^{\infty}\left(x^{\epsilon}e^{-x/b}\coth{x}-x^{\epsilon-1}e^{-x/b}\right)dx=\\\
&2^{-\epsilon-1}\Gamma(1+\epsilon)\left[\zeta\left(1+\epsilon,\frac{1}{2b}\right)+\zeta\left(1+\epsilon,1+\frac{1}{2b}\right)\right]-b^{\epsilon}\Gamma(\epsilon).\end{split}$
(A.5)

In order to take the limit $\epsilon\rightarrow 0$, we need the following
$\epsilon$ expansions:

$\displaystyle\begin{split}b^{\epsilon}&=1+\epsilon\ln b+\frac{1}{2}(\ln
b)^{2}\epsilon^{2}+\mathcal{O}(\epsilon^{3}),\\\
\Gamma(\epsilon)&=\frac{1}{\epsilon}-\gamma+\frac{1}{12}(6\gamma^{2}+\pi^{2})\epsilon+\mathcal{O}(\epsilon^{2}),\\\
\Gamma(1+\epsilon)&=1-\gamma\epsilon+\frac{1}{12}(6\gamma^{2}+\pi^{2})\epsilon^{2}+\mathcal{O}(\epsilon^{3}),\\\
\Gamma(\epsilon-1)&=-\frac{1}{\epsilon}+\gamma-1+\left(-\frac{\pi^{2}}{12}-\frac{\gamma^{2}}{2}-1+\gamma\right)\epsilon+\mathcal{O}(\epsilon^{2}),\\\
\zeta(\epsilon,h)&=\frac{1}{2}-h+\epsilon\zeta^{\prime}(0,h)+\frac{\epsilon^{2}}{2}\zeta^{\prime\prime}(0,h)+\mathcal{O}(\epsilon^{3}),\\\
\zeta(1+\epsilon,h)&=\frac{1}{\epsilon}-\psi(h)+\mathcal{O}(\epsilon),\\\
\zeta(\epsilon-1,h)&=-\frac{1}{2}h^{2}+\frac{1}{2}h-\frac{1}{12}+\epsilon\zeta^{\prime}(-1,h)+\frac{\epsilon^{2}}{2}\zeta^{\prime\prime}(-1,h)+\mathcal{O}(\epsilon^{3}),\end{split}$
(A.6)

where $\gamma$ is the Euler-Mascheroni constant and $\psi(z)$ the digamma
function, defined as $\psi(x)\equiv\Gamma^{\prime}(x)/\Gamma(x)$.

With the identities above, and in the limit $\epsilon\rightarrow 0$, (A.5)
gives

$\lim_{\epsilon\rightarrow
0}\int_{0}^{\infty}\left(x^{\epsilon}e^{-x/b}\coth{x}-x^{\epsilon-1}e^{-x/b}\right)dx=-\psi\left(\frac{1}{2b}\right)-b-\ln(2b),$
(A.7)

where I have used the identity $\psi(1+x)=\psi(x)+1/x$. As promised, the
divergent pieces $\propto\epsilon^{-1}$ have canceled out.

Similarly, one can compute the derivative with respect to $b$ in the limit
$a\ll 1$. This is

$\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
b}=\frac{bm_{e}^{4}}{12\pi^{2}}\int_{0}^{\infty}\frac{e^{-s}}{s}\left(1-\frac{3}{2}\frac{\coth(bs)}{bs}+\frac{3}{2}\csch^{2}(bs)\right)ds.$
(A.8)

With $x\equiv bs$, and introducing the regulator $x^{\epsilon}$, the integral
becomes

$\frac{bm_{e}^{4}}{12\pi^{2}}\lim_{\epsilon\rightarrow
0}\int_{0}^{\infty}e^{-x/b}\left(x^{\epsilon-1}-\frac{3}{2}x^{\epsilon-2}\coth
x+\frac{3}{2}x^{\epsilon-1}\csch^{2}x\right)dx.$ (A.9)

Using (A.3), we get

$\displaystyle\begin{split}&\frac{bm_{e}^{4}}{12\pi^{2}}\Biggl{[}-\gamma+\ln
b-(1-\gamma)\left(\frac{3}{4b^{2}}+\frac{1}{2}\right)+\left(\frac{3}{4b^{2}}+\frac{1}{2}\right)\ln
2+6\zeta^{\prime}\left(-1,\frac{1}{2b}\right)-\frac{3}{2b}\ln(2b)\\\
&-\gamma\left(\frac{3}{4b^{2}}-\frac{1}{2}\right)-\left(\frac{3}{4b^{2}}-\frac{1}{2}\right)\ln
2+6\zeta^{\prime}\left(-1,\frac{1}{2b}\right)-\frac{3}{b}\ln\Gamma\left(\frac{1}{2b}\right)+\frac{3}{2b}\ln
2\pi\Biggl{]}.\end{split}$ (A.10)

Therefore, the two derivatives are

$\displaystyle\begin{split}\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
a}&=\frac{am_{e}^{4}}{12\pi^{2}}\left(-\psi\left(\frac{1}{2b}\right)-b-\ln
2b\right),\\\ \frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
b}&=\frac{bm_{e}^{4}}{12\pi^{2}}\left(\ln
2b-\frac{3}{4b^{2}}+12\zeta^{\prime}\left(-1,\frac{1}{2b}\right)+\frac{3}{2b}\ln\frac{\pi}{b}-\frac{3}{b}\ln\Gamma\left(\frac{1}{2b}\right)-\frac{1}{2}\right).\end{split}$
(A.11)

In the supercritical limit $b\gg 1$, $\psi(1/2b)\approx-2b$ and
$\Gamma(1/2b)\approx 2b$, therefore (A.11) becomes

$\displaystyle\begin{split}\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
a}&=\frac{am_{e}^{4}}{12\pi^{2}}\left(b-\ln b\right),\\\
\frac{\partial\mathcal{L}_{EM}^{(1)}(a,b)}{\partial
b}&=\frac{bm_{e}^{4}}{12\pi^{2}}\ln b.\end{split}$ (A.12)

## References

  * [1] Bohr, N., and Rosenfeld, L. (1933). On the question of the measurability of electromagnetic field quantities. Quantum theory and measurement, 479.
  * [2] Bronstein, M. P. (2005). Quantentheorie schwacher Gravitationsfelder, Phys. Ztschr. der Sowjetuion, 9, 140–157 (1936)
  * [3] Feynman, R. (2018). Feynman lectures on gravitation. CRC Press.
  * [4] Eppley, K., and Hannah, E. (1977). The necessity of quantizing the gravitational field. Foundations of Physics, 7, 51-68.
  * [5] Page, D. N., and Geilker, C. D. (1981). Indirect evidence for quantum gravity. Physical Review Letters, 47(14), 979.
  * [6] Mari, A., De Palma, G., and Giovannetti, V. (2016). Experiments testing macroscopic quantum superpositions must be slow. Scientific reports, 6(1), 22777.
  * [7] Belenchia, A., Wald, R. M., Giacomini, F., Castro-Ruiz, E., Brukner, Č., and Aspelmeyer, M. (2018). Quantum superposition of massive objects and the quantization of gravity. Physical Review D, 98(12), 126009.
  * [8] Mattingly, J. (2006). Why Eppley and Hannah’s thought experiment fails. Physical Review D, 73(6), 064025.
  * [9] Baym, G., and Ozawa, T. (2009). Two-slit diffraction with highly charged particles: Niels Bohr’s consistency argument that the electromagnetic field must be quantized. Proceedings of the National Academy of Sciences, 106(9), 3035-3040.
  * [10] Rydving, E., Aurell, E., and Pikovski, I. (2021). Do Gedanken experiments compel quantization of gravity?. Physical Review D, 104(8), 086024.
  * [11] Calmet, X., Graesser, M., and Hsu, S. D. (2004). Minimum length from quantum mechanics and classical general relativity. Physical review letters, 93(21), 211101.
  * [12] Krauss, L. M., and Wilczek, F. (2014). Using cosmology to establish the quantization of gravity. Physical Review D, 89(4), 047501.
  * [13] Dyson, F. (2013). Is a graviton detectable?. International Journal of Modern Physics A, 28(25), 1330041.
  * [14] Rothman, T., and Boughn, S. (2006). Can gravitons be detected?. Foundations of Physics, 36, 1801-1825.
  * [15] Boughn, S., and Rothman, T. (2006). Aspects of graviton detection: graviton emission and absorption by atomic hydrogen. Classical and quantum gravity, 23(20), 5839.
  * [16] Gertsenshtein, M. E. (1962). Wave resonance of light and gravitional waves. Sov Phys JETP, 14, 84-85.
  * [17] Ejlli, D., and Thandlam, V. R. (2019). Graviton-photon mixing. Physical Review D, 99(4), 044022.
  * [18] Carney, D., Domcke, V., and Rodd, N. L. (2024). Graviton detection and the quantization of gravity. Physical Review D, 109(4), 044009.
  * [19] Anninos, P., Rothman, T., and Palessandro, A. (2024). Graviton-Photon Oscillations in an Expanding Universe. arXiv preprint arXiv:2401.08346.
  * [20] Palessandro, A., and Rothman, T. (2023). A simple derivation of the Gertsenshtein effect. Physics of the Dark Universe, 40, 101187.
  * [21] Zel’dovich, Y. B. (1973). Electromagnetic and gravitational waves in a stationary magnetic field. Zh. Eksp. Teor. Fiz, 65, 1311.
  * [22] Weisskopf, V. (1936). On the electrodynamics of the vacuum on the basis of the quantum theory of the electron. Kong. Dans. Vid. Selsk. Mat. Fys. Medd, 14, 24.
  * [23] Anninos, P. (1998). Plane-symmetric cosmology with relativistic hydrodynamics. Physical Review D, 58(6), 064010.
  * [24] Tsagas, C. G. (2004). Electromagnetic fields in curved spacetimes. Classical and Quantum Gravity, 22(2), 393.
  * [25] Berestetskii, V. B., Lifshitz, E. M., and Pitaevskii, L. P. (1982). Quantum Electrodynamics: Volume 4 (Vol. 4). Butterworth-Heinemann.
  * [26] Domcke, V., and Garcia-Cely, C. (2021). Potential of radio telescopes as high-frequency gravitational wave detectors. Physical review letters, 126(2), 021104.
  * [27] Heisenberg, W., and Euler, H. (1936). Folgerungen aus der diracschen theorie des positrons. Zeitschrift für Physik, 98(11-12), 714-732.
  * [28] Dittrich, W., and Gies, H. (2000). Probing the quantum vacuum: perturbative effective action approach in quantum electrodynamics and its application (Vol. 166). Springer Science and Business Media.
  * [29] Lundin, J. (2010). QED and collective effects in vacuum and plasmas (Doctoral dissertation, Umeå universitet. Institutionen för fysik).
  * [30] Kim, C. M., and Kim, S. P. (2021). Magnetars as laboratories for strong field QED. arXiv preprint arXiv:2112.02460.
  * [31] Medvedev, M. V. (2023). Plasma modes in QED super-strong magnetic fields of magnetars and laser plasmas. Physics of Plasmas, 30(9).
  * [32] Anastassopoulos, V., A. Liolios, J. I. Collar, C. J. Hailey, H. Bräuninger, Igor Garcia Irastorza, A. Dermenev et al. ”arXiv: New CAST Limit on the Axion-Photon Interaction.” Nature Phys. 13, no. arXiv: 1705.02290 (2017): 584-590.
  * [33] Natarajan, C. M., Tanner, M. G., and Hadfield, R. H. (2012). Superconducting nanowire single-photon detectors: physics and applications. Superconductor science and technology, 25(6), 063001.
  * [34] Bai, P., Zhang, Y. H., and Shen, W. Z. (2017). Infrared single photon detector based on optical up-converter at 1550 nm. Scientific Reports, 7(1), 15341.
  * [35] Palessandro, A., and Sloth, M. S. (2020). Gravitational absorption lines. Physical Review D, 101(4), 043504.
  * [36] Martin, J., and Brandenberger, R. H. (2001). Trans-Planckian problem of inflationary cosmology. Physical Review D, 63(12), 123501.
  * [37] Niemeyer, J. C. (2001). Inflation with a Planck-scale frequency cutoff. Physical Review D, 63(12), 123502.
  * [38] Bedroya, A., and Vafa, C. (2020). Trans-Planckian censorship and the swampland. Journal of High Energy Physics, 2020(9), 1-34.
  * [39] Bedroya, A., Brandenberger, R., Loverde, M., and Vafa, C. (2020). Trans-Planckian censorship and inflationary cosmology. Physical Review D, 101(10), 103502.
  * [40] Penrose, R. (1969). Gravitational collapse: The role of general relativity. Nuovo Cimento Rivista Serie, 1, 252.
  * [41] Penrose, R. (1973). Naked singularities. Annals of the New York Academy of Sciences, 224(1), 125-134.
  * [42] Brahma, S. (2020). Trans-Planckian censorship, inflation, and excited initial states for perturbations. Physical Review D, 101(2), 023526.
  * [43] Jacobson, T. (1995). Thermodynamics of spacetime: the Einstein equation of state. Physical Review Letters, 75(7), 1260.
  * [44] Padmanabhan, T. (2010). Thermodynamical aspects of gravity: new insights. Reports on Progress in Physics, 73(4), 046901.
  * [45] Verlinde, E. (2011). On the origin of gravity and the laws of Newton. Journal of High Energy Physics, 2011(4), 1-27.
  * [46] Maldacena, J. (1999). The large-N limit of superconformal field theories and supergravity. International journal of theoretical physics, 38(4), 1113-1133.


# GraspCaps: Capsule Networks Are All You Need
for Grasping Familiar Objects

Tomas van der Velde, Hamidreza Kasaei All authors are with the Department of
Artificial Intelligence, University of Groningen, The Netherlands
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

As robots become more accessible outside of industrial settings, the need for
reliable object grasping and manipulation grows significantly. In such dynamic
environments it is expected that the robot is capable of reliably grasping and
manipulating novel objects in different situations. In this work we present
GraspCaps: a novel architecture based on Capsule Networks for generating per-
point grasp configurations for familiar objects. In our work, the activation
vector of each capsule in the deepest capsule layer corresponds to one
specific class of object. This way, the network is able to extract a rich
feature vector of the objects present in the point cloud input, which is then
used for generating per-point grasp vectors. This approach should allow the
network to learn specific grasping strategies for each of the different object
categories. Along with GraspCaps we present a method for generating a large
object grasping dataset using simulated annealing. The obtained dataset is
then used to train the GraspCaps network. We performed an extensive set of
experiments to assess the performance of the proposed approach regarding
familiar object recognition accuracy and grasp success rate on challenging
real and simulated scenarios.

##  I INTRODUCTION

As robots become more and more accessible outside of industrial settings
(e.g., homes, hospitals, shopping malls), the need for reliable object
grasping and manipulation grows significantly. In such dynamic environments,
it is expected that the robot is capable of reliably grasping and manipulating
novel objects in different situations (see Fig. 1).

A large body of recent efforts for object grasping has focused on solving 4DoF
$(x,y,z,yaw)$ object-agnostic grasping, where the gripper is forced to
approach objects from above. The major drawbacks of these approaches are that
(1) they do not take into account the semantic function or label of the
object, and (2) they inevitably limit the interaction possibilities with the
object. For instance, they are not able to distinguish objects from each other
and are not able to grasp a horizontally placed plate. These limitations
motivate the study of “learning to grasp familiar objects”, where the robot is
able to recognize the label of the object and its gripper is free to approach
objects from any arbitrary direction it can reach.

Towards this end, we formulate object grasping as a supervised learning
problem to grasp familiar objects. The primary assumption is that new objects
which are geometrically similar to known objects can be grasped in similar
ways using object-aware grasping [1, 2]. Object aware grasping allows the
network to specifically generate grasps based on the features and geometry of
the specific object it is attempting to grasp, as opposed to object agnostic
grasping, which generates grasp vectors based on the input to the network
without any deeper knowledge on the features of the object it is attempting to
grasp. Since capsule networks are able to extract intrinsic parameters of the
input data, we’ve designed a novel architecture called GraspCaps, that
receives a point cloud of the object as an input and generates per-point grasp
configurations and a single semantic category label as outputs. Our approach
takes the activation of one capsule in the capsule network and processes that
activation to generate per-point grasp vectors and fitness values. The network
architecture will be explained in more detail in Section III. To the best of
our knowledge, this is the first network architecture utilizing a capsule
network for object-aware grasp synthesis. The contributions of this thesis can
be summarized as follows:

Figure 1: In this example, a service robot is instructed to perform a cleaning
the table task. In each execution cycle, the robot needs to process the input
point cloud, predict reliable grasp configurations, and finally grasp and
place objects into the baskets to accomplish this task successfully. Figure 2:
Diagram of the GraspCaps architecture. The architecture is divided into
several modules: the Feature extraction module, the Capsule module, and the
Grasping module. The object classification is determined by looking at the
activation of each capsule. The capsule with the highest activation is chosen
as the winner, and the corresponding object is determined to be the object in
the input data. Layers shown in yellow are fully-connected layers using Leaky
ReLU activation. Layers shown in red use sigmoidal activation. The output of
the $1024\times 4$ grasping head is normalized to obtain a unit quaternion,
which is not shown in the diagram. Output dimensions of the network are shown
in rectangular blocks with dotted outline.

  * •

We propose a novel architecture for object-aware grasping that receives a
point cloud of an object as input and produces a semantic category label and
point-wise grasp synthesis as outputs. To the best of our knowledge, this is
the first grasping model based on the capsule network.

  * •

We propose an algorithm for generating 6D grasp vectors from point clouds and
create a synthetic grasp dataset consisting of $4,576$ samples with
corresponding object labels and target grasp vectors.

  * •

We perform extensive sets of experiments in both simulated and real-robot
settings to validate the performance of the proposed approach.

##  II Related Work

Current research into processing point clouds can be split up in two
categories: (i) approaches that first transform the point cloud into a
different data structure [3, 4, 5], (ii) approaches that directly process the
point cloud [6, 7, 8, 9, 10]. Our method falls into the second category.

Processing the point set directly has a number of advantages, since no
overhead is added by transforming the point set, and there is no chance of any
information loss in the conversion. However, point sets are by definition
unordered, which makes extracting local structures and identifying similar
regions non-trivial. PointNet [6] was one of the first architectures to
effectively use point set data for training a neural network in an object
recognition task. By design the PointNet architecture is mostly invariant to
point order, which benefits point sets since extracting a natural order from
these sets is non-trivial. However, this does limit the performance of
PointNet as it cannot recognize local structures in point sets. In prior
research the importance of order in data for the performance of neural
networks has been illustrated [11], hence order should not be fully
disregarded. PointNet++ [7] improves upon PointNet by recognizing local
structures in the data. Our network architecture is based in part on the
architecture used by [12], which makes the insight to split up the PointNet
architecture into several distinct modules.

Later research showed successful results working with point sets by
transforming the point set to be processed by a convolutional neural network.
PointCNN [8] preprocesses the input data by applying a $\chi$-transform on the
point set. DGCNN [10] and Point-GNN [9] employ layer architectures that
transform the point set into a graph representation and apply convolution to
the resulting graph edges.

Several approaches have been successful in processing point sets using a CNN
by first transforming the point set into a more regular data structure, such
as a 3D voxel grid [5], top-down view [3, 13, 14], or multi-view 2D images
[4]. The resulting data structures can be processed with existing deep neural
network architectures. These conversions come with significant limitations
however, as there is a sizable loss in information when converting the point
cloud to a different structure, whether that be in the form of losing natural
point densities when converting to a voxel grid, or the loss of spatial
relations between points when converting to a top-down image. Additionally,
the generated voxel grids might be more voluminous than the original point
set, as it is likely that many of the voxels remain empty [6]. Due to these
considerations we decided to base the GraspCaps architecture in part on the
architecture proposed by [10], as it is able to process the point cloud
directly, and the graph representation obtained during execution is able to
retain the spatial relation between points present in the input point cloud.

In the field of grasp generation, S4G [15] extended the PointNet architecture
to generate 6D grasps based on the input point set. Grasp pose detection (GPD)
[16] was developed to generate and evaluate the fitness of grasps. It takes a
point cloud as its input and generates a number of grasps which are then
filtered on fitness. The network then classifies the grasp candidate as either
successful, or unsuccessful. PointNetGPD [17] builds upon the idea of GPD and
expands on it by employing the PointNet architecture to evaluate the fitness
of grasps. GraspNet [18] uses a variable autoencoder to generate a set of
grasps for an object using a point cloud. It evaluates the fitness of the
generated grasps using an encoder-decoder network. It is able to generate
grasps with a success rate comparable to PointNetGPD.

In contrast to GPD-based approaches, our method generates point-wise grasp
configurations and is able to classify familiar objects.

##  III NETWORK ARCHITECTURE

As shown in Fig. 2, the GraspCaps architecture consists of three main modules:
the feature extraction module, the capsule module, and the grasp synthesis
module. The Feature Extraction Module takes a normalized $1024\times 3$
dimensional point cloud as its input, and processes the point cloud using the
feature extraction module from the Dynamic graph CNN (DGCNN) [10] network to
generate a $1024\times 1$ dimensional feature vector. Specifically, it
processes a point cloud by use of four EdgeConv layers. An EdgeConv layer
first transforms the point cloud into $N$ undirected local graphs of $k$
nodes, where $N$ is the number of points in the input point cloud, and $k$ is
an integer representing the number of closest neighbouring points that are
used for constructing the local graph. The network then applies a shared
multi-layer perceptron (MLP) on the edges of the graphs. This process is
repeated for each of the four EdgeConv layers in the feature extraction
module. The output of all five EdgeConv layers is concatenated and processed
using a shared MLP and pooled to a $1024\times 1$ feature vector using an
adaptive max-pool operation.

The Capsule Module of our network consists of a single fully connected layer,
a primary capsule layer, and a secondary capsule layer. The fully connected
layer consists of 256 neurons using sigmoidal activation that transform the
feature vector for use in the primary capsule layer. The primary capsule layer
consists of 16 capsules, each containing 4 neurons. The secondary capsule
layer consists of 8 capsules each containing 30 neurons, with each capsule
corresponding to one specific shape class of object. The primary and secondary
capsule layers are connected through routing by agreement [19]. Capsules in
deeper layers encode more meaningful features. Our network is trained such
that each capsule in the deepest layer corresponds to one class of object. To
determine the class of the object, the Euclidean norms of the activations in
the secondary capsule layer are ranked. The capsule with the highest norm is
then chosen as the winner and its activation is used for further processing in
the next module.

The Grasping Module consists of four fully-connected heads. Their architecture
is based on the architecture of the reconstruction network as described by
[19], each containing three fully connected layers. The first two layers
consist of 512 and 1024 neurons respectively, using leaky ReLU activation. The
third layer size is dependent on the output shape of the network and uses
linear activation. The reconstruction head has an output shape of $1024\times
3$ to generate point locations in 3D space. The grasp head has an output shape
of $1024\times 4$ to generate quaternion rotation vectors for each point. The
quality and width heads both have an output shape of $1024\times 1$, as they
only need to output a scalar per point. The output of the grasping head is
normalized to obtain a unit quaternion vector.

All parameters were chosen by performing extensive parameter sweeps.

###  III-A Loss functions

As the network performs several tasks at once, a custom loss function was
written. The loss function can be broken down into three main components:
Margin loss ($\ell_{\text{margin}}$) [19], Reconstruction loss
($\ell_{\text{recon}}$) [19], and Grasping loss ($\ell_{\text{grasp}}$).

Margin loss is used to train the capsule activation to be representative of
the class it encapsulates:

$\displaystyle\ell_{\text{margin}}(\mathbf{v}_{i})=$ $\displaystyle
T_{i}\max(0,m^{+}-||\mathbf{v}_{i}||)^{2}+$ (1)
$\displaystyle\lambda(1-T_{i})\max(0,||\mathbf{v}_{i}||-m^{-})^{2}$

where $L$ is the loss, and $T_{i}$ is the existence of the object the capsule
corresponds to in the input data, which is set to $1$ if the object is
present, and $0$ if the object is absent. The $\lambda$ used is a constant
scaling factor that is set to $0.5$. It is used to scale down the shrinking of
the activity vectors in the event that the output of the capsule is incorrect,
which stops the initial learning from shrinking the activation vectors [19].
$m^{+}$ and $m^{-}$ are constants set to $0.9$ and $0.1$ respectively.
$\mathbf{v}_{i}$ is the activation vector of capsule $i$. By calculating the
Euclidean norm, $||\mathbf{v}_{i}||$, we obtain a scalar that can be treated
as the probability that the corresponding class is present in the input.

Reconstruction loss is used as a regularizer that ensures that the capsule
learns information relevant its corresponding object class. In our
architecture it is defined as the mean squared error loss between the input
point set and the reconstructed point set. It is scaled down by a factor
$\beta$ such that it doesn’t overtake the other losses and functions only as a
regularizer.

The grasping loss is based on the loss function defined in [5], which is
composed of separate loss functions for rotation, grasp quality, and the grasp
width. The first alteration that was made from the loss function is the
inclusion of $T_{i}$, which behaves identical to the $T_{i}$ described in the
margin loss above. Including this factor in the grasping loss ensures that the
grasping network does not train on capsule activations that do not correspond
to the correct object. The second alteration from the loss function is the
definition of the quality loss $\ell_{\text{quality}}(q_{i},\hat{q}_{i})$,
which we extend to allow any real-valued ground truth value $q_{i}\in[0,1]$
instead of the binary ground truth $q_{i}\in\\{0,1\\}$ used by [5]. Using this
alteration the network is able to use $q_{i}$ like a fitness value, and
distinguish usable but imperfect grasps from the best-fitting grasps. Bad or
colliding grasps are heavily penalized during data generation, so their values
are relatively small, which reduces the risk of the network training on bad
grasps. The grasping loss is defined as:

$\displaystyle\ell_{\text{grasp}}(\mathbf{g_{i}},\mathbf{\hat{g}}_{i})=$
$\displaystyle T_{i}[\ell_{\text{quality}}(q_{i},\hat{q}_{i})+$ (2)
$\displaystyle
q_{i}(\ell_{\text{rotation}}(\mathbf{r}_{i},\mathbf{\hat{r}}_{i})+\alpha\ell_{\text{width}}(w_{i},\hat{w}_{i}))]$

where both $\ell_{\text{quality}}$ and $\ell_{\text{width}}$ are defined as
the mean squared error loss. The rotation loss is defined in Equation (3).
$\alpha$ is a constant set to $0.001$. As the end-effector of the robotic arm
we are using is a symmetrical two-fingered gripper, a grasp vector rotated by
$180^{\circ}$ around the grippers wrist results in effectively the same grasp
[5]. Therefore, we define the rotational loss function to consider both grasps
correct:

$\ell_{\text{rotation}}(\mathbf{r},\mathbf{\hat{r}})=\min(\ell_{\text{quat}}(\mathbf{r},\mathbf{\hat{r}}),\ell_{\text{quat}}(\mathbf{r}\pi,\mathbf{\hat{r}}))$
(3)

where
$\ell_{\text{quat}}(\mathbf{r},\mathbf{\hat{r}})=1-|\mathbf{r}\cdot\mathbf{\hat{r}}|$
is defined to use the inner product to calculate the distance between the
ground truth rotation $\mathbf{r}$ and the generated rotation
$\mathbf{\hat{r}}$. The loss function that is used for training the network
incorporates all of these loss functions as follows:

$\mathcal{L}(\mathbf{t}_{i},\mathbf{\hat{t}}_{i})=\ell_{\text{margin}}(\mathbf{v}_{i})+\beta\ell_{\text{recon}}(\mathbf{p}_{i},\mathbf{\hat{p}}_{i})+\ell_{\text{grasp}}(\mathbf{g_{i}},\mathbf{\hat{g}}_{i})$
(4)

where $\mathbf{t}_{i}$ is the output of the network, consisting of the capsule
output $\mathbf{v}_{i}$, the reconstruction of the input point set
$\mathbf{p}_{i}$, and the generated grasp $\mathbf{g}_{i}$.
$\mathbf{\hat{t}}_{i}$ is the target vector, composed of the input point set
$\mathbf{\hat{p}}_{i}$, and the target grasp $\mathbf{\hat{g}}_{i}$. $\beta$
is a scaling constant set to $0.0005$.

##  IV Generating Grasp Synthesis

Due to the niche our network exists in (simultaneous recognition and grasping
on a point cloud), no existing benchmark datasets were found that met our
requirements, so instead we created our own. In particular, to generate a
large dataset with valid grasp configurations as targets, an algorithm was
needed to create valid grasps based on point cloud data.

Figure 3:  During the dataset generation process, only the orientation and
opening width of the gripper are altered. The location of the grasp center is
determined by choosing a random point in the point cloud before converging
using simulated annealing.

In this work, the gripper is simulated as a combination of three separate
boxes: two for the fingers, and one for the base (see Fig. 3). Initially, we
randomly place an object inside the workspace of the robot and capture the
object’s point cloud. The point cloud of the object is then converted to a
watertight mesh for more efficient collision checking. The original point
cloud is retained to calculate the fitness of grasps. The algorithm then
initializes the grasp in a random rotation, either with the gripper facing
down towards the object or facing the object from the side. After
initializing, the orientation and opening width of the gripper are iteratively
updated to converge on a well-fitting grasp. A new state is accepted if the
grasp has a higher fitness than the current state, or by random chance using
simulated annealing [20].

In simulated annealing, the algorithm starts out with a high temperature. This
temperature corresponds with a high chance of accepting new states, even if
they have a lower fitness than the current state. The temperature of the
system linearly decreases during execution and worse states are less and less
likely to be accepted. In the final iterations simulated annealing behaves
almost identical to stochastic gradient descent. The high temperature at the
start of the process allows the algorithm to escape local maxima and converge
to the global optimal solution, while the low temperature at the end of the
process ensures that the algorithm converges. In terms of grasp generation it
allows the system to escape usable but imperfect grasps and reach better
grasps. The only state transition that is not able to occur even when
accepting random state transitions is to go from a non-colliding state to a
colliding state. We have also considered bounds on the rotation to ensure the
algorithm does not generate grasps that are difficult for the robotic arm to
reach.

The fitness of each state is determined by five factors. The first factor
considers what percentage of points is located between the two fingers of the
gripper. This should increase the likelihood that the algorithm converges on
grasping a large part of the object. The second factor determines the minimum
distance between a finger and the points between the fingers. It does this for
each finger. The third factor looks at how closely the normals of the fingers
of the gripper overlap with the normals of the object. If the normals are
completely opposed to each other the gripper is perpendicular to the object,
and the grasp should have a high chance of success. The fourth factor is
defined as the distance between the two grippers. Since the grasp should be as
tight as possible, a small penalty is added to the fitness for the distance
between the two fingers. The last factor adds a base fitness based on the
distance of the grasping point to the center of the object. The closer the
point is to the center of the object, the higher its score. Examples of
generated grasp synthesis for different objects are shown in Fig. 3.

##  V Experimental Results

###  V-A Grasp dataset

|
---|---
Figure 4:  Our experimental setup in simulation and real robot: (left) $90$
simulated objects semantically grouped in the simulated Gazebo environment;
(right) The real robot setup as used for the experiments. Placed in front of
the robot are all items used to test the network.

As shown in Fig. 4, we used $90$ simulated household objects to generate a
large synthetic grasp dataset [21]. Furthermore, to address the familiar
object recognition task, we grouped all objects into 8 distinct object
categories based on shape similarities, e.g. milk cartons are grouped with
juice cartons and other boxes. The obtained synthetic dataset is composed of
$4,576$ samples ($572$ per class). All samples are made by placing an object
with random orientation and location in Gazebo [22] and taking a single sensor
measurement using a simulated Microsoft Kinect camera to obtain a point cloud.

In order to train the grasp-generation section of the network efficiently, 120
grasp points are generated per object centered on a random points in the point
cloud. Each generated grasp consists of a rotation in quaternion format, a
quality measurement, and the opening width of the gripper. By effectively
generating two datasets, we are able to first train the classification section
of the network, after which we train the grasping section. This resulted in
better performance than training the network on a single one-sample-one-grasp
dataset in an end-to-end manner.

###  V-B Ablation Studies

Through a series of ablation experiments, we investigate the impact the
capsule module has on familiar object recognition and grasping. As a baseline,
we considered a network based on the GraspCaps architecture, but lacking the
capsule layer present in GraspCaps. This allows the network to generate object
agnostic grasps and allows us to observe whether capsule networks are a viable
method of capturing the information needed to generate usable object aware
grasps. The classification module is based on the architecture described in
[10]. Both networks were first trained on the generated dataset of 8 classes
for 300 epochs, after which they had their weights frozen in the feature
extraction and classification modules and were trained on the 120 grasp per
sample dataset for an additional 200 epochs. Networks trained on the real-life
dataset were first pre-trained on the synthetic dataset for 300 epochs.

The network was trained with batches of 16 point clouds, and optimized using
the ADAM optimizer [23] with a constant learning rate of 0.00001 ($10^{-5}$).
Each of the objects’ point cloud is pre-processed by centering it around the
point $\operatorname{x}=0$, $\operatorname{y}=0$, and normalizing the data to
the range $[-1,1]^{3}$. During training, the point cloud is augmented by
adding random noise from a normal distribution in the range $[-0.005,0.005]$.
This emulates the sensor noise often present in the real world and should let
the network generalize better to unseen data.

The swift increase in grasp accuracy shows that both networks quickly converge
to satisfactory performance. The most interesting part of these results is
comparing the grasping accuracy on the synthetic dataset with the results
obtained on the real-world dataset; in the synthetic dataset, both networks
converge to approximately 94% accuracy, with GraspCaps outperforming the
ablated network by a difference of less than 1%. When looking at the real-
world dataset results however, the difference is much more pronounced. Again,
GraspCaps achieves an accuracy of 94% while the ablated network converges to a
significantly lower accuracy of 89%. This is a strong indication that the
GraspCaps architecture has a higher generalization ability than the ablated
network. The real dataset is more noisy than the synthetic dataset, so the
results indicate that the GraspCaps architecture is more resistant to varying
input data and noise than the ablated network, leading to superior performance
in the real world.

###  V-C Results in Simulation

To evaluate the performance of the proposed approach in a robotic grasping
scenario, we integrated the GraspCaps architecture into our dual-arm robot
setup in the Gazebo simulator. The robot will be required to identify the
class of the object present, and generate a grasp configuration that can be
used to successfully pick up the object. The robot will then pick up the
object and attempt to place it into the basket. To execute the grasp, we
select the arm that is closest to the object.

|
---|---
Figure 5: Left: Raw output of the fitness head of the GraspCaps network on an
object from the vertical cylinder class. Right: The output when smoothed using
our Gaussian filter.

We increase the probability of correctly classifying the object in the input
point set by pre-processing the input. We take the full point set of the
object, which is often well over the required 1024 points, and take multiple
permutations of 1024 points to give to the network. This decreases the chance
of the network miss-classifying an object due to an unfortunate selection of
points. The classification of the object is determined by a majority vote on
the neural network output. In the event of a tie the algorithm is re-run until
consensus is reached. The output of the network will be post-processed by
smoothing the output of the fitness head of the network to identify regions
with high fitness that are suitable for grasping. An example of the effect of
the smoothing can be found in Fig. 5.

We found that, although both the GraspCaps network and the comparison network
achieved an equal classification performance of 80%, the GraspCaps network
outperformed the ablated comparison network in terms of grasping success with
percentages of 71% for the GraspCaps network versus 53% for the ablated
network.

###  V-D Performance on novel objects

Figure 6: Set of novel objects shown in the Gazebo.

To test the generalizability of the network, we tested the network performance
on a set of seven novel objects from the 3DGEMS dataset [24] shown in Fig. 6.

TABLE I: Performance of the GraspCaps on novel objects.  Object | Classification | Grasping success
---|---|---
label | frequency
Bottle | V Cylinder | 5/5 (100%) | 4/5 (80%)
Clock | Hardware | 2/5 (40%) | 5/5 (100%)
| Box | 2/5 (40%) |
| V Object | 1/5 (20%) |
Mouse | Box | 5/5 (100%) | 5/5 (100%)
Beer Mug | V Object | 3/5 (60%) | 4/5 (80%)
| V Cylinder | 1/5 (20%) |
| Hardware | 1/5 (20%) |
Tea box | Box | 5/5 (100%) | 1/5 (20%)
Tissue box | Hardware | 3/5 (60%) | 2/5 (40%)
| Box | 2/5 (40%) |
Hammer | H Cylinder | 5/5 (100%) | 4/5 (80%)
Average | — | — | 25/35 (71%)
TABLE II: Performance of ablated network on novel objects.  Object | Classification | Grasping success
---|---|---
label | frequency
Bottle | V Cylinder | 3/5 (60%) | 3/5 (60%)
| V Object | 2/5 (40%) |
Clock | Hardware | 3/5 (60%) | 5/5 (100%)
| Box | 1/5 (20%) |
| V Object | 1/5 (20%) |
Mouse | Box | 2/5 (40%) | 5/5 (100%)
| V Object | 2/5 (40%) |
| Box | 1/5 (20%) |
Beer Mug | V Cylinder | 3/5 (60%) | 1/5 (20%)
| V Object | 1/5 (20%) |
| Hardware | 1/5 (20%) |
Tea box | Box | 5/5 (100%) | 1/5 (20%)
Tissue box | Box | 4/5 (80%) | 2/5 (40%)
| Hardware | 1/5 (20%) |
Hammer | H Cylinder | 5/5 (100%) | 3/5 (60%)
Average | — | — | 20/35 (57%)

A number of the novel objects, such as the glass bottle and the tea box, can
be easily sorted into the classes the network was trained on. Other objects
however, such as the computer mouse and digital clock, have more abstract
shapes and may be difficult to sort into the existing classes. For these
objects specifically it will be very interesting to see how the network will
adapt and if it will be able to successfully grasp the objects.

Both the GraspCaps architecture and the ablated architecture will attempt to
grasp each of the objects five times to get a clear indication of the
generalization capabilities of both networks. The experimental setup and task
will be equal to the setup in Gazebo described earlier this section. When
comparing the results of the GraspCaps network shown in Table I with the
results of the ablated network in Table II it can be seen that the GraspCaps
network outperforms the ablated network with an average of 71% grasping
accuracy for the GraspCaps architecture compared to an average of 57% for the
ablated network.

During the experiments on grasping the tissue and tea boxes, both networks
generated grasp configurations that pierced the box with one of the gripper
fingers in the majority of attempts, which caused the grasp to fail. The
grasps that were generated on the boxes were in most cases correctly oriented,
with the grippers parallel to the sides of the box. There is some sense to the
generated grasps piercing the tissue box, as there is a significant opening in
the center that could fit one of the fingers of the gripper, which the network
could have noticed. An explanation for the tea box is harder to find. One
possibility is that, since the tea box is rather large and has equally sized
sides, both networks were unable to find grasp configurations that did not
collide with the box. Adding large cubic objects to the dataset could
potentially solve this problem. As can be seen in Tables I and II, all other
objects had relatively high grasping success rates during evaluation of the
network. This indicates that on average the network generalizes well for novel
objects.

###  V-E Results on robotic platform

In this round of experiments, we design a pick and place scenario to evaluate
the performance of the proposed approach in real-world scenarios. We
integrated the GraspCaps into our dual arm setup [25][1][26], and instructed
the robot to perform the pick and place task. The GraspCaps network has been
tested in a real world setting and has been found to perform well on objects
that exist in its dataset, as well as on a number of novel objects. Three
examples of grasping objects in various situations are shown in Fig. 7.
Experimental results showed that the network was able to accurately pick up
and place objects in the basket in both isolated scenario (Fig. 7 left and
center) and pile of objects (Fig. 7 right), where a number of objects were
placed on a table with random rotation and significant overlap, causing a
number of objects to be partially or fully obscured from view. The network
successfully picked up the objects in the pile with correct orientation of the
gripper and was able to drop objects of multiple shapes into the basket
without collision. A video demonstrating the performance of the GraspCaps
network in both isolated object scenarios and a pile scenario has been
attached to the paper as a supplementary
material111https://youtu.be/duuEDnk6HNw. The video contains both successful
and unsuccessful attempts, as well as novel objects that were not present in
the training data.

Figure 7:  In both grasping isolated (left and center) and pile of objects
(right) scenarios, the robot was able to grasp the target object and put it
into the basket successfully.

##  VI CONCLUSIONS

We have presented GraspCaps, a novel simultaneous recognition and grasping
model based on capsule network. Along with GraspCaps we have presented a new
dataset for simultaneous recognition and grasping on point cloud data and an
algorithm for generating grasps using simulated annealing. GraspCaps has been
found to perform well in both simulated and real-life settings on a variety of
familiar and novel objects. For future research it would be interesting to
extend the network with an additional head that generates an affordance mask,
such that the network can be trained to grasp objects only at specific parts,
for example, grasping the handles of a pan, or the handle of a knife.

## References

  * [1] S. H. Kasaei, N. Shafii, L. S. Lopes, and A. M. Tomé, “Interactive open-ended object, affordance and grasp learning for robotic manipulation,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 3747–3753.
  * [2] N. Shafii, S. H. Kasaei, and L. S. Lopes, “Learning to grasp familiar objects using object view recognition and template matching,” in _2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2016, pp. 2895–2900.
  * [3] K. Santhakumar and H. Kasaei, “Lifelong 3d object recognition and grasp synthesis using dual memory recurrent self-organization networks,” _Neural Networks_ , vol. 150, pp. 167–180, 2022.
  * [4] H. Kasaei and M. Kasaei, “Mvgrasp: Real-time multi-view 3d object grasping in highly cluttered environments,” _arXiv preprint arXiv:2103.10997_ , 2021\.
  * [5] M. Breyer, J. J. Chung, L. Ott, R. Siegwart, and J. Nieto, “Volumetric grasping network: Real-time 6 dof grasp detection in clutter,” _arXiv preprint arXiv:2101.01132_ , 2021.
  * [6] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 652–660.
  * [7] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” _arXiv preprint arXiv:1706.02413_ , 2017.
  * [8] Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen, “Pointcnn: Convolution on $\chi$-transformed points,” _Advances in neural information processing systems_ , vol. 31, pp. 820–830, 2018.
  * [9] W. Shi and R. Rajkumar, “Point-gnn: Graph neural network for 3d object detection in a point cloud,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 1711–1719.
  * [10] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, “Dynamic graph cnn for learning on point clouds,” _Acm Transactions On Graphics (tog)_ , vol. 38, no. 5, pp. 1–12, 2019.
  * [11] O. Vinyals, S. Bengio, and M. Kudlur, “Order matters: Sequence to sequence for sets,” _arXiv preprint arXiv:1511.06391_ , 2015.
  * [12] A. Cheraghian and L. Petersson, “3dcapsule: Extending the capsule architecture to classify 3d point clouds,” in _2019 IEEE Winter Conference on Applications of Computer Vision (WACV)_. IEEE, 2019, pp. 1194–1202.
  * [13] S. Kumra, S. Joshi, and F. Sahin, “Gr-convnet v2: A real-time multi-grasp detection network for robotic grasping,” _Sensors_ , vol. 22, no. 16, p. 6208, 2022.
  * [14] D. Morrison, P. Corke, and J. Leitner, “Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach,” _arXiv preprint arXiv:1804.05172_ , 2018.
  * [15] Y. Qin, R. Chen, H. Zhu, M. Song, J. Xu, and H. Su, “S4g: Amodal single-view single-shot se (3) grasp detection in cluttered scenes,” in _Conference on robot learning_. PMLR, 2020, pp. 53–65.
  * [16] A. ten Pas, M. Gualtieri, K. Saenko, and R. Platt, “Grasp pose detection in point clouds,” _The International Journal of Robotics Research_ , vol. 36, no. 13-14, pp. 1455–1473, 2017.
  * [17] H. Liang, X. Ma, S. Li, M. Görner, S. Tang, B. Fang, F. Sun, and J. Zhang, “Pointnetgpd: Detecting grasp configurations from point sets,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 3629–3635.
  * [18] A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational grasp generation for object manipulation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 2901–2910.
  * [19] S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic routing between capsules,” _arXiv preprint arXiv:1710.09829_ , 2017.
  * [20] S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi, “Optimization by simulated annealing,” _science_ , vol. 220, no. 4598, pp. 671–680, 1983.
  * [21] H. Kasaei and S. Xiong, “Lifelong ensemble learning based on multiple representations for few-shot object recognition,” _arXiv preprint arXiv:2205.01982_ , 2022.
  * [22] N. Koenig and A. Howard, “Design and use paradigms for gazebo, an open-source multi-robot simulator,” in _2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No. 04CH37566)_ , vol. 3. IEEE, 2004, pp. 2149–2154.
  * [23] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [24] A. Rasouli and J. K. Tsotsos, “The effect of color space selection on detectability and discriminability of colored objects,” _arXiv preprint arXiv:1702.05421_ , 2017.
  * [25] S. H. Kasaei, M. Oliveira, G. H. Lim, L. S. Lopes, and A. M. Tomé, “Towards lifelong assistive robotics: A tight coupling between object perception and manipulation,” _Neurocomputing_ , vol. 291, pp. 151–166, 2018\.
  * [26] H. Kasaei, S. Luo, R. Sasso, and M. Kasaei, “Simultaneous multi-view object recognition and grasping in open-ended domains,” _arXiv preprint arXiv:2106.01866_ , 2021.