# Diagnostic Tomography of Applied Holography

D. V. Khveshchenko  Department of Physics and Astronomy, University of North
Carolina, Chapel Hill, NC 27599

###### Abstract

The single-particle behavior in $d\geq 1$-dimensional Fermi gases with a large
number $N$ of species and strong short-range $s$-wave scattering is discussed
in the $2d$ ’tomographic’ framework of a (pseudo)holographic correspondence
with a certain $3d$ gravity of the $AdS_{3}$ type. However, due to the
intrinsically topological nature of such a bulk theory its dynamics reduces to
a purely boundary one and so, akin to its $SYK/AdS_{2}$ counterpart, this
formal correspondence neither represents a genuine case of, nor endorses the
hypothetical generalized holographic duality.

Holography forever (yes/no?)

After having had been abundantly present in the information space of strongly
correlated systems for over $15$ years, the so-called ’applied holographic’ -
a.k.a. ’bottom-up’ $AdS/CMT$ (where CMT stands for Condensed Matter Theory
but, in reality, often means $non-AdS/non-CFT$) - approach [1] has been
finally disappearing under the radar, as of lately. Once presented as
spectacular (including quantitative, despite typically operating well outside
the regime of applicability of any semiclassical approximation in the bulk
gravity theory) successes in ’explaining’ such popular condensed matter
systems as the superconducting cuprates in their normal state, the bold claims
of the early holographic studies are neither being elaborated upon even by the
enthusiastic proponents, nor any longer scrutinized by the skeptics alike. In
fact, nowadays even the remaining staunch advocates for applied holography
tend to report on those physical phenomena which this approach $does$ $not$ \-
as opposed to those which is $does$ \- capture [2].

At its inception, though, this novel approach offered a seemingly
straightforward and tempting to faithfully follow - ’no questions asked’ -
practical recipe which strove to provide a universal tool for studying a broad
variety of strongly-correlated quantum many-body systems [1]. And under its
self-proclamation of being uniquely suited for those situations where the
conventional condensed matter techniques were claimed to fail it has been
opportunistically entertained in great many ways, thus creating a rather
peculiar culture of prioritizing technical convenience over (self)critical
judgment. Most frustratingly, the answer to the central question: ’If
generalized ($non-AdS/non-CFT$) holography were indeed valid, then $why$ would
that be?’ has neither been seriously demanded, nor pursued with any
determination.

Although somewhat delayed, the eventual demise of such a shortcut to bypassing
the usual burden of proof, of course, should have been just a matter of time.
However, it has also been suggested that, rather ironically, some of the
general holographic claims might turn out ’to be right, albeit for the wrong
reason’ [3].

For one, much of the late holographic discourse has been limited to the far
more humble (and, incidentally, much less prone to criticism) topic as linear
hydrodynamics, one of the central issues being a possible (non)existence of
fundamental bounds for the various kinetic coefficients and (non)universal
relations between them [4]. However, this activity, too, has been gradually
withering out, also due to the realization that such relations readily arise
within the standard theory of quantum transport and with no reference to any
holographic conjecture whatsoever [5].

That said, it is quite conceivable that the truly systematic - as opposed to
any heuristic - approach to generic many-body quantum systems could indeed be
developed in the form of a path integral over some Wigner function-type
collective field variable acting in the corresponding phase-space. A
subsequent reduction of this formally exact description to the first few
moments of the Wigner’s function, alongside their (non-linear)
(hydro)dynamics, can then be thought of as establishing a holography-like
relationship between the $2d+1$-dimensional ’bulk’ and its boundary defined in
some lesser ($d+1$, as the lowest) number of dimensions [6].

In practice, such a (pseudo)holographic picture is likely to have much in
common with the so-called ’geometric bosonization’ technique that was
discussed three decades ago [7] and was recently resurrected (either with [8]
or without [9] any reference to Ref.[7]). This prospective direction is still
awaiting for its further development, though.

Another holographic spin-off topic is signified by the Sachdev-Ye-Kitaev (SYK)
model [10] and related (random or not [11]) $0+1$-dimensional models which are
often portrayed as the water-proof example of holographic duality. Curiously
enough, this characterization would typically be made despite readily
acknowledging that in $1+1$ dimensions the pertinent Jackiw-Teitelboim (JT)
bulk gravity appears to be non-dynamical and is fully described by the
boundary degrees of freedom, thereby revealing its intrinsically topological
and effectively $0+1$-dimensional nature.

Apart from some subtle high-energy features which require an introduction of
extra bulk non-gravitational (matter) fields [12], the $0+1$-dimensional
boundary dynamics determines all the long-distance bulk correlations, low-
temperature thermodynamics, etc. Moreover, this correspondence turns out to be
very non-specific, as the dual $AdS_{2}$ geometry emerges universally in the
near-extremal regime of a generic $d+1$-dimensional bulk gravity [1].

Thus, although in the above cases certain many-body properties can indeed look
like being ostensibly holographic, such correspondence appears to represent a
mere equivalence between the systems of, effectively, same dimensionality. And
while still being potentially useful, such topological - akin to that in the
Quantum Hall Effect - duality does not quite rise to the same level as
genuine, ’non-topological’, one where the two dual systems do, in fact, belong
in different dimensions. In that regard, the former may be viewed merely as
the result of using a redundant, non-minimal, description, while the latter
would indeed be radically new and requiring a major paradigm shift.

The present note aims to add to the list of pseudo-holographic correspondences
by suggesting that a certain class of Fermi systems with short-range, yet
potentially strong, interactions (e.g., neutral Fermi gases) could be formally
mapped onto the $2+1$-dimensional gravity of the $AdS_{3}$ variety. However,
the latter happens to be topological as well [13], and, therefore, is neither
specific enough to positively identify the boundary system in question, nor
practically useful to allow for its properties to be accessible only via the
bulk.

Tomographic fermions

The physics of weakly dispersing bands, where correlation are strong and
interactions can possibly dominate over kinetic energy, has been the topic of
many theoretical and experimental studies, starting with the Hubbard model and
the Fractional Quantum Hall effect, and culminating, in the recent years, with
the discovery of flat bands in twisted bi/tri-layer graphene and other
systems.

It appears, however, that this fundamental problem finds its cleaner
implementation in cold neutral Fermi gases. Unlike the former where the
relevant interactions are often described by fixed intermediate (rather than
truly strong) couplings under the conditions of broken translational and
rotational invariances, the latter can be precisely tuned into the extreme
(unitary) regime while maintaining translation and rotational invariance.

By allowing for the long-ranged Coulomb forces this model can be further
applied to semiconductor heterostructures. However, even in its most
minimalistic formulation the full solution to this general problem still
remains to be constructed. Arguably, it should have been the first target for
applying any technique pretending to provide a new insight into such more
complex scenarios as, e.g., the ’strange-metallic’ normal state of the
superconducting cuprates.

Lately, much attention has been paid to the random matrix-type models whose
hallmark feature is a dominant role of the so-called ’melonic’ diagrams [10,
11, 12]. In the spaceless SYK-type models, a justification of the melonic
approximation requires a large number $N$ of the fermionic flavors, both, in
the random (original SYK) as well as the non-random (Witten-Gurau)
(non)colored tensor models.

Restoring spatial correlations and generalizing to the case of arbitrary
$q\geq 4$-fermion couplings, the pertinent $d+1$-dimensional melonic theory is
described by the Luttinger-Ward functional

$\displaystyle F=\sum^{N}_{a=1}\ln
det(\partial_{\tau}-\epsilon_{a}({\bf\nabla}_{\bf
x}))+~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$ $\displaystyle-N\int_{\tau_{1},{\bf
x}_{1}}\int_{\tau_{2},{\bf x}_{2}}(\Sigma(\tau_{12},{{\bf
x}_{12}}G(\tau_{12},{{\bf x}_{12}})-~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
$\displaystyle-{1\over q}U(\tau_{12},{\bf x}_{12}){G}^{q}({\bf
x}_{12},\tau_{12}))~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$ (1)

formulated in terms of the bi-local field variables $G$ and $\Sigma$
corresponding to the single-particle Green function and its self-energy. The
dispersion functions $\epsilon_{a}({\bf k})=v_{a}(k-k_{Fa})$ vanishing at the
corresponding (Luttinger) Fermi surfaces (FS) endow the fermions with spatial
dynamics while the coupling function $U(\tau,{\bf r})$ replaces the square of
the SYK entangling amplitude averaged over a random ensemble (if any).

While the fine details of this function would be instrumental for determining
(non-universal) fermion structures at the lattice scale, the long-distance
(potentially universal) properties can be fully characterized by the overall
coupling strength and power-laws of its spatio-temporal decay.

The Schwinger-Dyson equations derived from Eq.(1) read

$\displaystyle(\partial_{\tau_{1}}-\epsilon_{a}(-i{\bf\nabla_{\bf
x1}}))G(\tau_{12},{{\bf x}_{12}})$ $\displaystyle+\int_{\tau_{3},{\bf
x}_{3}}{\Sigma}(\tau_{13},{{\bf x}_{13}}){G}(\tau_{32},{{\bf
x}_{32}})=\delta({\bf
x}_{12})\delta(\tau_{12})~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$ (2)

and

$\Sigma(\tau,{\bf r})=U(\tau,{\bf r})G^{q/2}(\tau,{r})G^{q/2-1}(-\tau,-{r})$
(3)

which expression for the self-energy is formally similar to that of the
$2^{nd}$ order Born approximation in the Fermi gas with short-range (yet,
potentially strong) scattering.

In the limit of a totally flat (degenerate) dispersion relations
$\epsilon_{a}({\bf k})=const$ and strongly localized interaction,
$U({\tau,{\bf r}})=J^{2}\delta({\bf r})$, the solution of Eq.(2) localizes in
space

$G_{syk-r}(\tau,{\bf r})\sim{sgn(\tau)\over(J\tau)^{\Delta}}\delta({\bf r})$
(4)

with $\Delta=2/q$, as in the standard SYK model [10]. Correspondingly, its
Fourier transform, $G_{syk-r}(\omega,{\bf k})\sim
sgn(\tau)/J^{2/q}\omega^{1-2/q}$, becomes independent of momentum.

Introducing bare dispersions $\epsilon_{a}({\bf k})\neq const$ and/or a non-
zero spatial range of interactions favors non-local solutions. For $q=4$ one
such solution was proposed in Ref.[14] where the use of Eqs.(2,3) was
justified by introducing $N\gg 1$ radially nested FS, while the only allowed
processes of fermion scattering would be resonant, the energies of the
incoming and outgoing scattered particles obeying the resonant condition
$\sum^{q/2}_{a=1}\epsilon_{a}({\bf k}_{a})=\sum^{q}_{b=q/2+1}\epsilon_{b}({\bf
k}_{b})$.

A related version of this model was studied in Ref.[15] by exploiting the
’tomographic’ expansion of the propagator

$\displaystyle G(\tau,{\bf
r})\approx{k_{F}^{(d-1)/2}\over\Omega_{r}^{1/2}}\sum_{\pm}e^{\pm
i(k_{F}r-\pi(d-1)/4)}{g(\tau,\pm r)}$ (5)

where $\Omega_{r}=S_{d-1}r^{(d-1)}$ is the surface area of the
$d-1$-dimensional sphere and $S_{d-1}=2\pi^{d/2}/\Gamma(d/2)$.

Expanding the $G$ factors in Eqs.(2,3) and discarding all but the $s$-wave
amplitudes (consistent with the assumed short-range nature of interactions),
one arrives at the equation for the $1+1$-dimensional propagator $g(\tau,r)$.
Although the radial variable $r$ takes values on the positive semi-axis, the
convergent and divergent spherical waves can be combined into one chiral (uni-
directional) mode defined in the entire unbounded domain $-\infty<r<\infty$,
by analogy with the standard analysis of the Kondo effect. In contrast, for
$d=1$ the spatial coordinate is initially unbounded and the fermion system is
non-chiral.

In the tomographic representation Eq.(2) then reduces to its ’radial’
counterpart

$\displaystyle(\partial_{\tau}+\partial_{r})g(\tau,r)+\int_{\tau^{\prime},r^{\prime}}\sigma(\tau-\tau^{\prime},r-r^{\prime})g(\tau^{\prime},r^{\prime})~{}~{}~{}~{}~{}~{}~{}~{}~{}$
$\displaystyle=\delta(\tau)\delta(r)~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$
(6)

where the effectively $1+1$-dimensional self-energy reads

$\sigma(\tau,{r})=U(\tau,r)\sum_{\xi_{a},\zeta_{a}}e^{i\sum{k}_{a}{r}}\prod_{a=2}^{q}{g(\eta_{a}\tau,\zeta_{a}{r})}\Omega_{r}^{(4-q)/2}$
(7)

where $\xi_{a},\zeta_{a}=\pm 1$ are two-valued sign factors. Among the
different terms in the sum (7) the most important are the non-oscillating ones
with $\sum_{a}{k}_{a}=0$. Under such a selection rule only the terms with zero
total $1d$ momentum should be kept. Moreover, in the case of a common Fermi
velocity the resonant scattering condition gets automatically fulfilled, too.

For the sake of generality it would be instructive to consider the interaction
function that decays algebraically in, both, space and time,
$U(\tau,{r})={J^{2}/\tau^{2\alpha}{r}^{2\beta}}$ with some positive $\alpha$
and $\beta$. In Eq.(7) it appears to be multiplied with the extra power
${(4-q)(d-1)/2}$ of $r$ which stems from the tomographic expansion (5).

For comparison, in Ref.[15] the ’defect’ power law governing the spatial
dependence of correlations was found to be $(2-q)(d-1)/2$, which mishap can be
traced back to missing another factor of $\Omega_{r}$ which is due to the
conversion of the $d$-dimensional spatial into the $1d$ radial
$\delta$-function in the r.h.s. of Eq.(6).

Neglecting the derivative terms in Eq.(6) gives rise to the generalized SYK-
like integral equation where the power-law decay of the interaction function
prompts the use of factorized solution

$g(\tau,{r})\sim{sgn\tau\over\tau^{2\Delta_{\tau}}}{1\over{r}^{2\Delta_{r}}}$
(8)

which is dominated by the self-energy (7) and governed by the exponents

$\Delta_{\tau}={1-\alpha\over
q},~{}~{}~{}~{}~{}~{}\Delta_{r}={1-\beta+(d-1)(1-q/4)\over q}$ (9)

Such a solution was first proposed in the context of multi-dimensional SYK
generalizations [16] and later reproduced in Ref.[15] (with no reference to
Ref.[16] and, in fact, incorrectly, see above). Also, the factorized solution
was utilized in a somewhat different - but mathematically similar - context of
the replica-(a)symmetric SYK configurations [17].

In the Fourier transform of the effectively $1d$ propagator

$g(\omega,{k})={1\over i\omega-\epsilon(k)-\sigma(\omega,k)}$ (10)

the bare (derivative) terms are combined with the Fourier-transformed self-
energy

$\sigma(\omega,{k})={\lambda}\omega^{1-2\Delta_{\tau}}k^{1-2\Delta_{r}}$ (11)

Balancing the bare against the self-energy terms allows one to estimate the
effective dynamical critical index.

Namely, in the complimentary regimes $\tau\gg r$ and $\tau\gg r$ one reads off
the naive estimates $z_{>}=(1-2\Delta_{r})/2\Delta_{\tau}$ and
$z_{<}=2\Delta_{r}/(1-\Delta_{\tau})$ which, in general, would be
inconsistent, as $z_{>}\geq 1$ while $z_{<}\leq 1$. In fact, for any $d$, $q$,
and $\alpha,\beta>0$ the minimal attainable value of
${z}_{>,min}=(1+\beta)/(1-\alpha)$ is always greater than the maximal
${z}_{<,max}=(1-\beta)/(1+\alpha)$.

However, for $\alpha=\beta=0$ and $q=4$ (but arbitrary $d$) both estimates
converge to the same value of $z=1$. With this unique choice of parameters one
identifies the regime where $\Delta_{\tau}=\Delta_{r}=1/4$ and the self-energy
$\sigma(\omega,{k})={\lambda}(\omega k)^{1/2}$ governed by a dimensionless
amplitude $\lambda\sim(J/\mu)^{2}$ with $\mu=v_{F}k_{F}$ appears to be
marginally on par with the bare terms in (10).

These findings are consistent with the general expectation that, in contrast
to the $0+1$-dimensional SYK/tensor models where all the $q$-particle
couplings are strongly relevant in the infrared, in their (effectively)
$1+1$-dimensional generalizations the only marginal could be the interactions
with the lowest non-trivial value of $q=4$.

Incidentally, in this special case there exists an elegant exact solution to
the integral equation (6) [18]

$g(\tau,r)={1\over{\sqrt{(iv_{+}\tau-r)(iv_{-}\tau-r)}}}$ (12)

where $v_{\pm}=v(1\pm\lambda)$. Eq.(12) is uniquely tailored to the case of
$q=4$ and does not allow for any immediate generalization.

Also, in the non-chiral case of $d=1$ the arguments presented in Ref.[19]
suggest that the ’charge’ mode tends to develop a gap due the generic back-
scattering processes. Therefore, the following discussion will be limited to
the dimensions $d>1$ where the FS is continuous and the chiral tomographic
fermions remain gapless.

The Fourier transform of (12)

$g(\omega,{k})={1\over{\sqrt{(i\omega-v_{+}k)(i\omega-v_{-}k)}}}$ (13)

features the branch-cut singularity while exhibiting the intact dynamical
critical exponent $z=1$. Instead of an anomalous power-law decay, though,
Eq.(13) demonstrates the behavior akin to spin-charge separation. Accordingly,
the corresponding spectral function $Img(\omega,k)$ is manifestly non-Fermi
liquid (NFL)-like, showing a double-peak structure with the square-root
singularities at $\omega=v_{\pm}k$.

The propagator given by Eqs.(5,12) differs markedly from the approximate
(semi-numerical) solution [14] obtained with the use of the mixed $\tau-{k}$
representation in the momentum-space variant of the SYK model

$G_{syk-p}(\tau,{\bf
k})\sim{sgn\tau\over(J\tau)^{1/2}}e^{-O(T/J)\epsilon(k)\tau}$ (14)

At low temperatures (13) is strongly localized in the coordinate space,
regardless of its dimension, which behavior is suggestive of $z=\infty$ and
the near-extremal $AdS_{2}\times R^{d}$ geometry of the corresponding bulk
dual [1].

Likewise, Eq.(12) differs from the semi-local holographic propagator

$G_{semi-loc}(\omega,{\bf k})={1\over(i\omega)^{2\delta_{k}}-\epsilon(k)}$
(15)

where the momentum-dependent exponent $\delta_{k}=(a+b{\bf k}^{2})^{1/2}$
varies continuously with the scaling dimension of the bulk fermion [20]. It
has been used extensively in constructing the various holographic
phenomenologies which, however, encounter significant problems with restoring
the proper FS.

Boundary-bound bulk dual

Both, the $2d$\- and $3d$-dimensional, gravities are intrinsically topological
and, therefore, all their gauge invariant bulk degrees of freedom are
effectively determined by the state of the boundary [21] Among other things,
this implies that there are no gravitational waves and the list of the
classical background solutions is limited to the maximally symmetric spaces of
constant curvature: Minkowski, de-Sitter, or anti-de-Sitter.

It was conjectured in Ref.[6], the pertinent class of quantum
$1+1$-dimensional systems can be related to a formally $2+1$-dimensional
phase-space geometry, the third (’holographic’) coordinate then being played
by the conjugate momentum $p$.

Somewhat surprisingly, there has been no consistent attempts to incorporate
the $1d$ phenomenon of spin-charge separation into the holographic framework.
For one thing, the putative gravity dual corresponding to the
$1+1$-dimensional chiral SYK was found in Ref.[18] to attain the maximal $0+1$
SYK-like many-body quantum chaos quantified by the Lyapunov exponent
$\lambda_{L,max}=2\pi T$ right where the model loses its chiral nature
($v_{-}\to 0$).

Therefore, it is conceivable that a viable background geometry could be rid of
the ubiquitous near-extremal black hole which exhibits the universal
$AdS_{2}\times R^{d}$ metric as in its presence the boundary state would
likely remain maximally chaotic for $T\to 0$ and arbitrary values of the
entangling couplings [10, 12].

As far as the task of identifying the pertinent geometry is concerned, there
exists a well-known connection [13] between the $2+1$-dimensional gravity with
a negative cosmological constant $\Lambda=-1/l^{2}$

$S={l\over 16\pi\kappa}\int d\tau drdp{\sqrt{g}}(R-\Lambda)$ (16)

where $l$ and $\kappa$ are the AdS radius and Newton’s constant, respectively,
and the (double-sided) Chern-Simons model with the action

$S={l\over 16\pi\kappa}Tr\int d\tau
dxdp\epsilon^{\mu\nu\lambda}({\hat{A}}_{\mu}^{\pm}\partial_{\nu}{\hat{A}}_{\lambda}^{\pm}+{\hat{A}}_{\mu}^{\pm}{\hat{A}}_{\nu}^{\pm}{\hat{A}}_{\lambda}^{\pm})$
(17)

Under the identification of the $3d$ metric

$g_{\mu\nu}={l^{2}\over 4}(A_{\mu}^{+}-A_{\mu}^{-})(A_{\nu}^{+}-A_{\nu}^{-})$
(18)

The equation of motion derived from (17) imposes the null curvature condition

$\partial_{\mu}{\hat{A}}_{\nu}^{\pm}+{\hat{A}}_{\mu}^{\pm}{\hat{A}}_{\nu}^{\pm}-(\mu\leftrightarrow\nu)=0$
(19)

which then becomes the Einstein equation, $R_{\mu\nu}-2g_{\mu\nu}\Lambda=0$
complemented by that of zero torsion.

The chiral connections ${\hat{A}}_{\mu}^{\pm}$ can be expanded in the basis of
generators ${\hat{L}}^{\pm}_{0,\pm 1}$ of the algebra $SL(2,R)\times
SL(2,R)=SO(2,2)$ which obey the commutation relations
$[{\hat{L}}^{\pm}_{n},{\hat{L}}^{\pm}_{m}]=(n-m){\hat{L}}^{\pm}_{n+m}$ and are
normalized as $Tr{\hat{L}}^{\pm}_{n}{\hat{L}}^{\pm}_{m}={1\over
2}\delta_{n0}\delta_{m0}-\delta_{n1}\delta_{m,-1}$. Furthermore, the
asymptotic symmetry algebra is, in fact, enhanced to the product of two chiral
Virasoro algebras.

Upon parameterizing the solutions of (18) in terms of an arbitrary group
element ${\hat{\chi}}_{\pm}$, functions $p_{\pm}(x_{\pm})$, and the conjugate
chemical potentials $\mu_{\pm}={\delta H^{\pm}\over\delta p_{\pm}}$

${\hat{A}}^{\pm}(x,p,\tau)={\hat{\chi}}^{-1}_{\pm}(p){\hat{L}}_{0}(\mu_{\pm}d\tau\pm
p_{\pm}dx){\hat{\chi}}_{\pm}(p)$ (20)

which becomes the continuity equation
${\dot{p}}_{\pm}\mp\mu_{\pm}^{\prime}=0$. By choosing the appropriate form of
the Hamiltonian $H^{\pm}=\sum_{l}a_{l}H^{\pm}_{l}$ where all
$H^{\pm}_{l}=\sum_{k=1}^{l}b_{k}p_{\pm}^{k}\mu_{\pm}^{l-k}$ are in involution
($[H_{k},H_{l}]=0$) one can then reproduce certain families of
$1+1$-dimensional integrable (KdV, mKdV, Gardner, Benjamin-Ono, and other)
equations [21].

On the gravity side, different saddle points of the coherent states path
integral governed by (16) can be identified as globally distinguishable (but
locally $AdS_{3}$) classical solutions, the two competing minima being the
thermal $AdS_{3}$ and Banados-Teitelboim- Zanelli (BTZ) black hole [22].

In particular, the non-chiral spinless Luttinger liquid (LL) with $k=1$ can be
reproduced by introducing the original Brown-Henneaux boundary conditions with
constant $\mu_{\pm}\sim p_{\pm}$, the outer/inner BTZ horizons being located
at $p_{>/<}=(p_{+}\pm p_{-})/2$. The dual metric

$\displaystyle
ds^{2}={p^{2}dp^{2}\over(p^{2}-p^{2}_{+})(p^{2}-p^{2}_{-})}+(p^{2}-p_{+}^{2})({p^{2}-p^{2}_{-})\over
p^{2}}d\tau^{2}~{}~{}~{}~{}~{}$ $\displaystyle+p^{2}(dr-{p_{+}p_{-}\over
p^{2}}d\tau)^{2}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$ (21)

describes a rotating BTZ black hole with the event horizon but no curvature
singularity. In particular, the non-rotating solution with $p_{<}=0$ can then
be used to recover the left-right symmetric LL (cf. the above discussion of
the case $d=1$).

For static, yet non-constant, $p_{\pm}(x)$ the corresponding boundary
solutions possess non-trivial global charges given by the chiral integrals of
motion $H_{k}^{\pm}$ while their bulk counterparts can be regarded as black
holes with multi-graviton excitations (’soft hair’) [21]. By introducing
higher order terms $H^{\pm}_{k}$ with $k\geq 3$ one can generate new black
hole configurations.

The most general solution can be obtained by acting on the ground state (e.g.,
BTZ black hole) with elements of the asymptotic symmetry group commuting with
the Hamiltonian. In this way, one can construct various constant curvature
(locally $AdS_{3}$) space-times with anisotropic Lifshitz scaling and
$z=2k-1$. This opens up the possibility of studying the systems with $z>1$
without the need of bulk geometries which are asymptotically Lifshitz space-
times. Namely, in the case of a higher-spin symmetry $SL(M,R)$ with $M>2$ the
list of attainable gravitational backgrounds includes the asymptotically
Lobachevsky, Schroedinger, warped $AdS$, etc. space-times [23].

Despite a whole range of possibilities it would be rather unlikely for the
list of the (potentially) relevant gravity theories to include the Einstein-
Maxwell-dilaton theory or for the relevant metrics to be even nearly as exotic
as, e.g., Bianchi VII that has been repeatedly invoked in the holographic
studies [1].

In the context of the $1+1$-dimensional SYK model, however, the one-sided
chirality with the causality cone falling within the range of velocities
bounded by $v_{\pm}$ and the lack of conformal symmetry make it more difficult
to identify the proper geometries.

To that end, it was speculated in Ref.[24] that the properties of this model
may be somewhat similar to the non-chiral CFTs with large central charges. In
particular, it was conjectured that their gravity dual, if any, could be
represented by a space-time rotating faster than the speed of light at its
boundary. Also, the discrete (un)folding of the radial dimension performed in
the process of constructing the ’tomographic’ representation might be
suggestive of some orbifold-like underlying geometry.

Some additional insight can be gained by comparing the boundary propagator
(12) to the two-point function (in the Euclidean signature) of a bulk fermion
of dimension $\Delta$. In the semiclassical regime it would be governed by the
exponential of the action computed on the $3d$ geodesic ancored at the
boundary end points

${g}(\tau,r)\sim\exp(-\Delta\int{\sqrt{g_{pp}dp^{2}+g_{\tau\tau}d\tau^{2}+g_{rr}dr^{2}}}|_{p\to
0})$ (22)

Fourier transforming this expression to the space-time domain one then obtains

$g(\omega,k)\sim\exp(-\int dp{\sqrt{g_{pp}({\Delta^{2}\over
l^{2}}+{\omega^{2}\over g_{\tau\tau}}+{k^{2}\over g_{rr}}}})~{}~{}~{})$ (23)

Leaving a systematic identification of the pertinent metric to the future
discussion, by a direct inspection one finds that the propagator (12,13) can
be reproduced with the use of the metric

$ds^{2}={dp^{2}\over p^{2}}+p^{2}(dr_{-}^{2}+\Delta v^{2}d\tau^{2})$ (24)

where $r_{-}=r-i\tau$ and the bulk fermion dimension $\Delta=1$. The
logarithmic divergence in the momentum integral in (22,23) gets cut off at the
low momenta $p\sim 1/{\sqrt{r_{-}^{2}+\delta v^{2}\tau^{2}}}$, thereby
resulting in Eqs.(12,13).

In the limit of vanishing interaction ( $\lambda=\delta v/2v\to 0$) the chiral
boundary propagator and its Fourier transform return to their free
counterparts $g_{0}(\tau,r)=1/(r-i\tau)$ and
$g_{0}(\omega,k)=1/({i\omega-k})$, respectively.

Strange theories of strange (or not so?) metals

Since its inception, one of the most recurrent themes in applied holography
was the problem of ’strange metals’, in reference to such electronic materials
as the cuprates, pnictides, heavy fermion compounds, twisted bi/tri-layer
graphene, etc.

Thermodynamics of Eq.(1) is governed by the free energy ${F\sim}T^{1+1/z}$ in
any dimension $d\geq 1$, thus producing a linear (possibly, $log$-enhanced)
specific heat $C=-T\partial^{2}F/\partial T^{2}\sim T$ for $z=1$, albeit with
a prefactor different from that in the ordinary Fermi gas.

For comparison, with the use of the solution (13) the longitudinal optical
conductivity can be readily evaluated in the same ‘no vertex correction’
approximation

$\displaystyle\sigma(\Omega)={\pi\mu\over
2T}\int{d\epsilon_{k}d\omega\over\cosh^{2}(\omega/T)}ImG(\omega+\Omega,{\bf{k}})ImG(\omega,{\bf{k}})$
$\displaystyle\sim{\lambda\mu T\over{(\lambda
T)^{2}+\Omega^{2}}}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}$ (25)

from which one recovers the Drude peak, $\sigma(\Omega)\sim\mu\delta(\Omega)$,
in the vanishing coupling limit.

Albeit not immediately obvious, the source of momentum relaxation is provided
by the intrinsic randomness built into the action (1). And while the customary
substitution $\Omega\to T$ may not be generally justifiable (as it is in the
’no momentum drag’ regime), the d.c. resistivity shows a linear behavior with
increasing temperature, in agreement with the results of Ref.[14].

Furthermore, in Ref.[14] the slope of the linear in $T$ rate of momentum
relaxation was found to be universal, which prediction appears to be
consistent with the data on a host of materials [25] (see, however, [26]).
Despite this intriguing observation, in the later work on the subject the
remnant FS model was abandoned in favor of the more conventional coordinate
space generalization of the SYK model with some additional single-particle
randomness that, alongside the random entangling correlations, yields the
resistivity $\rho(T)=\rho_{0}+\alpha T$ [27].

The above results can be contrasted against the earlier attempts to
accommodate the general SYK scenario into the physics of cuprates and other
’strange metals’ by exploiting the ultra-local behavior with $z=\infty$. This
universality class was argued to be characterized by such hallmarks as
universal (’Planckian’) linear-$T$ equilibration/thermalization rate, maximal
chaos, and saturated bounds for the kinetic coefficients [25].

Yet another thrust towards the multi-dimensional generalizations of the SYK
model delivered the resistivity exponent $4/q$, thus suggesting some
’accidentally linear’ temperature dependence for $q=4$ [28]. However, the
general possibility of other values of this exponent should caution one
against invoking this observation to explain the robust linear-$T$ dependence
observed in the broad range of strongly correlated systems.

Meanwhile, in the still other ’bottom up’ holographic scenaria the
thermodynamic and transport properties would be derived from such a popular
workhorse as the Einstein-Maxwell-dilaton model which also paves the way for
the Lifshitz and hyperscaling-violating (HV) background geometries which are
parametrized by the dynamical index $z$ and the HV exponent $\theta$. Besides,
yet another exponent would often be introduced for the effective charge
renormalization [29].

The title of the aforementioned Ref.[2] states that the popular (Gubser-Rocha
[30]) holographic model where the dynamical index $z$ and HV exponent $\theta$
are both infinite while their ratio is finite, $\theta/z=-1$, does $not$
explain the cuprates (presumably, suggesting that some alternate schemes might
still do). Specifically, in Ref.[2] it was proposed to resolve the dichotomy
between the $T$-dependences of resistivity and Hall angle observed in the
cuprates by postulating a $T$-dependent carrier density $n\sim T$, alongside
the generic relaxation time $\tau\sim 1/T^{2}$.

Interestingly, though, despite including the earlier work of Ref.[31] in their
list of references (item [36] in Ref.[2]) the authors of Ref.[2] did not seem
to realize that in Ref.[31] the same scenario had been proposed (incidentally,
the actual journal reference [36] in Ref.[2] was cited incorrectly).

Nor, did the authors of Ref.[2] seem to appreciate that, apart from the
conductivity $\sigma\sim 1/T$ and the Hall angle $\tan\theta_{H}\sim 1/T^{2}$,
the scheme proposed in Ref.[31] renders the linear (possibly, $\log$\-
enhanced) specific heat and reproduces the experimentally observed
magnetoresistivity, ${\Delta\rho\over\rho}\sim\theta^{2}_{H}\sim 1/T^{4}$,
spin susceptibility (both, at the momentum $Q=(\pi,\pi)$ and momentum-
integrated), as well as the Hall Lorentz, Seebeck, and Nernst coefficients,
the latter satisfying the relations $S\sim(T/e\sigma){d\sigma/d\mu}$ and
$\nu_{N}\sim(T/eB){d\theta_{H}/d\mu}$ where $\mu$ is a chemical potential.
Altogether, the above predictions appear to be in a generally better agreement
with the data than the competing holographic phenomenologies [32] even without
introducing an additional charge exponent.

Also, the proposal of Ref.[31] differs from such historic scenaria as that of
two distinct scattering times: $\tau_{tr}\sim 1/T$ and $\tau_{H}\sim 1/T^{2}$
pertaining to the relaxation of longitudinal and transverse [33] or charge-
symmetric vs anti-symmetric [34] currents, respectively, or that based on the
marginal Fermi liquid phenomenology [35].

Although the scenaria of $T$-dependent carrier density were indeed discussed
at the early stages of the high-$T_{c}$ saga [36], the peculiar linear in $T$
behavior of the carrier density can be seen as problematic enough to view the
above agreement with experiment as no more than fortuitous.

Conceivably, however, in light of the gradually emerging consensus about the
by and large mundane (hence, non-NFL) behavior in most of the cuprates phase
diagram [37] some of the seemingly unresolved issues might eventually turn out
to be moot. On the other hand, such a conciliatory resolution could still be
challenged farther down the road [38].

The bottom line (up front)

Applied holography purportedly offers a unique insight into the largely
uncharted territory of ’super-strongly coupled’ systems with a nearly or even
completely flat dispersion. The majority of the recent and ongoing exercises
on this topic exploits the various variants of the ’ultra-local’ scenario
represented by the SYK model and its (pseudo)holographic dual which has been
identified as the JT gravity. In contrast, the present note focuses on the
holographic aspects (or lack thereof) of the intrinsically ’linear-radial’
tomographic representation models.

On the practical side, a valid instance of holographic correspondence could
then allow one to replace the conventional (usually impossible) exact or
diagrammatic calculations in the strongly-interacting quantum boundary theory
with the technically much simpler task of solving some classical equations $a$
$la$ Einstein in the higher-dimensional bulk.

However, if the nature of the holography in question is topological, one can
hardly gain any new knowledge by exploiting the (in this case, trivial) bulk
gravity, thus largely negating any potential benefits of the alternate bulk
description. In fact, all the essential properties of the SYK models can be
understood without much input from the dual $AdS_{2}$ gravity, thus making
their correspondence secondary to the (asymptotic) exact solubility of either
model. The only ’holographic’ aspect that appears to be essential for solving
both models is the very existence of the ’conformal’ solution (4), reminiscent
of the first such example found in the theory of phase transitions in helium
[39].

To summarize, in the present note it is argued that a broad family of $d\geq
1$-dimensional ’tomographic’ models allows for the unifying effective
$1+1$-dimensional description which, in turn, can be reproduced in the context
of some bulk gravity of the $AdS_{3}$ type. Once again, the intrinsically
topological nature of the latter theory does not provide any significant
insight that could have not been gathered in the framework of the boundary
model itself.

On a more general note, the intrinsic topological aspect of any example of the
$AdS/CMT$ holographic correspondence could be traced back to the popular
interpretation of the extra holographic coordinate as that of the Wilson’s
renormalization group (RG) scale [1]. Despite being intuitively appealing,
this identification may run into a potential problem with double counting of
the effects of the boundary degrees of freedom which are supposed to generate
that RG flow without any reference to the bulk fields. Indeed, any coarse-
graining RG procedure (either in the Wilson’s or in tensor network sense) can
be executed solely within the boundary theory. Introducing an extra
holographic coordinate would then be redundant and, therefore, subject to a
gauge condition and/or an associated constraint.

The above discussion implies that any purported discovery of a novel $AdS/CMT$
holographic duality should be calling for an inquiry into whether it really
involves the systems in different dimensions ( ’It from Qubit’, as per the
popular motto [1]) or is it merely topological (’All from Hall’, as per the
above slogan’s obscure rival [40]). Contrary to the former, the ’HALLographic’
correspondence would be relating pairs of systems of the (de facto) same
dimensionality and, therefore, would be unlikely to serve as a preferred means
of gaining some exclusive insight into the properties of one party
(’boundary’) by instead studying its counterpart (’bulk’).

The hospitality at the Aspen Center for Physics funded by National Science
Foundation through the grant PHY-2210452 and at the International Institute of
Physics in Natal, Brazil supported by the Simons Foundation are gratefully
acknowledged.

## References

  * [1] M. Ammon and J. Erdmenger, Gauge/Gravity Duality: Foundations and Applications (Cambridge University Press, 2015); J. Zaanen, Y. Liu, Y.-W. Sun, and K. Schalm, Holographic Duality in Condensed Matter Physics (Cambridge University Press, 2015); S. Hartnoll, A. Lucas, and S. Sachdev, Holographic Quantum Matter (The MIT Press, 2018).
  * [2] Y. Ahn, M. Baggioli, H.-S. Jeong, K.-Y. Kim,
arXiv:2307.04433; https://pdf.pirsa.org/files/23060098.pdf

  * [3] D. V. Khveshchenko, Lith. Journal of Physics, 56, 125 (2016), arXiv:1603.09741; ibid 55, 208 (2015), arXiv:1404.7000.
  * [4] Sean A. Hartnoll, arXiv:1405.3651; A. Amoretti et al, JHEP 1507 (2015) 102; M. Baggioli and O. Pujolas, Phys. Rev. Lett. 114, 251602 (2015); R.A. Davison and B. Gouteraux, JHEP 1509 (2015) 090; JHEP 1501 (2015) 039; R. A. Davison, B.Gouteraux, S. A. Hartnoll, arXiv:1507.07137; M. Blake, JHEP09(2015)010.
  * [5] S. Chakrabarty and Z. Nussinov, Ann. of Physics 443, 168970 (2022), arXiv:2107.10222; arXiv:2303.00021.
  * [6] D. V. Khveshchenko, Lith. J. of Phys., Vol. 61, p. 233, 2021, arXiv:2102.01617.
  * [7] D. V. Khveshchenko, Phys. Rev. B 49, 6893 (1994), arXiv:cond-mat/9401012; ibid B 52, 4833 (1995), arXiv:cond-mat/9409118.
  * [8] L. V. Delacretaz, Y.-H. Du, U. Mehta, and D. T. Son, Phys. Rev. Res. 4, 033131 (2022), arXiv:2203.05004; https://pirsa.org/23100071; T. Park, L. Balents, arXiv:2310.04636.
  * [9] D. V. Else, T. Senthil, Phys. Rev. Lett. 127, 086601 (2021), arXiv:2010.10523; D. V. Else, R. Thorngren, and T. Senthil, Phys. Rev. X 11, 021005 (2021),78, arXiv:2007.07896; D. V. Else, Phys. Rev. B 108, 045107 (2023), arXiv:2301.10775; arXiv:2307.02526; S. Han, F. Desrochers , and Y. B. Kim, arxiv:2306.14955.
  * [10] S. Sachdev and J. Ye., Phys. Rev.Lett. 70, 3339 (1993); O. Parcollet et al, Phys. Rev. B58, 3794 (1998); O. Parcollet and A. Georges,Phys.Rev. B59, 5341 (1999); A. Georges, O. Parcollet, and S. Sachdev, Phys.Rev. B63, 134406 (2001); S. Sachdev, ibid 105, 151602 (2010); Phys.Rev.X5, 041025 (2015); A. Kitaev, KITP seminars, 2015; arXiv:1711.08169; A. Kitaev and S.J. Suh, arXiv:1711.08467;1808.07032; I. R.Klebanov and G. Tarnopolsky, Phys Rev D95, 046004 (2017), S. Giombi, I. R. Klebanov, and G. Tarnopolsky, ibid 96, 106014 (2017); J. Maldacena, S.H. Shenker and D. Stanford, JHEP 08 (2016) 106; J. Maldacena and D. Stanford, Phys. Rev. D 94, 106002 (2016); J. Maldacena, D. Stanford and Z.Yang, arXiv:1606.01857; D. Stanford and E.Witten, JHEP10(2017)008.
  * [11] E. Witten, arXiv:1610.09758; R.Gurau, Nucl. Phys. B, 2017, 916, 386, arXiv:1611.04032; 1705.08581.
  * [12] J. Polchinski and V. Rosenhaus, JHEP 1604, 001 (2016); D. J. Gross and V. Rosenhaus, JHEP 1705 (2017) 092, ibid 1712 (2017) 148; G.Sarosi, arXiv:1711.08482; H. W. Lin, J.Maldacena, and Y. Zhao, arXiv:1904.12820; Y.Gu, X-L. Qi, and D. Stanford, J. High Energ. Phys. (2017) 2017: 125; Y. Gu, A. Lucas, and X-L. Qi, SciPost Phys. 2, 018 (2017); J. High Energ. Phys. (2017) 2017: 120; Y. Gu and A.Kitaev, J. High Energ. Phys. (2019) 2019: 75; 11 D. Bagrets, A. Altland and A. Kamenev, Nucl. Phys. B 911, 191 (2016); ibid 921, 727 (2017); T. G. Mertens, G. J. Turiaci and H. L. Verlinde, JHEP08(2017) 136; T. G. Mertens, JHEP 1805 (2018) 036; Z. Yang, arXiv:1809.08647; D.V.Khveshchenko, Lith. J. of Physics, 2022 Vol. 62 No. 2, arXiv:2205.11478; ibid v.59, p.104 (2019), arXiv:1905.04381.
  * [13] A. Staruszkiewicz, Acta Phys.Polon. 24 (1963) 735–740, 37; S. Deser, R. Jackiw, and G.’t Hooft, Ann. Phys. 152 (1984) 220; S. Deser and R. Jackiw, Annals Phys. 153 (1984) 405–416; A. Achucarro and P. K. Townsend, Phys. Lett., vol. B180, p. 89, 1986; E. Witten, Nucl. Phys., vol. B311, p. 46, 1988; Commun. Math. Phys. 121, 351 (1989).
  * [14] Aavishkar A. Patel, Subir Sachdev, Phys. Rev. Lett. 123, 066601 (2019), arxiv:1906.03265; P. Cha, A. A. Patel, E. Gull, and Eun-Ah Kim, arxiv:1910.07530.
  * [15] D. Miserev, J. Klinovaja, D. Loss, Phys. Rev. B 103, 075104 (2021); ibid B 106, 134417 (2022); arXiv:2303.16732.
  * [16] D. V. Khveshchenko, Sci Post Phys. 5, 012 (2018), arXiv:1705.03956; Condens. Matter 2018, 3(4), 40, arXiv:1805.00870.
  * [17] I. Aref’eva, M. Khramtsov, M. Tikhanovskaya, I. Volovich, JHEP 07 (2019) 113, arXiv:1811.04831.
  * [18] B. Lian, S. L. Sondhi, Z. Yang, JHEP09 28 (2019) 29067, arXiv:1906.03308.
  * [19] A. Yunge, A. De Martino, and A. Tagliacozzo, Phys.Rev. B108, 075428 (2023).
  * [20] H. Liu, J. McGreevy, D. Vegh, Phys.Rev.D83, 065029,2011, arxiv:0903.2477; N. Iqbal, H. Liu, Fortsch.Phys.57, 367,2009, arXiv:0903.2596; T. Faulkner, H. Liu, J. McGreevy, D. Vegh, Phys.Rev.D83, 125002,2011, arxiv:0907.2694; arxiv:1101.0597; M. Čubrović, J. Zaanen, K. Schalm, JHEP10 29017, arXiv:1012.5681; M. Cubrovic, Y. Liu, K. Schalm, Ya-Wen Sun, J. Zaanen, Phys.Rev. D84 086002, arXiv:1106.1798; F. HerCek, V. Gecin, M. Cubrovic, SciPost Phys. Core 6, 027 (2023), arXiv:2208.05920.
  * [21] J.D. Brown and M. Henneaux, Commun. Math. Phys. 104, 207 (1986); O. Coussaert, M.Henneaux, and P. van Driel, arxiv:gr-qc/9506019; R.E. Goldstein and D.M. Petrich, Phys. Rev. Lett. 67 (3203) 1991; H.Afshar et al, arxiv:1512.08233; arxiv:1611.09783; arxiv:1603.04824; M. Henneauxa, W. Merbisa and A. Ranjbara arxiv:1912.09465; S. Li, N. Toumbas, J. Troost, arxiv:1903.06501; A. Pérez, D. Tempo, R. Troncoso, arxiv:1605.04490; A.Perez arxiv:1711.02646; D. Grumiller1, W. Merbis, arxiv:1906.10694; E. Ojed and A.Pereza arxiv:1906.11226; A. Dymarsky and S.Sugishitaa arxiv:2002.08368; J.Matulich, S. Prohazka, and J. Salzer, arxiv:1903.09165; E. Ojeda and A. Perez, arxiv:1906.11226; J.Cotler and K. Jensen, arxiv:2006.08648; D. Melnikov, F. Novaes, A. Perezc and R. Troncoso, arxiv:1808.04034; S. Dutta, D. Mukherjee, S. Parihar, arxiv:2302.08471.
  * [22] M. Banados, C. Teitelboim, and J. Zanelli, Phys. Rev. Lett. 69 (1992) 1849, hep-th/9204099; M. Banados, M. Henneaux, C. Teitelboim, and J. Zanelli, Phys. Rev. D48 (1993) 1506, gr-qc/9302012.
  * [23] M.Henneaux and S.-J. Rey arxiv:1008.4579; G. Compere, and W. Song arxiv:1306.0014; M. Gutperle and Yi Li arxiv:1412.7085; M. Beccaria et al, arxiv:1504.06555; Grumiller, R. Troncoso arxiv:1607.05360; E. Ojeda and A. Perez, arxiv:2009.07829; P.M. Petropoulos, arxiv:1406.2328; P. M. Petropoulos, K.Siampos, arxiv:1510.06456; J. Gath, A. Mukhopadhyay, A. C. Petkou, arxiv:1506.04813; L. Ciambelli, et al, arxiv:1802.05286; arXiv:1802.06809; D. Hansen, J. Hartong, and N. A. Obers3, arxiv:2001.10277; J. de Boer and D. Engelhardt, arxiv:1604.05327; M. Rangamani et al, arxiv:0811.2049; A. Campoleoni et al, arxiv:1812.04019.
  * [24] Y. Hu and B. Lian, Phys. Rev. B 105, 125109 (2022), arxiv:2109.13263; M. Berkooz et al, Phys. Rev. D106,045007 (2022), arxiv:2111.06108; B. Lian, arxiv:2202.06169.pdf.
  * [25] R.A. Davison, K.Schalm, and J.Zaanen, Phys. Rev. B89, 245116 (2014); J. Zaanen, SciPost Phys. 6, 061 (2019); S. A. Hartnoll, A. P. Mackenzie, Rev. Mod. Phys.94.041002; arXiv:2107.07802.
  * [26] M.V. Sadovskii, Physics Uspekhi 64(2), 175-190 (2021), arXiv:2008.09300; JETP Letters 111, 188-192 (2020), arXiv:2001.01393.
  * [27] I. Esterlis, H. Guo, A. A. Patel, S. Sachdev, Phys. Rev. B 103, 235129 (2021), arXiv:2103.08615; H. Guo, A. A. Patel, I. Esterlis, S. Sachdev, Physical Review B 106, 115151 (2022), arXiv:2207.08841; A. A. Patel, H.Guo, I. Esterlis, S. Sachdev, Science 381, 790 (2023), arXiv:2203.04990.
  * [28] Ben-Zion, D., McGreevy, J., Phys.Rev.B97,155117; arXiv:1711.02686; Patel, A.A., McGreevy, J., Arovas, D.P., Sachdev, S., Phys. Rev. X8 2018, 021049; Chowdhury, D., Werman, Y., Berg, E., Senthil, T., ibid X8 2018 031024, arXiv:1801.06178.
  * [29] B. Gouteraux, JHEP 1401, 080 (2014); ibid 1404, 181 (2014); A. Donos and J. P. Gauntlett, JHEP 06 (2014) 007, ibid 11(2014)081; R. A. Davison and B. Gouteraux, JHEP 1509 (2015) 090; M. Blake, JHEP 09 (2015) 010.
  * [30] S. S. Gubser and F. D. Rocha, Phys. Rev. D81, 046001, 2010; arXiv:0911.2898.
  * [31] D. V. Khveshchenko, EPL 111 (2015) 1700, arXiv:1502.03375; Lith. Journal of Physics, 61, 1 (2021); arXiv:2011.11617.
  * [32] S.A. Hartnoll and A.Karch, Phys. Rev. B 91, 155126 (2015); A. Karch, JHEP 1406, 140 (2014), arXiv:1405.2926; A. Amoretti and D. Musso, JHEP 1509 (2015) 094; A. Amoretti et al, JHEP 1409 (2014) 160; ibid 1507 (2015) 102; ibid 06(2016)113; ibid 08 (2020); Phys. Rev. D91 (2015) 025002; Adv. in Phys. X, v.2, 409 (2017); Phys. Rev. Research 2, 023387 (2020).
  * [33] P.W.Anderson, Phys.Rev.Lett.67, 2092 (1991).
  * [34] P.Coleman, A.J.Schofeld,and A.M.Tsvelik,
Phys.Rev.Lett.76, 1324 (1996); D. K. Lee and P.A. Lee, J. of Phys. Cond.
Mat.9, 10 (1997).

  * [35] C.M.Varma et al, Phys.Rev.Lett.63, 1996 (1989).
  * [36] A.S. Alexandrov, V.V. Kabanov, N.F. Mott, Phys. Rev.Lett. 77 (1996) 4796; N. Luo, G.H. Miley, Physica C 371 (2002) 259.
  * [37] https://www.kitp.ucsb.edu/activities/quasipart23
  * [38] https://www.kitp.ucsb.edu/activities/gapless24
  * [39] A. Z. Patashinskii and V. L. Pokrovskii, JETP 19, 677 (1964).
  * [40] D. V. Khveshchenko, arXiv:2305.04399.


Orthogonal-view Microscope

for the Biomechanics Investigations of Aquatic Organisms

Brian T. Le $^{1,*}$; Katherine M. Auer $^{1,*}$; David A. Lopez $^{1,*}$; Justin P. Shum $^{1,*}$;  Brian Suarsana $^{1,*}$;

Ga-Young Kelly Suh $^{1}$; Per Niklas Hedde $^{2,\dag}$; and Siavash Ahrar $^{1,\dag}$.

(* Authors contributed equally; †Corresponding authors.)

(Version-6; Last updated: July. 24. 2023)

1. Department of Biomedical Engineering,
California State University Long Beach
1250 Bellflower Blvd. Long Beach, CA 90840, USA.

2. Beckman Laser Institute and Medical Clinic, University of California Irvine
Irvine, CA 92612, USA.

Corresponding contacts

E-mail: Siavash.Ahrar @ csulb.edu; Twitter: @SeeAvash

E-mail: Phedde @ uci.edu

Microscopes are essential for the biomechanical and hydrodynamical investigation of small aquatic organisms. We report a do-it-yourself microscope (GLUBscope) that enables the visualization of organisms from two orthogonal imaging planes - top and side views. Compared to conventional imaging systems, this approach provides a comprehensive visualization strategy of organisms, which could have complex shapes and morphologies. The microscope was constructed by combining custom 3D-printed parts and off-the-shelf components. The system is designed for modularity and reconfigurability. Open-source design files and build instructions are provided in this report. Additionally, proof-of-use experiments (particularly with Hydra) and other organisms that combine the GLUBscope with an analysis pipeline were demonstrated to highlight the system’s utility. Beyond the applications demonstrated, the system can be used or modified for various imaging applications.

DIY Microscope, Imaging, Orthogonal view, Biomechanics of organisms.

Specifications table

to |X|X[3,l]|
Hardware name     GLUBscope: the Orthogonal-view Microscope

Subject area

* Engineering and material science

Hardware type

* Imaging tools

Closest commercial analog

* No commercial analog is available

Open source license

* CC-BY-SA 4.0

Cost of hardware

*  $ 3000-4000

Source file repository

§ HARDWARE IN CONTEXT

Investigating biomechanics [1, 2, 3], hydrodynamics[4], and mechanobiology [5] of organisms is of great interest. Beyond their fundamental biological importance, these investigations could provide approaches
to mitigate or address the emerging threats to biodiversity due to climate change [6]. Therefore, broader access to imaging tools, specifically microscopes, for laboratory and field investigations could be critical. Unfortunately, most commercially available microscopes are expensive, difficult to deploy for field studies, and challenging to customize [7]. Thankfully, microscopes have benefited from the emergence of the do-it-yourself (DIY) and open hardware movement. The DIY movement has reduced costs, provided access to local fabrication, allowed for customization, and increased access to imaging instruments [8]. This report describes an open-sourced DIY orthogonal-view microscope for biomechanics and hydrodynamics investigation of aquatic organisms.

Over the past two decades, significant advancements have been made in developing low-cost microscopes and other DIY hardware. Salido et al. provide a comprehensive summary of and breakdown of these advances in their review [9]. Broadly, in one approach to achieve cost-effective microscopy mobile phones are used in conjunction with DIY hardware and optics. Examples of this approach include a mobile phone strategy for clinical microscopy [10], a wide-field fluorescence microscope [11], Foldscope [12], and CellScope [13]. Furthermore, the concept of mobile phone-based microscopy has expanded to techniques such as light-sheet [14], enhanced micro and nanoparticle imaging [15], and other applications [16]. A second approach combines DIY and open-source hardware to build microscopes from scratch, leveraging prototyping approaches (e.g., 3D printing and laser cutting). Del Rosario et al. provide key considerations for constructing microscopes using 3D printing [17]. Many exciting examples of DIY microscopes have been reported [18]. For example, the OpenFlexure project integrates optics with 3D-printed precision mechanical positioning, enabling automated microscopy that is accessible to a wide range of users [19]. In another example, the UC2 (You. See. Too.) initiative has developed a modular and open-source toolbox that facilitates the construction of DIY microscopes [20]. Moreover, researchers have successfully produced 3D-printed components enabling imaging and control of model organisms (e.g., temperature) [21], as well as microscopes tailored for controlling optofluidic applications[7]. Of particular interest in the study of marine organisms are PlanktoScope [22, 23] and the scale-free tracking microscope [24, 25].
Combined with DIY microscopes, advances in fluidics have been critical in enabling biomechanics investigations of whole organisms (C. elegans, zebrafish larvae, Drosophila larvae, and Hydra) under well-controlled conditions [5]. Specifically, microfluidics could immobilize, trap, or limit the organisms' movements to enable imaging or other functional recordings [26, 27]. These strategies are critical for many investigations due to organisms' propensity for shifting out of focus.  However, this approach may limit some biomechanics considerations. However, maintaining organisms (or key features) in the field of view can be challenging due to their shape or small rotations.

In this investigation, we developed an imaging system for the biomechanics investigation of aquatic organisms. The DIY microscope, GLUBscope, was built by combining custom 3D-printed parts and broadly available off-the-shelf components. One key feature of GLUBscope is the ability to view the samples from two orthogonal planes of imaging - a top view and a side view. With this approach, an organism could move out of a plane in one view yet remain visible in the orthogonal view. Due to their complex 3D shapes, viewing organisms from multiple angles could be critical in addressing questions related to biomechanics and hydrodynamics (e.g., surface attachment, elongation, or stretching in multiple planes). In this effort, we avoided the immobilization strategies and used chambers (3.5 mL volume) with larger volumes than the standard microfluidics often used for biomechanics investigations. The system was built for modularity and reconfigurability, such that various components could be readily exchanged. In this report and accompanying repository, we have shared design files, bill of materials, and build and operation instructions. Additionally, proof-of-use experiments with organisms (particularly Hydra) are provided to demonstrate the use of GLUBscope for biomechanics investigation of aquatic organisms.

§ HARDWARE DESCRIPTION

GLUBscope provides two orthogonal views of the samples, setting it apart from conventional imaging systems. To build the GLUBscope, off-the-shelf components, and DIY 3D-printed parts were used. The system was built on an optical breadboard measuring 12" x 18" x $1/2$". We note that almost all commercial mechanical components (e.g., optical breadboard) can be replaced with alternatives. They were used here due to their broad accessibility.

To position samples, an XYZ micromanipulator stage was used. Figure 1 presents images of the GLUBscope and a simplified system block diagram emphasizing the optical components. The 3D-printed parts were made via Snapmaker 2.0. These parts included two guide rails to move optical components to focus the image. Two cameras were used, one for capturing the top view (or vertical view) and another for the side view (or horizontal view). We used FLIR cameras and Arducam cameras, offering a choice between high-resolution/high-sensitivity and more cost-effective options. Camera control was achieved using Micromanager software and Arducam software. The study used various objectives ranging from 4X to 10X to suit different magnification requirements. Brightfield imaging of the samples was achieved by employing two bright white LEDs positioned below and on the side of the sample. For horizontal (side) illumination, a stand was developed. An optional diffuser holder was developed for top view illumination. LEDs were powered by a 9V battery and regulated using two resistors (variable or fixed at 350 $\Omega$) such that the system could be operated in the field without access to an electrical grid. The components were secured onto the optical breadboard via 1/4-inch 20 screws. The bill of materials, the designs for the 3D-printed parts, and other resources are provided through an Open Science Framework (OSF) page.

Potential uses of the GLUBscope include:

* Orthogonal view imaging (top and side).

* Characterize biomechanics or hydrodynamics of aquatic organisms.

* Customize and use as a field microscope.

* Use for pedagogy or classroom room setting.

§ DESIGN FILES SUMMARY

Editable versions of design files for the 3D-printed components are available from the project's Open Science Framework repository, <https://osf.io/9mw7d/>.

Following is the summary of files for the hardware.

to |X[2,1]|X|X|X[1.5,1]|
Design Filename     File Type     License     Location of the File

breadboard$\_$and$\_$battery$\_$holder      3D Model (.stl)     CC-BY-SA 4.0     OSF page
horizontal$\_$camera$\_$holder     3D Model (.stl)     CC-BY-SA 4.0     OSF page
horizontal$\_$camera$\_$path     3D Model (.stl)     CC-BY-SA 4.0     OSF page
horizontal$\_$camera$\_$post     3D Model (.stl)     CC-BY-SA 4.0     OSF page
horizontal$\_$camera$\_$stands     3D Model (.stl)     CC-BY-SA 4.0     OSF page
horizontal$\_$light     3D Model (.stl)     CC-BY-SA 4.0     OSF page
vertical$\_$camera$\_$adapter      3D Model (.stl)     CC-BY-SA 4.0     OSF page
vertical$\_$camera$\_$holder     3D Model (.stl)     CC-BY-SA 4.0     OSF page
vertical$\_$camera$\_$posts     3D Model (.stl)     CC-BY-SA 4.0     OSF page
vertical$\_$camera$\_$stands     3D Model (.stl)     CC-BY-SA 4.0     OSF page
diffuser holder v2     3D Model (.stl)     CC-BY-SA 4.0     OSF page

The supplementary material provides an animated rendering of the components' assembly.

§ BILL OF MATERIALS

A complete bill of materials is available from the OSF page with additional comments. The estimated costs and potential vendors are provided. We note that availability and costs are subject to suppliers. Alternative options are recommended when possible.

§ BUILD INSTRUCTIONS

Step-by-step instructions for the system build, and images are provided (Figures 2 and 3). Please note that, for the connections listed below relying on $1/4$” - 20 screws, it is possible to reverse the orientations while constructing the GLUBscope with no negative impact on build stability. Four screws are the minimum requirement for a secure connection between the base of a part and the optical breadboard. There are additional ports available for a more secure attachment. A degree-marked level (circular bubble level bullseye) can be used to ensure that the parts are level.

$\blacksquare$ SAFETY 1:  Please use proper personal protective equipment (PPE). Bright LEDs and other light sources could pose a safety risk.

$\blacksquare$ SAFETY 2: Recall that the bead of a 3D Printer can become hot. After the print is completed, remove the part from the bed by applying a gentle shear force. If a chisel is required to remove the part, please use appropriate PPE, such as cut-proof gloves and safety glasses.

$\blacksquare$ SAFETY 3: Please use appropriate ventilation and PPE when soldering.

Following are the instructions to build the GLUBscope.

Horizontal Path Assembly

* Place the M6 breadboard on a flat surface. Here the position (port) on the top left corner is referenced as coordinate $(1,1)$.

* Optional Step: Apply double-sided (set) screws in the 4 corners $(1,1),  (1,12),  (18,1),  (18,12)$. Attach optical construction rails to create an enclosure for the system to block ambient light.

* Roughly align the parts vertical_camera_stands in desired coordinates $(9,1),(9,10)$, spaced 166 mm apart or with 6 ports spaced between them.

* Secure the stands via four screws.

* Align horizontal_camera_stands centers at coordinates $(3,7),(8,7)$, spaced 58.44 mm outside to inside.

* Fasten screws into 4 corners. See Figure 2 A.

* Place horizontal_camera_path on top of horizontal_camera_stands and secure in place with screws. See Figure 2 B.

* Slide horizontal_camera_posts into horizontal_camera_path. See Figure 2 D and E.

* Attach FLIR camera with optical components (See Figure 2 E attached to horizontal_camera_holder.

* Slide horizontal_camera_holder with camera attached to horizontal_camera_post.

* Connect FLIR camera to USB 3.1 Gen 1 Micro-B to USB-A Locking Cable and then the computer. See Figure 2 H.

Vertical Path Assembly

* Align vertical_camera_posts with vertical_camera_stands and secure in place with screws.

* Slide vertical_camera_holder on vertical_camera_posts, with the short side closest to the user.

* Place vertical_camera_adapter  on the top of vertical_camera_holder and align ports. See Figure 2 I and J.

* Align FLIR camera in vertical_camera_adapter_flir. See Figure 2 K.

* Attach the optical path with desired lens, filter, and objective on the bottom port of vertical_camera_holder
Connect FLIR camera to a computer via USB 3.1 Gen 1 Micro-B to USB-A Locking Cable. See Figure 2 L.

* Alternatively, the Arducam camera can be used. To connect the camera to the computer a USB USB2.0 cable were used.

Lighting System Assembly

* Solder jumper wires to the 3W LEDs (flat form factor) for vertical illumination. Please note one or two LEDs may be needed. LEDs with a standard form factor can be used for the side illumination. These LEDs can be directly attached via jumper wires.

* Attach the 3W LED on the optical breadboard. Roughly align the LED with the objective for vertical imaging between $(12,6)$ and $(12,7)$. See Figure 3 A.

* Run wires underneath optical breadboard in port $(13,6)$.  The electrical breadboard connects these wires to the circuit and 9V battery.

* Use foam materials inside the rectangular opening of the horizontal_light for securing the side illumination LED. Then, attach horizontal_light to the optical breadboard via $(14,5), (14,6)$. Figure 3 B.

* Connect the LEDs to jumper cables. Roughly align the LED with the horizontal objective.

* Move the wires underneath optical breadboard via $(15,6)$ to be later connected to the electrical circuit.

* Attach the breadboard_and_battery_holder to optical breadboard via $(3,2), (3,3)$ and $(7,2),(7,3)$ positions.

* Attach the Micromanipulator stage into ports $(11,8)$ and $(13,8)$.

* Place the electrical breadboard and a 9V battery in their respective holders.

* Using two 350 $\Omega$ resistors (or two potentiometers) and necessary wires, create two series circuits to power the LEDs via the 9V battery. See Figure 3 F. Please note that it is important to select resistors (or two potentiometers) with appropriate power rating.

* Bring the wires from both LEDs to above the optical breadboard through ports $(8,3)$ and $(8,2)$ and then connect them to the electrical circuit. See Figure 3 F.

* Place the diffuser holder v2 above the 3W LED for vertical illumination. See Figure 3 D and E for the position.

* Place an optical diffuser inside of the diffuser holder v2 for better illumination.

§ OPERATION INSTRUCTIONS

§.§ Camera configurations:
The GLUBscope cameras can be operated with one or two computers. In this, we typically used two computers. The use of more high resolution (20 MP) and economical cameras was demonstrated. Two 20 MP cameras (FLIR) were used for the first option. These were connected via a USB 3.1 Gen 1 Micro-B to USB-A Locking cable and controlled via open-source Micromanager software.
Please note that it is important to select the correct hardware configuration files.
The OSF repository provides guides (written and video instructions) for the configuration selection. For the economical option, Arducam cameras (webcams) were used. These cameras were connected to the computer with a USB 2.0 cable . They were controlled via the AMCap software.

§.§ Sample positioning:
All components should be level for the best alignment. Next, turn on the LEDs to confirm that the cameras can collect live images. The rail guides are used for rough focus, and the XYZ micromanipulator stage is used to fine-focus the sample. Typically an edge of a slide or a cuvette can be used as a target for focusing an image. Please note that the light sources can be moved to better illuminate the entire field of view. In our experience, the LEDs could be too close to the sample, only illuminating a portion of the field. The optional diffusers (for the top view) improved the illumination. Please note that simultaneously viewing the same field of view (top and side) may require adjustments. Using a lower magnification for the top view was helpful to this aim. Our investigations used a 4X objective for the top view and a 10X for the side view. The working distance of the objectives was not an issue. This parameter should be considered for higher magnification objectives. Objectives can be changed based on a project's needs. Additionally, LEDs can be adjusted - the distance between the light source and the sample and brightness - to better illuminate the sample.

§.§ 3D printing:
Components were 3D printed via a
Snapmaker 2.0 with Polylactic Acid (PLA) filaments. The final version of guide rails was 3D printed with nylon composites to ensure mechanical durability.

§ VALIDATION AND CASE STUDIES

GLUBscope was validated and proof-of-use experiments were investigated with the following experiments.

Calibration slides: Standard slides (i.e., a 1951 Resolution Test Target slide and a 0.01mm calibration slide) were used to obtain top and side images. Images generated from the references are presented in (Figure 4 A and B). Please note that the top view of 0.01mm is illuminated via an LED without a diffuser. GLUBscope was also used as a fluorescence microscope. To this aim, the white LEDs were switched with bright green LEDs. 5 $\mu$m green fluorescent beads were used as the sample. Beads were suspended in DI water and were added to the cuvettes (3.5 mL volume, 10mm path length, 4 Clear Windows). Preliminary results are presented as supplementary material (SupFigure 1). We note this feature requires further optimizations to improve the excitation light source.

Organism Biomechanics: Proof-of-use experiments were conducted to demonstrate using GLUBscope in biomechanics investigations. In the first demonstration, freshwater polyps Hydra vulgaris were imaged using the top-view (4X) and side-view (10x) (see Figure 5A). A simple analysis pipeline was also developed to enable biomechanical investigationsFigure 5B. Using the video recordings from the microscope and DeepLabCut application [28], anatomical landmarks from the organism were tracked. These included features such as the foot, two points across the body column, tentacles, and the head Figure 5B. In the presented position plot, blue squares represent the starting position, and yellow triangles mark the final position. Changes in position are highlighted by lines connecting these two shapes with color coding encoding time. Moreover, two simple applications, Frame to segment and Segment to distance, were developed for additional biomechanical analysis. Using the position data generated by DeepLabCut, initial and final length segments and percentage elongation of the segments were calculated.
The source code for both applications is available from the OSF repository.

Next GLUBscope was used to visualize sand dollar larvae (Dendraster Excentricus) and sea anemones SupFigure 2. In the case of larvae, GLUBscope enabled the imaging of the complex structure of the larvae from multiple angles. While preliminary, this ability could provide a unique approach to visualizing the complex hydrodynamics of these organisms. Side visualization was critical for the final demonstration, imaging sea anemones since the organisms were attached to an opaque material that blocked light transmission from the top view.
Collectively, the three proof-of-use experiments demonstrate various applications of GLUBscope to study  aquatic organisms.

§ CONCLUSION

We demonstrated GLUBscope, a modular DIY microscope
made from 3D-printed parts and off-the-shelf components. GLUBscope was designed to enable imaging of samples from two orthogonal planes. This feature is useful for biomechanical investigation of aquatic organisms. Brightfield illumination was used for most of the study, and preliminary fluorescent imaging was also demonstrated. Better sources of illumination (brighter LEDs, or laser) similar to our prior efforts are needed  [29] are needed to advance this feature. To record from two planes of imaging, the implementation required two cameras. We demonstrated the use of high-resolution/high-sensitivity and more economical options. This demonstration is valuable since the cameras are the most expensive component. As part of the report, proof-of-use experiments and analysis pipelines (specifically biomechanical studies of Hydra) were demonstrated. Organisms in the current system were housed inside a cuvette (3.5 mL, 10 mm path) with four polished sides. In future experiments, microfluidics or other customized flow chambers, as described in [30], could be employed. It is important to note that investigating certain aspects of the hydrodynamics of freely moving aquatic organisms (e.g., sand-dollar larvae) may require modifications, larger volumes, and complex flow microenvironments. Among the existing DIY microscopes, the scale-free tracking system [24, 25] is an elegant approach for providing a free-path for the movement of organisms in one direction. Conventional chambers, micro and millifluidics can still serve as valuable tools for many investigations.

GLUBscope is of interest in the biomechanics and hydrodynamics of aquatic organisms. Beyond their biological importance, these investigations could be crucial in mitigating threats due to climate change [6]. Climate change threatens the planet's overall biodiversity. The recent Intergovernmental Panel on Climate Change (IPCC) reports specifically highlight a significant risk of permanent damage to aquatic organisms and their ecosystems [31]. Therefore, broader access to resources (e.g., microscopes) and closer investigation of the aquatic organisms and their responses to the emerging climate change challenges are needed.

List of Figures:

* Figure 1: GLUBscope - system overview.

* Figure 2: Build instructions part 1. hardware and optics.

* Figure 3: Build instructions part 2: LEDs, electronics, and stage.

* Figure 4: System verification via two standard slides.

* Figure 5: GLUBscope - Hydra case study.

List of Supplementary Videos:

* Supplementary Video 1: Hydra side view.

* Supplementary Video 2: Hydra top view.

* Supplementary Video 3: Hydra side view

* Supplementary Video 4: Hydra top view - tracking.

List of Supplementary Figures:

* Supplementary Figure -1: Preliminary results demonstrating fluorescent imaging.

* Supplementary Figure -2:  GLUBscope -  case studies

Animals and human rights: Not applicable.

CRediT author statement

Katherine Auer: Methodology, Data Curation, Validation, Formal analysis, Investigation. Brian Le: Methodology, Data Curation, Validation, Formal analysis, Investigation. David Lopez: Methodology, Data Curation, Validation, Formal analysis, Investigation. Justin P. Shum: Methodology, Data Curation, Validation, Formal analysis, Investigation.
Brian Suarsana: Methodology, Data Curation, Validation, Formal analysis, Investigation. Ga-Young Kelly Suh: Methodology, Validation, Resources. Per Niklas Hedde: Conceptualization, Methodology, Funding acquisition. Siavash Ahrar: Conceptualization, Methodology, Data Curation, Resources, Supervision, Project administration, Funding acquisition. All authors contributed to manuscript preparation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal conflict or competing interest.

The authors express gratitude to members of Ahrar-lab specifically Moses Villeda and Natalia Margaris, Prof. Rob Steele (UC Irvine), members of Pace-lab (CSULB), and CSULB Marine Lab specifically Yvette Ralph.

This work was supported in part by:
1. CSUPERB award to SA. 2. National Institutes of General Medical Sciences (R21GM135493) award to PNH.
Granting agencies are not responsible for the content of the manuscript.

[1]
Jason A Carter, Callen Hyland, Robert E Steele, and Eva-Maria S Collins.
Dynamics of mouth opening in hydra.
Biophysical journal, 110(5):1191–1201, 2016.

[2]
Anniek Stokkermans, Aditi Chakrabarti, Kaushikaram Subramanian, Ling Wang,
Sifan Yin, Prachiti Moghe, Petrus Steenbergen, Gregor Mönke, Takashi
Hiiragi, Robert Prevedel, et al.
Muscular hydraulics drive larva-polyp morphogenesis.
Current Biology, 32(21):4707–4718, 2022.

[3]
John R Szymanski and Rafael Yuste.
Mapping the whole-body muscle activity of hydra vulgaris.
Current Biology, 29(11):1807–1817, 2019.

[4]
Kit Yu Karen Chan and D Grünbaum.
Temperature and diet modified swimming behaviors of larval sand
Marine Ecology Progress Series, 415:49–59, 2010.

[5]
Anna A Kim, Adam L Nekimken, Sylvia Fechner, Lucy E O’Brien, and Beth L
Microfluidics for mechanobiology of model organisms.
In Methods in cell biology, volume 146, pages 217–259.
Elsevier, 2018.

[6]
Kit Yu Karen Chan.
Biomechanics of larval morphology affect swimming: insights from the
sand dollars dendraster excentricus, 2012.

[7]
Nagaraj Nagalingam, Aswin Raghunathan, Vikram Korede, Edwin FJ Overmars,
Shih-Te Hung, Remco Hartkamp, Johan T Padding, Carlas S Smith, and
Huseyin Burak Eral.
Low-cost fluorescence microscope with microfluidic device fabrication
for optofluidic applications.
HardwareX, 14:e00415, 2023.

[8]
Tobias Wenzel.
Open hardware: From diy trend to global transformation in access to
laboratory equipment.
PLoS biology, 21(1):e3001931, 2023.

[9]
Jesus Salido, Gloria Bueno, Jesus Ruiz-Santaquiteria, and Gabriel Cristobal.
A review on low-cost microscopes for o pen s cience.
Microscopy research and technique, 85(10):3270–3283, 2022.

[10]
David N Breslauer, Robi N Maamari, Neil A Switz, Wilbur A Lam, and Daniel A
Mobile phone based clinical microscopy for global health
PloS one, 4(7):e6320, 2009.

[11]
Hongying Zhu, Oguzhan Yaglidere, Ting-Wei Su, Derek Tseng, and Aydogan Ozcan.
Cost-effective and compact wide-field fluorescent imaging on a
Lab on a Chip, 11(2):315–322, 2011.

[12]
James S Cybulski, James Clements, and Manu Prakash.
Foldscope: origami-based paper microscope.
PloS one, 9(6):e98781, 2014.

[13]
Asa Tapley, Neil Switz, Clay Reber, J Lucian Davis, Cecily Miller, John Baptist
Matovu, William Worodria, Laurence Huang, Daniel A Fletcher, and Adithya
Mobile digital fluorescence microscopy for diagnosis of tuberculosis.
Journal of clinical microbiology, 51(6):1774–1778, 2013.

[14]
Per Niklas Hedde.
minispim—a miniaturized light-sheet microscope.
ACS sensors, 6(7):2654–2663, 2021.

[15]
Muhammad A Sami, Muhammad Tayyab, and Umer Hassan.
Excitation modalities for enhanced micro and nanoparticle imaging in
a smartphone coupled 3d printed fluorescent microscope.
Lab on a Chip, 22(19):3755–3769, 2022.

[16]
Lydia Skolrood, Yan Wang, Shengwei Zhang, and Qingshan Wei.
Single-molecule and particle detection on true portable microscopy
Sensors and Actuators Reports, 4:100063, 2022.

[17]
Mario Del Rosario, Hannah S Heil, Afonso Mendes, Vittorio Saggiomo, and Ricardo
The field guide to 3d printing in optical microscopy for life
Advanced Biology, 6(4):2100994, 2022.

[18]
Johannes Hohlbein, Benedict Diederich, Barbora Marsikova, Emmanuel G Reynaud,
Séamus Holden, Wiebke Jahr, Robert Haase, and Kirti Prakash.
Open microscopy in the life sciences: quo vadis?
Nature Methods, 19(9):1020–1025, 2022.

[19]
Joel T Collins, Joe Knapper, Julian Stirling, Joram Mduda, Catherine Mkindi,
Valeriana Mayagaya, Grace A Mwakajinga, Paul T Nyakyi, Valerian L Sanga, Dave
Carbery, et al.
Robotic microscopy for everyone: the openflexure microscope.
Biomedical Optics Express, 11(5):2447–2460, 2020.

[20]
Benedict Diederich, René Lachmann, Swen Carlstedt, Barbora Marsikova,
Haoran Wang, Xavier Uwurukundo, Alexander S Mosig, and Rainer Heintzmann.
A versatile and customizable low-cost 3d-printed open standard for
microscopic imaging.
Nature communications, 11(1):5979, 2020.

[21]
Andre Maia Chagas, Lucia L Prieto-Godino, Aristides B Arrenberg, and Tom Baden.
The€ 100 lab: A 3d-printable open-source platform for fluorescence
microscopy, optogenetics, and accurate temperature control during behaviour
of zebrafish, drosophila, and caenorhabditis elegans.
PLoS biology, 15(7):e2002702, 2017.

[22]
T Pollina, AG Larson, F Lombard, H Li, D Le Guen, S Colin, C de Vargas, and
M Prakash.
Planktoscope: affordable modular quantitative imaging platform for
citizen oceanography. front.
Technological Advances for Measuring Planktonic Components of
the Pelagic Ecosystem: An Integrated Approach to Data Collection and
Analysis, 9:123, 2023.

[23]
Planktoscope project.
<https://www.planktoscope.org/discover>, Last accessed on

[24]
Deepak Krishnamurthy, Hongquan Li, Francois Benoit du Rey, Pierre Cambournac,
Adam G Larson, Ethan Li, and Manu Prakash.
Scale-free vertical tracking microscopy.
Nature Methods, 17(10):1040–1051, 2020.

[25]
Deepak Krishnamurthy, Rachel Pepper, and Manu Prakash.
Active sinking particles: sessile suspension feeders significantly
alter the flow and transport to sinking aggregates.
Journal of the Royal Society Interface, 20(199):20220537, 2023.

[26]
Brandon G Wong, Adrian Paz, Michael A Corrado, Brian R Ramos, Amanda Cinquin,
Olivier Cinquin, and Elliot E Hui.
Live imaging reveals active infiltration of mitotic zone by its stem
cell niche.
Integrative Biology, 5(7):976–982, 2013.

[27]
Adriana San-Miguel and Hang Lu.
Microfluidics as a tool for c. elegans research.
WormBook: The Online Review of C. elegans Biology [Internet],

[28]
Tanmay Nath, Alexander Mathis, An Chi Chen, Amir Patel, Matthias Bethge, and
Mackenzie Weygandt Mathis.
Using deeplabcut for 3d markerless pose estimation across species and
Nature protocols, 14(7):2152–2176, 2019.

[29]
Per Niklas Hedde, Brian T Le, Erika L Gomez, Leora Duong, Robert E Steele, and
Siavash Ahrar.
Spim-flow: An integrated light sheet and microfluidics platform for
hydrodynamic studies of hydra.
Biology, 12(1):116, 2023.

[30]
Per Niklas Hedde, Leonel Malacrida, Siavash Ahrar, Albert Siryaporn, and Enrico
sidespim–selective plane illumination based on a conventional
inverted microscope.
Biomedical Optics Express, 8(9):3918–3937, 2017.

[31]
Climate change 2022: Impacts, adaptation, and vulnerability. contribution of
working group ii to the sixth assessment report of the intergovernmental
panel on climate change, 2022.

GLUBscope - system overview. (A,B) Photographs of the finished GLUBscope. In this configuration two different cameras are used. (C) System block diagram of the major optical components for the GLUBscope.

Build instructions part 1. hardware and optics.  (A:C)  Positioning of components (posts and guide rails) for imaging.  (D:H) Horizontal or side-view imaging components. (I:L) Vertical or top-view imaging components.

Build instructions part 2: LEDs, electronics, and stage. (A:E) LEDs for vertical and side illuminations. Use of diffuser and holder are highligted in pannels D and E. In this example, bright field LEDs are demonstrated. (F) Electronic components to power the LEDs. (G) Fully assembled GLUBscope.

System verification via standard slides. (A) Reference slide-1 from top and side view. (B) Reference slide-2 from top and side view

GLUBscope - Hydra case study. (A) Hydra visualized from both top and side views. (B) Sample pipeline of tracking anatomical features (via DeepLabCut)and additional applications developed to estimate segment lengths and elongations. Plot demonstrates the staring and the final position of the anatomical features.

Supplementary Figure-1: fluorescent imaging. (A) Green fluorescent beads (5 microns) were mixed in DIY water and vortexed inside a cuvette (3.5 mL volume and path length 1 cm). An off-the-shelf Green LED was used to illuminate the sample. (B) A region of interest (ROI) and a sample line for intensity measurements were selected. (C) The intensity profile along the sample line indicating the bead clusters. The signal to noise could be improved by using brighter excitation light source and reducing background .

Supplementary Figure -2: GLUBscope -  case studies. (A-B) top and side views of sand dollar larvae (C) Sea anemone attached to a rock inside the sample holder and on the stage. (D) Sea anemone imaged from the side view with GLUBscope.


# DAM-GAN : Image Inpainting using Dynamic Attention Map based on Fake Texture
Detection

###### Abstract

Deep neural advancements have recently brought remarkable image synthesis
performance to the field of image inpainting. The adaptation of generative
adversarial networks (GAN) in particular has accelerated significant progress
in high-quality image reconstruction. However, although many notable GAN-based
networks have been proposed for image inpainting, still pixel artifacts or
color inconsistency occur in synthesized images during the generation process,
which are usually called fake textures. To reduce pixel inconsistency disorder
resulted from fake textures, we introduce a GAN-based model using dynamic
attention map (DAM-GAN). Our proposed DAM-GAN concentrates on detecting fake
texture and products dynamic attention maps to diminish pixel inconsistency
from the feature maps in the generator. Evaluation results on CelebA-HQ and
Places2 datasets with other image inpainting approaches show the superiority
of our network.

Index Terms—  CNN, Computer Vision, GAN, Image Inpainting, Image Completion

##  1 Introduction

Image inpainting, or image completion, is a task about image synthesis
technique aims to filling occluded regions or missing pixels with appropriate
semantic contents. The main objective of image inpainting is producing
visually authentic images with less semantic inconsistency using computer
vision-based approaches. Traditional methods relied on a patch-based matching
approach using the measurement of cosine similarity [1]. Recently, the
remarkable capability of generative adversarial networks (GAN) [2] has boosted
image inpainting performance based on convolutional neural networks (CNN).
Because of its hierarchical design, GAN with encoder-decoder structure has
exceptional reconstruction ability compared to previous approaches. The
decoder synthesizes visual images from the feature level as the encoder learns
how to extract feature representations from images. Currently, GAN-based
approaches constitute a dominant stream in image inpainting [3, 4, 5, 6, 7,
8].

However, despite GAN’s high image restoration performance, some pixel
artifacts or color inconsistency called ’fake texture’ inevitably occur in the
process of decoding [9, 10]. Fake pixels cause degradation of image
restoration performance by dropping the appearance consistency in the
synthesized image. To tackle this issue, we introduce dynamic attention map
(DAM) that detects fake textures in feature map and highlights them by
generating an attention mask (or attention map) [11] for image inpainting.
Unlike existing GAN-based inpainting methods requiring high computational cost
for generating attention map [12, 8], our proposed DAM blocks exploit
learnable convolutional layers for detecting fake texture and converting it
into an attention map for each different scale of each decoding layer. We
reported the comparisons on CelebA-HQ and Places2 datasets and showed that
outcome of our DAM-GAN demonstrating higher quality than other existing
inpainting methods including GAN-based approaches.

Fig. 1: Overview of GAN-based image inpainting using our proposed dynamic
attention map (DAM) module.

Fig. 2: Overall structure of generator $G$ in DAM-GAN. The coarse
reconstruction part has a 3-level structure, and the DAM reconstruction part
has a 4-level structure, including skip-connection and dynamic attention map
(DAM) blocks.

##  2 Related Works

Traditional image inpainting methods were based on the exemplar-search
approach, which divides image into patches to refill missing areas with other
patches according to similarity computations such as PatchMatch [1]. Recently,
progressive improvement of deep learning based generative models have
demonstrated high feasibility for image synthesis. Especially GAN [2]
demonstrates brilliant performance in image inpainting tasks. Context Encoders
(CE) [3] adopted encoder-decoder based GAN for image inpainting and Globally
and Locally (GL) [4] incorporates global and local generators to maintain
pixel consistency of output images. Contextual Attention (CA) [6] imitated the
traditional patch-based method using GAN to take advantage of the basic
concept of conventional exemplar-based methods. However, CE [3], GL [4] and CA
[6] have limitations on refilling irregular regions because of their local
region based discriminators. Since they are usually specialized in
reconstructing rectangular masks, images with free-shaped masks will decrease
the quality of outputs. To tackle this limitations, recent inpainting
approaches tend to remove local discriminator on architecture [13].

Partial conv [14] did not employ GAN for inpainting, but solved the problem of
generalization on irregular masks. It propose rule-based binary mask which is
updated layer by layer in encoder-decoder network and showed high feasibility
of refilling irregular masks. This mask-based inpainting approach is advanced
in Gated conv [7] by adopting GAN and replacing rule-based mask with learnable
mask. Both Partial conv [14] and Gated conv [7] put forward a mask-based
weights map for feature maps in the decoding process, similar to attention map
[11] based method.

##  3 Proposed Method

Given original ground-truth $\hat{x}$, we apply occlusion mask to make input
image $x$. The GAN-based inpainting network contains generator $G$ and
discriminator $D$. Through the inpainting process in encoder-decoder
architecture of generator, the output image $G(x)$ is obtained. In this
section, we introduce our inpainting network with our proposed DAM module and
loss functions for training our model.

Fig. 3: An architecture of DAM block. Four DAM blocks are located in each
layer of the decoder. Inside the i-th block, weight map ${M}_{i}$ is trained
on ${L}_{DAM}$. $\otimes$ and $\oplus$ denote element-wise multiplication and
summation between feature maps.

###  3.1 GAN Framework

The goal of generator $G$ is to fill missing parts with appropriate contents
by understanding the input image $x$ (encoding) and synthesizing the output
image $G(x)$ (decoding). Fig. 2 describes the overall architecture of
generator $G$. The coarse reconstruction stage begins by filling pixels with a
rough texture. The DAM reconstruction then uses DAM blocks to restore the
coarse output ${G}_{C}(x)$ with detailed contents. We defined the residual
convolution layer by combining residual block [15] and convolution layer, and
we adopted concatenation-based skip-connection [16] and dilated convolution
[17] in the middle of the generator. Skip-connections have a notable effect on
reducing vanishing gradient problems and maintaining spatial information of
reconstructed images, and dilated convolution increases the receptive field to
enhance the efficiency of the computations.

Discriminator $D$ serves as a criticizer that distinguishes between real and
synthesized images. Adversarial training between $G$ and $D$ can further
improve the quality of synthesized image. Because local discriminator has
critical limitations on handling irregular mask as mentioned in section 2., we
use one global discriminator for adversarial training our model. We employed
the global discriminator from CA [6].

###  3.2 Dynamic Attention Map Block

Dynamic attention map (DAM) block located in each last four decoding layers
from the generator $G$. The structure and function of DAM block are
illustrated in Fig. 3. The concatenated feature $[{T}_{i},{S}_{i}]$ passes
through a 1x1 convolutional filter, and input feature ${F}_{i}$ is obtained.

Similar to fakeness prediction in [9], fakeness map ${M}_{i}$ is produced
through 1x1 convolutional filters and sigmoid function from feature ${F}_{i}$.
Then, we can use ${M}_{i}$ as an attention map like [11]. After element-wise
multiplication of ${M}_{i}\otimes{F}_{i}$, the output feature
${F^{\prime}}_{i}$ is obtained. Then element-wise sum
${F}_{i}\oplus{F^{\prime}}_{i}$ becomes the final output ${T}_{i-1}$, which is
upsampled and passed to the upper layer in the decoder. Fakeness map ${M}_{i}$
is trainable dynamically in each layer from decoder using DAM loss
$\mathcal{L}_{DAM}$, which is expressed in section 3.

###  3.3 Loss Functions

Reconstruction Loss  Image inpainting models are trained in pixel space by
reducing the pixel-wise distance between ground-truth and synthesized images.
We train both the coarse part and DAM part in the reconstruction process as
shown in Eq. 1.

$\mathcal{L}_{re}=|\hat{x}-{G}_{C}(x)|_{1}+|\hat{x}-G(x)|_{1}$ (1)

Where $x$ and $\hat{x}$ represent masked image and ground-truth, ${G}_{C}(x)$
and $G(x)$ denote coarse and final output.

Adversarial Loss  Generator and discriminator are trained in a competitive
relationship in a mini-max game to perform their roles. Generator $G$ tries to
reconstruct the input image $x$ into inpainted image $G(x)$ as similar to the
real image $\hat{x}$. Otherwise, discriminator $D$ is trained to distinguish
real image $\hat{x}$ from fake image $x$ as possible. The adversarial loss is
denoted in Eq. 2. $D$ aims to maximize the adversarial loss
$\mathcal{L}_{adv}$ while $G$ tries to minimize it.

$\mathcal{L}_{adv}={\mathbb{E}}_{\hat{x}}[\log
D(\hat{x})]+{\mathbb{E}}_{x}[\log(1-D(G(x)))]$ (2)

Fig. 4: Comparisons on CelebA-HQ [18] and Places2 [19] using the centered mask
and random masks. From left to right: (a) Ground-truth, (b) Input, (c) CE [3],
(d) CA [6], (e) Partial [14], (f) Gated [7] and (g) Our DAM-GAN.

DAM Loss  In each decoder layer, our proposed DAM block generates an attention
map based on pixels from detected fake texture. We can consider those fake
regions as pixels that will be highlighted during the image synthesis process.
By computing the gray-scale pixel difference between real and inpainted
images, the DAM block dynamically learns fake prediction functions from a
ground-truth fakeness map. Then all pixels are divided by 255 to normalize
them between [0, 1]. Formally, DAM loss can be described as in Eq. 3.

$\begin{gathered}{M}^{GT}_{j}=grayscale(|{G(x)}_{j}-{\hat{x}}_{j}|)/255.0\\\
\mathcal{L}^{j}_{DAM}=|{M}_{j}-{M}^{GT}_{j}|_{1}\qquad\mathcal{L}_{DAM}=\sum_{j=0}^{3}\mathcal{L}^{j}_{DAM}\end{gathered}$
(3)

Where $j\subseteq\\{0,1,2,3\\}$, the j-th ground-truth mask ${M}^{j}_{GT}$ is
obtained from the difference between real image (${\hat{x}}^{j}$) and
inpainted image (${G(x)}^{j}$) resized with the same size of ${M}^{j}$.

Table 1: Quantitative comparison results of image inpainting with CE [3], CA [6], Partial [14], Gated [7] and our DAM-GAN on CelebA-HQ dataset. The highest performances are marked in bold. | Center | Free
---|---|---
|  PSNR | SSIM | PSNR | SSIM
CE |  22.56 |  0.864 |  27.20 |  0.939
CA |  23.06 |  0.875 |  28.23 |  0.954
Partial |  23.57 |  0.884 |  29.04 |  0.952
Gated |  24.04 |  0.892 |  29.11 |  0.952
Ours |  24.77 |  0.903 |  29.49 |  0.960

Full Objective Loss  The total loss function of DAM-GAN is defined as in Eq.
4. The hyper-parameters ${\lambda}_{re},{\lambda}_{adv}$ and ${\lambda}_{DAM}$
denotes predefined weights for each component.

$\mathcal{L}_{total}={\lambda}_{re}\cdot\mathcal{L}_{re}+{\lambda}_{adv}\cdot\mathcal{L}_{adv}+{\lambda}_{DAM}\cdot\mathcal{L}_{DAM}$
(4)

##  4 Experiments

###  4.1 Implementation Details

Our model was trained on two datasets: CelebA-HQ and [18] Places2 [19]. We
randomly divided the 30,000 images in CelebA-HQ dataset into a training set of
27,000 images and a validation set of 3,000 images. In Places2 dataset, we
select same categories as [20] in training set and tested our model on
validation set. All images are resized to 128 $\times$ 128\. To prepare input
images for our model, we defined the centered mask and random mask. The
centered mask has 64 $\times$ 64 size fixed in the center of the image, and
the random mask has an irregular shape following the mask generation approach
in [21]. We used an ADAM optimizer [22] in this experiment, and hyper-
parameters are set to ${\lambda}_{re}=1,{\lambda}_{adv}=0.001$ and
${\lambda}_{DAM}=0.005$.

###  4.2 Quantitative Results

As mentioned in CA [6], image inpainting lacks a definable numerical metric
for assessing inpainting results. Therefore, evaluation metrics in this field
are still limited to pixel-level image quality metrics. The most commonly used
quality comparison metrics in GAN tasks are the peak signal-to-noise ratio
(PSNR) and the structural similarity index (SSIM) [23]. We conducted
quantitative comparisons on CelebA-HQ as shown in Table 1. with four state-of-
the-art inpainting benchmarks: CE [3], CA [6], Partial [14] and Gated [7].
However, since the local discriminator of CA [6] cannot handle random masks,
we conducted all experiments using only a global discriminator.

###  4.3 Qualitative Results

We displayed the results of our image inpainting and other four image
inpainting approaches on CelebA-HQ and Places2 in Fig. 4. It shows that our
DAM-GAN trained with a dynamic attention map based on fakeness detection
produces better visual image inpainting results than other models.

##  5 CONCLUSION

In this paper, we proposed a deep image inpainting generative model with
dynamic attention map (DAM) blocks for weighting specific pixels in a feature
map based on fake detection map. During training, the network learns itself
how to refill missing regions using dynamically-learnable maps. We
demonstrated that our DAM-GAN outperformed other inpainting models in terms of
inpainting performance by comparing quantitative results.

##  6 ACKNOWLEDGEMENTS

This work was supported by Institute of Information & communications
Technology Planning & Evaluation(IITP) grant funded by the Korea
government(MSIT) (No.B0101-15-0266, Development of High Performance Visual
BigData Discovery Platform for Large-Scale Realtime Data Analysis),
(No.2017-0-00897, Development of Object Detection and Recognition for
Intelligent Vehicles) and (No.2018-0-01290, Development of an Open Dataset and
Cognitive Processing Technology for the Recognition of Features Derived From
Unstructured Human Motions Used in Self-driving Cars)

## References

  * [1] C. Barnes, E. Shechtman, A. Finkelstein, et al.,  “Patchmatch: A randomized correspondence algorithm for structural image editing,”  ACM Trans. Graph., vol. 28, no. 3, pp. 24, 2009.
  * [2] I. Goofellow, J. Pouget-Abadie, M. Mirza, et al.,  “Generative adversarial nets,”  Advances in neural information processing systems, vol. 27, 2014\.
  * [3] D. Pathak, P. Krahenbuhl, J. Donahue, et al.,  “Context encoders: Feature learning by inpainting,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2536–2544.
  * [4] S. Iizuka, E. Simo-Serra, and H. Ishikawa,  “Globally and locally consistent image completion,”  ACM Transactions on Graphics (ToG), vol. 36, no. 4, pp. 1–14, 2017\.
  * [5] Y. Song, C. Yang, Z.Lin, et al.,  “Contextual-based image inpainting: Infer, match, and translate,”  in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 3–19.
  * [6] J. Yu, Z. Lin, J. Yang, et al.,  “Generative image inpainting with contextual attention,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5505–5514.
  * [7] J. Yu, Z. Lin, J. Yang, et al.,  “Free-form image inpainting with gated convolution,”  in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4471–4480.
  * [8] N. Wang, S. Ma, J. Li, et al.,  “Multistage attention network for image inpainting,”  Pattern Recognition, vol. 106, pp. 107448, 2020.
  * [9] Y. Huang, F. Juefei-Xu, R. Wang, et al.,  “Fakelocator: Robust localization of gan-based face manipulations,”  arXiv preprint arXiv:2001.09598, 2020.
  * [10] X. Zhang, S. Karaman, and S.F. Chang,  “Detecting and simulating artifacts in gan fake images,”  in 2019 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 2019, pp. 1–6.
  * [11] H. Zhang, I. Goodfellow, D. Metaxas, et al.,  “Self-attention generative adversarial networks,”  in International conference on machine learning. PMLR, 2019, pp. 7354–7363.
  * [12] C. Xie, S. Liu, C. Li, et al.,  “Image inpainting with learnable bidirectional attention maps,”  in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 8858–8867.
  * [13] Y. Fang, Y. Li, X. Tu, et al.,  “Face completion with hybrid dilated convolution,”  Signal Processing: Image Communication, vol. 80, pp. 115664, 2020\.
  * [14] G. Liu, F.A. Reda, K.J. Shih, et al.,  “Image inpainting for irregular holes using partial convolutions,”  in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 85–100.
  * [15] K. He, X. Zhang, S. Ren, et al.,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [16] O. Ronneberger, P. Fischer, and T. Brox,  “U-net: Convolutional networks for biomedical image segmentation,”  in International Conference on Medical image computing and computer-assisted intervention. Springer, 2015, pp. 234–241.
  * [17] L.C. Chen, G. Papandreou, I. Kokkinos, et al.,  “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,”  IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 4, pp. 834–848, 2017.
  * [18] T. Karras, T. Aila, S. Laine, et al.,  “Progressive growing of gans for improved quality, stability, and variation,”  arXiv preprint arXiv:1710.10196, 2017.
  * [19] B. Zhou, A. Lapedriza, A. Khosla, et al.,  “Places: A 10 million image database for scene recognition,”  IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 6, pp. 1452–1464, 2017.
  * [20] H. Liu, Z. Wan, W. Huang, et al.,  “Pd-gan: Probabilistic diverse gan for image inpainting,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9371–9381.
  * [21] H. Park and D. Kim,  “Acn: Occlusion-tolerant face alignment by attentional combination of heterogeneous regression networks,”  Pattern Recognition, vol. 114, pp. 107761, 2021.
  * [22] D.P. Kingma and J. Ba,  “Adam: A method for stochastic optimization,”  arXiv preprint arXiv:1412.6980, 2014.
  * [23] Z. Wang, A.C. Bovik, H.R. SheikhHamid, et al.,  “Image quality assessment: from error visibility to structural similarity,”  IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.