Joule- Thomson expansion of $AdS$ black holes in

Einstein Power- Yang- Mills gravity

Anindya Biswas111Electronic address<EMAIL_ADDRESS>
Department of Physics,

Ranaghat College, Ranaghat, India.

Abstract

In this paper we study Joule-Thomson $(JT)$ expansion of non-linearly charged
$AdS$ black holes in Einstein-power-Yang-Mills (EPYM) gravity in $D$
dimensions. Within the framework of extended phase space thermodynamics we
identify the cosmological constant as thermodynamic pressure and the black
hole mass with the enthalpy and derive the Joule- Thomson coefficient $\mu$.
Furthermore we have presented equations for inversion curves and the exact
expression for the minimum inversion temperature. We also have calculated the
ratio between the minimum of inversion $T_{i}^{min}$ and the critical
temperature $T_{c}$ and obtained the analytic expression for the ratio
$\frac{T_{i}^{min}}{T_{c}}$ that depends explicitly on the non- linearity
parameter $q$ and dimension $D$. We consider the isenthalpic curves in the
$T-P$ plane for different values of the fixed black hole mass and obtain
heating and cooling region. Finally we have dealt with two limiting masses
which characterizes the process of Joule-Thomson expansion in the $EPYM$ black
holes.

##  1 Introduction

For past few decades the research towards the quantum theory of gravity has
been emerging successfully with the pioneering work of Hawking and Bekenstein
[1]\- [3] in the branch of black hole physics. The discovery of black hole
radiation [2] has opened up a new avenue to understand black hole space- time
by introducing its temperature and entropy which are considered in any other
thermodynamic systems. The study of thermodynamic properties of black holes
have got immense success by the discovery of black hole phase transition in
anti-de Sitter $(AdS)$ space-time with a negative cosmological constant
following Hawking and Page [4]. This particular phase transition is occurred
between a stable Schwarzschild- $AdS$ black hole and the thermal AdS space and
is called Hawking- Page $(HP)$ phase transition. With the advent of Anti de-
Sitter/ Conformal field theory $(AdS/CFT)$ correspondence [5], $HP$\- phase
transition of black holes in $AdS$ space- time have shown an interesting
connection with the confining- deconfining phase transition of $\mathcal{N}=4$
Yang-Mills gauge theory [6], [7]. The authors of [8, 9] have observed that for
charged $AdS$ black holes [i.e., Reissner-Nordstr$\ddot{o}$m $AdS$ $(RNAdS)$
black holes] the first order phase transition is happened between small and
large black hole in a fixed charge ensemble is like van der Waals liquid- gas
phase transition. The analogy has got much attention when the cosmological
constant $\Lambda$ is identified with the thermodynamic pressure [10, 11, 12]
in $AdS$ space. Variation of $\Lambda$ in first law of black hole
thermodynamics suggest that the mass of an $AdS$ black hole should be
interpreted as the enthalpy of the space time [10, 11] rather than internal
energy in contrast to the standard notion of classical thermodynamics.
Treating cosmological constant as pressure $P$ and its conjugate thermodynamic
volume $V$ complete the similarity between $AdS$ black holes and the Van der
Waals fluid in an extended phase space with a $PdV$ term in the first law of
black hole thermodynamics. In this context many authors have explored various
thermodynamic phenomena like holographic heat engine [13], compressibility of
rotating class of black holes [14], universality class of black hole phase
transition at critical point [15]-[17] etc. Another thermodynamic issue which
has got much attention recently is the Joule- Thomson expansion of black
holes. The Joule- Thomson expansion of black holes in $AdS$ space- time was
first explored in [18]. In the domain of classical thermodynamics Joule-
Thomson expansion simply describes the expansion of non- ideal gas at high
pressure regime passes through the porous plug into the low pressure regime.
Since this is an adiabatic expansion so the temperature changes and the
enthalpy remains same at the initial and final equilibrium state. As a result
one can get heating and cooling effect due to this process at some inversion
point where the inversion curves intersect the isenthalpic curves in the $T-P$
plane. After the discoveries of Joule- Thomson expansion in the background of
charged $AdS$ black holes in [18] and for Kerr- $AdS$ black holes in [19],
several studies have been made for various classes of black holes in different
theories of gravity [20]\- [66]. The Joule- Thomson process which has been
studied so far in the literature are very much consistent but the inversion
curve in all those papers are different from the results of van der Waals gas
[18].

Recently, much interests have shown in the research of gravitational theories
where nonlinearity in the Maxwell fields are taking into account. The black
hole solutions in nonlinear electrodynamics are quite fascinating due to its
nonsingular nature [67]. In this context the Born- Infeld electrodynamics
[68], [69] attracts lot of interest because it smoothed out the divergences at
the origin arise due to linear electric field. The class of black hole
solutions in power maxwell invariant $(PMI)$ theory are obtained with
Lagrangian density is given by $(F_{\mu\nu}F^{\mu\nu})^{q}$, where $q$ is an
arbitrary rational number [70]. After studying the solutions of black holes in
Einstein $PMI$ gravity many authors have explored other non-linear model where
nonabelian Yang- Mills field coupled through gravity in general relativity.
The authors in [71] have investigated the possible black hole solutions which
are sourced by the power of Yang-Mills $(YM)$ invariant as
$(F_{\mu\nu}^{(a)}F^{(a)\mu\nu})^{q}$ 111where $F_{\mu\nu}^{(a)}$ is the $YM$
field with its internal index $1\leq a\leq{\frac{1}{2}}(D-1)(D-2)$, setting
$q=1$ recovers the $D$ dimensional Einstein- Yang- Mills $(EYM)$ black holes
in $AdS$ space- time [72, 73].. The study of Van der Waals like phase
transition and the critical behaiviour in the extended thermodynamics of $AdS$
black holes in Einstein- Power Maxwell and Einstein Power- Yang- Mills
theories have been investigated in [74, 75, 76]. The Joule- Thomson expansion
for higher dimensional non- linearly charged AdS black hole in Einstein-$PMI$
gravity has been explored in [54]. Motivated by those analysis, here we are
intended to study Joule- Thomson process for black holes in the Einstein
Power- Yang- Mills $(EPYM)$ gravity and explore the impact of non- linear
parameter $q$ in the process of $JT$ expansion.

The paper is organised as follows. In section $(2)$ we briefly discuss the
solutions and the thermodynamic properties of $EPYM$ black holes in $D$ space-
time dimensions. Then in section $(3)$ we present Joule- Thomson expansion of
$EPYM$ black holes with nonzero Yang- Mills magnetic charge $Q$ and non-
linearity parameter $q$, and investigate Joule- Thomson coefficient, the
inversion curves, the isenthalpic curves and ratio between the minimum of
inversion temperature and the critical temperature. Finally section. $(4)$
contains the conclusion of this paper.

##  2 Black Holes in Einstein Power- Yang- Mills Gravity

We consider the $D$ dimensional action given in [71] for Einstein-power-Yang-
Mills $(EPYM)$ gravity with a cosmological constant $\Lambda$, is given by
$(8\pi G=1)$

$I=\frac{1}{2}\int
d^{D}x\sqrt{-g}\Big{(}\mathcal{R}-\frac{(D-2)(D-1)}{3}\Lambda-\mathcal{F}^{q}\Big{)},$
(1)

where $\mathcal{F}$ is the $YM$ invariant

$\displaystyle\mathcal{F}$ $\displaystyle=$ $\displaystyle
Tr(F_{\lambda\sigma}^{(a)}F^{(a)\lambda\sigma},$ $\displaystyle Tr(.)$
$\displaystyle=$ $\displaystyle\Sigma_{a=1}^{\frac{(D-1)(D-2)}{2}}(.).$ (2)

$R$ is the Ricci Scalar and $q$ is a positive real parameter. The $YM$ field
is defined as

$F_{\lambda\sigma}^{(a)}=\partial_{\mu}A_{\nu}^{(a)}-\partial_{\nu}A_{\mu}^{(a)}+\frac{1}{2\sigma}C_{(b)(c)}^{(a)}A_{\mu}^{(b)}A_{\mu}^{(c)}.$
(3)

Here $C_{(b)(c)}^{(a)}$ are the structure constants of $\frac{(D-1)(D-2)}{2}$
parameter Lie group $G$ and $\sigma$ is a coupling constant, $A_{\mu}^{(a)}$
are the $SO(D-1)$ gauge group $YM$ potentials. The metric ansatz for $D$
dimensions is chosen as

$ds^{2}=-f(r)dt^{2}+\frac{dr^{2}}{f(r)}+r^{2}d\Omega^{2}_{(D-2)}.$ (4)

Here $d\Omega^{2}_{D-2}$ is the line element of unit $(D-2)$ sphere. Under the
condition $q\neq\frac{(D-1)}{4}$, the metric function $f(r)$ of $D$\-
dimensional $EPYM$ black hole with negative cosmological constant can be
represented by

$f(r)=1-\frac{2m}{r^{D-3}}-\frac{\Lambda}{3}r^{2}+\frac{[(D-3)(D-2)Q^{2}]^{q}}{(D-2)(4q-D+1)r^{4q-2}}.$
(5)

Here $m$ is a parameter related with the black hole mass and $Q$ is the charge
parameter associated with the Yang Mills fields. In order to satisfy the Weak
Energy Condition (WEC) of the Power- Yang- Mills term, one must take $q>0$
[71]. A bound on $q$ has been discussed in [71] to satisfy certain energy
conditions including causality condition is $\frac{D-1}{4}\leq
q<\frac{D-1}{2}$. For $q=1$ the above black hole solutions reduce to the
Einstein-Yang-Mills black holes in higher dimensions [72, 73].

(a)

(b)

Figure 1: Plot of the metric function $f(r)$ with r for various dimensions
$D$.

The location of event horizon $r_{+}$ is given by the largest positive real
root of $f(r_{+})=0$ from Eq.(5). Here the study of the positive real roots of
the function $f(r)$ has been done graphically in Fig.1. This analysis shows
that the line element Eq.(5) describes a naked singularity for $m<m_{ext}$ and
a black hole with an outer event horizon and an inner Cauchy horizon for
$m>m_{ext}$. Finally, for $m=m_{ext}$, the horizon is degenerate and Eq.(5)
represents an extremal black hole as shown in two subplots of Fig.1 for $D=4$
and $D=8$ dimensions. One should also note that the non- linearity parameter
$q$ is tuned in such a way that the above discussion about black hole
solutions having two horizons is valid only in the limit $q>{\frac{D-1}{4}}$.
On the other hand single curve in each subplot drawn for $q<\frac{D-1}{4}$,
where we obtain black holes with single horizon with out any extremal limit.
Two sample subplots here produced in Fig.1 to show graphically how the zeros
of the function $f(r)$ are appeared with $m$ values and that imposes a bound
on $q$. Here we assume $m$ is always to be larger than $m_{ext}$ and, so the
metric Eq.(5) describes an $AdS$ black hole, with an event horizon at
$r=r_{+}$.

Now the Hawking temperature $T$, mass $M$ and entropy $S$ of the black hole
can be presented as [75],

$\displaystyle T$ $\displaystyle=$
$\displaystyle\frac{f^{\prime}(r_{+})}{4\pi}=\frac{D-3}{4\pi
r_{+}}+\frac{2(D-1)P}{3}r_{+}-\frac{(D-2)^{q-1}[(D-3)Q^{2}]^{q}}{4\pi
r_{+}^{4q-1}},$ (6) $\displaystyle M$ $\displaystyle=$
$\displaystyle\frac{(D-2)\omega_{D-2}}{8\pi}m$ (7) $\displaystyle=$
$\displaystyle\frac{(D-2)\omega_{D-2}}{48\pi}\Big{(}8\pi
Pr_{+}^{D-1}+\frac{3(D-2)^{q-1}[(D-3)Q^{2}]^{q}}{(4q-D+1)}r_{+}^{D-4q-1}+3r_{+}^{D-3}\Big{)},$
$\displaystyle S$ $\displaystyle=$
$\displaystyle\frac{\omega_{D-2}r_{+}^{D-2}}{4}.$ (8)

According to [75] the $YM$ potential can be given as

$\Phi_{Q}=\frac{\omega_{D-2}q[(D-2)(D-3)Q^{2}]^{q}}{8\pi(4q-D+1)Q}r_{+}^{D-4q-1},$
(9)

with $\omega_{D}=\frac{2\pi^{\frac{D+1}{2}}}{\Gamma{({\frac{D+1}{2}}})}$ being
the volume of the unit $D$\- sphere and the thermodynamic pressure $P$ is
connected with the cosmological constant $\Lambda$ through the relation
$P=-\frac{\Lambda}{8\pi}$ in the extended phase- space thermodynamics.

The Smarr relation for $EPYM$ black hole in the extended phase space is
obtained by using all the above quantities and considering the mass $M$ as the
enthalpy of the black hole [10],

$M=\frac{D-2}{D-3}TS+\frac{2q-1}{(D-3)q}\Phi_{Q}Q-\frac{2}{D-3}VP.$ (10)

However equations (6), (7), (8), (9) for all those thermodynamic quantities
must satisfy the 1st law of thermodynamics

$dM=TdS+\Phi_{Q}dQ+VdP.$ (11)

The thermodynamic volume can be derived from the relation $V=(\frac{\partial
M}{\partial P})_{S,\Phi_{Q}}$ is given by

$V=\frac{(D-2)\omega_{D-2}}{6}r_{+}^{D-1}.$ (12)

From equations (6) and (7) one can get the equation of state of this black
hole

$P=\frac{T}{2(D-1)r_{+}}+\frac{(D-2)^{q-1}[(D-3)Q^{2}]^{q}}{8\pi(D-1)r_{+}^{4q}}-\frac{3(D-3)}{8\pi(D-1)r_{+}^{2}}.$
(13)

Following [12] the thermodynamic critical point for $EPYM$ black hole can be
studied by using the conditions $\big{(}\frac{\partial P}{\partial
r_{+}}\big{)}_{T_{c}}=\big{(}\frac{\partial^{2}P}{\partial
r_{+}^{2}}\big{)}_{T_{c}}=0$ and these correspond to the following results for
critical temperature, critical pressure and the critical horizon radius
222Detailed analysis regarding $P-V$ criticality and Van der Waals like phase
transition of $EPYM$ black holes are given in [75].

$\displaystyle T_{c}$ $\displaystyle=$
$\displaystyle\Big{(}\frac{D-3}{2\pi}\Big{)}\Big{(}\frac{4q-2}{4q-1}\Big{)}\Bigg{\\{}\frac{[(D-2)(D-3)]^{1-q}}{2q(4q-1)Q^{2q}}\Bigg{\\}}^{\frac{1}{4q-2}},$
(14) $\displaystyle P_{c}$ $\displaystyle=$
$\displaystyle\Big{(}\frac{3}{8\pi}\Big{)}\Big{(}\frac{D-3}{D-1}\Big{)}\Big{(}\frac{4q-2}{4q}\Big{)}\Bigg{\\{}\frac{[(D-2)(D-3)]^{1-q}}{2q(4q-1)Q^{2q}}\Bigg{\\}}^{\frac{1}{2q-1}},$
(15) $\displaystyle r_{c}$ $\displaystyle=$
$\displaystyle\Bigg{\\{}\frac{[(D-2)(D-3)]^{1-q}}{2q(4q-1)Q^{2q}}\Bigg{\\}}^{\frac{1}{4q-2}}.$
(16)

At this point we would like to make some comments on the limit of $q$ that is
$\frac{D-1}{4}\leq q<\frac{D-1}{2}$ [71]. As we have observed earlier that the
lower bound $\frac{D-1}{4}<q$, is precise in order to get black hole solutions
with both the inner and outer horizon radius like $RNAdS$ black holes [9]. The
upper bound on $q$ has appeared to satisfy all the energy conditions including
causality condition as well [71]. Henceforth our following discussion will be
confined within the bound $\frac{D-1}{4}\leq q<\frac{D-1}{2}$. However one can
extend this parameter space to include the values of $q$ beyond the upper
limit to study all thermodynamics features of $EPYM$ black holes.

##  3 The Joule- Thomson Expansion

In the domain of classical thermodynamics Joule- Thomson process is occurred
in a way when a gas is allowed to pass through the porous plug from high
pressure region to relatively low pressure region keeping enthalpy of the gas
is constant. So one does measure the temperature change relative to the change
of the pressure in this expansion process. An interesting phenomena happens
during this process is the heating and cooling of the gas. As the mass $M$ of
$AdS$ black hole is identified with the enthalpy $H$, in Joule- Thomson
expansion enthalpy remains constant at the initial and final state and one
would get an isenthalpic curve which denotes the locus of all the points
representing equilibrium states of the constant enthalpy so the constant mass.
The Joule- Thomson coefficient is described by the value of the slope of an
isenthalpic curve on a $T-P$ diagram at some point is denoted by $\mu$.
According to [18] one can write

$\mu=\Big{(}\frac{\partial T}{\partial
P}\Big{)}_{H}=\frac{1}{C_{P}}\Bigg{[}T\Big{(}\frac{\partial V}{\partial
T}\Big{)}_{P}-V\Bigg{]}.$ (17)

On the other hand the locus of all the maxima of the isenthalpic curves is
described as inversion curve, whereas this inversion curve divide the
isenthalpic curve into two portion where $\mu>0$ is the cooling region and the
$\mu<0$ is the region of heating. Inversion temperature can be obtained by
taking $\mu=0$ that is,

$T_{i}=V\Big{(}\frac{\partial T}{\partial V}\Big{)}_{P}.$ (18)

Now we are interested to calculate $J-T$ coefficient using equation (17)
considering all the thermodynamic parameters from Eqs. (6), (8) of $EPYM$
black holes

$\mu=\frac{2r_{+}\Big{[}3(4q+D-2)\\{(D-2)(D-3)Q^{2}\\}^{q}-(D-2)r_{+}^{4q-2}\\{(D-2)(D-1)(8\pi
r_{+}^{2}P+3)-6\\}\Big{]}}{3\Big{[}3\\{(D-2)(D-3)Q^{2}\\}^{q}-(D-2)r_{+}^{4q-2}\\{3(D-3)+8\pi(D-1)Pr_{+}^{2}\\}\Big{]}}.$
(19)

$(a)$ $D=4$, $Q=1$,

$(b)$ $D=4$, $Q=1$,

$(c)$ $D=8$, $Q=1$,

$(d)$ $D=8$, $Q=1$,

Figure 2: The plot of Joule-Thomson coefficient $\mu$ and temperature $T$ of
the Einstein Power- Yang- Mills black hole with respect to $r_{+}$ for
different values of the nonlinear parameter $q$ at $P=1$.

Therefore in Fig.2 we have plotted the $J-T$ coefficient $\mu$ and the
temperature $T$ given in Eqs. (19) and (6) respectively with the horizon
radius $r_{+}$ for different values of non- linear parameter $q$ at some fixed
pressure $P$ and charge $Q$. In those $\mu-r_{+}$ plot we have obtained two
special points, one is divergence point and the other one is zero point in
both the plots drawn for $D=4$ and $D=8$ dimensions. However in Fig. 2(b) and
2(d) one can see the points where $T=0$, these points denote the position of
the horizon where the black holes become extremal. Another interesting fact
one should note that the divergent point in $\mu-r_{+}$ plot coincide with the
zero point of the Hawking temperature. The zero point of $\mu$ versus $r_{+}$
curve simply signifies the inversion point of Joule- Thomson expansion. As the
values of the non- linearity parameters $q$ increases the position of the
horizons for both the divergent and the zero point in $\mu-r_{+}$ curves move
to the larger value as shown in Fig. 2(a) and 2(c). The observation here is
quite different from the result discussed in [54] for the case of nonlinearly
charged $AdS$ black hole in Power Invariant Einstein Maxwell gravity, where
horizon radius decreases for divergent and the inversion point with increasing
values of the nonlinear parameter.

Now we derive the inversion pressure $P_{i}$ and temperature $T_{i}$ of these
$EPYM$ black holes by using Eqs. (13) and (18) and following the Ref. [18],

$\displaystyle P_{i}$ $\displaystyle=$ $\displaystyle\frac{3}{8\pi
r_{+}^{2}(D-2)^{2}(D-1)}\Big{[}(4q+D-2)\Big{\\{}(D-3)(D-2)Q^{2}\Big{\\}}^{q}r_{+}^{2-4q}$
(20) $\displaystyle-$ $\displaystyle D(D-2)(D-3)\Big{]},$
$T_{i}=\frac{\Big{[}2q\Big{\\{}(D-3)(D-2)Q^{2}\Big{\\}}^{q}r_{+}^{2-4q}-(D-3)(D-2)\Big{]}}{2\pi(D-2)^{2}r_{+}}.$
(21)

From the above Eqs. (20) and (21) inversion curves in the $T-P$ plane for
different values of the space time dimension $D$, the charge $Q$ and the non-
linearity parameter $q$ are presented in Fig.3. We have illustrated four plots
in Fig. 3$(a)$\- 3$(d)$ for fixed charge $Q=1$, to see the dependence of
inversion curve on the non- linearity parameter $q$, where the inversion
temperature increases monotonically with the inversion pressure and the slope
of the curve increases with the increasing value of $q$. However one can
immediately see that the inversion temperature for a fixed inversion pressure
in each dimension increases with the increasing value of the non- linearity
parameter and it happens at any value of the pressure. The effect of Yang-
Mills charge $Q$ on the inversion curve has been depicted in the Fig. 3$(e)$\-
3$(h)$. The slope of the inversion curve increases with the increasing value
of charge for fixed value of non- linearity parameter $q$. In contrast to the
effect of $q$ over a fixed charge $Q$ at low pressure the inversion
temperature decreases with the magnetic $YM$ charge $Q$. Whereas the slope of
inversion curve increases with $Q$ for high pressure.

(a) $D=4$, $Q=1$,

(b) $D=5$, $Q=1$,

(c) $D=6$, $Q=1$,

(d) $D=8$, $Q=1$,

(e) $D=4$, $q=1.4$,

(f) $D=5$, $q=1.9$,

(g) $D=6$, $q=2.4$,

(h) $D=8$, $q=3.4$,

Figure 3: Inversion Curve for $EPYM$ black hole for various values of $D$, $Q$
and $q$.

Now the minimum inversion temperature $T_{i}^{min}$ can be derived by putting
$P_{i}=0$ in Eq.(20). After solving Eq.(20) one will get the horizon radius
$r_{i}^{min}$ as

$r_{i}^{min}=\Bigg{[}\frac{(4q+D-2)\Big{\\{}(D-2)(D-3)\Big{\\}}^{q-1}Q^{2q}}{D}\Bigg{]}^{\frac{1}{4q-2}}.$
(22)

Substituting the value of $r_{i}^{min}$ from Eq. (22) into Eq.(21) one should
obtain the following expression for minimum inversion temperature

$T_{i}^{min}=\frac{D-3}{2\pi}\frac{2q+1}{4q+D-2}\Bigg{[}\frac{(4q+D-2)\Big{\\{}(D-2)(D-3)\Big{\\}}^{q-1}Q^{2q}}{D}\Bigg{]}^{\frac{1}{2-4q}}.$
(23)

The ratio between minimum of inversion temperature and critical temperature
obtained from Eq.(23) and Eq.(14) respectively could be given in the following
form

$\frac{T_{i}^{min}}{T_{c}}=\frac{1}{2}(2qD)^{\frac{1}{4q-2}}\Big{(}\frac{4q-1}{4q+D-2}\Big{)}^{\frac{4q-1}{4q-2}}.$
(24)

The above ratio does not depend on the $YM$ charge $Q$, it is only a function
of dimension $D$ and the non-linearity parameter $q$, so parameter $q$ plays a
crucial role in determining the ratio Eq.(24) for some fixed $D$.

Figure 4: Plot for $\frac{T_{i}^{min}}{T_{c}}$ vs. $q$ for various values of
$D$.

One can now easily estimate the ratio $\frac{T_{i}^{min}}{T_{c}}$ from Eq.(24)
for several values of $q$ and $D$. It can be checked immediately from Eq.(24)
that the ratio $\frac{T_{i}^{min}}{T_{c}}$ is one half in case of $D=4$ and
$q=1$ which is in full agreement with the results for other $AdS$ black holes
studied in former literature [18, 19, 25, 28, 48]. Here we discuss the ratio
Eq.(24) for other $D$ and $q$ values just by inspecting the plot given in
Fig.4. For $D=4$ and $q\leq 1$ we get $\frac{T_{i}^{min}}{T_{c}}\leq 0.5$,
whereas for $q>1$ we get the ratio $\frac{T_{i}^{min}}{T_{c}}>0.5$ and finally
it goes to $1/2$ asymptotically when $q$ tends to $\infty$. So one might think
off this as the effect of non linear $YM$ fields initially on the ratio
$\frac{T_{i}^{min}}{T_{c}}$ but the effect will be diminishing for higher
values of $q$ which is quite surprising here. Again black holes in higher
space- time dimensions also show same behaviour as $D=4$ case and in every
dimension we get a $q$ where the ratio becomes $1/2$.

As it is known from [10] that the $ADM$ mass of the black holes are identified
with its enthalpy, so to study the Joule- Thomson expansion which is an
isenthalpic phenomenon in an extended phase space can be considered as a
process where the mass remains fixed. We can obtain the isenthalpic curves in
the $T-P$ diagram by using equation of state Eq.(13) and Eq.(7). Here we are
interested to study isenthalpic curves for various combinations of $D$ and
$q$, which have been plotted in Fig.5. In each subplot we also have drawn
inversion curve which intersect isenthalpic curves at its maximum point.
Intersection point being the maximum point of the fixed mass curve, the
inversion temperature curve divide them into two parts. The positive slope
parts $(\mu>0)$ in the isenthalpic curve represents cooling region in the
throttling process, while for the negative slope part $(\mu<0)$ represents
heating process. In the following we would like to discuss how a parameter
space can be chosen in order to get those constant mass curves with suitable
nature for $JT$ expansion and that put a bound on the masses of the black
holes.

Here we focus on black hole horizon where naked singularity can be avoided by
choosing black holes of $ADM$ mass greater than its extremal mass $(M_{ext})$.
The numerical values of extremal masses $M_{ext}$ for different values of non-
linearity parameter $q$, space- time dimensions $D$ and $Q=1$ at zero
cosmological constant $\Lambda$ i.e. $P=0$ have been presented in Table 1. As
discussed in [29, 36] $AdS$ black holes with an $ADM$ mass $M$ whose Hawking
temperature $T_{0}$ at vanishing pressure is larger than the minimum inversion
temperature $T_{i}^{min}$. So black hole with the given set of parameters
$(q,D,Q)$ one should get those curves for constant mass cut by the inversion
curve at its maximum point. Here we present numerical values of the mass
$M_{i}^{min}$ correspond to the temperature $T_{i}^{min}$ in table 1. However
for black holes with mass $M\leq M_{i}^{min}$ the isenthalpic curves have no
inversion point so cooling- heating transition may not happen in this case.
Following [29, 36] we also discuss the other limit of the mass $M^{max}$ again
for zero pressure correspondence to the temperature $T_{0}^{max}$ where two
isenthalpic curves might intersect at a particular point in the $T-P$ plane.
Here one could restrict the situation of getting intersection between two
different mass curves at some particular point by considering the mass $M\leq
M^{max}$. We have also calculated $M^{max}$ for different set of values of $q$
and $D$ and listed up in table 1. Finally one can get regular isenthalpic
curves as depicted in Fig.5 for $AdS$ $EPYM$ black holes with $ADM$ mass
should lie in the interval $M_{i}^{min}<M<M^{max}$. It is to be noted that the
previously mentioned characteristic masses $M_{i}^{min}$ and $M^{max}$ are
both decreasing as the non- linearity parameters are increasing for a
particular dimension. Furthermore we have drawn two more plots in Fig.6 for
higher value of $YM$ charge $Q$. The nuerical values of characteristic masses
$M_{i}^{min}$ and $M^{max}$ have come out quite large as shown in table 2, in
contrast to the characteristics masses of lower $Q$ black holes given in table
1.

Table 1: Value of three masses $M_{ext}$, $M_{i}^{min}$, $M^{max}$ for various nonlinrarity parameter $q$ and dimension $D$ for fixed charge $Q=1$. $D$ | $q$ | $M_{ext}$ | $M_{i}^{min}$ | $M^{max}$
---|---|---|---|---
4 | 0.8 | 2.6727 | 2.68626 | 2.81229
4 | 1.4 | 0.747234 | 0.776084 | 0.89411
5 | 1.05 | 14.0587 | 14.1443 | 15.1776
5 | 1.9 | 3.25977 | 3.45882 | 4.45754
6 | 1.3 | 67.406 | 67.8167 | 73.948
6 | 2.4 | 13.6617 | 14.7081 | 20.8005
8 | 1.8 | 1404.69 | 1412.34 | 1571.39
8 | 3.4 | 234.062 | 256.549 | 420.795
Table 2: Value of three masses $M_{ext}$, $M_{i}^{min}$, $M^{max}$ for various nonlinrarity parameter $q$ and dimension $D$ for fixed charge $Q=1.5$. $D$ | $q$ | $M_{ext}$ | $M_{i}^{min}$ | $M^{max}$
---|---|---|---|---
4 | 0.8 | 2.6727 | 2.68626 | 2.81229
4 | 1.4 | 0.747234 | 0.776084 | 0.89411

(a) $D=4$, $q=0.8$, $Q=1.0$.

(b) $D=4$, $q=1.4$, $Q=1.0$.

(c) $D=5$, $q=1.05$, $Q=1.0$.

(d) $D=5$, $q=1.9$, $Q=1.0$.

(e) $D=6$, $q=1.3$, $Q=1.0$.

(f) $D=6$, $q=2.4$, $Q=1.0$.

(g) $D=8$, $q=1.8$, $Q=1.0$.

(h) $D=8$, $q=3.4$, $Q=1.0$.

Figure 5: Isenthalpic curve for various values of $D$, and $q$ for $Q=1$.

(a) $D=4$, $q=1.4$, $Q=1.5$.

(b) $D=8$, $q=3.4$, $Q=1.5$.

Figure 6: Isenthalpic curve for various values of $D$, and $q$ for $Q=1.5$.

##  4 Conclusion

In the present paper, we systemically investigated the Joule-Thomson effect of
$D$ dimensional $AdS$ black hole in Einstein Power- Yang Mills gravity in the
extended phase space. The cosmological constant and the mass are identified
with the pressure and the enthalpy of the black hole respectively. Since the
Joule- Thomson expansion is an isenthalpic process then the mass would remain
constant before and after the process. At first we have presented an
analytical expression for the Joule- Thomson coefficient $\mu$ which diverges
at the points where the Hawking temperature of the black holes become zero.
After that we have analysed $T_{i}-P_{i}$ inversion curve in Fig.3 for various
non- linearity parameter $q$, space- time dimension $D$ and $YM$ charge $Q$.
It has been shown how inversion curves behave with respect to the various
parameters of the $EPYM$ black holes. In Fig.3$(e)$\- 3$(h)$ we have obtained
different behaviour of the inversion curves, the inversion temperature
decreases with the magnetic $YM$ charge $Q$ at low pressure while it increases
with $Q$ for high pressure. Fig.3$(a)$\- 3$(d)$ express that the slope of the
inversion curves gradually increases by increasing $q$ for particular $D$ and
$Q$. Next we present the analytic expression for minimum of the inversion
temperature $T_{i}^{min}$ at $P_{i}=0$. Furthermore we also have derived the
ratio $\frac{T_{i}^{min}}{T_{c}}$ which explicitly depends upon $q$ and $D$
only. Using Fig.4 we also have investigated the ratio
$\frac{T_{i}^{min}}{T_{c}}$ for $EPYM$ balck holes for $D=4$ and higher
dimensions. For any dimensions the value of the ratio become $1/2$ at certain
value of $q$ and then goes asymptotically $1/2$ at infinite $q$. It needs
further investigation why the ratio $T_{i}^{min}/T_{c}\rightarrow 1/2$ as
$q\rightarrow\infty$, if it is considered that adding more non- linearity in
to the theory makes the ratio $T_{i}^{min}/T_{c}$ greater than $1/2$. After
that we have plotted the isenthalpic curves for different masses in Figs.5 and
6. However the inversion curve divides the isenthalpic curves in the $T-P$
plane into two regions. The region above the inversion curve leads to the
cooling region with positive slope of $JT$ coefficient $\mu$ for isenthalpic
curves. The region under the inversion curve is the heating region with
negative slope. We get intersection point between isenthalpic curves and the
inversion curve for a particular $ADM$ mass of the black hole at a maximum
point of the isenthalpic curve. Finally we should conclude by mentioning that
the limit of the masses $M_{i}^{min}<M<M^{max}$ have taken in order to get
isenthalpic curves where cooling and heating regions are obtained on both
sides of the inversion curve. On the other hand the intersection between two
isenthalpic curves for two different masses in the $T-P$ plane could be
avoided if one must consider the limit $M<M^{max}$. We have thoroughly
calculated and tabulated the numerical values of $M_{i}^{min}$ and $M^{max}$
in table 1 and 2 which are necessary to understand Joule- Thomson effect of
$EPYM$ black holes by analysing all isenthalpic curves in Figs.5 and 6 for
different values of $q$, $D$ and $Q$.

## References

  * [1] S.W. Hawking, Nature 248: 30 (1974).
  * [2] S.W. Hawking, Commun. Math. Phys.4: 199 (1975).
  * [3] J. D. Bekenstein, Phys. Rev. D 7: 2333 (1973).
  * [4] S. Hawking and D. N. Page, Commun. Math. Phys. 87 (1983) 577.
  * [5] J. M. Maldacena, Int. J. Theor. Phys. 38 (1999) 1113 [hep-th/9711200].
  * [6] E. Witten, Adv. Theor. Math. Phys. 2 (1998) 253, hep-th/9802150.
  * [7] E. Witten, Adv. Theor. Math. Phys. 2 (1998) 505, hep-th/9803131.
  * [8] A. Chamblin, R. Emparan, C. Johnson, and R. Myers, Phys.Rev. D 60 (1999) 064018, [hep-th/9902170].
  * [9] A. Chamblin, R. Emparan, C. Johnson, and R. Myers, H, Phys.Rev. D 60 (1999) 104026, [hep-th/9904197].
  * [10] D. Kastor, S. Ray, and J. Traschen, Class.Quant.Grav. 26 (2009) 195011, [arXiv:0904.2765].
  * [11] B. Dolan, Class.Quant.Grav. 28 (2011) 125020, [arXiv:1008.5023].
  * [12] D. Kubiznak and R. B. Mann, JHEP 1207 (2012) 033, [arXiv:1205.0559].
  * [13] C. V. Johnson, Class. Quant. Grav. 31 (2014)205002
  * [14] B. P. Dolan, Class. Quant. Grav. 28, 235017 (2011)
  * [15] S. W. Wei and Y. X. Liu, Phys. Rev. D 87 (2013) no.4, 044014.
  * [16] R. Banerjee and D. Roychowdhury, Phys. Rev. D 85 (2012) 044040.
  * [17] C. Niu, Y. Tian and X. N. Wu, Phys. Rev. D 85 (2012) 024017.
  * [18] Özgür Ökcü and E. Aydıner, The European Physical Journal C, 77 (1):24 (2017) [arXiv:1611.06327 [gr-qc]]
  * [19] Özgür Ökcü and E. Aydıner, Eur. Phys. J. C 78, no. 2, 123 (2018) [arXiv:1709.06426 [gr-qc]].
  * [20] M. Chabab, H. El Moumni, S. Iraoui and K. Masmar, Astrophys. Space Sci. 362, no.10, 192 (2017) [arXiv:1701.00872 [hep-th]].
  * [21] H. Ghaarnejad, E. Yaraie and M. Farsam, Int. J. Theor. Phys. 57, no.6, 1671-1682 (2018) [arXiv:1802.08749 [gr-qc]].
  * [22] R. D’Almeida and K. P. Yogendran, [arXiv:1802.05116 [hep-th]].
  * [23] M. Chabab, H. El Moumni, S. Iraoui, K. Masmar and S. Zhizeh, LHEP 02, 05 (2018) [arXiv:1804.10042 [gr-qc]].
  * [24] A. Rizwan C.L., N. Kumara A., D. Vaid and K. M. Ajith, Int. J. Mod. Phys. A 33, no.35, 1850210 (2019) [arXiv:1805.11053 [gr-qc]].
  * [25] J. X. Mo and G. Q. Li, Class. Quant. Grav. 37, no.4, 045009 (2020) [arXiv:1805.04327 [grqc]].
  * [26] J. Liang, W. Lin and B. Mu, [arXiv:2103.03119 [gr-qc]].
  * [27] K. Hegde, A. Naveena Kumara, C. L. A. Rizwan, A. K. M. and M. S. Ali, [arXiv:2003.08778 [gr-qc]].
  * [28] J. X. Mo, G. Q. Li, S. Q. Lan and X. B. Xu, Phys. Rev. D 98, no.12, 124032 (2018) [arXiv:1804.02650 [gr-qc]].
  * [29] Z. W. Zhao, Y. H. Xiu and N. Li, Phys. Rev. D 98, no.12, 124003 (2018) [arXiv:1805.04861 [gr-qc]].
  * [30] S. Q. Lan, Phys. Rev. D 98, no.8, 084014 (2018) [arXiv:1805.05817 [gr-qc]].
  * [31] S. W. Wei and Y. X. Liu, Nucl. Phys. B, 114700 (2019) [arXiv:1708.08176 [gr-qc]].
  * [32] X. M. Kuang, B. Liu and A. Ovgun, Eur. Phys. J. C 78, no.10, 840 (2018) [arXiv:1807.10447 [gr-qc]].
  * [33] J. Pu, S. Guo, Q. Q. Jiang and X. T. Zu, Chin. Phys. C 44, no.3, 035102 (2020) [arXiv:1905.02318 [gr-qc]].
  * [34] C. H. Nam, Eur. Phys. J. C 78, no.7, 581 (2018)
  * [35] C. Li, P. He, P. Li and J. B. Deng, Gen. Rel. Grav. 52, no.5, 50 (2020) [arXiv:1904.09548 [gr-qc]].
  * [36] D. Mahdavian Yekta, A. Hadikhani and O. Okcu, Phys. Lett. B 795, 521-527 (2019) [arXiv:1905.03057 [hep-th]].
  * [37] S. Hyun and C. H. Nam, Eur. Phys. J. C 79, no.9, 737 (2019) [arXiv:1908.09294 [gr-qc]].
  * [38] C. H. Nam, Gen. Rel. Grav. 53, no.3, 30 (2021) [arXiv:1906.05557 [gr-qc]].
  * [39] C. H. Nam, Eur. Phys. J. C 78, no.12, 1016 (2018)
  * [40] M. Rostami, J. Sadeghi, S. Miraboutalebi, A. A. Masoudi and B. Pourhassan, Int. J. Geom. Meth. Mod. Phys. 17, no.09, 2050136 (2020) [arXiv:1908.08410 [gr-qc]].
  * [41] A. Haldar and R. Biswas, EPL 123, no.4, 40005 (2018)
  * [42] S. Guo, J.. Q. Jiang, [arXiv:1905.03604 [gr-qc]].
  * [43] S. Q. Lan, Nucl. Phys. B 948, 114787 (2019)
  * [44] J. Sadeghi and R. Toorandaz, Nucl. Phys. B 951, 114902 (2020)
  * [45] S. Bi, M. Du, J. Tao and F. Yao, Chin. Phys. C 45, no.2, 025109 (2021) [arXiv:2006.08920 [gr-qc]].
  * [46] H. Ranjbari, M. Sadeghi, M. Ghanaatian and G. Forozani, Eur. Phys. J. C 80, no.1, 17 (2020) [arXiv:1911.10803 [hep-th]].
  * [47] S. Guo, Y. Han and G. P. Li, [arXiv:1912.09590 [hep-th]].
  * [48] R. K., C. L. A. Rizwan, A. Naveena Kumara, D. Vaid and M. S. Ali, Phys. Dark Univ. 32, 100825 (2021) [arXiv:2002.03634 [gr-qc]].
  * [49] C. H. Nam, Eur. Phys. J. Plus 135, no.2, 259 (2020)
  * [50] Y. Meng, J. Pu and Q. Q. Jiang, Chin. Phys. C 44, no.6, 065105 (2020)
  * [51] S. Guo, Y. Han and G. P. Li, Class. Quant. Grav. 37, no. 8, 085016 (2020)
  * [52] M. Ghanaatian, M. Sadeghi, H. Ranjbari and G. Forozani, Mod. Phys. Lett. A 35, no.24, 2050203 (2020) [arXiv:1906.00369 [hep-th]].
  * [53] S. Guo, Y. Han and G. P. Li, Mod. Phys. Lett. A 35, no.14, 2050113 (2020)
  * [54] Z. W. Feng, X. Zhou, G. He, S. Q. Zhou and S. Z. Yang, Commun. Theor. Phys. 73, no.6, 065401 (2021) [arXiv:2009.02172 [gr-qc]].
  * [55] U. Debnath, [arXiv:2006.02920 [gr-qc]].
  * [56] Y. Cao, H. Feng, W. Hong and J. Tao, [arXiv:2101.08199 [gr-qc]].
  * [57] Y. l. Huang and S. Guo, [arXiv:2009.09401 [hep-th]].
  * [58] M. Zhang, C. M. Zhang, D. C. Zou and R. H. Yue, [arXiv:2102.04308 [hep-th]].
  * [59] N. Chen, [arXiv:2003.00247 [gr-qc]].
  * [60] A. Jawad and S. Chaudhary, Mod. Phys. Lett. A 35, no.39, 2050326 (2020) 5
  * [61] J. Liang, B. Mu and P. Wang, [arXiv:2104.08841 [gr-qc]].
  * [62] U. Debnath, Phys. Lett. B 810, 135807 (2020) [arXiv:2010.02102 [gr-qc]].
  * [63] B. Mirza, F. Naeimipour and M. Tavakoli, Front. in Phys. 9, 33 (2021) [arXiv:2105.05047 [gr-qc]].
  * [64] J. P. M. Graca, E. F. Capossoli and H. Boschi-Filho, [arXiv:2105.04689 [gr-qc]].
  * [65] Rui Yina, Jing Lianga and Benrong Mu, [arXiv:2105.09173v1 [gr-qc]] (2021)
  * [66] Chao-Ming Zhang, Ming Zhang and De-Cheng Zou, [arXiv:2106.00183v1 [hep-th]] (2021)
  * [67] Eloy Ayón-Beato and A. Garcia, Phys. Rev. Lett. 80, 5056 (1998);
Eloy Ayón-Beato and A. Garcia, Phys. Lett. B 464, 25 (1999);
Alberto Garcia, Gen. Rel. Grav. 31, 629 (1999);
M. Cataldo and A. Garcia, Phys. Rev. D 61, 084003 (2000)

  * [68] M. Born, Proc. Roy. Soc. Lond. A 143, 410 (1934).
  * [69] M. Born and L. Infeld, Proc. Roy. Soc. Lond. A 144, 425 (1934).
  * [70] H. Maeda, M. Hassaïne, and C. Martínez, Phys. Rev. D 79 (2009) 044012;
M. Hassaïne, and C. Martínez Class. Quantum Grav. 25 (2008) 195023;
M. Hassaïne, and C. Martínez, Phys. Rev. D 75 (2007) 027502.

  * [71] S. H. Mazharimousavi and M. Halilsoy, Phys. Lett. B 681, 190 (2009) [arXiv:0908.0308 [gr- qc]].
  * [72] S. H. Mazharimousavi and M. Halilsoy, Phys. Rev. D 76 (2007) 087501;
  * [73] S. H. Mazharimousavi and M. Halilsoy, Phys. Lett. B 659 (2008) 471.
  * [74] S. H. Hendi and M. H. Vahidinia, Phys. Rev. D 88, 084045 (2013), [arXiv:1212.6128 [hep- th].
  * [75] M. Zhang, Z.-Y. Yang, D.-C. Zou, W. Xu, and R.-H. Yue, Gen. Rel. Grav. 47, 14 (2015), [arXiv:1412.1197 [hep-th]]
  * [76] P. K. Yerra, B. Chandrasekhar, Mod. Phys. Lett. A 34 (2019) 27, 1950216, [arXiv:1806.08226 [hep-th]]


# Deep Proximal Learning for High-Resolution Plane Wave Compounding

Nishith Chennakeshava, Ben Luijten, Massimo Mischi, Yonina C. Eldar, and Ruud
J. G. van Sloun,

###### Abstract

Plane Wave imaging enables many applications that require high frame rates,
including localisation microscopy, shear wave elastography, and ultra-
sensitive Doppler. To alleviate the degradation of image quality with respect
to conventional focused acquisition, typically, multiple acquisitions from
distinctly steered plane waves are coherently (i.e. after time-of-flight
correction) compounded into a single image. This poses a trade-off between
image quality and achievable frame-rate. To that end, we propose a new deep
learning approach, derived by formulating plane wave compounding as a linear
inverse problem, that attains high resolution, high-contrast images from just
3 plane wave transmissions. Our solution unfolds the iterations of a proximal
gradient descent algorithm as a deep network, thereby directly exploiting the
physics-based generative acquisition model into the neural network design. We
train our network in a greedy manner, i.e. layer-by-layer, using a combination
of pixel, temporal, and distribution (adversarial) losses to achieve both
perceptual fidelity and data consistency. Through the strong model-based
inductive bias, the proposed architecture outperforms several standard
benchmark architectures in terms of image quality, with a low computational
and memory footprint.

###### Index Terms:

Plane Wave Compounding, Machine Learning, Ultrasound, Signal Processing

##  I Introduction

Figure 1: Unfolded deep proximal learning, obtained by unfolding the recurrent
proximal gradient scheme to a K layered neural network.

In Plane Wave (PW) imaging, multiple low resolution images are acquired using
steered transmissions, which are then coherently compounded to obtain a high
quality image with enhanced lateral resolution and contrast. Compounding more
steered plane wave acquisitions improves image quality [1], but comes at the
cost of temporal resolution, i.e. frame-rate. This poses challenges for
applications that rely on high frame rates, such as elastography, Ultrasound
Localisation Microscopy (ULM) [2], or sensitive Doppler. One way to improve
temporal resolution is to reduce the number of steered PWs transmitted per
acquisition, but this negatively impacts lateral resolution. In this paper, we
propose a model-based Deep Learning (DL) solution that aims to relax this
trade-off between lateral resolution and frame-rate, achieving high resolution
and contrast by compounding just 3 PW transmissions.

Deep Learning has revolutionised many domains. It is also increasingly being
used in ultrasound based applications [3, 4]. Convolutional Neural Nets
(CNNs), such as ResNets [5] and UNets [6] have produced excellent results in
denoising tasks [7]. Other applications of Deep Neural Networks (DNNs) in
ultrasound include the weighting of channel signals (adaptive beamforming by
deep learning) [8], probabilistic sub-sampling for compressed sensing [9], and
coherent PW compounding [10]. Related to this work, Gasse et al. (2017) [10]
use a CNN in tandem with Maxout activation functions, to coherently compound 3
beamformed images, acquired using 1 PW each, and map them towards an image
acquired using 31 PWs. Most of the work in the domain of plane wave
compounding is achieved by using a variation of the ResNet [11] or the UNet
architectures [12], sometimes in combination with a discriminator in a
Generative Adversarial Network-like (GAN) training strategy [13, 14]. There
are also works exploring channel-based compounding, in pursuit of better image
quality [15]. However, generic convolutional neural networks like a UNet, or a
ResNet may sometimes yield solutions that are either physically invalid, or
are visually implausible. In other words, they may violate the physical
measurement model. Such Neural Networks (NNs) are also typically over-
parameterised [16]. By training light weight models, we tend to limit or avoid
the under-specification problem, as there are fewer degrees of freedom in the
NN architecture [17, 18, 19].

This study expands upon the initial work of [20], by incorporating physics-
based inductive biases. We do this by employing physics-based signal priors in
the design of our NN, such that we may obtain a network that is compact, yet
performs very well when compared to benchmarks (a UNet, ResNet, and [10]). In
addition, we take further advantage of the signal’s noise characteristics in
this work. Therefore, this problem is formulated as an ill-posed, linear
inverse problem that we solve using data-driven techniques. We achieve an
additional boost in the spatial resolution and contrast of the resulting image
by training towards images acquired using a higher frequency, and many more
PWs. Furthermore, to promote consistency between frames we implement a frame-
to-frame loss, which suppresses uncorrelated noise between the frames.

We begin by describing the methods employed in deriving the unfolded proximal
gradient descent scheme in Section II, followed by details about the data
acquisition technique, and the volume of data collected in Section III. We
then describe the training strategy in Section IV, and showcase the results in
Section V. We discuss the results in Section VI, and finally conclude the
paper in Section VII.

Figure 2: Illustration of data and loss in the training loop. Pink indicates
the flow of loss values, and black indicates the flow of data, from input to
output. The $U$ blocks indicate the unfoldings of the proximal network, up to
$K$ unfoldings. The block to the right hand side of the image with a neuron
indicates the discriminator. As we need frame t-1 and t for the calculation of
all loss terms, the graphic also displays the training loop across frames in
an acquisition.

##  II Methods

###  II-A Signal model and problem formulation

We model the problem of compounding multiple plane waves as an inverse problem
in which we aim to recover an underlying high-resolution (HR), high-contrast
image from a set of low-resolution (LR) noisy measurements with multiple
angled plane waves. We denote $\mathbf{x}\in\mathbb{R}^{N}$ as the vectorised
high-resolution beamformed RF image, and $\mathbf{y}\in\mathbb{R}^{NM}$ the
vectorised measurement of low-resolution beamformed RF images from $M$
transmitted plane waves. We consider these measurements to be acquired
according to the following linear model:

$\mathbf{y}=\mathbf{A}\mathbf{x}+\mathbf{n}.$ (1)

Here,

$\mathbf{A}=\begin{pmatrix}\mathbf{A}_{1}\\\ \mathbf{A}_{2}\\\ \vdots\\\
\mathbf{A}_{M}\end{pmatrix},$ (2)

and

$\mathbf{y}=\begin{pmatrix}\mathbf{y}_{1}\\\ \mathbf{y}_{2}\\\ \vdots\\\
\mathbf{y}_{M}\end{pmatrix},$ (3)

where $\mathbf{y}_{m}$ is the vectorised, beamformed RF image belonging to the
$m^{\textrm{th}}$ steered plane wave transmission, $\mathbf{n}$ $\in$
$\mathbb{R}^{NM}$ is a noise vector which is assumed to follow a Gaussian
distribution with zero mean and diagonal covariance, and $\mathbf{A}$ $\in$
$\mathbb{R}^{NM\times N}$ is a block matrix, with its blocks $\mathbf{A}_{1}$,
$\mathbf{A}_{2}$,…, $\mathbf{A}_{M}$ being the measurement matrices of
individual PW acquisitions. These measurement matrices capture the
transformation between our underlying high-resolution RF image $\mathbf{x}$
and each of the plane wave measurements, and are a function of their
respective point spread functions. We assume that they follow a convolutional
Toeplitz structure through which we may re-write application of
$\mathbf{A}_{M}$ as a convolution with an anisotropic ‘blurring’ kernel.

Solving equation (1) for $\mathbf{x}$ is an ill-posed problem, and hence
direct maximum-likelihood estimation may lead to solutions that are not
consistent with our prior belief of what HR images look like (anatomical and
visual plausibility). To incorporate such priors, we estimate $\mathbf{x}$
through Maximum A-Posteriori (MAP) estimation:

$\hat{\mathbf{x}}:=\operatorname*{arg\,max}_{\mathbf{x}}p(\mathbf{x}|\mathbf{y})\propto\operatorname*{arg\,max}_{\mathbf{x}}p(\mathbf{y}|\mathbf{x})p_{\theta}(\mathbf{x}),$
(4)

where $\mathbf{\hat{\mathbf{x}}}$ is the estimated high-resolution ultrasound
image, and $p(\mathbf{y}|\mathbf{x})$ is the likelihood according to the
linear measurement model in (1), having a Gaussian distribution on the model
errors. Here, $p_{\theta}(\mathbf{x})$ is a probability density function that
expresses prior beliefs about the distribution of HR images $\mathbf{x}$,
where $\theta$ are learnt parameters.

Under a Gaussian model on $p(\mathbf{y}|\mathbf{x})$, taking the log of (4)
results in:

$\hat{\mathbf{x}}=\operatorname*{arg\,min}_{\mathbf{x}}\left\lVert\mathbf{y}-\mathbf{A}\mathbf{x}\right\rVert_{2}^{2}-\log
p_{\theta}(\mathbf{x})$ (5)

where $\log p_{\theta}(\mathbf{x})$ acts as a regulariser $R(\mathbf{x})$ for
the optimisation problem.

###  II-B Proximal gradient iterations

Assuming $p_{\theta}(\mathbf{x})$ is given, we proceed by formulating a
proximal gradient solver for (5). This results in an iterative algorithm that
alternates between a gradient update step with respect to the measurement
model (data consistency), and a proximal step that pushes the intermediate
solutions in the proximity of the regulariser. One iteration is given by:

$\displaystyle\mathbf{\hat{x}}^{(k+1)}$
$\displaystyle=\mathcal{P}(\mathbf{\hat{x}}^{(k)}-\mu\mathbf{A}^{T}(\mathbf{A}\mathbf{\hat{x}}^{(k)}-\mathbf{y})),$
(6)

where $\mathcal{P}^{(k)}$ is the proximal operator of the regularizer
$R(\mathbf{x})$ norm [3], and $\mu$ is a step size. We can rewrite (6) in a
compact form, separating contributions of $\hat{\mathbf{x}}$ and $\mathbf{y}$,
as

$\displaystyle\mathbf{\hat{x}}^{(k+1)}$
$\displaystyle=\mathcal{P}_{\theta}^{(k)}(\mathbf{\hat{x}}^{(k)}-\mu^{(k)}\mathbf{A}^{T}(\mathbf{A}\mathbf{\hat{x}}^{(k)}-\mathbf{y}))$
(7)
$\displaystyle=\mathcal{P}_{\theta}^{(k)}(\mu^{(k)}\mathbf{A}^{T}\mathbf{y}+(\mathbf{I}-\mu^{(k)}\mathbf{A}^{T}\mathbf{A})\mathbf{\hat{x}}^{(k)})$
$\displaystyle=\mathcal{P}_{\theta}^{(k)}(\mathbf{W}^{(k)}\mathbf{y}+\mathbf{V}^{(k)}\mathbf{\hat{x}}^{(k)}).$

where we denote $\mathbf{W}^{(k)}=\mu^{(k)}\mathbf{A}^{T}$ and
$\mathbf{V}^{(k)}=\mathbf{I}-\mu^{(k)}\mathbf{A}^{T}\mathbf{A}$.

For particular distributions $p_{\theta}(\mathbf{x})$ the proximal operator
$\mathcal{P}_{\theta}$ has a closed form. For example, if
$p_{\theta}(\mathbf{x})$ is a Laplace distribution, equivalent to $\ell_{1}$
regularisation on $\mathbf{x}$, $\mathcal{P}$ takes the well-known form of a
soft-thresholding operator.

###  II-C Deep unfolding

In this work, instead of formalising regularizers analytically, we learn the
proximal operator directly from an empirical data distribution, similar to
[21]. To that end, we unfold the iterations of (6) into a K-layered neural
network $U_{\theta}$, following [22, 23, 24] and shown in Fig. 1. Unfolding
allows us to learn both the proximal operator (under some parameterisation
$\theta$) and the parameters of the gradient update, $\mathbf{V}^{(k)}$ and
$\mathbf{W}^{(k)}$, in an end-to-end fashion.

Exploiting the block convolutional structure of $\mathbf{A}$, we implement
$\mathbf{W}^{(k)}$ as:

$\mathbf{W}^{(k)}\mathbf{y}:=\mathbf{w}_{1}^{(k)}\circledast\mathbf{y}_{1}+\mathbf{w}_{2}^{(k)}\circledast\mathbf{y}_{2}+...+\mathbf{w}_{M}^{(k)}\circledast\mathbf{y}_{M},$
(8)

and, likewise:

$\mathbf{V}^{(k)}\mathbf{\hat{x}}^{(k)}=\mathbf{v}^{(k)}\circledast\mathbf{\hat{x}}^{(k)},$
(9)

where $\circledast$ denotes a convolutional operation,
$\\{\mathbf{w}_{1}^{(k)},...,\mathbf{w}_{M}^{(k)}\\}$ and $\mathbf{v}^{(k)}$
are learned convolutional kernels, where we adopted a size of $3\times 3$.

We model $\mathcal{P}_{\theta}^{(k)}$ using a U-Net-style architecture with 9
convolutional layers with Leaky ReLU activations (see Table IV, in appendix A
for architectural details). As mentioned before, making
$\mathcal{P}_{\theta}^{(k)}$ trainable avoids the need to devise hand-designed
priors with an analytical proximal operator [25], and instead allows for data-
driven proximal mappings that are learned from the training data distribution.
As an additional advantage, unrolling the iterative scheme into a fixed-
complexity trained computational graph yields a fast solution that avoids the
problem of ambiguous computational complexity in iterative algorithms.

##  III Data Acquisition

For this study we acquired both in-vivo and in-silico data. The in-vivo data
from 2 healthy volunteers was acquired using a Vantage system (Verasonics
Inc., WA, USA), with the L11-4v and the L11-5v transducers. It was collected
across 3 separate scan sessions, containing scans of the carotid artery in
various alignments, muscle tissue, and the Achilles tendon. The in-silico data
consists of 20 randomly spaced point scatterers, at random distances from each
other, created using the Verasonics simulation mode. Thirty of the in-silico
images also contained simulated speckle, made using 10,000 point scatterers of
low reflectivity.

An overview of the driving schemes and imaging parameters for both the LR
input data and HR target data is given in Table I. The LR inputs (using 3
plane waves with a center frequency of 4 MHz) and HR targets (using 75 plane
waves with a center frequency of 12 MHz) were acquired in successive frames.
All 75 transmissions are equally spaced between the steering angles of
$-18^{o}$ and $18^{o}$. All channel data was converted to beamformed RF using
pixel-based beamforming. To further improve image quality in the HR
acquisitions, we performed minimum variance (MV) beamforming, as opposed to
regular delay-and-sum (DAS) for the LR acquisitions. To reduce motion between
our input-target pairs, we gather the data using an acquisition sequence that
gathers the low frequency image in the first frame, and the high frequency
image in the subsequent frame.

TABLE I: Parameters of the US probes, driving schemes, and beamforming methods for the high-resolution (HR) targets and low-resolution (LR) inputs. Parameter | LR inputs | HR targets
---|---|---
Number of elements | 128 | 128
Array pitch | 0.3 mm | 0.3 mm
Array aperture | 38.4 mm | 38.4 mm
Transmit Frequency | 4 MHz | 12 MHz
Number of plane waves | 3 ($-10.2^{o}$, $0^{o}$, and $10.2^{o}$) | 75 ($-18^{o}$ to $18^{o}$)
Image reconstruction | pixel-based DAS | pixel-based MV + compounding

The training data contains 465 input-target pairs, one input being a stack of
3 LR single PW images, and the target being the corresponding HR image
acquired subsequently. It contains a mix of in-vivo and in-silico data, at a
ratio of approximately 3:1 (345 in-vivo and 120 in-silico). The validation set
contains 60 input-target pairs, and the test set contains 45 input-target
pairs (30 in-vivo and 15 in-silico).

In addition to this, due to the increased attenuation of the high-frequency
targets, and therefore a lack of SNR at depth, we only include image pixels up
to a set depth ($\sim$2.33 cm) for training. In addition, we exclude the first
$\sim$0.3 cm, which predominantly consists of ringing. Note that for
inference, we evaluate the full depth.

##  IV Training Strategy

Figure 3: Log-compressed and envelope-detected set of 3 in-vivo (first row)
and in-silico (second row) low-frequency input images, followed by the
corresponding high frequency ground truth, compounded using 75 plane waves,
and the deep network output. The first row depicts a longitudinal profile at
the point of bifurcation of the carotid artery. All images were taken from the
test set.

In this section, the training strategy is described in detail. To promote high
fidelity in the image domain, we guide the optimization by adding an RF-to-
image transformation function, $\mathcal{Q}$, to the computational graph
before computing the losses. This way we emphasise a high resolution image in
the image domain, after envelope detection and log-compression. We define
$\mathcal{Q}$ as:

$\mathcal{Q}(\mathbf{x})=20\log_{10}\left(\epsilon+\frac{|Hilbert(\mathbf{x})|}{\max(|Hilbert(\mathbf{x})|)}\right),$
(10)

where $\epsilon$ is a small constant, and the values are clipped outside the
range of interest (-60dB to 0dB). We combine 3 loss functions that capture
various aspects of desired outputs: a pixel-wise distance loss, an adversarial
loss, and a frame-to-frame loss (see Fig. 2 for an overview).

###  IV-A Pixel-Based Distance Loss

We start with a pixel-based distance loss, which computes the average per-
pixel $\ell_{1}$ distance between the reconstructed image
$\mathcal{Q}(\hat{\mathbf{x}})=\mathcal{Q}(U_{\theta}(\mathbf{y}_{t}))$, and
the target image $\mathcal{Q}(\mathbf{x})$:

$\displaystyle\mathcal{L}_{\ell_{1}}$
$\displaystyle=\mathbb{E}_{(\mathbf{y},\mathbf{x})\sim
p_{data}}\left\lVert\mathcal{Q}(\mathbf{x}_{t})-\mathcal{Q}(U_{\theta}(\mathbf{y}_{t}))\right\rVert_{1}$
(11)

where $t$ indicates the frame number. Although we have framed the above in
terms of expectation values, in practice, we approximate it by sampling a
fixed dataset, i.e., the training data.

###  IV-B Adversarial Loss

We also include an adversarial loss to promote perceptually adequate outputs.
Such adversaries are often used in GANs: deep generative models trained in a
min-max game to map simple Gaussian-distributed noise vectors into outputs
that follow a desired target distribution (e.g. an empirical distribution of
natural images). Here we use a similar setup, and train our deep unfolded
proximal gradient network in a min-max game with a neural adversary to produce
outputs that, given the particular input data, fall within the distribution of
high-resolution ultrasound images as sampled in our training set. By not just
assessing outputs but rather input-output pairs, we mitigate hallucination of
output image features that fall within the distribution of HR images, but are
not consistent with the input data.

Our neural adversary, $D_{\psi}$, outputs its belief that the image data
spanned by its receptive field belongs to the set of high-resolution images.
Here, $D_{\psi}$ is fully convolutional and has a receptive field smaller than
the image dimensions (see Table V, in appendix A for architectural details),
thus outputting probabilities for multiple distinct image regions (i.e. like a
patchGAN [26]). This directs the focus of the adversary towards local image
quality features, rather than global semantic features.

We train $D_{\psi}$ by minimizing the binary cross-entropy classification loss
between its predictions and the labels (target or generated), evaluated on
batches that contain both target images and generated images. At the same
time, the deep compounding network $U_{\theta}$ is trained to maximize this
loss, thereby attempting to fool the adversary. This leads to the following
optimization problem across the training data distribution $p_{data}$:

$\begin{aligned}
\hat{\theta},\hat{\mathbb{\psi}}&=\operatorname*{arg\,max}_{\mathbb{\psi}}\operatorname*{arg\,min}_{\mathbb{\theta}}\left\\{-\mathcal{L}_{adv}\right\\}\\\
&=\operatorname*{arg\,max}_{\mathbb{\psi}}\operatorname*{arg\,min}_{\mathbb{\theta}}\mathbb{E}_{(\mathbf{y}_{t},\mathbf{x}_{t})\sim
p_{data}}\\\
&\qquad\left\\{\log(D_{\psi}([\mathcal{Q}(\mathbf{y}_{t}),\mathcal{Q}(\mathbf{x}_{t})]))+\right.\\\
&\left.\qquad\qquad\log(1-D_{\psi}([\mathcal{Q}(\mathbf{y}_{t}),\mathcal{Q}(U_{\theta}(\mathbf{y}_{t}))]))\right\\},\end{aligned}$
(12)

where $\theta$ are the parameters of the compounding network and $\psi$ are
the parameters of the discriminator $D_{\psi}$.

Figure 4: This figure compares different compounding methods against the
target. We compare our approach against standard coherent compounding,
compounding by a U-Net, and the architecture proposed by Gasse et al. in [10].
The first row depicts a longitudinal profile of the carotid artery at the
point of bifurcation. The second row depicts a longitudinal profile of the
carotid artery. All images were taken from the test set.

###  IV-C Frame-to-Frame Loss

In addition to the $\ell_{1}$ distance, and the adversarial loss mentioned
above, we also implement an $\ell_{2}$ distance on consecutive frames
($(t-1)^{th}$ and $t^{th}$ frame) as predicted by $U_{\theta}$. This is done
to promote consistency between frames while suppressing (frame-wise)
uncorrelated noise:

$\mathcal{L}_{ftf}=\mathbb{E}_{(\mathbf{y}_{t},\mathbf{y}_{t-1})\sim
p_{data}}\left\lVert\mathcal{Q}(U_{\theta}(\mathbf{y}_{t-1}))-\mathcal{Q}(U_{\theta}(\mathbf{y}_{t}))\right\rVert_{2}^{2}.$
(13)

###  IV-D Training details and parameters

The total loss function is a weighted sum of the aforementioned losses, and
given by:

$\mathcal{L}_{tot}=\lambda_{1}\cdot\mathcal{L}_{adv}+\lambda_{2}\cdot\mathcal{L}_{\ell_{1}}+\lambda_{3}\cdot\mathcal{L}_{ftf}$
(14)

where $\lambda_{i}$ are weight terms, with $\lambda_{1}=1.0$,
$\lambda_{2}=1.0$, and $\lambda_{3}=0.001$ in our experiments, determined
empirically.

We train the network in a greedy manner [27, 28], i.e. fold-by-fold. In the
first iteration, we only train the first fold. We then add the next fold,
freezing the first, and continue training. This process continues until all
folds are trained. Note that the last convolutional layer of the architecture
(see Fig. 2) is trained in every iteration. Each fold is trained for 1800
epochs, with a batch size of 1. We use the Adam optimiser [29] with a learning
rate of $1\times 10^{-5}$, and with a $\beta_{1}=0.5$. All other optimiser
parameters are left at default, as described in [29].

All of the networks were created using Python 3 and TensorFlow 2 [30], and
trained using an NVIDIA 2080 Ti GPU.

##  V Results

Figure 5: This figure plots the performance of the proposed network as the
volume of in-silico data in the training set in increased. The performance on
the in-vivo and in-silico test sets, are shown separately. Notably, in-vivo
performance improves when adding a significant amount of in-silico images to
the train set. Due to vast differences in the distributions of in-silico and
in-vivo data, we plot the trends separately such that trend lines are
comprehensible for both in-vivo and combined test sets. Figure 6: The graphs
displays the beam profiles of a point scatterer generated by a in-silico, and
the subsequent prediction on that data. Figure 7: The graph displays the
relationship between the number of unfoldings in the deep proximal network,
and the performance on the test set. Due to vast differences in the
distributions of in-silico and in-vivo data, we plot the trends separately
such that trend lines are comprehensible for both in-vivo and combined test
sets.

The results presented in this section were obtained using a network with 5
folds (unless otherwise stated) using the aforementioned framework to
construct and train an unfolded proximal gradient network.

The first row of Fig. 3 shows the log-compressed and envelope detected input
images, obtained using single 4 MHz transmits, along with the corresponding
image target acquired using 75 PW transmits at 12 MHz, followed by the
compounded image obtained using the proposed method. Perceptually, the
resulting image has a similar fidelity as the target, showing high structural
resolution and contrast.

In the second row of Fig. 3, we display the result of our unfolded proximal
gradient network when applied to in-silico point scatterers, showing strong
contrast, resolution, and side-lobe suppression. Fig. 6 displays a comparison
of the beam profile of a point scatterer, between different methods of
beamforming and compounding methods.

Figure 4 show a comparison between the various techniques of compounding the
given set of LR images. We compare the resulting images from standard coherent
compounding, the target (MV), the prediction by the proposed network, a UNet,
and the CNN by Gasse et al. [10].

Table II quantifies the Mean Absolute Error (MAE) values, the number of
parameters, and the un-optimised inference times of the proposed network (with
5 folds), a UNet-like network (UNet Lite; see Table VI in appendix A for
architectural details) with a similar number of parameters, a full UNet [6],
and the CNN of [10], containing about 60% more free parameters than the
proposed network. We chose UNets as the benchmark because it is shown to be a
very good baseline for such problems [31]. Furthermore, most other works rely
on a variation of a neural network with skip connections, so the UNet is a
good common denominator. This comparison is made on the test data that was
gathered for use in this project, as described in Section III. The benchmark
networks were trained using the same training strategy as the proposed
network, albeit not in a greedy training strategy.

Table III shows the results of an ablation study evaluating the contribution
of the individual loss terms in (14) to the results. In both Table II and
Table III, we separate the calculation of the MAE between the in-vivo and in-
silico test samples.

Additionally, we performed experiments to ascertain the volume of in-silico
data that should be added to the training loop in order to obtain the best
performance. Fig. 5 quantifies the result of such experiments, showing the MAE
metric as a function of the number of in-silico training data to in-silico
training data. We also performed an empirical search for the optimal number of
layers, and trained the network with 3, 5, 7, and 10 folds. The result of this
search is shown in Fig. 7, which displays the MAE metric as a function of the
number of unfoldings of the proximal network.

TABLE II: The table displays a comparison made between the proposed Unfolded Proximal Gradient Scheme, U-Net Lite (a network with a similar number of parameters), a full UNet, and the CNN published by Gasse et al. (2017). We compare the Mean Absolute Error (MAE) (including mean and variance across images in the test set), the number of parameters, and inference times (tested on an NVIDIA 3080 Ti). Network | Mean Absolute Error (mean $\pm$ var) | No. of | Inference
---|---|---|---
In-Vivo | In-Silico | Parameters | Time
Unfolded Proximal | 1.1$\times 10^{-1}$ $\pm$ 2.6$\times 10^{-4}$ | 7.1$\times 10^{-3}$ $\pm$ 1.48$\times 10^{-6}$ | $\sim$ 175 k | $\sim$ 20 ms
Gradient
UNet | 1.16$\times 10^{-1}$ $\pm$ 9.8$\times 10^{-8}$ | 1.1$\times 10^{-2}$ $\pm$ 9.7$\times 10^{-7}$ |  $\sim$ 1175 k |  $\sim$ 8 ms
UNet Lite | 1.34$\times 10^{-1}$ $\pm$ 1.93$\times 10^{-3}$ | 1.5$\times 10^{-2}$ $\pm$ 4.94$\times 10^{-6}$ |  $\sim$ 160 k |  $\sim$ 6 ms
CNN by | 1.4$\times 10^{-1}$ $\pm$ 1.96$\times 10^{-3}$ | 1.8$\times 10^{-2}$ $\pm$ 5.65$\times 10^{-6}$ | $\sim$ 280 k | $\sim$ 6 ms
Gasse et al. (2017)
TABLE III: The following table shows an ablation study, evaluating the contribution of loss terms as given in equation (14). We present the Mean Absolute Error (MAE), reporting both its mean and standard deviation across the test set. Losses | Training | Mean Absolute Error (mean $\pm$ var)
---|---|---
Pixel-based distance | Adversary | Frame-to-Frame |  | In-Vivo | In-Silico
✓ | ✓ | ✓ | Greedy | 1.1$\times 10^{-1}$ $\pm$ 2.6$\times 10^{-4}$ | 7.1$\times 10^{-3}$ $\pm$ 1.48$\times 10^{-6}$
- | ✓ | ✓ | Greedy | 1.3$\times 10^{-1}$ $\pm$ 1.03$\times 10^{-3}$ | 7.4$\times 10^{-3}$ $\pm$ 1.18$\times 10^{-6}$
✓ | - | ✓ | Greedy | 1.03$\times 10^{-1}$ $\pm$ 4.79$\times 10^{-4}$ | 7.34$\times 10^{-3}$ $\pm$ 1.16$\times 10^{-6}$
✓ | ✓ | - | Greedy | 1.12$\times 10^{-1}$ $\pm$ 2.48$\times 10^{-4}$ | 7.43$\times 10^{-3}$ $\pm$ 1.19$\times 10^{-6}$

We also re-trained the network after the initial greedy training process, but
received no appreciable increase in the quantitative or qualitative metrics of
the consequent results.

##  VI Discussion

We first start with Fig. 5, which makes the distinction between testing the
network with in-silico, and in-vivo samples. Due to the stark differences in
the distribution of the images, the variance of the combined test set can be
quite high. Therefore, we plot two trend lines, showing that results improve
for both in-vivo images, and simulated point scatterer data. This indicates
that adding in-silico data in the training loop can also help improve the
quality of not just in-silico inputs, but also in-vivo data. It also shows
that augmenting limited in-vivo training data with in-silico data is a valid
approach.

Fig. 6 shows that the proposed network does indeed improve the beam profile of
point scatterers. The network not only manages to improve the lateral
localisation of the point scatterer, but also reduces the noise in its
periphery, as seen in the input profile and the regularly compounded profile.

While augmenting the in-vivo dataset with a sizeable volume of in-silico
samples improves results, eventually hitting a point of diminishing returns,
the same cannot be said of adding folds to the network. In Fig. 7, we show
that increasing the number of folds in a network does not necessarily lead to
better results. But rather that the optimum number of folds may lay with an
intermediary number of unfoldings, which is in line with other works that deal
with unfolded networks [24].

In Fig. 3, we see that the network does succeed in improving the spatial
resolution and contrast of the input signal, with the image quality
approaching the ground truth. We also have predictions by the benchmark
networks compared to the unfolded network shown in Fig. 4. Although the UNet
performs quite well when considering the metrics from Table II, visually, we
see that the results are hazy and undefined. Whereas the CNN is noisy, and is
potentially hallucinating the noise that we observe in the predicted image. We
can therefore hypothesise that the UNet based networks are perhaps under-
specified, and have therefore learnt a shortcut to optimise the problem at
hand. While the CNN performs better than the UNet visually, it comes at the
cost of a larger computational cost, and a more noticeable noise profile.

A similar effect is in action in Table III, when $\lambda_{1}=0.0$
(adversarial loss weight term). The MAE for the in-vivo samples indicate that
the results are better, however, qualitatively/perceptual performance is
worse. This is because the adversarial loss tends to promote reconstruction of
the high frequency, crisp details of an image, while the $\ell_{1}$ loss is in
practice dominated by errors in the low frequency components. Thus, without a
perceptual penalty on the sharpness of an image (via the adversary), the
resulting image tends to be blurry and hazy. Fig. 8, in appendix B displays
the images that are obtained from networks that have been trained with the
$\mathcal{L}_{\ell_{1}}$ and the $\mathcal{L}_{adv}$ components turned off
individually.

While we employ a combination of both pixel-based and distribution-based loss,
the neural network optimises for the lowest value of loss, rather than the
best visual representation of the US signal data. Although it is advantageous
to use such a combination, it presents us with a problem that has been
encountered in US imaging before, and demonstrates why it is equally important
to visually inspect the resulting images in addition to relying on metrics.
More importantly, the benchmark networks lack any of the physics-based design
considerations that the proposed network possesses.

These results allude to the advantages of using model based networks in
challenging ill-posed problems, as it appears to promote stability by virtue
of its architecture. We also find that it is crucial to transform the output
of the unfolded network using an RF-to-Image transformation function (equation
(10)) before the calculation of the losses (much more trivial to balance loss
terms), or passing it on to the discriminator.

##  VII Conclusions

We proposed a model-based deep neural architecture for high-resolution plane
wave compounding, incorporating strong physics-based inductive biases.
Following the unfolding approach, we observe that the proposed network out-
performs several benchmarks by approximately 5%, using the MAE metric.

Perceptually, the proximal network produces images with enhanced fidelity,
containing high structural resolution and contrast, as shown in Figs. 3 and 4.
We also derive a high quality prediction on the point scatterers as seen in
the second row of Fig. 3, with a high degree of side-lobe suppression,
resolution, and contrast. The same is confirmed using a 1D lateral profile of
the point scatterers. We also further demonstrate that it is beneficial to
include a significant number of in-silico samples in the training of the
network, and that increasing the number of folds beyond 5 does not necessarily
lead to better performance, which is in line with other works that have
trained unfolded networks [24]. Replacing the proximal operator with other
potential candidates is a great avenue for further research, along with
enhanced data augmentation.

While it is typically infeasible to make use of MV beamforming in real-time
applications, we have presented a neural architecture that can greatly enhance
the resolution and contrast of ultrasound images, over DAS beamformed images,
with great potential for real-time applications. We achieve the above results
with a relatively low parameter count when compared to other architectures. We
conclude that the model-based approach proposed here can produce better
results under a comparatively small computational foot-print.

## Acknowledgment

The authors would like to thank Dr. Elik Aharoni, Ronnie Rosen, and Oded Drori
for their guidance and help in collecting the in-vivo data. We would also like
to thank the Weizmann Institute of Science for accommodating a visit in the
spirit of international collaboration.

## References

  * [1] M. Tanter and M. Fink, “Ultrafast imaging in biomedical ultrasound,” _IEEE transactions on ultrasonics, ferroelectrics, and frequency control_ , vol. 61, no. 1, pp. 102–119, 2014.
  * [2] R. J. van Sloun, O. Solomon, M. Bruce, Z. Z. Khaing, H. Wijkstra, Y. C. Eldar, and M. Mischi, “Super-resolution ultrasound localization microscopy through deep learning,” _IEEE Transactions on Medical Imaging_ , vol. 40, no. 3, pp. 829–839, 2020.
  * [3] R. J. van Sloun, R. Cohen, and Y. C. Eldar, “Deep learning in ultrasound imaging,” _Proceedings of the IEEE 108 (1)_ , pp. 11–29, 2019.
  * [4] M. A. L. Bell, J. Huang, D. Hyun, Y. C. Eldar, R. van Sloun, and M. Mischi, “Challenge on ultrasound beamforming with deep learning (cubdl),” in _2020 IEEE International Ultrasonics Symposium (IUS)_. IEEE, 2020, pp. 1–5.
  * [5] C. Szegedy, S. Ioffe, V. Vanhoucke, and A. A. Alemi, “Inception-v4, inception-resnet and the impact of residual connections on learning,” in _Thirty-First AAAI Conference on Artificial Intelligence_ , 2017.
  * [6] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.
  * [7] Y. Wang, B. Yu, L. Wang, C. Zu, D. S. Lalush, W. Lin, X. Wu, J. Zhou, D. Shen, and L. Zhou, “3d conditional generative adversarial networks for high-quality pet image estimation at low dose,” _Neuroimage_ , vol. 174, pp. 550–562, 2018.
  * [8] B. Luijten, R. Cohen, F. J. De Bruijn, H. A. Schmeitz, M. Mischi, Y. C. Eldar, and R. J. Van Sloun, “Adaptive ultrasound beamforming using deep learning,” _IEEE Transactions on Medical Imaging_ , 2020.
  * [9] I. A. Huijben, B. S. Veeling, and R. J. van Sloun, “Deep probabilistic subsampling for task-adaptive compressed sensing,” in _International Conference on Learning Representations_ , 2019.
  * [10] M. Gasse, F. Millioz, E. Roux, D. Garcia, H. Liebgott, and D. Friboulet, “High-quality plane wave compounding using convolutional neural networks,” _IEEE transactions on ultrasonics, ferroelectrics, and frequency control_ , vol. 64, no. 10, pp. 1637–1639, 2017.
  * [11] S. Khan, J. Huh, and J. C. Ye, “Universal plane-wave compounding for high quality us imaging using deep learning,” in _2019 IEEE International Ultrasonics Symposium (IUS)_. IEEE, 2019, pp. 2345–2347.
  * [12] B. Guo, B. Zhang, Z. Ma, N. Li, Y. Bao, and D. Yu, “High-quality plane wave compounding using deep learning for hand-held ultrasound devices,” in _International Conference on Advanced Data Mining and Applications_. Springer, 2020, pp. 547–559.
  * [13] X. Zhang, J. Li, Q. He, H. Zhang, and J. Luo, “High-quality reconstruction of plane-wave imaging using generative adversarial network,” in _2018 IEEE International Ultrasonics Symposium (IUS)_. IEEE, 2018, pp. 1–4.
  * [14] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in _Advances in neural information processing systems_ , 2014, pp. 2672–2680.
  * [15] S. Rothlübbers, H. Strohm, K. Eickel, J. Jenne, V. Kuhlen, D. Sinden, and M. Günther, “Improving image quality of single plane wave ultrasound via deep learning based channel compounding,” in _2020 IEEE International Ultrasonics Symposium (IUS)_. IEEE, 2020, pp. 1–4.
  * [16] J. P. Cohen, M. Luck, and S. Honari, “Distribution matching losses can hallucinate features in medical image translation,” in _International conference on medical image computing and computer-assisted intervention_. Springer, 2018, pp. 529–536.
  * [17] A. D’Amour, K. Heller, D. Moldovan, B. Adlam, B. Alipanahi, A. Beutel, C. Chen, J. Deaton, J. Eisenstein, M. D. Hoffman _et al._ , “Underspecification presents challenges for credibility in modern machine learning,” _arXiv preprint arXiv:2011.03395_ , 2020.
  * [18] R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, “Shortcut learning in deep neural networks,” _Nature Machine Intelligence_ , vol. 2, no. 11, pp. 665–673, 2020.
  * [19] R. Cohen, Y. Zhang, O. Solomon, D. Toberman, L. Taieb, R. J. van Sloun, and Y. C. Eldar, “Deep convolutional robust pca with application to ultrasound imaging,” in _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 3212–3216.
  * [20] N. Chennakeshava, B. Luijten, O. Drori, M. Mischi, Y. C. Eldar, and R. J. van Sloun, “High resolution plane wave compounding through deep proximal learning,” in _2020 IEEE International Ultrasonics Symposium (IUS)_. IEEE, 2020, pp. 1–4.
  * [21] M. Mardani, Q. Sun, D. Donoho, V. Papyan, H. Monajemi, S. Vasanawala, and J. Pauly, “Neural proximal gradient descent for compressive imaging,” _Advances in Neural Information Processing Systems_ , vol. 31, pp. 9573–9583, 2018.
  * [22] K. Gregor and Y. LeCun, “Learning fast approximations of sparse coding,” in _Proceedings of the 27th International Conference on International Conference on Machine Learning_ , 2010, pp. 399–406.
  * [23] V. Monga, Y. Li, and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” _arXiv preprint arXiv:1912.10557_ , 2019.
  * [24] O. Solomon, R. Cohen, Y. Zhang, Y. Yang, Q. He, J. Luo, R. J. van Sloun, and Y. C. Eldar, “Deep unfolded robust PCA with application to clutter suppression in ultrasound,” _IEEE transactions on medical imaging 39(4)_ , pp. 1051–1063, 2019.
  * [25] O. Scherzer, “The use of morozov’s discrepancy principle for tikhonov regularization for solving nonlinear ill-posed problems,” _Computing_ , vol. 51, no. 1, pp. 45–60, 1993.
  * [26] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1125–1134.
  * [27] Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle _et al._ , “Greedy layer-wise training of deep networks,” _Advances in neural information processing systems_ , vol. 19, p. 153, 2007.
  * [28] E. Belilovsky, M. Eickenberg, and E. Oyallon, “Greedy layerwise learning can scale to imagenet,” in _International conference on machine learning_. PMLR, 2019, pp. 583–593.
  * [29] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [30] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org. [Online]. Available: https://www.tensorflow.org/
  * [31] A. Hauptmann and J. Adler, “On the unreasonable effectiveness of cnns,” _arXiv preprint arXiv:2007.14745_ , 2020.

## Appendix A

Given below are the details of the neural network architectures used in this
work. Table IV details on the architecture of the adopted proximal network,
Table V gives the architecture of the discriminator, and Table VI provides the
UNet Lite network used in the benchmarks.

TABLE IV: The architectural design of the neural proximal operator. Layer # | Type | Kernel Size | Activation | Output Shape
---|---|---|---|---
1 | Convolutional | 3x3 | LeakyRelu | (2048x128x1)
2 | Convolutional | 3x3 | LeakyRelu | (2048x128x8)
3 | Convolutional | 3x3 | LeakyRelu | (2048x128x16)
4 | Convolutional | 3x3 | LeakyRelu | (1024x64x32)
5 | Convolutional | 3x3 | LeakyRelu | (1024x64x32)
6 | Dropout(0.3) | - | - | (1024x64x32)
7 | Concatenate | - | - | (2048x128x40)
8 | Convolutional | 3x3 | LeakyRelu | (2048x128x16)
9 | Convolutional | 3x3 | LeakyRelu | (2048x128x32)
10 | Convolutional | 3x3 | LeakyRelu | (2048x128x1)
TABLE V: The architectural design of the neural discriminator used for adversarial training. Layer # | Type | Kernel Size | Activation | Output Shape
---|---|---|---|---
1 | Concatenate | - | - | (2048x128x4)
2 | Convolutional | 3x3 | LeakyRelu | (1024x64x32)
3 | Convolutional | 3x3 | LeakyRelu | (512x32x64)
4 | Convolutional | 3x3 | LeakyRelu | (256x16x128)
5 | Convolutional | 3x3 | Sigmoid | (256x16x1)
TABLE VI: The architectural design of the baseline Unet Lite. Layer # | Type | Kernel Size | Activation | Output Shape
---|---|---|---|---
1 | Concatenate | - | - | (2048x128x3)
2 | Convolutional | 3x3 | LeakyRelu | (2048x128x16)
3 | Convolutional | 3x3 | LeakyRelu | (2048x128x32)
4 | Convolutional | 3x3 | LeakyRelu | (1024x64x64)
5 | Convolutional | 3x3 | LeakyRelu | (1024x64x64)
6 | Convolutional | 3x3 | LeakyRelu | (2048x128x64)
7 | Dropout(0.3) | - | - | (2048x128x64)
8 | Concatenate | - | - | (2048x128x96)
9 | Convolutional | 3x3 | LeakyRelu | (2048x128x60)
10 | Convolutional | 3x3 | LeakyRelu | (2048x128x16)
11 | Convolutional | 3x3 | LeakyRelu | (2048x128x1)

## Appendix B

Figure 8 qualitatively displays the effect of the individual components of the
total loss function. It shows us that a lack of the $\mathcal{L}_{adv}$ term
tends to provide results that do not have highly defined features. On the
other hand, a prediction made using a network trained without the
$\mathcal{L}_{\ell_{1}}$ term yields results that lack contrast.

Figure 8: The images show the effect of switching off parts of the total
objective function on the final prediction. It displays standard coherent
compounding, the target image, the proposed network, and two other iterations
of the proposed network where $\mathcal{L}_{\ell_{1}}$ and $\mathcal{L}_{adv}$
are set to zero, individually. The anatomy is a longitudinal profile of the
carotid artery. These images have been taken from the test set.