# Pluripolar hulls and fine holomorphy

Jan Wiegerinck  KdV Institute for Mathematics
University of Amsterdam
Science Park 105-107
P.O. box 94248, 1090 GE Amsterdam
The Netherlands<EMAIL_ADDRESS>
###### Abstract.

Examples by Poletsky and the author and by Zwonek show the existence of
nowhere extendable holomorphic functions with the property that the pluripolar
hull of their graphs is much larger than the graph of the respective
functions, and contains multiple sheets. We will explain this phenomenon by
fine analytic continuation of the function over part of a Cantor-type set
involved. This gives more information on the hull, and allows for weakening
and effectiveness of the conditions in the original examples.

###### Key words and phrases:

finely holomorphic function, pluripolar hull, fine analytic continuation

###### 2020 Mathematics Subject Classification:

30G12, 30A14, 31C10, 31C40, 32U05, 32U15

Part of this work was done while the author visited Jagiellonian University in
Krakow. He is grateful to Armen Edigarian, Lukasz Kosinski, and Włodzimierz
Zwonek for the invitation and for very stimulating discussions.

##  1\. Introduction

We study the relation between pluripolar hulls of graphs of certain
holomorphic functions and fine holomorphy. These notions are recalled in
Section 2. Pluripolar hulls are of importance because of an old result of
Zeriahi [21] that roughly states that a pluripolar set
$X\subset{\mathbb{C}}^{n}$ is complete if it is an $F_{\sigma}$ as well as
$G_{\delta}$ and equal to its pluripolar hull. The last two conditions are
necessary too. Recall that $X\subset{\mathbb{C}}^{n}$ is pluripolar if it is
contained in the $-\infty$ set of a plurisubharmonic function, and it is
complete if it equals the $-\infty$ set of such a function.

Levenberg, Martin, and Poletsky in [15] conjectured that the pluripolar hull
of a graph of an analytic function $f$ is equal to the graph of the maximal
analytic extension of $f$ in the sense of Weierstrass. This conjecture was
refuted in [5]. Other counterexamples to this conjecture showed remarkable
behavior: the hull could extend far beyond the domain of the maximal extension
and could consist of many sheets, [4, 17, 22]. The connection with fine
holomorphy was made in [8]. This eventually led to Theorem 2.2, which together
with the observation in [3] that the examples from [5, 7] are all instances of
finely analytic continuation, suggested the following refined conjecture.

######  Conjecture 1.1.

[20] The pluripolar hull of a graph of an analytic function $f$ consists of
the graph of the (possibly multi-valued) maximal finely analytic extension of
$f$.

In [20] the author boldly stated without much proof that in all the known
examples it can be seen that the pluripolar hull occurs because of finely
analytic continuation. The purpose of this paper is to corroborate this
statement and the conjecture.

Let ${\mathcal{C}}={\mathcal{C}}(a,b)$, $a=(a_{0},a_{1},a_{2},\cdots)$,
$b=(b_{0},b_{1},b_{2},\cdots)$, $a_{j}<b_{j}$, be a Cantor set in the real
line formed by omitting from the interval $[a_{0},b_{0}]$ open intervals
$(a_{i},b_{i})$, $i\geqslant 1$, in such a way that
$[a_{i},b_{i}]\cap[a_{j},b_{j}]=\varnothing$ ($i>j\geqslant 0$) and the
maximal length of an interval in
$[a_{0},b_{0}]\setminus\bigcup_{j=1}^{N}(a_{j},b_{j})$ tends to 0 for
$N\to\infty$. Without loss of generality, we can and will assume that for all
$i$ we have $b_{i+1}-a_{i+1}\leqslant b_{i}-a_{i}$.

In [17] Poletsky and the author studied the holomorphic function $f_{a,b}$
defined by

$f_{a,b}(z)=\frac{z-b_{0}}{z-a_{0}}\prod_{i=1}^{\infty}\frac{z-a_{i}}{z-b_{i}}$

and its square roots, which are both well-defined single valued functions on
the complement of ${\mathcal{C}}(a,b)$. The main result here was that it is
possible to choose the Cantor set in such a way that the _pluripolar hull_ in
${\mathbb{C}}^{2}$ of the graph $\Gamma$ of each of these roots consists of
precisely the graphs of the two square roots above the complement of the
Cantor set. What happens above the Cantor set remained unclear. Then Zwonek
[22] constructed holomorphic functions on the unit disc that can not be
extended over the disc, but with the property that the pluripolar hull of
their graph consists of infinitely many points above
${\mathbb{C}}\setminus\\{|z|=1\\}$, see also Siciak, [19]. In [4] similar
results were obtained for functions on the complement of certain Cantor-like
sets and, moreover, the hull above these Cantor sets was partially described.

We show that a computable condition on Cantor sets ${\mathcal{C}}(a,b)$ in
terms of properties of the sequences $a$ and $b$ guarantees that the
pluripolar hull of a square root of $f_{a,b}$ indeed comes about because of
fine analytic extension of this root over part of the Cantor set. This is done
in Section 4. In particular, it follows, as in [4], that the pluripolar hull
also contains points above this part of the Cantor set. Under some extra
conditions we show in Section 5 that above this part the hull consists of
precisely the graphs of the fine extension of the roots. In Section 3 we deal
with some elementary properties of $f_{a,b}$ and its square roots, and correct
an error in [17].

In the final Section 6 we follow up on the examples in [22] and [19]. In these
articles Blaschke products $B$ with extra conditions on their zeroset are
constructed. The domain of existence of $B$ is the unit disc and for every
holomorphic function $f$ on a larger domain than the unit disc, the hull of
the graph of $fB$ is strictly larger than the graph of $fB$ over the unit disc
and may contain infinitely many sheets. We show that the Blaschke product $B$
and therefore the functions $fB$ admit finely analytic continuation over part
of the unit circle. As in [4] the condition is local, and altogether less
restrictive than the ones in [22, 19]. Moreover, we obtain information on the
hull over this part of the unit circle.

##  2\. Finely holomorphic functions and pluripolar hulls

Recall that the fine topology on ${\mathbb{R}}^{n}$, in particular on
${\mathbb{R}}^{2}={\mathbb{C}}$ is the coarsest topology that makes all
subharmonic functions continuous. See [10, 11, 2, 1] for background on the
fine topology. On finely open sets one can define _finely holomorphic
functions_. For our purposes the following definition is the most convenient.
See [12, 13, 14] for more information on finely holomorphic functions.

######  Definition 2.1.

Let $\Omega$ be finely open subset of ${\mathbb{C}}$. A function
$f:\Omega\to{\mathbb{C}}$ is called _finely holomorphic_ (or _finely
analytic_) on $\Omega$ if for every $z\in\Omega$ there exists a fine
neighborhood $K_{z}\subset\Omega$, which can be taken to be compact in the
Euclidean topology, with the property that $f|_{K_{z}}$ is a uniform limit of
rational functions with poles off $K_{z}$.

Note that if $\Omega$ is Euclidean open, then finely holomorphic functions on
$\Omega$ are just ordinary holomorphic functions. Finely open sets of the form
$\\{z\in D:h(z)>c\\}$, where $h$ is a subharmonic function on an open set $D$
in ${\mathbb{C}}$, form a base of the fine topology.

Next let us recall that the (${\mathbb{C}}^{n}$-)pluripolar hull of a
pluripolar set $X\subset{\mathbb{C}}^{n}$ is defined as

$X^{*}_{{\mathbb{C}}^{n}}\operatorname{\overset{def}{=}}\\{z\in{\mathbb{C}}^{n}:\
\forall u\in\operatorname{PSH}({\mathbb{C}}^{n})\ u|_{X}=-\infty\Rightarrow
u(z)=-\infty\\}.$

Here $\operatorname{PSH}({\mathbb{C}}^{n})$ denotes the plurisubharmonic
functions on ${\mathbb{C}}^{n}$. See [16] for background and properties of
pluripolar hulls.

Note that if $X$ is a sufficiently large part of the graph of a holomorphic
function $f$, (the projection on the domain of $f$ should have positive
capacity), then $X^{*}_{{\mathbb{C}}^{n}}$ contains the graph of any analytic
extension of $f$.

Edlund and Jöricke, [8] observed a connection between pluripolar hulls of
graphs and fine analytic extension. This eventually led to the following
result [9, Theorem 4.5].

######  Theorem 2.2.

Let $E$ be a set of positive capacity in ${\mathbb{C}}$ and let $f$ be a
finely holomorphic function on a fine neighborhood of $E$. Let $X$ denote the
graph of $f$ over $E$. Then $X^{*}_{{\mathbb{C}}^{n}}$ contains the graph of
the (possibly multi-valued) maximal finely analytic continuation of $f$.

##  3\. The product $f_{a,b}$ and its square roots

We denote by $d(z,A)$ the Euclidean distance from $z\in{\mathbb{C}}$ to
$A\subset{\mathbb{C}}$. Writing

$f_{a,b}(z)=\left(1+\frac{a_{0}-b_{0}}{z-a_{0}}\right)\prod_{i=1}^{\infty}\left(1+\frac{b_{i}-a_{i}}{z-b_{i}}\right)$

we see that the product converges uniformly on sets

$\Omega_{\delta}=\\{z:d(z,{\mathcal{C}}(a,b))\geqslant\delta\\},\quad\delta>0,$

because $0\leqslant\sum_{i=1}^{\infty}(b_{i}-a_{i})\leqslant b_{0}-a_{0}$.
Hence $f_{a,b}$ represents a holomorphic function on the complement of
${\mathcal{C}}(a,b)$. It is bounded at infinity and therefore extends to be
holomorphic at infinity. The Laurent expansion on $\\{|z|>|a_{0}|+|b_{0}|\\}$
is

(3.1) $f_{a,b}(z)=1+\frac{c_{1}}{z}+\frac{c_{2}}{z^{2}}+\cdots,$

with $c_{1}=-(b_{0}-a_{0})+\sum_{i=1}^{\infty}(b_{i}-a_{i})$, so that $c_{1}$
equals the length (1-dimensional Lebesgue measure) of the Cantor set.

If ${\mathcal{C}}(a,b)$ has length zero it was claimed in [17] that $f_{a,b}$
is constant. This is in general not true. Consider $\log f_{a,b}$ a branch of
the logarithm, which is well-defined in a neighborhood of $\infty$. Its
derivative equals

$\left(\frac{1}{z-b_{0}}-\frac{1}{z-a_{0}}\right)-\sum_{i=1}^{\infty}\left(\frac{1}{z-b_{i}}-\frac{1}{z-a_{i}}\right)=\frac{1}{z}\sum_{k=1}^{\infty}\frac{b_{0}^{k}-a_{0}^{k}-\sum_{j=1}^{\infty}b_{j}^{k}-a_{j}^{k}}{z^{k}}.$

If ${\mathcal{C}}(a,b)$ has length zero, then the first term ($k=1$) in this
sum vanishes, but the others generally need not.

Let

$f^{N}=f^{N}_{a,b}=\frac{z-b_{0}}{z-a_{0}}\prod_{i=1}^{N}\frac{z-a_{i}}{z-b_{i}}$

be a partial product. There exist two holomorphic branches $g^{N}_{+}$ and
$g^{N}_{-}$ of $\sqrt{f^{N}}$ on

$({\mathbb{C}}\setminus[a_{0},b_{0}])\cup\left(\bigcup_{j=1}^{N}(a_{j},b_{j})\right)$

with $\lim\limits_{z\to\infty}g^{N}_{+}(z)=1$ and
$\lim\limits_{z\to\infty}g^{N}_{-}(z)=-1$.

On

$({\mathbb{C}}\setminus{\mathbb{R}})\cup(a_{0},b_{0})\setminus\left(\bigcup_{j=1}^{N}[a_{j},b_{j}]\right)$

there are two branches of $\sqrt{f^{N}}$: $\tilde{g}^{N}_{+}$ with
$\lim\limits_{z\to\infty\atop\operatorname{Im}z>0}\tilde{g}^{N}_{+}(z)=1$, and
$\tilde{g}^{N}_{-}$ with
$\lim\limits_{z\to\infty\atop\operatorname{Im}z>0}\tilde{g}^{N}_{-}(z)=-1$.

In particular, it follows that
$\lim\limits_{z\to\infty\atop\operatorname{Im}z<0}\tilde{g}^{N}_{\pm}(z)=\mp
1$, as $\tilde{g}^{N}_{\pm}=g^{N}_{\mp}$ on the lower half-plane. The four
functions $g^{N}_{\pm}$, $\tilde{g}^{N}_{\pm}$ are analytic extensions of each
other and together form a 2-valued holomorphic function on
${\mathbb{C}}\setminus\\{a_{j},b_{j},j=0\ldots N\\}$.

Clearly $g^{N}_{\pm}$ both converge uniformly on compact sets to holomorphic
square roots $g_{\pm}$ of $f_{a,b}$ on

(3.2)
$D=({\mathbb{C}}\setminus[a_{0},b_{0}])\cup\left(\bigcup_{j=1}^{\infty}(a_{j},b_{j})\right),$

while $\tilde{g}^{N}_{\pm}$ both converge uniformly on compact sets to square
roots $\tilde{g}_{\pm}$ of $f_{a,b}$ on

(3.3) $\tilde{D}={\mathbb{C}}\setminus{\mathbb{R}}.$

Obviously $g_{\pm}=\tilde{g}_{\pm}$ on the upper half-plane and
$g_{\pm}=-\tilde{g}_{\pm}$ on the lower half-plane. The maximal analytic
extension of each of these functions is a single valued function on $D$.

##  4\. Fine holomorphy on the Cantor set

We start with a condition on ${\mathcal{C}}(a,b)$ that ensures that $f_{a,b}$
converges uniformly on compact sets in a set $E$ that contains a subset of
${\mathcal{C}}(a,b)$ and is sufficiently big, namely that it is a fine
neighborhood of certain points in ${\mathcal{C}}(a,b)$. To make the connection
with [17] we will write $b_{j}-a_{j}=e^{-jc_{j}}$, where $\\{c_{j}\\}_{j}$ is
an increasing sequence of positive numbers tending to $\infty$.

The following lemma is obvious.

######  Lemma 4.1.

Let $(p_{n})_{n}$ be a sequence of positive numbers with converging sum. Then

(4.1) $\sum_{n=1}^{\infty}\frac{b_{n}-a_{n}}{|z-b_{n}|}\quad\text{ and
}\quad\left(1+\frac{a_{0}-b_{0}}{z-a_{0}}\right)\prod_{i=1}^{\infty}\left(1+\frac{b_{i}-a_{i}}{z-b_{i}}\right)$

converge uniformly on compact sets in

(4.2) $E_{N}=\bigcap_{n=N}^{\infty}\\{z:b_{n}-a_{n}\leqslant
p_{n}|z-b_{n}|\\}\setminus\\{b_{1},\ldots,b_{N-1}\\},\quad N>0.$

Note that only if $\sum\frac{e^{-jc_{j}}}{p_{j}}$ is finite,
$E_{N}\cap{\mathcal{C}}(a,b)$ will eventually be nonempty.

The following Lemma is a variation of Lemma 2 and Example 14 in [4].

######  Lemma 4.2.

Let ${\mathcal{C}}(a,b)$ be a Cantor set formed by sequences $a$, $b$ as
before and let $\\{c_{j}\\}_{j}$ be an increasing sequence tending to $\infty$
with $b_{j}-a_{j}=e^{-jc_{j}}$, and such that

(4.3) $\sum_{j=1}^{\infty}\frac{1}{jc_{j}}<\frac{1}{2}.$

Then for $N$ sufficiently large, the set $E_{N}$ in (4.2) contains a fine
neighborhood $E$ of a nonempty subset of $C(a,b)$.The functions in (4.1)
converge uniformly on compact subsets of $E$.

###### Proof.

We can assume $b_{0}-a_{0}=1$. Let $p_{j}=e^{-jc_{j}/2}$ and set

(4.4)
$F_{N}=\bigcup_{i=1}^{\infty}[a_{i},b_{i}]\cup\bigcup_{n=N}^{\infty}\\{z\in{\mathbb{C}}:\
|z-b_{n}|\leqslant(b_{n}-a_{n})/p_{n}\\}.$

Note that $F_{N}$ is compact.

We have $\operatorname{Cap}[a_{j},b_{j}]=e^{-jc_{j}}/4$ and
$\operatorname{Cap}(\\{|z-b_{j}|\leqslant(b_{j}-a_{j})/p_{j}\\})=e^{-jc_{j}/2}$.
We now use Theorem 5.1.4 in [18], which estimates the capacity of a union of
Borel sets. It reads in our setting

(4.5)
$\begin{split}\frac{1}{\log(\frac{1}{\operatorname{Cap}(F_{N})})}&\leqslant\sum_{j=1}^{\infty}\frac{1}{\log(4e^{jc_{j}})}+\sum_{j=N}^{\infty}\frac{1}{\log(e^{jc_{j}/2})}\\\
&=\sum_{j=1}^{\infty}\frac{1}{jc_{j}+\log
4}+\sum_{j=N}^{\infty}\frac{2}{jc_{j}}.\end{split}$

Let $\varepsilon>0$. By taking $N$ sufficiently large, the final sum
$\sum_{j=N}^{\infty}$ in this equation can be made smaller than $\varepsilon$.
Then

(4.6)
$\frac{1}{\log(\frac{1}{\operatorname{Cap}(F_{n})})}\leqslant\sum_{j=1}^{\infty}\frac{1}{jc_{j}}+\varepsilon,$

or

(4.7)
$\operatorname{Cap}(F_{N})\leqslant\exp\left(\frac{-1}{\sum_{j=1}^{\infty}\frac{1}{jc_{j}}+\varepsilon}\right)<\frac{1}{4},$

if $\sum\frac{1}{jc_{j}}\leqslant 1/2$ and $\varepsilon$ is sufficiently
small.

Define the compact set $J_{N}:=[0,1]\cup F_{N}$. Let

$u(z):=g_{F_{N}}-g_{J_{N}},$

where $g_{X}$ denotes the Green function of $X$ with pole at infinity. Being
the difference of two subharmonic functions, $u$ is finely continuous,
therefore $\tilde{E}=\\{u>0\\}$ is a finely open set. Since the complement of
$F_{N}$ is contained in $E_{N}\cup\\{b_{1},\ldots,b_{N-1}\\}$, and $u\equiv 0$
on $F_{N}$, the functions in (4.1) converge uniformly on compact sets in the
finely open set $E=\tilde{E}\setminus\\{b_{1},\ldots,b_{N-1}\\}$.

It remains to show that $E$ contains points of ${\mathcal{C}}(a,b)$. We have

$u(\infty)=\log(\operatorname{Cap}J_{N})-\log(\operatorname{Cap}F_{N}),$

cf. [18, Thm 5.2.1]. Since $J_{N}$ contains $[0,1]$,
$\operatorname{Cap}J_{N}\geqslant 1/4$, hence $u(\infty)>0$ if $N$ is
sufficiently large. Let $\varphi$ be the Riemann map of the unit disc to the
complement $\Omega$ of $J_{N}$ that maps $0$ to $\infty$. Observe that the
boundary of $\Omega$ is locally connected and regular for the Dirichlet
problem, therefore $\varphi$ extends continuously to the unit circle and
$G_{J_{N}}$ is a continuous function with $G_{J_{N}}\equiv 0$ on $J_{N}$. Note
that for every $\theta$ the curve $r\mapsto\varphi(re^{i\theta})$,
($0\leqslant r\leqslant 1$) is non-thin at $\varphi(e^{i\theta})\in J_{N}$.

Let $v=u\circ\varphi$. Then $v$ is harmonic on the unit disc and
$v(0)=u(\infty)>0$. Since

$v(0)=\frac{1}{2\pi}\int_{0}^{2\pi}v^{*}(e^{i\theta})\,d\theta,$

where $v^{*}(e^{i\theta})=\lim_{r\uparrow 1}v(re^{i\theta})$ a.e., it follow
that $\lim_{r\uparrow 1}v(re^{i\theta})>0$ on a subset $\Gamma$ of positive
measure on the unit circle. Using the above observations, we find for
$\theta\in\Gamma$

$u\circ\varphi(e^{i\theta})=g_{F_{N}}(\varphi(e^{i\theta}))-g_{J_{N}}(\varphi(e^{i\theta}))=\liminf_{r\uparrow
1}g_{F_{N}}(\varphi(re^{i\theta}))-g_{J_{N}}(\varphi(re^{i\theta}))=\lim_{r\uparrow
1}v(re^{i\theta})>0.$

It follows that $\varphi(e^{i\theta})\in J_{N}\setminus
F_{N}\subset{\mathcal{C}}(a,b)$ and we are done.

∎

######  Remark 4.3.

It is a well-known consequence of Wiener’s criterion that relatively finely
open subsets of $[0,1]$ have positive Lebesgue measure. We don’t need this
fact, but it strengthens the connection with [4, Lemma 2].

######  Theorem 4.4.

Assume that ${\mathcal{C}}(a,b)$ satisfies the conditions of Lemma 4.2. Let
$E\subset{\mathcal{C}}(a,b)$ be the set constructed in the lemma. The
functions $f_{a,b}$ and $\tilde{g}_{\pm}$ extend over $E$ as finely
holomorphic functions from the upper half plane to the lower half plane.

###### Proof.

By Lemma 4.2 the functions $f^{N}$, and therefore the functions $g^{N}_{\pm}$
converge uniformly on compact sets in a finely open neighborhood $E$ of points
of ${\mathcal{C}}(a,b)$ to $f_{a,b}$, respectively $\tilde{g}_{\pm}$. By the
definition it follows that their limits are finely holomorphic on $E$. These
limits clearly coincide with $f_{a,b}$, respectively $\tilde{g}_{\pm}$ on
$E\cap({\mathbb{C}}\setminus{\mathbb{R}})$ and are therefore the sought for
finely holomorphic extensions. ∎

######  Remark 4.5.

One may wonder if, perhaps, $f_{a,b}$ extends as a finely holomorphic function
to the complement of the set of poles $\\{b_{j}\\}$. However, this is not the
case. The polar set $\\{b_{j}\\}$ is of the first Baire category in its
closure ${\mathcal{C}}(a,b)$ and then Theorem 3.4. in [14] states that
$f_{a,b}$ would extend finely holomorphically over part of $\\{b_{j}\\}$,
which is clearly not the case.

######  Corollary 4.6.

With $D$ and $\tilde{D}$ as in (3.2), (3.3) and assuming that
${\mathcal{C}}(a,b)$ satisfies the conditions of Lemma 4.2, the four finely
analytic function elements $\tilde{g}_{\pm}$ on $\tilde{D}\cup E$ and
$g_{\pm}$ on $D$ are (finely) holomorphic continuations of each other. Let $X$
denote the graph of $g_{+}$, then $X^{*}_{{\mathbb{C}}^{2}}$ contains the
graphs of $\tilde{g}_{\pm}$ over $\tilde{D}\cup E$ and the graph of $g_{-}$
over $D$. In particular $X^{*}_{{\mathbb{C}}^{2}}$ consists of precisely two
points over ${\mathbb{C}}\setminus{\mathcal{C}}(a,b)$ and at least two points
over $E$.

###### Proof.

Note that $\tilde{D}\cup E$ is finely connected, and $D$ is (finely)
connected. Hence the first statement follows directly from Theorem 4.4 and the
fact that $g_{\pm}=\tilde{g}_{\pm}$ on the upper half plane and
$g_{\pm}=-\tilde{g}_{\pm}$ on the lower half plane. That
$X^{*}_{{\mathbb{C}}^{2}}$ consists of at most two points $(z,w)$ for
$z\in{\mathbb{C}}\setminus{\mathcal{C}}(a,b)$ is part of Theorem 3.6. in [17].
The proof of this part only uses that $\\{c_{n}\\}_{n}$ tends to $\infty$.
Theorem 4.5 in [9] shows that points of the form $(z,\tilde{g}_{\pm}(z))$
belong to $X^{*}_{{\mathbb{C}}^{2}}$ if $z\in E$. ∎

##  5\. The pluripolar hull above $E$

As a proof of principle we will show in this section that under a strong
condition on the size of the deleted intervals, for every $z\in E$ the fiber
$X^{*}_{{\mathbb{C}}^{2}}\cap(\\{z\\}\times{\mathbb{C}})$ equals $\\{(z,w):\
w^{2}=f_{a,b}(z)\\}$. Here we have written $f_{a,b}$ also for its finely
analytic extension over $E$.

######  Lemma 5.1.

Let ${\mathcal{C}}(a,b)$ be a Cantor set as before, with
$b_{j}-a_{j}=e^{-jc_{j}}$ and $c_{j}=(j+2)!$, in particular the condition of
Lemma 4.2 is satisfied. Let $E$ and $F_{N}$ be the sets constructed in Lemma
4.2 and $f=f_{a,b}$ the finely holomorphic function on
$V=({\mathbb{C}}\setminus{\mathcal{C}}(a,b))\cup E$. Then for $N$ sufficiently
large there is a plurisubharmonic function $v$ on ${\mathbb{C}}^{2}$ such that
$\\{v=-\infty\\}\cap([a_{0},b_{0}]\setminus
F_{N}\times{\mathbb{C}})=\Gamma_{f}\cap([a_{0},b_{0}]\setminus
F_{N}\times{\mathbb{C}})$.

###### Proof.

Without loss of generality we can assume that $[a_{0},b_{0}]=[0,1]$. Set
$p_{j}=e^{-jc_{j}/2}$. Let

$f_{n}(z)=\frac{z-b_{0}}{z-a_{0}}\prod_{i=1}^{n}\frac{z-a_{i}}{z-b_{i}}=\frac{P_{n}(z)}{Q_{n}(z)},$

where $P_{n}$ and $Q_{n}$ are monic polynomials of degree $n+1$. We have for
$C>1$

(5.1) $\displaystyle|Q_{n}(z)|\leqslant(C+1)^{n+1}\quad\text{if $|z|<C$},$ for
$z\in[a_{0},b_{0}]\setminus F_{N}$ we have (5.2)
$\displaystyle|P_{n}(z)|<1,\quad|Q_{n}(z)|<1,$ and (5.3)
$\displaystyle|Q_{n}(z)|\geqslant\prod_{j=1}^{n}\frac{e^{-jc_{j}}}{p_{j}}=\prod_{j=1}^{n}e^{-jc_{j}/2}.$

Thus on $[a_{0},b_{0}]\setminus F_{N}$ we have the following estimates

(5.4)
$\begin{split}|f(z)-f_{n}(z)||Q_{n}(z)|&\leqslant\left|\left(\prod_{j=n+1}^{\infty}\left(1+\frac{b_{i}-a_{i}}{z-b_{i}}\right)-1\right)P_{n}(z)\right|\\\
&\leqslant\prod_{j=n+1}^{\infty}(1+p_{j})-1\leqslant\exp(\sum_{j=n+1}^{\infty}p_{j})-1\leqslant
2p_{n+1},\end{split}$

if $n\geqslant 2$.

Next for every $\delta>0$ and $|w-f(z)|\geqslant\delta$

(5.5) $|w-f_{n}(z)||Q_{n}(z)|\geqslant\delta\prod_{j=1}^{n}e^{-jc_{j}/2}.$

Now consider the plurisubharmonic functions

$v_{n}(z,w)=\log(|w-f_{n}(z)||Q_{n}(z)|)\quad\text{ on ${\mathbb{C}}^{2}$.}$

Note that

(5.6) $\frac{(n+1)c_{n+1}}{nc_{n}}=\frac{(n+1)(n+2)}{n}\to\infty,\quad\text{as
$n\to\infty$.}$

Let $\\{e_{n}\\}_{n}$ be a sequence of positive numbers such that $\sum e_{n}$
is finite but $\sum e_{n}\frac{-(n+1)c_{n+1}}{nc_{n}}=-\infty$, and form

(5.7) $v(z,w)=\sum_{n=1}^{\infty}\frac{\max\\{v_{n}(z,w),\log
p_{n+1}\\}}{nc_{n}}e_{n}.$

Because $v_{n}(z,w)\leqslant(n+2)\log(C+1)$ on $\\{|z|,|w|<C\\}$, for $C$
sufficiently large, the function $v(z,w)$ is plurisubharmonic on
${\mathbb{C}}^{2}$. For $z\in{\mathcal{C}}(a,b)\setminus F_{N}$ the function
$v(z,w)$ has the following properties. If $w=f(z)$, then
$v_{n}(z,w)\leqslant\log 2+\log p_{n+1}$ and therefore $v(z,f(z))=-\infty$. On
the other hand, if $|w-f(z)|\geqslant\delta$, then there exists $n_{0}>0$ with

(5.8)
$v_{n}(z,w)\geqslant\log(\delta)+\sum_{j=1}^{n}(-jc_{j}/2)\geqslant\log\delta-
nc_{n},$

therefore $v(z,w)$ is finite.

∎

######  Theorem 5.2.

Let ${\mathcal{C}}(a,b)$ be a Cantor set that satisfies the conditions of
Lemma 5.1. Let $X$ denote the graph of $g_{+}$ over
${\mathbb{C}}\setminus{\mathcal{C}}(a,b)$ and let $g$ denote the maximal
finely analytic extension of $g_{+}$. Then
$X^{*}_{{\mathbb{C}}^{2}}\cap\\{(D\cup
E)\times{\mathbb{C}}\\}=\Gamma_{g}\cap\\{(D\cup E)\times{\mathbb{C}}\\}$.

###### Proof.

Because of Corollary 4.6, we only have to prove that for $z\in E$
$X^{*}_{{\mathbb{C}}^{2}}\cap(\\{z\\}\times{\mathbb{C}}=\\{(z,g_{+}(z)),(z,g_{-}(z))\\}$.
The set $E$ has positive length and hence positive capacity. The function $v$
constructed in the previous lemma is plurisubharmonic function and equals
$-\infty$ on $\\{(z,f_{a,b}(z)),z\in E\\}$. Then by Theorem 2.2 it equals
$-\infty$ on the graph of $f_{a,b}$. Consider the function
$u(z,w)=v(z,w^{2})$. The function $u$ is plurisubharmonic and equals $-\infty$
on the graph of $g_{+}$ and for $z\in E$ we have $u(z,w)=-\infty$ if and only
if $w$ is one of the two values of $g(z)$. ∎

##  6\. Blaschke products

In this section we will consider Blaschke products of the form

(6.1)
$B(z)=z^{l}\prod_{j=1}^{\infty}\left(\frac{|a_{j}|}{a_{j}}\frac{a_{j}-z}{1-\overline{a_{j}}z}\right)=z^{l}\prod_{j=1}^{\infty}\left(\frac{1}{|a_{j}|}\frac{a_{j}-z}{1/\overline{a_{j}}-z}\right)$

on the unit disc ${\mathbb{D}}$. In the next lemma we formulate a condition
similar to (4.3) that guarantees that such a Blaschke product extends as a
finely holomorphic function over a part $E$ of the boundary of ${\mathbb{D}}$,
while $E$ is in the closure of the zeros of $B$. The lemma leads then to an
easy proof of an extension of Theorem 1 in [22].

It is well known that under the Blaschke condition
$\sum_{k}(1-|a_{k}|)<\infty$ the product (6.1) represents a function that is
holomorphic on ${\mathbb{D}}\cup({\mathbb{D}}^{e}\setminus X)$, where
${\mathbb{D}}^{e}=\\{z:|z|>1\\}$ and $X=\\{1/\overline{a_{j}}\\}$, the set of
poles of $B$.

######  Lemma 6.1.

Let $S=\\{e^{i\varphi}:\ \alpha\leqslant\varphi\leqslant\beta\\}$ be a proper
segment of $\partial{\mathbb{D}}$, $(a_{j})_{j}$ a sequence in ${\mathbb{D}}$
that satisfies the Blaschke condition and such that
$S\subset\overline{\\{a_{j},j=1,2,\ldots\\}}$. Let $B$ be the Blaschke product
with zeros $\\{a_{j}\\}$. Suppose that there exists an increasing sequence of
positive numbers $\\{c_{j}\\}$ such that $\sum_{j=1}^{\infty}\frac{1}{jc_{j}}$
is finite and that

(6.2) $\left|a_{j}-\frac{1}{\overline{a_{j}}}\right|=e^{-jc_{j}}.$

Then there exists a finely open set $E\subset{\mathbb{C}}$ that meets $S$ with
the property that $B$ extends as a finely holomorphic function over $E$.

###### Proof.

The proof goes much the same as the proof of Lemma 4.2. First observe that

$\left|\frac{1}{|a_{j}|}\frac{a_{j}-z}{1/\overline{a_{j}}-z}-1\right|=\left|\frac{1}{|a_{j}|}\frac{a_{j}-1/\overline{a_{j}}}{1/\overline{a_{j}}-z}-(1-1/|a_{j}|)\right|\leqslant\frac{1}{|a_{j}|}\frac{|a_{j}-1/\overline{a_{j}}|}{|1/\overline{a_{j}}-z|}+\frac{(1-|a_{j}|)}{|a_{j}|}.$

It follows that the product (6.1) converges uniformly on compact sets in

$E_{N}=\left(\bigcap_{j=N}^{\infty}\\{z:|1/\overline{a_{j}}-z|\geqslant|a_{j}-1/\overline{a_{j}}|/e^{-jc_{j}/2}\\}\right)\setminus\\{1/\overline{a_{1}},1/\overline{a_{2}},\ldots,1/\overline{a_{N-1}}\\}.$

Let

$F_{N}=\bigcup_{j=N}^{\infty}\\{z\in{\mathbb{C}}:|z-1/\overline{a_{j}}|\leqslant
e^{-jc_{j}/2}\\}.$

Put $J_{N}=S\cup F_{N}$ Note that $J_{N}$ is compact, its complement in
${\mathbb{C}}\cup\\{\infty\\}$ is simply connected and
$\operatorname{Cap}J_{N}\geqslant\operatorname{Cap}S=\sin((\beta-\alpha)/4)$.
As in the proof of Lemma 4.2 we can take $N$ so large that
$\operatorname{Cap}F_{N}<\operatorname{Cap}J_{N}$. Now form again
$u(z):=g_{F_{N}}-g_{J_{N}}$ and the proof continues just like the proof of
Lemma 4.2. ∎

######  Theorem 6.2.

Let $S=\\{e^{i\varphi}:\ \alpha\leqslant\varphi\leqslant\beta\\}$ be a proper
segment in $\partial{\mathbb{D}}$ and let $B$ be a Blaschke product with zero
set $\\{a_{k},b_{l}\\}$, where $\alpha\leqslant\arg a_{k}\leqslant\beta$ and
$\arg b_{k}\notin[\alpha,\beta]$. Suppose that the $a_{k}$ satisfy (6.2).

Let $U$ be a domain with $S\subset U$ and $E\subset S$ the set constructed in
Lemma 6.1. For $f$ holomorphic on $U$ put $g=fB$ on
$(U\setminus(\partial{\mathbb{D}}\cup X))\cup E$. Then

  * •

$\Gamma^{*}_{g|_{{\mathbb{D}}\cap U}}\supset\Gamma_{g|_{(U\setminus(S\cup
X))\cup E}}$

  * •

$\Gamma^{*}_{g|_{{\mathbb{D}}^{e}\cap U}}\supset\Gamma_{g|_{(U\setminus(S\cup
X))\cup E}}$

In fact both $\Gamma^{*}_{g|_{{\mathbb{D}}\cap U}}$ and
$\Gamma^{*}_{g|_{{\mathbb{D}}^{e}\cap U}}$ contain the graph of the maximal
finely analytic extension of $g$.

###### Proof.

The function $f$ is holomorphic on $U$ and $B$ is finely holomorphic on
${\mathbb{C}}\setminus(\partial{\mathbb{D}}\cup X)\cup E$. therefore $g$ is
finely holomorphic on the fine domain $(U\setminus(S\cup X))\cup E$. Theorem
2.2 now gives the result. ∎

######  Corollary 6.3.

Let $B$ be a Blaschke product as in Theorem 6.2 with the additional property
that it admits no holomorphic extension outside ${\mathbb{D}}$. Take
$f(z)=p.v.\log(z+2)$ on $|z|<2$. Then of $g=fB$ cannot be extended outside
${\mathbb{D}}$ but $\Gamma^{*}_{g|_{{\mathbb{D}}\cap U}}$ contains the graph
of the maximal finely holomorphic extension of $g$. In particular, all fibers
$\Gamma^{*}_{g|_{{\mathbb{D}}\cap U}}\cap(\\{z\\}\times{\mathbb{C}})$ will be
infinite, and most of them will be polar sets in the complex line
$\\{z\\}\times{\mathbb{C}}$.

######  Remark 6.4.

  1. (1)

In [22] Zwonek proves his version of Corollary 6.3 under the following
condition on $B$:

(6.3) $\lim_{r\to\infty}\operatorname{Cap}(\\{z\in{\mathbb{D}}:|B(z)|\leqslant
r\\})=0.$

Siciak [19] replaced this condition by the seemingly weaker condition: There
exists $r_{0}$ with $0<r_{0}<1$ such that

$\operatorname{Cap}(\\{z\in{\mathbb{D}}:|B(z)|\leqslant r_{0}\\})<1.$

In fact it is not known whether Siciak’s condition is really strictly weaker.
In Proposition 7 of [22] two conditions are given that together guarantee
(6.3). Our condition is equivalent to Zwonek’s second condition; we don’t need
his first.

  2. (2)

In both [22] and [19] the conditions on the capacity and the zeros of the
Blaschke product are on all of ${\mathbb{D}}$, whereas in Theorem 6.2 and its
corollary the condition on the zeros is only required in a sector.

  3. (3)

In [4] similar results as in Theorem 6.2 and its corollary are proved by
potential theoretic methods. The explanation by fine holomorphy is new.

######  Remark 6.5.

While it would be possible to obtain more precise information of the
pluripolar hull of the graph of the Blaschke product along the lines of Lemma
5.1 and Theorem 5.2, it is not clear how to do this in general for the
products $fB$. We will not pursue this further.

## References

  * [1] Armitage, D. H. and Gardiner, S. J. Classical potential theory, Springer Monographs in Mathematics. Springer-Verlag London, Ltd., London, 2001.
  * [2] Doob, J.L. Classical Potential Theory and Its Probabilistic Counterpart, Grundl. math. Wiss. 262, Springer, 1984.
  * [3] Edigarian, A., El Marzguioui, S. and Wiegerinck, J. The image of a finely holomorphic map is pluripolar. Ann. Polon. Math. 97 (2010), 137–149.
  * [4] Edigarian, A., Siciak, J. and Zwonek, W. Bounded holomorphic functions with multiple sheeted pluripolar hulls. Studia Math. 175 (2006), no. 3, 233–-247.
  * [5] Edigarian, A. and Wiegerinck, J. Graphs that are not complete pluripolar. Proc. Amer. Math. Soc. 131 (2003), no. 8, 2459–-2465.
  * [6] Edigarian, A. and Wiegerinck, J. The pluripolar hull of the graph of a holomorphic function with polar singularities. Indiana Univ. Math. J. 52 (2003), no. 6, 1663–-1680.
  * [7] Edigarian, A. and Wiegerinck, J. Determination of the pluripolar hull of graphs of certain holomorphic functions. Ann. Inst. Fourier (Grenoble) 54 (2004), 2085–2104.
  * [8] Edlund, T. and Jöricke, B. The pluripolar hull of a graph and fine analytic continuation. Ark. Mat. 44 (2006), no. 1, 39–-60.
  * [9] El Marzguioui, S and Wiegerinck, J. Continuity properties of finely plurisubharmonic functions and pluripolarity. Indiana Univ. Math. J. 59 (2010), no. 5, 1793–-1800.
  * [10] Fuglede, B. Finely Harmonic Functions, Springer LNM. 289, Springer, Berlin, 1972.
  * [11] Fuglede, B. Fine topology and finely holomorphic functions. 18th Scandinavian Congress of Mathematicians (Aarhus, 1980), pp. 22–38, Progr. Math. 11, Birkhäuser, Boston, Mass., 1981.
  * [12] Fuglede, B. Sur les fonctions finement holomorphes. Ann. Inst. Fourier (Grenoble) 31, (1981), no. 4, vii, 57–88.
  * [13] Fuglede, B. Finely holomorphic functions. A survey. Rev. Roumaine Math. Pures Appl. 33, (1988), no. 4, 283–-295.
  * [14] Fuglede, B., Groot, A. and Wiegerinck, J. Domains of existence for finely holomorphic functions. Potential Anal. 51 (2019), no. 3, 469–-481.
  * [15] Levenberg, N., Martin, G. and Poletsky, E. A. Analytic disks and pluripolar sets. Indiana Univ. Math. J. 41 (1992), 515–-532.
  * [16] Levenberg, N. and Poletsky, E. A. Pluripolar hulls.  Michigan Math. J. 46 (1999), no. 1, 151–-162.
  * [17] Poletsky, E. A. and Wiegerinck, J. Graphs with multiple sheeted pluripolar hulls. Ann. Polon. Math. 88 (2006), no. 2, 161–-171.
  * [18] Ransford, T. Potential Theory in the Complex Plane, London Math. Soc. Student texts 28 Cambridge University Press, 1995.
  * [19] Siciak, J. Pluripolar continuation of graphs of Blaschke products. In: Finite or infinite dimensional complex analysis and applications, 305–-317, Kyushu University Press, Fukuoka, 2005.
  * [20] Wiegerinck, J. Plurifine potential theory. Ann. Polon. Math. 106 (2012), 275–-292.
  * [21] Zeriahi, A. Ensembles pluripolaires exceptionnels pour la croissance partielle des fonctions holomorphes. Ann. Polon. Math. 50 (1989), 81–91.
  * [22] Zwonek, W. A note on pluripolar hulls of graphs of Blaschke products. Potential Anal. 22 (2005), 195–-206.


# Can we discern microlensed gravitational-wave signals from the signal of
precessing compact binary mergers?

Kyungmin Kim<EMAIL_ADDRESS>Department of Physics, Ewha Womans
University, 52 Ewhayeodae-gil, Seodaemun-gu, Seoul 03760, Republic of Korea
Korea Astronomy and Space Science Institute, 776 Daedeok-daero, Yuseong-gu,
Daejeon 34055, Republic of Korea  Anna Liu  Department of Physics, The Chinese
University of Hong Kong, Shatin, New Territories, Hong Kong

###### Abstract

Microlensed gravitational waves (GWs) are likely observable by recognizing the
signature of interference caused by
$\sim\\!\mathcal{O}(10\textrm{--}100)~{}\textrm{ms}$ time delays between
multiple lensed signals. However, the shape of the anticipated microlensed GW
signals might be confused with the modulation appearing in the waveform of GWs
from precessing compact binary mergers. Their morphological similarity may be
an obstacle to template-based searches to correctly identifying the origin of
observed GWs and it seamlessly raises a fundamental question, _can we discern
microlensed GW signals from the signal of precessing compact binary mergers?_
We discuss the feasibility of distinguishing those GWs via examining simulated
GW signals with and without the presence of noise. We find that it is
certainly possible if we compare signal-to-noise ratios (SNRs) computed with
templates of different hypotheses for a given target signal. We show that
proper parameter estimation for lensed GWs enables us to identify the targets
of interest by focusing on a half number of assumptions for the target signal
than the SNR-based test.

_Introduction._ —Like electromagnetic waves propagating in the universe,
gravitational waves (GWs) can be gravitationally lensed by massive objects
between the distant source and the observer [1, 2, 3, 4, 5, 6, 7, 8].
Consequently, for the gravitational lensing of GWs, it is possible to consider
all known types of lensing configurations governed by not only the relative
position of the source to the lens but also the size, shape, and distribution
of the lens mass.

In this Letter, we focus on the microlensing of GWs (GW microlensing,
hereafter) expected to occur with stellar objects of masses
$\lesssim\\!10^{5}M_{\odot}$ acting as the lens.111Many works (e.g., [9, 10,
11, 12, 13]) have referred microlensing to the lensing producing interference
patterns by the lens mass within $[1,10^{5}]M_{\odot}$ and we follow the
convention adopted in the literature. Meanwhile, the scale of lensing can be
further separated by the size of the Einstein radius $\theta_{E}$ of the lens,
i.e., if $\theta_{E}\\!\sim\\!\textrm{milli-arcsec}$ or $\sim\\!\textrm{micro-
arcsec}$, it corresponds to millilensing or microlensing, respectively (see
[14], for further discussion). In specific, if the time delay between multiple
lensed GWs is $\sim\\!\mathcal{O}(10\textrm{--}100)~{}\textrm{ms}$, the short
time delays yield the waveforms of those lensed GWs to interfere with each
other so that the interference pattern, also known as _beating pattern_ ,
becomes a characteristic signature of GW microlensing. [15, 12, 11, 13]
Therefore, we can anticipate that detecting microlensed GWs is achievable by
recognizing beating patterns from GW signals.

On the other hand, the precessional motion of a compact binary
system—originated by the misaligned component spins against the orbital
angular momentum of the binary—can introduce amplitude modulation and phase
shift to a GW signal not assuming precession (see Fig. 1, for example). In
particular, at first glance at the example signals in the figure, the
modulation induced by the precession (the second top panel), mainly shown in
the inspiral phase corresponding to $t\\!<\\!0~{}\textrm{s}$, looks quite
similar to the beating pattern of a microlensed GW signal (the third top
panel).

The morphological similarity between the two different types of GWs may be an
obstacle to template-based searches requiring appropriate assumptions on the
intrinsic and extrinsic parameters of the source binary system. Therefore,
their similarity seamlessly raises a fundamental question: _can we discern
microlensed GW signals from the signal of precessing compact binary mergers?_
To answer this question, we start by describing how we prepare the example
signals in Fig. 1 for the sake of this work. We will designate each signal as
a microlensed signal and a precessing signal shortly.

_Preparation of GW signals of interest._ —For the preparation of test signals
of interest, we utilize PyCBC [16, 17, 18] and choose a state-of-the-art
phenomenological waveform model, IMRPhenomXPHM [19], not only to incorporate
the effect of multipole harmonic modes of GW signal but also to prepare both
nonprecessing and precessing GW signals in the same manner with and without
the parameters for the precessing binary.

Figure 1: Example noise-free GW signals studied in this Letter. The top and
bottom panels show signals in the time domain and the frequency domain,
respectively. From the left to the right, each column shows the case of
unlensed nonprecessing, unlensed precessing, microlensed nonprecessing, and
microlensed precessing signals, respectively. The time of $h(t)$ is given with
respect to the merging time at $t=0~{}\textrm{s}$. In the plots of $|h(f)|$,
we display a counter example signal (dashed line) and the sensitivity curves
of the Advanced LIGO (aLIGO)’s design power spectral density (PSD) (dotted
dark grey) and the Einstein Telescope (ET)’s design PSD [20] (dotted light
grey) for comparison.

First, we generate unlensed signals of nonprecessing binary and precessing
binary ($h_{\textrm{N}}$ and $h_{\textrm{P}}$, respectively) with the common
and precessing parameters summarized in Table 1. Note that the spin parameters
are randomly chosen but to satisfy the spin magnitude
$s\\!=\\!(s_{x}^{2}\\!+\\!s_{y}^{2}\\!+\\!s_{z}^{2})^{1/2}\\!\leq\\!1$.

Table 1: Parameters used for the generation of test signals of interest. Case | Parameter | Value
---|---|---
Common | Mass 1, $m_{1}$ | $30M_{\odot}$
Mass 2, $m_{2}$ | $10M_{\odot}$
Luminosity distance, $d_{L}$ | $500~{}\textrm{Mpc}$
Inclination angle, $\iota$ | $80~{}\textrm{deg}$
Sampling rate | $8192~{}\textrm{Hz}$
Precessing | Spin 1, $\vec{s}_{1}\\!=\\!(s_{1x},s_{1y},s_{1z})$ | (0.76, 0.06, 0.55)
Spin 2, $\vec{s}_{2}\\!=\\!(s_{2x},s_{2y},s_{2z})$ | (-0.19, 0.74, -0.33)
Angle between total angular | $160~{}\textrm{deg}$
momentum and line of sight, $\theta_{JN}$
Microlensed | Source position, $y$ | 0.9
Log-redshifted lens mass, | $3.2$
$\log_{10}\\!(M_{lz}/M_{\odot})$

Following the choice, the effective precession spin parameter $\chi_{p}$ [21]
of the precessing binary can be calculated as $\chi_{p}\\!\simeq\\!0.76$ via
the chi_p function of PyCBC with the given component masses and the $x$\- and
$y$-components of component spins.

Next, we simulate a microlensed signal $h_{\textrm{ML}}$ via
$h_{\textrm{ML}}(f)\\!=\\!F(f)h(f)$ adopting the thin-lens approximation,
where $h(f)$ in the right-hand side denotes an unlensed signal, either
$h_{\textrm{N}}$ or $h_{\textrm{P}}$, given in the frequency domain and $F(f)$
is the amplification factor given as a function of frequency $f$. To specify
$F(f)$ enabling us to simulate a $h_{\textrm{ML}}(f)$ in a simple means, we
take the point-mass lens model in the geometrical optics limit222As discussed
in [8, 13], the sensitive frequency band of current ground-based GW detectors
corresponds to where the geometrical optics limit is valid, although treating
GW microlensing in wave optics is a general treatment for the typical lens
mass $\lesssim 10^{5}~{}M_{\odot}$ of microlensing. which makes $F(f)$ can be
written as

$F(f)=\sqrt{|\mu_{+}|}\\!-\\!i\sqrt{|\mu_{-}|}e^{2\pi if\Delta t}~{},$ (1)

where $\mu_{+}$ and $\mu_{-}$ are the magnification factor for each of the two
lensed signals and $\Delta t$ is the time delay between them: $\mu_{\pm}$ and
$\Delta t$ are given as

$\mu_{\pm}=\frac{1}{2}\pm\frac{y^{2}+2}{2y\sqrt{y^{2}+4}}~{},$ (2)

and

$\Delta
t=\frac{4GM_{lz}}{c^{3}}\left[\frac{y\sqrt{y^{2}+4}}{2}+\ln\left\\{\frac{\sqrt{y^{2}+4}+y}{\sqrt{y^{2}+4}-y}\right\\}\right]~{},$
(3)

respectively, where $y$ is a dimensionless parameter for the source position
and $M_{lz}\\!=\\!M_{l}(1+z)$ is the redshifted lens mass.333The reader can
refer [8] for more details on the lensing configuration and formulation.

We then convert the signals in the frequency domain to the time domain via the
inverse Fourier transformation. Note that we empirically choose $y=0.9$ and
$M_{lz}=10^{3.2}M_{\odot}$ in order to make an apparently similar
$h_{\textrm{ML}}(t)$ to given $h_{\textrm{P}}(t)$; the parameters give
$\mu_{+}\simeq 1.21$, $\mu_{-}\simeq-0.21$, and $\Delta t\simeq
58~{}\textrm{ms}$.

Figure 2: The same signals of Fig. 1 but injected into the noise data of the
PSD of aLIGO’s design sensitivity. For visual convenience, we plot whitened
signals for both bandpassed $s(t)$ and $|s(f)|$. We further normalize $s(t)$
with the peak amplitude at $t=0~{}\textrm{s}$. We also present the spectrogram
of each signal in the bottom panel.

Finally, regarding realistic detection scenario, we inject the time-domain
signals into the noise data $n(t)$ to simulate the strain data $s(t)$ such as
$s(t)\\!=\\!h(t)\\!+\\!n(t)$. For $n(t)$, we consider a noise model, the power
spectral density (PSD) of the Advanced LIGO (aLIGO)’s design sensitivity [22],
that also can be acquired from the pycbc.psd module of PyCBC. In addition, to
obtain the final form of $s(t)$, we employ bandpass filtering following the
conventional signal processing for the LIGO-Virgo data [23]: For our example
signals, we apply an empirically determined optimal frequency window
$[20\textrm{Hz},400\textrm{Hz}]$ for the bandpass filter. We present the
resultant $s(t)$ and $|s(f)|$ in the top and middle panels of Fig. 2,
respectively.

_Comparison between signals in noise._ —Let us discuss the feasibility of
distinguishing signals of interest by comparing the signals in given noise.
Hereafter, ‘un’, ‘up’, ‘ln’, and ‘lp’ stand for the unlensed nonprecessing,
unlensed precessing, (micro)lensed nonprecessing, and (micro)lensed precessing
signals, respectively.

We can see from Fig. 2 that the presence of noise obviously hinders
recognizing the difference between signals shown in the noise-free $h(t)$ or
$|h(f)|$. The difficulty in identifying the difference is also shown in the
spectrogram presented in the bottom panel of Fig. 2: We can find that the
brightness—representing the energy content of the signal [24]—of the ‘un’
signal (the first panel) changes like the ‘up’ signal (the second panel) as
the time and frequency evolve. We also observe a few faint vertical nodes
appear on both signals.

From the spectrograms of ‘ln’ and ‘lp’ signals (the third and fourth panels,
respectively), we can still see some brightness changes for
$t<-0.1~{}\textrm{s}$ similar to ‘un’ and ‘up’ signals. However, we can
observe two peaks, the primary peak at $t=0~{}\textrm{s}$ and the secondary
peak at $t\sim-0.06~{}\textrm{s}$, which well agree with $\Delta t\simeq
58~{}\textrm{ms}$ but not shown in the spectrograms of ‘un’ and ‘up’. Despite
this, this makes us to hardly identify whether a microlensed signal is a
precessing signal or a nonprecessing signal. On top of that, the only
recognizable differences between the third and fourth spectrograms are the
amount of brightness and apparent visibility of two peaks or the presence of a
dark vertical node at slightly different times, i.e.,
$t\\!\sim\\!-0.2~{}\textrm{s}$ for ’ln’ and at $t\\!\sim\\!-0.15~{}\textrm{s}$
for ‘lp’; these differences do not tell us about the original intrinsic
property of a microlensed signal.

Table 2: Matched-filter signal-to-noise ratios (SNR) of noisy signals corresponding to the examined pairs of template and target. We also tabulate the noise-free cases for comparison. For the computation of SNR of noise-free targets, we use the PSD of aLIGO’s design sensitivity. We find that SNRs of homogeneous pairs are advantageous for obtaining higher SNRs than heterogeneous pairs as expected. We also see SNRs of precessing targets are slightly higher than nonprecessing targets. Noise | Homogeneous BBH pairs (template-target) | Heterogeneous BBH pairs (template-target)
---|---|---
Nonprecessing target | Precessing target | Nonprecessing target | Precessing target
un–un | un–ln | ln–ln | up–up | up–lp | lp–lp | up–un | up–ln | lp–ln | un–up | un–lp | ln–lp
Free | $24.5$ | $27.2$ | $29.4$ | $35.6$ | $39.5$ | $42.7$ | $8.5$ | $10.9$ | $11.2$ | $12.3$ | $14.0$ | $16.3$
aLIGO | $24.3$ | $26.4$ | $28.4$ | $34.0$ | $36.4$ | $40.1$ | $9.6$ | $12.2$ | $4.8$ | $13.0$ | $14.1$ | $4.9$

To quantitatively examine the distinguishability, we compute matched-filter
signal-to-noise ratio (SNR) and summarize resulting SNRs in Table 2. For this
examination, we configure multiple template-target pairs with the four-types
signals studied thus far: We take whitened noise-free signals as templates and
whitened noisy signals as targets. We classify the pairs into two categories,
homogeneous BBH pairs and heterogeneous BBH pairs, defining homogeneous or
heterogenous to represent whether the same nonprecessing/precessing binary
system is supposed for both template and target or not: Following the
definition, ‘xn-xn’ and ‘xp-xp’ correspond to homogeneous pairs and ‘xn-xp’
and ‘xp-xn’ correspond to heterogeneous pairs regardless of ‘x’, where ‘x’ can
be either ‘u’ or ‘l’. We then collect each pair of the two categories into two
additional subcategories of nonprecessing or precessing target signals.

From the SNRs of homogeneous BBH pairs, we see that SNRs for precessing
targets are commonly $\sim\\!40\%$ higher than corresponding SNRs for
nonprecessing targets. We observe that testing mirolensed templates for
microlensed targets shows the highest SNR among the pairs of each subcategory.
We also find that unlensed templates are applicable even for microlensed
targets because SNRs of ‘un-ln’ and ‘up-lp’ are higher than SNRs of unlensed
targets.

Obtaining higher SNR for precessing target is still shown from the SNRs of
heterogeneous BBH pairs. However, when we compare SNRs of heterogeneous BBH
pairs to corresponding homogeneous pairs for the same target, we can see that
using heterogeneous templates to opposite targets gives much lower SNRs. In
particular, as shown from ‘lp-ln’ and ‘ln-lp’ pairs in aLIGO noise, computing
SNR with heterogeneous microlensed template is obviously disadvantageous for
finding microlensed targets because it is much lower than the detection
criterion $\geq 8$ for the matched-filter SNR of a single detector [25].

The result of this SNR-based test provides an answer for our question: It is
feasible to distinguish a microlensed nonprecessing signal from an unlensed
precessing signal if an adequate homogeneous template is tested and higher SNR
is obtained than other templates. For example, for ‘ln’ target, we obtain SNRs
as $28.4$, $26.4$, and $12.2$ with ‘ln’, ‘un’, and ‘up’ templates,
respectively. Similarly, we get SNRs for ‘up’ target as $34.0$ and $13.0$ with
‘up’ and ‘un’ templates, respectively. Moreover, we find discriminating a
microlensed nonprecessing target from a microlensed precessing target is also
possible when we compare SNRs of ‘ln’ and ‘lp’ targets, e.g., ‘ln-ln’ ($28.4$)
and ‘lp-ln’ ($4.8$) or ‘lp-lp’ ($40.1$) and ‘ln-lp’ ($4.9$).

In practice, the analysis methods using templates (e.g., PyCBC and GstLAL
[26]) are the _de facto_ standard for searching GWs from compact binary
mergers. Therefore, our results suggest that utilizing the standard methods
with an adequate template bank consisting of microlensed waveforms including
precession will be a promising approach for determining if a GW event is
microlensed or not.

On the other hand, the results also imply that, to distinguish the target of
interest, we have to repeat computing SNR multiple times regarding all
possible hypotheses for the nature of a single target. However, since the
template-based searches require hundreds thousands of template waveforms,
e.g., $\sim\\!\textrm{250,000}$ templates for GW150914 [27], demanding such
repeated computation will inevitably increase the total computation time of
our test at least $\mathcal{N}$ times more, where $\mathcal{N}$ is the number
of hypotheses. For example, if we want to identify whether a target is an
unlensed/microlensed nonprecessing/precessing signal, we have to test the four
hypotheses, ‘un’, ‘up’, ‘ln’, and ‘lp’ and repeat the same analysis of finding
a template giving the highest SNR for each hypothesis. Then, the whole
procedure for finally picking up the most promising template from four of them
will take at least $4\times$ more computation time in total compared to the
typical search for a single BBH signal.

_Parameter estimation for microlensed signals._ —We conduct parameter
estimation (PE) to demonstrate whether a proper PE can help us to disclose the
origin of a given target. In particular, we focus on ‘ln’ and ‘lp’ signals
because the apparent similarity—showing the modulation in waveform and double
peaks around the merging time—between them makes us hardly identify their
intrinsic origin.

Figure 3: Posterior probability density functions (PDFs) of the parameter (a)
$K$, (b) $d_{2}^{\mathrm{eff}}$, (c) $\Delta t$, and (d) $\chi_{p}$. The
precessing and nonprecessing signals lead to consistent results with the
injected values (orange lines). Note that, for $\chi_{\textrm{P}}$, we present
the posterior PDF of precessing signal only because $\chi_{p}\\!=\\!0$ for
nonprecessing signals [21].

The PE method [14] implemented in this work is based on a phenomenological
lensed waveform model with specifically rewriting $F(f)$ in a parametric form
such as

$F(f,d^{\textrm{eff}}_{j},t_{j},n_{j})=\sum_{j=1}^{K}\frac{d_{L}}{d_{j}^{\textrm{eff}}}e^{2\pi
ft_{j}-i\pi n_{j}}~{},$ (4)

where $K$ refers to the number of lensed signals and
$d^{\textrm{eff}}_{j}\equiv d_{L}/\sqrt{\mu_{\textrm{rel},j}}$ is the
effective luminosity distance of $j^{\textrm{th}}$ lensed signal for the given
true luminosity distance to the source $d_{L}$.
$\mu_{\textrm{rel},j}\\!\equiv\\!|\mu_{j}/\mu_{1}|$ and $t_{j}$ are
respectively defined as the relative magnification and the relative time delay
of $j^{\textrm{th}}$ signal with respect to the first lensed signal. $n_{j}$
denotes the Morse phase index [28] of $j^{\textrm{th}}$ signal. The advantage
of using the parameterized $F(f,d^{\textrm{eff}}_{j},t_{j},n_{j})$ is that we
can determine $F$ for an arbitrary number of individual lensed signal with the
relative lensing observables without specifying the observables such as $y$
and $M_{lz}$. We can then write the $h_{\textrm{ML}}$ as the superposition of
multiple lensed GW signals such as

$h_{\textrm{ML}}(f)=\left(\sum_{j=1}^{K}\frac{d_{L}}{d^{\mathrm{eff}}_{j}}e^{2\pi
ift_{j}-i\pi n_{j}}\right)h(f)~{}.$ (5)

Consequently, depending on the result of inferred $K$, we can determine
whether the analyzed GW signal is multiply lensed or not. Note that
$K\\!=\\!2$ for the case of a point-mass lens, so that $j\\!=\\!1,2$ leading
$n_{j}$ becomes $n_{1}\\!=\\!0$ and $n_{2}\\!=\\!0.5$ for the first and second
lensed signals, respectively.

For this PE, we prepare another two target strain data—injecting the same
$h_{\textrm{ln}}$ and $h_{\textrm{lp}}$ into the Gaussian noise444The noise
data used for generating noisy strain data of the previous section is also
known as _zero-noise_ data (see [29, 30], for example). of the aLIGO’s design
sensitivity PSD using a Bayesian inference library, bilby [31, 32]—and infer
the lensing parameters together with the source parameters. In computing the
likelihood, we adopt the nested sampling technique with pymultinest sampler
[33, 34] and use two waveform models, IMRPhenomXHM [35] and IMRPhenomXPHM, for
the template of nonprecessing and precessing targets, respectively. In Fig. 3,
we present resulting probability density functions (PDFs) of the posteriors of
4 selected parameters, $K$, $d^{\textrm{eff}}_{2}$, $\Delta t$, and
$\chi_{p}$, where $\Delta t\\!\equiv\\!t_{2}$.

Figure 4: The same signals shown in Fig. 1 but injected into the noise data of
the PSD of ET’s design sensitivity.

We find that our PE correctly recovers the number of lensed signals $K$ as $2$
for both ‘ln’ and ‘lp’ signals (see Fig. 3(a)). From this result, we confirm
that precession does not affect identifying microlensed events. We also see
that, for both precessing and nonprecessing cases, the resulting posteriors of
$d^{\textrm{eff}}_{2}$ and $\Delta t$ include the injected values, i.e.,
$d^{\textrm{eff}}_{2}\\!\simeq\\!1200~{}\textrm{Mpc}$ and $\Delta t\simeq
58~{}\textrm{ms}$ (see Fig. 3(b) and Fig. 3(c), respectively) within their
distributions. On top of that, since we can recover $\chi_{p}$ from the
microlensed precessing signal in agreement with the injected value $\sim 0.76$
(Fig. 3(d)), our PE helps us to identify a precessing signal correctly.

When we compare the precessing and nonprecessing cases of
$d^{\textrm{eff}}_{2}$ and $\Delta t$, we can observe the range of posterior
distribution for precessing signal is narrower than that for nonprecessing
one. For this observation, we presume the performance of our PE is sensitive
to SNR because it turned out that the SNRs of those two signals calculated
during PE are $\textrm{SNR}_{\textrm{lp-lp}}=30.5>\textrm{SNR}_{\textrm{ln-
ln}}=15.6$.

We conclude that the PE-based analysis can inform us about whether a target
signal is microlensed or not (via $K$) and whether the correctly identified
microlensed signals is from a nonprecessing or a precessing binary (via
$\chi_{p}$). Moreover, for the identification of the targets of interest, we
find the PE-based analysis is more beneficial than the SNR-based test in
practice because, by recovering $K$, we can focus on two hypotheses related to
the precession only without further estimation on the other two hypotheses
associated with the microlensing.

_Discussions._ —In this Letter, we have supposed the GW microlensing occurs
with a point-like lens mass of $M_{lz}\\!=\\!10^{3.2}M_{\odot}$ in the
geometrical optics limit for simplicity. However, if we consider (i) extended
lens mass, e.g., singular isothermal sphere, in quasi-geometrical optics
approximation [36] or (ii) more complex microlensing configurations in wave
optics, e.g., studied in [15, 12], we expect more than two lensed signals
which may introduce more complex beating patterns on GW signals. Therefore,
debating the distinguishability in more realistic microlensing scenarios with
such treatments/configurations will be worth to devote.

On the other hand, when we look at the same signals but injected into the
noise model of ET’s design sensitivity [20] (see Fig. 4), the identification
becomes obviously much easier than the case of aLIGO’s noise. Indeed, we can
see quite similar shaped signals from the whitened bandpassed $s(t)$s compared
to the shape seen in the corresponding noise-free $h(t)$s in Fig. 1.
Additionally, from the spectrogram, we can observe a similar pattern in the
signals corresponding to the signals in Fig. 2 but much clearer ones, although
distinguishing nonprecessing and precessing signals from the microlensed cases
is still quite hard because both signals are amplified and have double peaks
near the merger time. Despite this, if a proper PE-based analysis will be
accompanied with the identification of microlensed event as done in this work,
it will be definitely possible to recognize whether the microlensed event is
from a precessing or nonprecessing binary.

In principle, one could do more comprehensive studies such as a systematic
study for various physical and lensing parameters as done in [37]. However,
our results have demonstrated already that the precessional effect is not
degenerate with beating patterns and have shown that the signals of interest
are distinguishable with both SNR- and PE-based tests. For the result, we
presume that the methods adopted in our tests are good enough at capturing the
difference between the intrinsic effect introducing the modulation in
precessing signals and the extrinsic effect resulting in the interference of
multiple lensed signals. Nonetheless, finding a decisive criterion for the
discrimination will be helpful for future observations on GWs containing
various intrinsic/extrinsic natures. For example, it is also known that
another intrinsic degree of freedom, the eccentricity, of a compact binary
merger can introduce modulation in GW waveforms (see Figure 1 of [38], for
example) and it makes us confused in identifying whether the modulation is
caused by precessing or eccentric binaries [39, 40, 41]. Thus, exploring the
distinguishability between the beating pattern of microlensed GWs and other
modulations originated by diverse intrinsic properties of compact binary
systems will be another interesting venue.

_Acknowledgements._ —The authors thank Otto A. Hannuksela, Tjonnie G. F. Li,
and Eungwang Seo for fruitful discussions and comments. This work is supported
by the National Research Foundation of Korea (NRF) grant funded by the
Ministry of Science and ICT of the Korea Government (NRF-2020R1C1C1005863).
K.K. also acknowledges that the computational work reported in this Letter was
partly performed on the KASI Science Cloud platform supported by Korea
Astronomy and Space Science Institute.

## References

  * Ohanian [1974] H. C. Ohanian, On the focusing of gravitational radiation, Int. J. Theor. Phys. 9, 425 (1974).
  * Bliokh and Minakov [1975] P. V. Bliokh and A. A. Minakov, Diffraction of light and lens effect of the stellar gravitation field, Ap&SS 34, L7 (1975).
  * Bontz and Haugan [1981] R. J. Bontz and M. P. Haugan, A diffraction limit on the gravitational lens effect, Ap&SS 78, 199 (1981).
  * Thorne [1983] K. S. Thorne, The theory of gravitational radiation: An introductory review, in  _Gravitational Radiation_ , edited by N. Deruelle and T. Piran (Amsterdam: North Holland, 1983).
  * Deguchi and Watson [1986] S. Deguchi and W. D. Watson, Diffraction in gravitational lensing for compact objects of low mass, Astrophys. J. 307, 30 (1986).
  * Schneider  _et al._ [1992] P. Schneider, J. Ehlers, and E. E. Falco,  _Gravitational Lenses_ (Springer, New York, 1992) p. 112.
  * Nakamura and Deguchi [1999] T. T. Nakamura and S. Deguchi, Wave Optics in Gravitational Lensing, Prog. Theor. Phys. Suppl. 133, 137 (1999).
  * Takahashi and Nakamura [2003] R. Takahashi and T. Nakamura, Wave effects in gravitational lensing of gravitational waves from chirping binaries, Astrophys. J. 595, 1039 (2003), arXiv:astro-ph/0305055 .
  * Hannuksela  _et al._ [2019] O. Hannuksela, K. Haris, K. Ng, S. Kumar, A. Mehta, D. Keitel, T. Li, and P. Ajith, Search for gravitational lensing signatures in LIGO-Virgo binary black hole events, Astrophys. J. Lett. 874, L2 (2019), arXiv:1901.02674 [gr-qc] .
  * Abbott  _et al._ [2021] R. Abbott _et al._ (LIGO Scientific, VIRGO), Search for Lensing Signatures in the Gravitational-Wave Observations from the First Half of LIGO–Virgo’s Third Observing Run, Astrophys. J. 923, 14 (2021), arXiv:2105.06384 [gr-qc] .
  * Kim  _et al._ [2021] K. Kim, J. Lee, R. S. H. Yuen, O. A. Hannuksela, and T. G. F. Li, Identification of Lensed Gravitational Waves with Deep Learning, Astrophys. J. 915, 119 (2021), arXiv:2010.12093 [gr-qc] .
  * Seo  _et al._ [2022] E. Seo, O. A. Hannuksela, and T. G. F. Li, Improving Detection of Gravitational-wave Microlensing Using Repeated Signals Induced by Strong Lensing, Astrophys. J. 932, 50 (2022).
  * Kim  _et al._ [2022] K. Kim, J. Lee, O. A. Hannuksela, and T. G. F. Li, Deep Learning–based Search for Microlensing Signature from Binary Black Hole Events in GWTC-1 and -2, Astrophys. J. 938, 157 (2022), arXiv:2206.08234 [gr-qc] .
  * Liu  _et al._ [2023] A. Liu, C.-F. Wong, S. Leong, A. More, O. Hannuksela, and T. Li, A phenomenological description of gravitational-wave millilensing (2023), LIGO-P2200365.
  * Diego  _et al._ [2019] J. Diego, O. Hannuksela, P. Kelly, T. Broadhurst, K. Kim, T. Li, G. Smoot, and G. Pagano, Observational signatures of microlensing in gravitational waves at LIGO/Virgo frequencies, Astron. Astrophys. 627, A130 (2019), arXiv:1903.04513 [astro-ph.CO] .
  * Dal Canton  _et al._ [2014] T. Dal Canton _et al._ , Implementing a search for aligned-spin neutron star-black hole systems with advanced ground based gravitational wave detectors, Phys. Rev. D 90, 082004 (2014), arXiv:1405.6731 [gr-qc] .
  * Usman  _et al._ [2016] S. A. Usman _et al._ , The PyCBC search for gravitational waves from compact binary coalescence, Class. Quant. Grav. 33, 215004 (2016), arXiv:1508.02357 [gr-qc] .
  * Nitz  _et al._ [2020] A. Nitz, I. Harry, D. Brown, _et al._ , gwastro/pycbc: Pycbc release v1.16.11 (2020).
  * Pratten  _et al._ [2021] G. Pratten _et al._ , Computationally efficient models for the dominant and subdominant harmonic modes of precessing binary black holes, Phys. Rev. D 103, 104056 (2021), arXiv:2004.06503 [gr-qc] .
  * Hild  _et al._ [2011] S. Hild _et al._ , Sensitivity Studies for Third-Generation Gravitational Wave Observatories, Class. Quant. Grav. 28, 094013 (2011), arXiv:1012.0908 [gr-qc] .
  * Schmidt  _et al._ [2015] P. Schmidt, F. Ohme, and M. Hannam, Towards models of gravitational waveforms from generic binaries II: Modelling precession effects with a single effective precession parameter, Phys. Rev. D 91, 024043 (2015), arXiv:1408.1810 [gr-qc] .
  * Barsotti  _et al._ [2018] L. Barsotti, S. Gras, M. Evans, and P. Fritschel,  _Advanced LIGO anticipated sensitivity curves_ , Tech. Rep. LIGO-T1800044 (LIGO Scientific Collaboration, 2018).
  * Abbott  _et al._ [2020] B. P. Abbott _et al._ (LIGO Scientific, Virgo), A guide to LIGO–Virgo detector noise and extraction of transient gravitational-wave signals, Class. Quant. Grav. 37, 055002 (2020), arXiv:1908.11170 [gr-qc] .
  * Chatterji  _et al._ [2004] S. Chatterji, L. Blackburn, G. Martin, and E. Katsavounidis, Multiresolution techniques for the detection of gravitational-wave bursts, Class. Quant. Grav. 21, S1809 (2004), arXiv:gr-qc/0412119 .
  * Abbott  _et al._ [2018] B. P. Abbott _et al._ (KAGRA, LIGO Scientific, Virgo, VIRGO), Prospects for observing and localizing gravitational-wave transients with Advanced LIGO, Advanced Virgo and KAGRA, Living Rev. Rel. 21, 3 (2018), arXiv:1304.0670 [gr-qc] .
  * Messick  _et al._ [2017] C. Messick, K. Blackburn, P. Brady, P. Brockill, K. Cannon, R. Cariou, S. Caudill, S. J. Chamberlin, J. D. E. Creighton, R. Everett, C. Hanna, D. Keppel, R. N. Lang, T. G. F. Li, D. Meacher, A. Nielsen, C. Pankow, S. Privitera, H. Qi, S. Sachdev, L. Sadeghian, L. Singer, E. G. Thomas, L. Wade, M. Wade, A. Weinstein, and K. Wiesner, Analysis Framework for the Prompt Discovery of Compact Binary Mergers in Gravitational-wave Data, Phys. Rev. D 95, 042001 (2017), arXiv:1604.04324 [astro-ph.IM] .
  * Abbott  _et al._ [2016] B. P. Abbott _et al._ (LIGO Scientific, Virgo), GW150914: First results from the search for binary black hole coalescence with Advanced LIGO, Phys. Rev. D 93, 122003 (2016), arXiv:1602.03839 [gr-qc] .
  * Janquart  _et al._ [2021] J. Janquart, O. A. Hannuksela, K. Haris, and C. Van Den Broeck, A fast and precise methodology to search for and analyse strongly lensed gravitational-wave events, Mon. Not. Roy. Astron. Soc. 506, 5430 (2021), arXiv:2105.04536 [gr-qc] .
  * Rodriguez  _et al._ [2014] C. L. Rodriguez, B. Farr, V. Raymond, W. M. Farr, T. B. Littenberg, D. Fazi, and V. Kalogera, Basic Parameter Estimation of Binary Neutron Star Systems by the Advanced LIGO/Virgo Network, Astrophys. J. 784, 119 (2014), arXiv:1309.3273 [astro-ph.HE] .
  * Favata  _et al._ [2022] M. Favata, C. Kim, K. G. Arun, J. Kim, and H. W. Lee, Constraining the orbital eccentricity of inspiralling compact binary systems with Advanced LIGO, Phys. Rev. D 105, 023003 (2022), arXiv:2108.05861 [gr-qc] .
  * Ashton  _et al._ [2019] G. Ashton _et al._ , BILBY: A user-friendly Bayesian inference library for gravitational-wave astronomy, Astrophys. J. Suppl. 241, 27 (2019), arXiv:1811.02042 [astro-ph.IM] .
  * Romero-Shaw  _et al._ [2020a] I. M. Romero-Shaw _et al._ , Bayesian inference for compact binary coalescences with bilby: validation and application to the first LIGO–Virgo gravitational-wave transient catalogue, Mon. Not. Roy. Astron. Soc. 499, 3295 (2020a), arXiv:2006.00714 [astro-ph.IM] .
  * Skilling [2006] J. Skilling, Nested sampling for general Bayesian computation, Bayesian Analysis 1, 833 (2006).
  * Buchner [2016] J. Buchner, PyMultiNest: Python interface for MultiNest, Astrophysics Source Code Library, record ascl:1606.005 (2016), ascl:1606.005 .
  * García-Quirós  _et al._ [2020] C. García-Quirós, M. Colleoni, S. Husa, H. Estellés, G. Pratten, A. Ramos-Buades, M. Mateu-Lucena, and R. Jaume, Multimode frequency-domain model for the gravitational wave signal from nonprecessing black-hole binaries, Phys. Rev. D 102, 064002 (2020), arXiv:2001.10914 [gr-qc] .
  * Takahashi [2004] R. Takahashi, Quasigeometrical optics approximation in gravitational lensing, Astron. Astrophys. 423, 787 (2004), arXiv:astro-ph/0402165 .
  * Bondarescu  _et al._ [2022] R. Bondarescu, H. Ubach, O. Bulashenko, and A. P. Lundgren, Compact Binaries through a Lens: Silent vs. Detectable Microlensing for the LIGO-Virgo-KAGRA Gravitational Wave Observatories, (2022), arXiv:2211.13604 [gr-qc] .
  * Abbott  _et al._ [2019] B. P. Abbott _et al._ (LIGO Scientific, Virgo), Search for Eccentric Binary Black Hole Mergers with Advanced LIGO and Advanced Virgo during their First and Second Observing Runs, Astrophys. J. 883, 149 (2019), arXiv:1907.09384 [astro-ph.HE] .
  * Romero-Shaw  _et al._ [2020b] I. M. Romero-Shaw, P. D. Lasky, E. Thrane, and J. C. Bustillo, GW190521: orbital eccentricity and signatures of dynamical formation in a binary black hole merger signal, Astrophys. J. Lett. 903, L5 (2020b), arXiv:2009.04771 [astro-ph.HE] .
  * Romero-Shaw  _et al._ [2021] I. M. Romero-Shaw, P. D. Lasky, and E. Thrane, Signs of Eccentricity in Two Gravitational-wave Signals May Indicate a Subpopulation of Dynamically Assembled Binary Black Holes, Astrophys. J. Lett. 921, L31 (2021), arXiv:2108.01284 [astro-ph.HE] .
  * Romero-Shaw  _et al._ [2022] I. M. Romero-Shaw, P. D. Lasky, and E. Thrane, Four eccentric mergers increase the evidence that LIGO–Virgo–KAGRA’s binary black holes form dynamically, (2022), arXiv:2206.14695 [astro-ph.HE] .


# Inertial Proximal Deep Learning Alternating Minimization for Efficient
Neutral Network Training

###### Abstract

In recent years, the Deep Learning Alternating Minimization (DLAM), which is
actually the alternating minimization applied to the penalty form of the deep
neutral networks training, has been developed as an alternative algorithm to
overcome several drawbacks of Stochastic Gradient Descent (SGD) algorithms.
This work develops an improved DLAM by the well-known inertial technique,
namely iPDLAM, which predicts a point by linearization of current and last
iterates. To obtain further training speed, we apply a warm-up technique to
the penalty parameter, that is, starting with a small initial one and
increasing it in the iterations. Numerical results on real-world datasets are
reported to demonstrate the efficiency of our proposed algorithm.

Index Terms—  Nonconvex alternating minimization, Penalty, Inertial method,
Network training.

##  1 Introduction

The deep neural network has achieved great success in computer vision and
machine learning. Mathematically, training a $L$-layer neural network can be
formulated as:

$\small\min_{{\bf W}_{1},{\bf W}_{2},\ldots,{\bf W}_{L}}\\{\mathcal{L}({\bf
y},\sigma_{L}({\bf W}_{L}...\sigma_{1}({\bf W}_{1}{\bf
a}_{0})))+\sum_{l=1}^{L}R_{l}({\bf W}_{l})\\},$

where ${\bf a}_{0}$ denotes the training sample and ${\bf y}$ denotes the
labelled set, $R_{l}$ is the regularization in the $l$-th layer, and
$\sigma_{l}$ is the $l$-th layer’s activation. The absence of $R_{l}$ yields
the fully connected networks. The main workhorse of the deep neutral network
training task is the SGD [1] and its variants. The core part of SGD for
training neural networks is to compute the gradient with respect to ${\bf
W}_{l}$, i.e., the backpropagation [2]. For simplicity of presentation, we
assume $\sigma_{l}$ is a $\mathbb{R}\mapsto\mathbb{R}$ mapping. According to
the chain rule, it follows: $\frac{\partial\mathcal{L}}{\partial{\bf
W}_{1}}=\frac{\partial\mathcal{L}}{\partial\sigma_{L}}\frac{\partial\sigma_{L}}{\partial\sigma_{L-1}}\ldots\frac{\partial\sigma_{2}}{\partial\sigma_{1}}\frac{\partial\sigma_{1}}{\partial{\bf
W}_{1}}.$ If $|\frac{\partial\sigma_{l}}{\partial\sigma_{l-1}}|<1$, the
gradient decays fast or even vanishes when $L$ is large. This phenomenon,
which has been mentioned in [3], hurts the speed and performance of SGD in the
deep layer case. On the other hand, the convergence of stochastic training
method is based on the Lipschitz continuous assumption on the gradient, which
fails to hold for various applications. To overcome these drawbacks, papers
[4, 5, 6, 7] propose gradient free methods by the Alternating Direction
Methods of Multipliers (ADMM) or Alternating Minimization. The core idea of
this method is the decomposition of the training task into a sequence of
substeps which are just related to one-layer activations. Due to that each
substep can always find its global minimizer, the gradient free method can
achieve notable speedups [4, 5, 6]. Another advantage of the gradient free is
parallelism due to that ADMM and AM have natural conveniences to be
implemented concurrently. Besides the acceleration and parallelism, ADMM and
ADM also enjoy the advantage of mild theoretical guarantees to converge
compared with SGD: The theory of SGD is heavily dependent on the global
smoothness of $\mathcal{L}({\bf y},\sigma_{L}({\bf W}_{L}...\sigma_{1}({\bf
W}_{1}{\bf a}_{0})))$ with respect to ${\bf W}_{1},{\bf W}_{2},\ldots,{\bf
W}_{L}$, which usually fails to hold. This paper studies the alternating
minimization based method, namely Deep Learning Alternating Minimization
(DLAM) and aims to develop an improved version of DLAM.

The training task can be reformulated as a nonlinear constrained optimization
problem. DLAM is actually the AM method applied to its penalty. In [8], the
authors propose a new framework based on using convex functions to appropriate
a non-decreasing activation. The AM method is the state-of-art solver for the
convex optimization problem. In paper [9], the authors extend the DLAM to the
online learning with co-activation memory technique. By representing the
activation function as a proximal operator form, [10] proposed a new penalty
framework, which can also be minimized by AM. With rewriting equivalent
biconvex constraints to activation functions, [11] proposes AM for Fenchel
lifted networks. With a higher dimensional space to lift the ReLU function,
paper [6] develops a smooth multi-convex formulation with AM. In [12], the
authors develop the AM methodology for the nested neural network.

The contribution of this paper can be concluded as follows: 1) A novel
algorithm. We develop a new algorithm for deep neural networks training. In
each iteration, the initial technique is employed to predict an auxiliary
point with current and last iterate. For acceleration, we use a small penalty
parameter in the beginning iterations and then increase it to a larger one.
The DLAM can be regarded as a special case of this scheme. 2) Sound results.
Various proximal operaters are widely used in statistical learning comunity
[13], which bring various applications problems potentially could be solved by
the proposed algorithm in this work. We present the numerical experiments to
demonstrate the performance of our algorithm. The convergence is verified, and
comparisons with latest classical solvers are presented.

##  2 Problem Formulation and Algorithms

Notation: Given an integer $N>0$, $[N]:=\\{1,2,\ldots,N\\}$. Let
$(a^{k})_{k\geq 1}$ and $(b^{k})_{k\geq 1}$ be positive, $a^{k}=o(b^{k})$
means $\lim_{k}\frac{a^{k}}{b^{k}}=0$. We denote that ${\bf z}^{k}:=({\bf
z}_{1}^{k},\ldots,{\bf z}_{l}^{k},\ldots,{\bf z}_{L}^{k})$ and $[{\bf
z}^{k};{\bf z}_{l}]:=({\bf z}_{1}^{k+1},\ldots,{\bf z}_{l-1}^{k+1},{\bf
z}_{l},{\bf z}_{l+1}^{k},\ldots,{\bf z}_{L}^{k})$. Similar notations are
defined for ${\bf W}^{k}$ and ${\bf a}^{k}$. $\mathcal{L}(\cdot)$ represents
the loss function and $\|\cdot\|$ is the L2 norm. For a map
$\sigma(\cdot):\mathbb{R}^{d}\mapsto\mathbb{R}^{d}$ and vectors ${\bf w},{\bf
v}\in\mathbb{R}^{d}$, we denote $\mathcal{S}^{\sigma}({\bf w},{\bf
v},r):=\mathop{\mathrm{argmin}}_{{\bf x}\in\mathbb{R}^{d}}\\{\|\sigma({\bf
x})-{\bf w}\|^{2}+r\|{\bf x}-{\bf v}\|^{2}\\},$ where $r>0$ is a proxmial
parameter. If $\sigma({\bf x})={\bf x}$, then $\mathcal{S}^{\sigma}({\bf
w},{\bf v},r)=\frac{{\bf w}+r{\bf v}}{1+r}$; If $\sigma(\cdot)$ is set to be
the ReLU function [14], $[\mathcal{S}^{\sigma}({\bf w},{\bf v},r)]_{i}$ is one
of these three items $\\{[\frac{{\bf w}+r{\bf v}}{1+r}]_{i},[{\bf
v}]_{i},\min\\{{\bf v}_{i},0\\}\\}$ minimizing $|\sigma(\cdot)-{\bf
w}_{i}|^{2}+r|\cdot-{\bf v}_{i}|^{2}$ [15]. For loss function with
regularization term, we denote an operator as $\mathcal{P}^{\mathcal{L}}({\bf
w},{\bf v},r):=\mathop{\mathrm{argmin}}_{{\bf
x}\in\mathbb{R}^{d}}\\{\mathcal{L}({\bf w},{\bf x})+\frac{r}{2}\|{\bf x}-{\bf
v}\|^{2}\\}.$ If $\mathcal{L}({\bf w},{\bf x})$ is set to be
$\frac{1}{2}\|{\bf x}-{\bf w}\|^{2}$, then $\mathcal{P}^{\mathcal{L}}({\bf
w},{\bf v},r)=\frac{{\bf w}+r{\bf v}}{1+r}$; If $\mathcal{L}({\bf w},{\bf x})$
is set to be hinge loss $\max(0,1-w\cdot x)$, then
$\mathcal{P}^{\mathcal{L}}({\bf w},{\bf v},r)$ is one of these three items
$\\{v_{i},w_{i}^{-1},v_{i}+w_{i}^{-1}\\}$. More proximal operators are widely
used in statistical learning community [13].

Penalty formulation: We reformulate the $L$-layer training task as linearly
constrained optimization problem by introducing $({\bf a}_{l})_{l\in[L-1]}$
and $({\bf W}_{l})_{l\in[L]}$:

$\displaystyle\min_{{\bf z},{\bf W},{\bf a}}\\{\mathcal{L}({\bf y},{\bf
z}_{L})+\sum_{l=1}^{L}R_{l}({\bf W}_{l})\\},$ (1) s.t. $\displaystyle{\bf
z}_{l}={\bf W}_{l}{\bf a}_{l-1},~{}l\in[L];~{}{\bf a}_{l}=\sigma_{l}({\bf
z}_{l}),~{}l\in[L-1].$

If $L=1$, we then need to minimize a function with linear constraints, which
can be efficiently solved by ADMM. When $L>1$, Problem (1) is a nonlinear
constrained problem, which is difficult to solve directly. Thus, people
consider its penalty problem. Given a very large penalty parameter
$\bar{\rho}>0$, we aim to solve a reformulated problem:

$\displaystyle\min_{{\bf z},{\bf W},{\bf a}}\\{\Phi_{\bar{\rho}}({\bf z},{\bf
W},{\bf a}):=\mathcal{L}({\bf y},{\bf z}_{L})+\sum_{l=1}^{L}R_{l}({\bf
W}_{l})$ (2) $\displaystyle\quad+\frac{\bar{\rho}}{2}(\sum_{l=1}^{L}\|{\bf
z}_{l}-{\bf W}_{l}{\bf a}_{l-1}\|^{2}+\sum_{l=1}^{L-1}\|{\bf
a}_{l}-\sigma_{l}({\bf z}_{l})\|^{2})\\}.$

An extreme case is setting $\bar{\rho}=+\infty$, in which (2) is identical to
(1). Actually, even for linearly constrained nonconvex minimization, penalty
method is also a good choice due to that it enjoys much more mild convergent
assumption and easily-set parameters than the nonconvex ADMM [16]. In the
numerical experiments, we update $\bar{\rho}$ from iteration to iteration, and
$\rho_{k+1}$ is set to be $1.1\rho_{k}$ at the end of each iteration.

It should be noted that the formulation in this work is different from Zeng’s
work [15], there is no acitvation $a_{L}=\sigma_{L}(z_{L})$ before the last
layer, and further analysis is based on this formulation.

Inertial methods: The DLAM is actually the alternating minimization applied to
(2). In the nonconvex community, the inertial technique [17] (also called as
heavy-ball or momentum) is widely used and proved to be algorithmically
efficient [18, 19, 20, 21]. Besides acceleration and good practical
performance for nonconvex problems, the advantage of inertial technique is
illustrated by weaker conditions avoiding saddle points [22]. The procedure of
inertial method is quite simple, it uses linear combination of current and
last point for next iteration. For example, the gradient descent minimizing a
smooth function $f$ employs the inertial term ${\bf\hat{x}}^{k}={\bf
x}^{k}+\alpha({\bf x}^{k}-{\bf x}^{k-1})$ ($\alpha\geq 0$) as ${\bf
x}^{k+1}={\bf x}^{k}-\nabla f({\bf\hat{x}}^{k})$.

Algorithm: We employ the alternating minimization method.We use the inertial
technique for ${\bf z}^{k}$, ${\bf W}^{k}$ and ${\bf a}^{k}$,

$\small\left\\{\begin{array}[]{lll}\hat{{\bf z}}^{k}&=&{\bf z}^{k}+\alpha({\bf
z}^{k}-{\bf z}^{k-1})\\\ \hat{{\bf W}}^{k}&=&{\bf W}^{k}+\beta({\bf
W}^{k}-{\bf W}^{k-1})\\\ \hat{{\bf a}}^{k}&=&{\bf a}^{k}+\gamma({\bf
a}^{k}-{\bf a}^{k-1})\\\ {\bf
z}_{l}^{k+1}&\in&\displaystyle\mathop{\mathrm{argmin}}_{{\bf
z}_{l}}\\{\Phi_{{\rho}_{k}}([{\bf z}^{k};{\bf z}_{l}],{\bf W}^{k},{\bf
a}^{k})+\frac{\delta{\rho}_{k}}{2}\|{\bf
z}_{l}-{\bf\hat{z}}^{k}_{l}\|^{2}\\}\\\ {\bf
W}_{l}^{k+1}&\in&\displaystyle\mathop{\mathrm{argmin}}_{{\bf
W}_{l}}\\{\Phi_{{\rho}_{k}}({\bf z}^{k+1},[{\bf W}^{k};{\bf W}_{l}],{\bf
a}^{k})+\frac{\delta{\rho}_{k}}{2}\|{\bf W}_{l}-\hat{{\bf
W}}^{k}_{l}\|^{2}\\}\\\ {\bf
a}_{l}^{k+1}&\in&\displaystyle\mathop{\mathrm{argmin}}_{{\bf
a}_{l}}\\{\Phi_{{\rho}_{k}}({\bf z}^{k+1},{\bf W}^{k+1},[{\bf a}^{k};{\bf
a}_{l}])+\frac{\delta{\rho}_{k}}{2}\|{\bf a}_{l}-\hat{{\bf
a}}^{k}_{l}\|^{2}\\}\\\ \end{array}\right..$

We can see if $\alpha=\beta=\gamma=\delta=0$, the algorithm above then
degenerates the DLAM. We first use the linear combinations to predict two new
points $\hat{{\bf W}}^{k},\hat{{\bf a}}^{k}$ (inertial step). In the substeps
of updating ${\bf z}^{k+1}_{l}$, ${\bf W}^{k+1}_{l}$ and ${\bf a}^{k+1}_{l}$,
we use a proximal-point way, i.e., adding the regular terms
$\frac{\delta{\rho}_{k}}{2}\|{\bf z}_{l}-\hat{{\bf z}}^{k}_{l}\|^{2}$,
$\frac{\delta{\rho}_{k}}{2}\|{\bf W}_{l}-\hat{{\bf W}}^{k}_{l}\|^{2}$ and
$\frac{\delta{\rho}_{k}}{2}\|{\bf a}_{l}-\hat{{\bf a}}^{k}_{l}\|^{2}$ in the
minimizations to get the sufficient descent. Let $d_{l}$ is the width of the
$l$-layer, and we have ${\bf W}_{l}\in\mathbb{R}^{d_{l-1}\times d_{l}}$.

As we mentioned at the beginning of this section, $\bar{\rho}$ is the penalty
parameter, which shall be large to yield the sufficient closed approximate
problem. A natural problem is that large $\bar{\rho}$ leads to small change of
$({\bf z}_{L}^{k})_{k\geq 1}$ and $({\bf W}^{k})_{k\geq 1}$, which indicates
the algorithm slowed down in this case. To overcome this drawback, we set a
small $\rho_{0}>0$ as the initialization and then increase it in the
iterations as $\rho_{k+1}=\min\\{\theta\rho_{k},\bar{\rho}\\}$, where
$\theta>1$. Such techniques have been used in the image processing and
nonconvex linearly constrained problems [23, 24]. With increasing penalty
parameter strategy, our algorithm can be displayed in Algorithm 1.

Algorithm 1 Inertial Proximal Deep Learning Alternating Minimization for
Neutral Network Training (iPDLAM)

0: parameters $\alpha\geq 0$, $\beta\geq 0$, $\gamma>0$, $\delta>0$,
$\theta>1$, $\bar{\rho}\geq\rho_{0}>0$, $K>0$Initialization: $({\bf
z}_{l}^{0})_{l\in[L]}$, $({\bf a}_{l}^{0})_{l\in[L-1]}$, $({\bf
W}_{l}^{0})_{l\in[L]}$for $k=1,2,\ldots$, $K$ 1\. $\hat{{\bf z}}^{k}={\bf
z}^{k}+\alpha({\bf z}^{k}-{\bf z}^{k-1})$; 2. $\hat{{\bf W}}^{k}={\bf
W}^{k}+\beta({\bf W}^{k}-{\bf W}^{k-1})$; 3. $\hat{{\bf a}}^{k}={\bf
a}^{k}+\gamma({\bf a}^{k}-{\bf a}^{k-1})$; 4. ${\bf
z}_{l}^{k+1}=\mathcal{S}^{\sigma_{l}}({\bf a}_{l}^{k},\frac{{\bf
W}_{l}^{k}{\bf
a}_{l-1}^{k}+\delta{\bf\hat{z}}^{k}_{l}}{1+\delta},1+\delta),~{}i\in[L-1]$; 5.
${\bf z}_{L}^{k+1}=\mathcal{P}^{\mathcal{L}}({\bf y},\frac{{\bf W}_{L}^{k}{\bf
a}_{L-1}^{k}+\delta\hat{{\bf z}}^{k}_{L}}{1+\delta},\rho_{k}(1+\delta))$; 6.
Update ${\bf W}_{l}^{k+1}$; 7. Update ${\bf a}_{l}^{k+1}$; 8. Update
$\rho_{k+1}=\min\\{\theta\rho_{k},\bar{\rho}\\}$end for

##  3 Numerical Experiments

In this section, we present the numerical results of our algorithm. We follow
the experimental setup introduced by [7]. Specifically, we consider the DNN
training model (1) with ReLU activation, the squared loss, and the network
architecture being an MLPs with hidden layers, on the two datasets, MNIST [25]
and Fashion MNIST [26]. The specific settings were summarized as follows:

  * •

For the MNIST data set, we implemented a 784-(1500$\times$2)-10 MLPs (i.e.,
the input dimension d0 = 28 $\times$ 28 = 784, the output dimension d3 = 10,
and the numbers of hidden units are all 1500), and set
$\alpha=\beta=\gamma,\delta$ with different values to testify the proposed
algorithm iPDLAM. The sizes of training and test samples are 60000 and 10000,
respectively.

  * •

The parameters of the DLAM is adopted as the default parameters given in [15].
The learning rate of SGD and its variants (.i.e. RMSprop, Adam, Adadelta,
AMSGrad, Adamax.) is 0.001 (a very conservative learning rate to see if SGD
can train the DNNs). More greedy learning rates such as 0.01 and 0.05 have
also been used, and similar results of training are observed.

  * •

For each experiment, we used the same mini-batch sizes (512) and
initializations for all algorithms. Specifically, all the weights ${\bf
W}_{l}^{0}$ are randomly initialized from a Gaussian distribution with a
standard deviation of 0.01 and the bias vectors are initialized as vectors of
all 0.1, while the auxiliary variables ${\bf z}_{l}$ and state variables ${\bf
a}_{l}$ are initialized by a single forward pass.

The proposed algorithm is implemented based on PyTorch. And the experiments
are conducted on a desktop computer with Intel® Core™ CPU i7-8700k @ 4.70 GHz
, 16 GB memory, and running Ubuntu 18.04 Server OS. The CPU contains 12
physical cores. We use PyTorch 1.5.0 to implement the proposed algorithm.

###  3.1 The closed solutions of substeps in iPDLAM

Given the activation $\sigma_{l}(\cdot)$ to be ReLU function, the loss
function is $\ell_{2}$-norm, $R_{l}(\cdot)$ is 0, and the bias parameter added
to the network, the closed solution of substeps in iPDLAM is presented in this
subsection. The parameters are discussed in the following subsections. Besides
the proximal operators on ${\bf z}_{l}^{k+1}$ for $l\in[L]$, the closed
solutions for ${\bf W}_{l}^{k+1}$ and ${\bf b}_{l}^{k+1}$ for $l\in[L]$ are:

$\small{\bf W}_{l}^{k+1}=(({\bf z}_{l}^{k+1}-{\bf b}_{l}^{k})({\bf
a}_{l-1}^{k})^{T}+\delta{\bf\hat{w}}_{l}^{k})(({\bf a}_{l-1}^{k})({\bf
a}_{l-1}^{k})^{T}+\delta{\bf I})^{-1},$ ${\bf b}_{l}^{k+1}=({\bf
z}_{l}^{k+1}-{\bf W}_{l}^{k+1}{\bf a}_{l-1}^{k}+\delta*\hat{{\bf
b}}^{k}_{l})/(1+\delta),$ (3)

in which, ${\bf a}_{0}$ is the input data.

In the following, we present numerical experiments to verify the theoretical
analysis, the iPDLAM’s superem performance over baselines under different
$\alpha,\beta,\gamma$ and $\delta$. And further more, we discuss the variables
update order’s impect on the final performance.

###  3.2 Comparisons with DLAM

Fig. 1: Comparations with DLAM on Function values, training and test accuracy
versus the iteration when $\alpha=\beta=\gamma=0.2,0.5,0.7$ on MNIST (left)
and Fashion MNIST (right).

Firstly, we present the objective function values, training accuracy and test
accuracy when $\alpha=\beta=\gamma=0.2,0.5,0.7$ for the two datasets; $\delta$
is set as $1$. The maximum iteration is set as $200$. The results of test 1
are given in Figure 1. We can see that iPDLAM outperforms DLAM: iPDLAM can
achieve smaller objective function values and larger training and test
accuracy. It is worth mentioning that the curves of the function values are
not always descent. This fact does not contradict our theory due to that we
prove $(\xi_{k})_{k}$ is decreasing after sufficient iteration (Lemma 1)
rather than the function values.

From Table 1, we can see that the inertial term contributes when $0<\alpha<1$.
When $\alpha>1$, iPDLAM performs poorly. This phenomenon coincides with our
theoretical findings.

| MNIST | Fashion MNIST
---|---|---
$\alpha$ | FV | TrA | TeA | FV | TrA | TeA
0.1 | 0.14 | 95.4% | 88.2% | 0.14 | 94.6% | 93.2%
0.2 | 0.12 | 95.7% | 88.9% | 0.12 | 95.0% | 93.4%
0.3 | 0.13 | 95.8% | 89.2% | 0.13 | 95.5% | 93.7%
0.4 | 0.14 | 96.2% | 89.5% | 0.14 | 95.7% | 93.9%
0.5 | 0.14 | 96.4% | 89.3% | 0.14 | 96.2% | 94.1%
0.6 | 0.10 | 96.6% | 89.2% | 0.10 | 96.6% | 94.2%
0.7 | 0.03 | 96.8% | 89.6% | 0.09 | 96.3% | 94.2%
0.8 | 0.28 | 92.0% | 85.8% | 0.29 | 91.6% | 91.8%
0.9 | 0.61 | 95.1% | 87.6% | 0.64 | 94.6% | 93.0%
1.0 | 5510 | 89.0% | 78.1% | 5600 | 89.1% | 86.2%
1.1 | nan | nan | nan | nan | 10.4% | 10.4%
1.2 | nan | nan | nan | nan | 10.4% | 10.4%
Table 1: Function values, training accuracy and test accuracy for different
$\alpha$ ($=\beta=\gamma$) on MNIST and Fashion MNIST after 200 iterations.
_Function Values_ (FV), _Training Accuracy_ (TrA) and _Test Accuracy_ (TeA).

###  3.3 Robust performance on different values of $\delta$

In the second test, we use $\alpha=\beta=\gamma=0.7$, the parameter $\delta$
is set as $0.1,0.2,0.3,0.4,1$. The dataset is the MNIST. The results of the
second test are given in Figure 2. In the five cases, the training and test
accuracy versus the iterations actually perform very similar for the five
cases. The results show that the algorithm is insensitive to $\delta$.

###  3.4 Results against classical Deep Learning optimizers

We compare iPDLAM with SGD, AdaGrad[27], Adadelta[28], Adam and Adamax[29].
The training and test accuracy versus the iteration (epoch) for different
algorithms are reported in Figure 3. Although our algorithm cannot beat
classical algorithms on training accuracy, it performs better than most of
them on test accuracy. Our algorithm can learn a quite good parameter in very
small iterations (less than 10). Moreover, the proposed iPDLAM is alawys
better than classical optimizers at first 40 iterations, both in It is also
the fastest one to reach stable, even when SGD fails to converge (the blue one
on the bottom in Figure 3).

Based on the experiment results, it is recomended to adopt the iPDLAM to train
the neural networks at the first stage as a warm-up strategy, which is
expected to greatly reduce the total training time cost, and speed up the
training process.

Fig. 2: Training and test accuracy versus the iteration for different
$\delta$.

Fig. 3: Training accuracy versus the iteration and time for different
algorithms on MNIST.

###  3.5 The update order of variables

We compare the alternating minimization methods and use the following cyclic
orders to update the variables:

  * •

Reverse: ${\bf z}_{L}\rightarrow{\bf z}_{L-1}\rightarrow\ldots\rightarrow{\bf
z}_{1}\rightarrow{\bf W}_{L}\rightarrow{\bf
W}_{L-1}\rightarrow\ldots\rightarrow{\bf W}_{1}\rightarrow{\bf
a}_{L}\rightarrow{\bf a}_{L-1}\rightarrow\ldots\rightarrow{\bf
a}_{1}\rightarrow{\bf z}_{L}\ldots$

  * •

Increase: ${\bf z}_{1}\rightarrow{\bf z}_{2}\rightarrow\ldots\rightarrow{\bf
z}_{L}\rightarrow{\bf W}_{1}\rightarrow{\bf
W}_{2}\rightarrow\ldots\rightarrow{\bf W}_{L}\rightarrow{\bf
a}_{1}\rightarrow{\bf a}_{2}\rightarrow\ldots\rightarrow{\bf
a}_{L-1}\rightarrow{\bf z}_{1}\ldots$

  * •

Nested Reverse (NR): ${\bf z}_{L}\rightarrow{\bf W}_{L}\rightarrow{\bf
a}_{L-1}\rightarrow{\bf z}_{L-1}\rightarrow{\bf
W}_{L-1}\rightarrow\ldots\rightarrow{\bf a}_{2}\rightarrow{\bf
z}_{2}\rightarrow{\bf W}_{2}\rightarrow{\bf a}_{1}\rightarrow{\bf
z}_{1}\rightarrow{\bf W}_{1}\rightarrow{\bf z}_{L}\rightarrow\ldots$

The numerical results are reported in Figure 4. The variables update order has
little impact on the optimization process, which suggests iPDLAMs could be
further applied to parallel and distributed settings. Further experiments
results are provided in the supplymentary material, and the implementation of
this work is public available.

Fig. 4: The variables update order has a little impact on the optimization
process.

##  4 Conclusion

In this paper, we propose an improved alternating minimization, named iPDLAM,
for the neural network training. The development of the algorithm is based on
the inertial techniques applied to the penalty formulation of the training
task. Different from the stochastic training methods, our algorithm enjoys
solid convergence guarantee, and the numerical results show that the proposed
algorithm takes smaller iterations to reach the same training and test
accuracy compared with various classical training algorithms.

## References

  * [1] H. Robbins and S. Monro, “A stochastic approximation method,” The annals of mathematical statistics, pp. 400–407, 1951.
  * [2] D. E. Rumelhart, G. E. Hinton, R. J. Williams, et al., “Learning representations by back-propagating errors,” Cognitive modeling, vol. 5, no. 3, p. 1, 1988.
  * [3] S. Hochreiter, Y. Bengio, P. Frasconi, J. Schmidhuber, et al., “Gradient flow in recurrent nets: the difficulty of learning long-term dependencies,” 2001\.
  * [4] G. Taylor, R. Burmeister, Z. Xu, B. Singh, A. Patel, and T. Goldstein, “Training neural networks without gradients: A scalable admm approach,” in International conference on machine learning, pp. 2722–2731, 2016.
  * [5] Z. Zhang, Y. Chen, and V. Saligrama, “Efficient training of very deep neural networks for supervised hashing,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1487–1495, 2016.
  * [6] Z. Zhang and M. Brand, “Convergent block coordinate descent for training tikhonov regularized deep neural networks,” in Advances in Neural Information Processing Systems, pp. 1721–1730, 2017.
  * [7] J. Wang, F. Yu, X. Chen, and L. Zhao, “Admm for efficient deep learning with global convergence,” arXiv preprint arXiv:1905.13611, 2019.
  * [8] A. Askari, G. Negiar, R. Sambharya, and L. E. Ghaoui, “Lifted neural networks,” arXiv preprint arXiv:1805.01532, 2018.
  * [9] A. Choromanska, E. Tandon, S. Kumaravel, R. Luss, I. Rish, B. Kingsbury, R. Tejwani, and D. Bouneffouf, “Beyond backprop: Alternating minimization with co-activation memory,” stat, vol. 1050, p. 24, 2018.
  * [10] J. Li, C. Fang, and Z. Lin, “Lifted proximal operator machines,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4181–4188, 2019.
  * [11] F. Gu, A. Askari, and L. E. Ghaoui, “Fenchel lifted networks: A lagrange relaxation of neural network training,” arXiv preprint arXiv:1811.08039, 2018.
  * [12] M. Carreira-Perpinan and W. Wang, “Distributed optimization of deeply nested systems,” in Artificial Intelligence and Statistics, pp. 10–19, 2014.
  * [13] P. L. Combettes and J.-C. Pesquet, “Proximal splitting methods in signal processing,” in Fixed-point algorithms for inverse problems in science and engineering, pp. 185–212, Springer, 2011.
  * [14] V. Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807–814, 2010.
  * [15] J. Zeng, T. T.-K. Lau, S. Lin, and Y. Yao, “Global convergence of block coordinate descent in deep learning,” vol. 97 of Proceedings of Machine Learning Research, (Long Beach, California, USA), pp. 7313–7323, PMLR, 09–15 Jun 2019.
  * [16] T. Sun, D. Li, H. Jiang, and Z. Quan, “Iteratively reweighted penalty alternating minimization methods with continuation for image deblurring,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3757–3761, IEEE, 2019.
  * [17] B. T. Polyak, “Some methods of speeding up the convergence of iteration methods,” USSR Computational Mathematics and Mathematical Physics, vol. 4, no. 5, pp. 1–17, 1964.
  * [18] P. Ochs, Y. Chen, T. Brox, and T. Pock, “ipiano: Inertial proximal algorithm for nonconvex optimization,” SIAM Journal on Imaging Sciences, vol. 7, no. 2, pp. 1388–1419, 2014.
  * [19] T. Pock and S. Sabach, “Inertial proximal alternating linearized minimization (ipalm) for nonconvex and nonsmooth problems,” SIAM Journal on Imaging Sciences, vol. 9, no. 4, pp. 1756–1787, 2016.
  * [20] N. Loizou and P. Richtárik, “Momentum and stochastic momentum for stochastic gradient, newton, proximal point and subspace descent methods,” arXiv preprint arXiv:1712.09677, 2017.
  * [21] N. Loizou and P. Richtárik, “Accelerated gossip via stochastic heavy ball method,” in 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 927–934, IEEE, 2018.
  * [22] T. Sun, D. Li, Z. Quan, H. Jiang, S. Li, and Y. Dou, “Heavy-ball algorithms always escape saddle points,” IJCAI, 2019.
  * [23] Y. Wang, J. Yang, W. Yin, and Y. Zhang, “A new alternating minimization algorithm for total variation image reconstruction,” SIAM Journal on Imaging Sciences, vol. 1, no. 3, pp. 248–272, 2008.
  * [24] T. Sun, L. Qiao, and D. Li, “Bregman reweighted alternating minimization and its application to image deblurring,” Information Sciences, 2019.
  * [25] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, et al., “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  * [26] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017.
  * [27] J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research, vol. 12, no. Jul, pp. 2121–2159, 2011.
  * [28] M. D. Zeiler, “Adadelta: an adaptive learning rate method,” arXiv preprint arXiv:1212.5701, 2012.
  * [29] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” ICLR, 2014.