# Transformation formula for the reduced Bergman kernel and its application

Sahil Gehlawat**, Aakanksha Jain and Amar Deep Sarkar  ADS: Harish-Chandra
Research Institute, Prayagraj (Allahabad) 211019, India
<EMAIL_ADDRESS>SG: Department of Mathematics, Indian Institute of
Science, Bangalore 560 012, India<EMAIL_ADDRESS>AJ: Department of
Mathematics, Indian Institute of Science, Bangalore 560 012, India
<EMAIL_ADDRESS>
###### Abstract.

In this article, we prove the transformation formula for the reduced Bergman
kernels under proper holomorphic correspondences between bounded domains in
the complex plane. As a corollary, we obtain the transformation formula for
the reduced Bergman kernels under proper holomorphic maps. We also establish
the transformation formula for the weighted reduced Bergman kernels under
proper holomorphic maps. Finally, we provide an application of this
transformation formula.

###### Key words and phrases:

reduced Bergman space, Bergman space, reduced Bergman kernel, Bergman kernel,
transformation formula

###### 1991 Mathematics Subject Classification:

Primary: 30H20, 46E22; Secondary: 32H99

*The author is supported by the postdoctoral fellowship of Harish-Chandra Research Institute, Prayagraj (Allahabad).

**The author is supported by the CSIR SPM Ph.D. fellowship.

##  1\. Introduction

Recall that the Bergman space associated with a domain $D\subset\mathbb{C}$
consists of square integrable holomorphic functions on it. An important space
that has a close relationship with this space is the space of all holomorphic
functions whose derivatives are square-integrable with respect to the area
measure. This space can be associated with a closed subspace of the Bergman
space, which we call the reduced Bergman space. The reduced Bergman space is a
reproducing kernel Hilbert space with its reproducing kernel called the
reduced Bergman kernel; see below for the definitions.

The aim of this note is to prove the transformation formula for the reduced
Bergman kernels under proper holomorphic correspondences between bounded
planar domains. This is motivated by Bell’s transformation formula for the
Bergman kernel under proper mappings and correspondences (see [1], [2], [3],
[4] and [5, Theorem 16.5]). The transformation formula under proper mappings
will follow as a corollary. But we will see that more is true: the
transformation formula for the weighted reduced Bergman kernels under proper
holomorphic mappings between bounded planar domains. Finally, we will see an
application of this transformation formula; any proper holomorphic map from a
bounded planar domain to the unit disc is rational if the reduced Bergman
kernel of the domain is rational. The techniques of the proofs will be similar
to Bell’s [4] work on the transformation formula for the Bergman kernel and
its applications.

M. Sakai [6] defined the reduced Bergman kernel in the following way: Let $U$
be a bounded planar domain. For $\xi\in U$, consider the complex linear space

$AD(U,\xi)=\left\\{f\in\mathcal{O}(U):f(\xi)=0,\int_{U}\lvert
f^{\prime}(z)\rvert^{2}dA<\infty\right\\}$

equipped with the inner-product $\langle
f,g\rangle=\int_{U}f^{\prime}(z)\,\overline{g^{\prime}(z)}\,dA$, where $dA$ is
the area Lebesgue measure on $U$. Along with this inner-product, $AD(U,\xi)$
is a complex Hilbert space. It can be proved that $AD(U,\xi)\ni
f\longrightarrow f^{\prime}(\xi)\in\mathbb{C}$ is a bounded linear functional.
The Riesz representation theorem therefore gives a unique function
$M(\cdot,\xi)\in AD(U,\xi)$ such that $f^{\prime}(\xi)=\langle
f,M(\cdot,\xi)\rangle$ for all $f\in AD(U,\xi)$. The function
$\tilde{K}(z,\xi):=\frac{d}{dz}M(z,\xi)$ is defined as the reduced Bergman
kernel of $U$.

We will give an another equivalent definition of the reduced Bergman kernel
(and the weighted reduced Bergman kernel).

######  Definition 1.1.

Let $D$ be a domain in $\mathbb{C}$. The reduced Bergman space of $D$ is the
space of all the square-integrable holomorphic functions on $D$ whose
primitive exists on $D$, i.e.,

$\mathcal{D}(D)=\left\\{f\in\mathcal{O}(D):f=g^{\prime}\ \text{for some}\
g\in\mathcal{O}(D)\,\,\text{and}\,\,\int_{D}\lvert
f(z)\rvert^{2}dA(z)<\infty\right\\}.$

This is a Hilbert space with respect to the inner product

$\langle f,g\rangle:=\int_{D}f(z)\overline{g(z)}dA(z).$

For every $\zeta\in D$, the evaluation functional

$f\mapsto f(\zeta),\quad\quad f\in\mathcal{D}(D)$

is a bounded linear functional, and therefore $\mathcal{D}(D)$ is a
reproducing kernel Hilbert space. The reproducing kernel of $\mathcal{D}(D)$,
denoted by $\tilde{K}_{D}(\cdot,\cdot)$, is called the reduced Bergman kernel
of $D$. It satisfies the reproducing property:

$f(\zeta)=\int_{D}f(z)\overline{\tilde{K}_{D}(z,\zeta)}dA(z)$

for all $f\in\mathcal{D}(D)$ and $\zeta\in D$.

Note that the reduced Bergman space is a closed subspace of the Bergman space.

Now we are ready to state our main results. First, we recall the definition of
a proper holomorphic correspondence. Let $D_{1}$ and $D_{2}$ be domains in
$\mathbb{C}$, and $\pi_{1}:D_{1}\times D_{2}\longrightarrow D_{1}$,
$\pi_{2}:D_{1}\times D_{2}\longrightarrow D_{2}$ be the projections. If $V$ is
a complex sub-variety of $D_{1}\times D_{2}$, then consider the associated
multi-valued function $f:D_{1}\multimap D_{2}$ given by
$f(z)=\pi_{2}\pi_{1}^{-1}(z)=\\{w:(z,w)\in V\\}$. The map $f$ is called a
holomorphic correspondence and $V$ is called the graph of $f$. The
correspondence $f$ is said to be proper if the projection maps
$\pi_{1}:V\longrightarrow D_{1}$ and $\pi_{2}:V\longrightarrow D_{2}$ are
proper.

We remark that there exist sub-varieties $V_{1}$ and $V_{2}$ of $D_{1}$ and
$D_{2}$ respectively, and positive integers $p$ and $q$ such that
$\pi_{2}\pi_{1}^{-1}$ is locally given by $p$ holomorphic maps on
$D_{1}\setminus V_{1}$ which we will denote by $\\{f_{i}\\}_{i=1}^{p}$, and
$\pi_{1}\pi_{2}^{-1}$ is locally given by $q$ holomorphic maps on
$D_{2}\setminus V_{2}$ which we will denote by $\\{F_{i}\\}_{i=1}^{q}$. Note
that the sub-varieties $V_{1}$ and $V_{2}$ are discrete subsets of $D_{1}$ and
$D_{2}$ respectively. Proper holomorphic correspondences arise naturally in
the field of complex function theory.

######  Theorem 1.2.

Let $D_{1}$ and $D_{2}$ be bounded domains in $\mathbb{C}$. If
$f:D_{1}\multimap D_{2}$ is a proper holomorphic correspondence, then the
reduced Bergman kernels $\tilde{K}_{j}$’s associated with $D_{j}$’s, $j=1,2$,
transform according to

$\sum_{i=1}^{p}f_{i}^{\prime}(z)\tilde{K}_{2}(f_{i}(z),w)=\sum_{j=1}^{q}\tilde{K}_{1}(z,F_{j}(w))\overline{F_{j}^{\prime}(w)},$

for all $z\in D_{1}$ and $w\in D_{2}$, where $f_{i}$’s and $F_{j}$’s, and the
positive integers $p,q$ are as above.

Before we state the next result, we recall the definition of a proper
holomorphic map. Let $D_{1}$ and $D_{2}$ be domains in $\mathbb{C}$. A
holomorphic map $f:D_{1}\rightarrow D_{2}$ is called proper if $f^{-1}(K)$ is
compact in $D_{1}$ whenever $K$ is compact in $D_{2}$. Let $V\subset D_{2}$
denote the set of all critical values of $f$ and $\tilde{V}=f^{-1}(V)\subset
D_{1}$. Note that both $V$ and $\tilde{V}$ are discrete sets. It is well known
that $f:D_{1}\setminus\tilde{V}\longrightarrow D_{2}\setminus V$ is an
$m$-to-$1$ holomorphic covering map for some $m\in\mathbb{N}$, where $m$ is
called the multiplicity of $f$. We will denote the $m$ local inverses of $f$
on $D_{2}\setminus V$ by $\\{F_{k}\\}_{k=1}^{m}$. Now as a special case of
Theorem 1.2, we have the following corollary:

######  Corollary 1.3.

Let $D_{1}$ and $D_{2}$ be bounded domains in $\mathbb{C}$. If
$f:D_{1}\longrightarrow D_{2}$ is a proper holomorphic map, then the reduced
Bergman kernels $\tilde{K}_{1}$ and $\tilde{K}_{2}$ associated with $D_{1}$
and $D_{2}$ respectively, transform according to

$f^{\prime}(z)\tilde{K}_{2}(f(z),w)=\sum_{k=1}^{m}\tilde{K}_{1}(z,F_{k}(w))\overline{F_{k}^{\prime}(w)},$

for all $z\in D_{1}$ and $w\in D_{2}$, where $m$ is the multiplicity of $f$
and $F_{k}$’s are the local inverses of $f$.

We would like to take this opportunity to give a transformation formula for
kernels of a more general class of spaces, namely weighted reduced Bergman
spaces. First we give a definition:

######  Definition 1.4.

Let $D$ be a domain in $\mathbb{C}$ and $\nu$ be a positive measurable
function on $D$ such that $1/\nu\in L^{\infty}_{loc}(D)$. The weighted reduced
Bergman space of $D$ with weight $\nu$ is the space of all the $\nu$-square
integrable holomorphic functions on $D$ whose primitive exists on $D$, i.e.,

$\mathcal{D}^{\nu}(D)=\left\\{f\in\mathcal{O}(D):f=g^{\prime}\ \text{for
some}\ g\in\mathcal{O}(D)\,\,\text{and}\,\,\int_{D}\lvert
f(z)\rvert^{2}\nu(z)dA(z)<\infty\right\\}.$

This is a Hilbert space with respect to the inner product

$\langle f,g\rangle_{\nu}:=\int_{D}f(z)\,\overline{g(z)}\,\nu(z)\,dA(z).$

For every $\zeta\in D$, the evaluation functional

$f\mapsto f(\zeta),\quad\quad f\in\mathcal{D}^{\nu}(D)$

is a bounded linear functional, and therefore $\mathcal{D}^{\nu}(D)$ is a
reproducing kernel Hilbert space. The reproducing kernel of
$\mathcal{D}^{\nu}(D)$, denoted by $\tilde{K}^{\nu}_{D}(\cdot,\cdot)$, is
called the weighted reduced Bergman kernel of $D$ with weight $\nu$. It
satisfies the reproducing property:

$f(\zeta)=\int_{D}f(z)\,\overline{\tilde{K}^{\nu}_{D}(z,\zeta)}\,\nu(z)\,dA(z)$

for all $f\in\mathcal{D}^{\nu}(D)$ and $\zeta\in D$.

######  Remark 1.5.

The weighted reduced Bergman kernel $\tilde{K}^{\nu}_{D}(z,\zeta)$ of $D$ is
holomorphic in the first variable and anti-holomorphic in the second variable,
and

$\tilde{K}^{\nu}_{D}(z,\zeta)=\int_{D}\tilde{K}^{\nu}_{D}(\xi,\zeta)\overline{\tilde{K}^{\nu}_{D}(\xi,z)}\nu(\xi)dA(\xi).$

For a bounded domain $D\subset\mathbb{C}$,
$\tilde{K}^{\nu}_{D}(\zeta,\zeta)>0$ for every $\zeta\in D$.

######  Theorem 1.6.

Let $D_{1}$ and $D_{2}$ be bounded domains in $\mathbb{C}$ and $\nu$ be a
positive measurable function on $D_{2}$ such that $1/\nu\in
L^{\infty}_{loc}(D_{2})$. If $f:D_{1}\longrightarrow D_{2}$ is a proper
holomorphic map, then the weighted reduced Bergman kernels
$\tilde{K}^{\nu\circ f}_{1}$ and $\tilde{K}^{\nu}_{2}$ associated with $D_{1}$
and $D_{2}$ respectively, transform according to

$f^{\prime}(z)\tilde{K}^{\nu}_{2}(f(z),w)=\sum_{k=1}^{m}\tilde{K}^{\nu\circ
f}_{1}(z,F_{k}(w))\overline{F_{k}^{\prime}(w)},$

for all $z\in D_{1}$ and $w\in D_{2}$, where $m$ is the multiplicity of $f$
and $F_{k}$’s are the local inverses of $f$.

Bell [4] had shown that any proper holomorphic map $f:D\rightarrow\mathbb{D}$
from a bounded planar domain $D$ to the unit disc will be rational if the
Bergman kernel of $D$ is a rational function. As an application of the above
transformation formula, we will prove a similar result replacing the
hypothesis of the Bergman kernel of $D$ being rational to reduced Bergman
kernel of $D$ being rational. To be precise, we prove:

######  Theorem 1.7.

Suppose $D$ is a bounded domain in $\mathbb{C}$ whose associated reduced
Bergman kernel is a rational function, then any proper holomorphic mapping
$f:D\rightarrow\mathbb{D}$ must be rational.

Acknowledgements. The authors would like to thank Kaushal Verma for all the
helpful discussions and suggestions.

##  2\. Proof of Theorem 1.2

In order to prove the transformation formula for the reduced Bergman kernels
under proper holomorphic correspondences, we will need the following lemmas
and propositions. We first state a result which we are going to use (see [5,
Theorem 16.3]):

######  Result 2.1.

Let $D\subset\mathbb{C}$ be a bounded domain and $V\subset D$ be a discrete
subset. Suppose that $h$ is holomorphic on $D\setminus V$ and $h\in
L^{2}(D\setminus V)$, i.e.,

$\int_{D\setminus V}\lvert h\rvert^{2}dA<\infty,$

then $h$ has a removable singularity at each point of $V$ and $h\in L^{2}(D)$.

Let $f:D_{1}\multimap D_{2}$ be a proper holomorphic correspondence between
bounded planar domains $D_{1}$ and $D_{2}$. For functions $u$ and $v$ on
$D_{2}$ and $D_{1}$ respectively, define maps $\Gamma_{1}$ and $\Gamma_{2}$
by:

$\Gamma_{1}(u)=\sum_{i=1}^{p}f_{i}^{\prime}(u\circ f_{i})$
$\Gamma_{2}(v)=\sum_{j=1}^{q}F_{j}^{\prime}(v\circ F_{j})$

where $f_{i}$’s for $1\leq i\leq p$ and $F_{j}$’s for $1\leq j\leq q$ are as
in the definition of holomorphic correspondences defined locally on
$D_{1}\setminus V_{1}$ and $D_{2}\setminus V_{2}$ respectively.

######  Lemma 2.2.

Given $(D_{1},D_{2},f)$ as above, the $\Gamma_{i}$’s are bounded linear maps
that satisfy:

$\Gamma_{1}(A^{2}(D_{2}))\subset
A^{2}(D_{1})\quad\text{and}\quad\Gamma_{2}(A^{2}(D_{1}))\subset A^{2}(D_{2}),$

where $A^{2}(D_{i})$ denotes the Bergman space of $D_{i}$ for $i=1,2$.

###### Proof.

The linearity of maps $\Gamma_{i}$’s is straightforward. First, we will show
that the maps $\Gamma_{i}$’s preserve square integrable functions, and that
$\Gamma_{1}:L^{2}(D_{2})\rightarrow L^{2}(D_{1})$ and
$\Gamma_{2}:L^{2}(D_{1})\rightarrow L^{2}(D_{2})$ are bounded operators. Since
smooth functions with compact support are dense in $L^{2}$-space, it is
sufficient to work with these functions and the final conclusion will follow
by density property and using partition of unity.

Let $v\in L^{2}(D_{1})$ be a smooth function with very small support $U\subset
D_{1}\setminus V_{1}$ such that $f_{i}(U)\cap f_{j}(U)=\emptyset$ for $i\neq
j$. Set $\tilde{U}_{i}=f_{i}(U)$, and let $F_{j_{i}}$ be the inverse of
$f_{i}$ on $\tilde{U}_{i}$. Observe that

$\langle\Gamma_{2}(v),\Gamma_{2}(v)\rangle_{2}=\int_{D_{2}}\lvert\Gamma_{2}(v)(w)\rvert^{2}\,dA(w)\\\
\leq q\int_{D_{2}}\sum_{j=1}^{q}\lvert F_{j}^{\prime}(w)\rvert^{2}\lvert
v(F_{j}(w))\rvert^{2}dA(w)=q\sum_{i=1}^{p}\int_{\tilde{U}_{i}}\sum_{j=1}^{q}\lvert
F_{j}^{\prime}(w)\rvert^{2}\lvert v(F_{j}(w))\rvert^{2}dA(w).$

Since $F_{j}(\tilde{U}_{i})\cap U\neq\emptyset$ if and only if $j=j_{i}$,
therefore

$\langle\Gamma_{2}(v),\Gamma_{2}(v)\rangle_{2}\leq
q\sum_{i=1}^{p}\int_{\tilde{U}_{i}}\lvert
F_{j_{i}}^{\prime}(w)\rvert^{2}\lvert v\circ F_{j_{i}}(w)\rvert^{2}dA(w).$

Set $F_{j_{i}}(w)=z$ on $\tilde{U}_{i}$, we get $w=f_{i}(z)$ and

$\langle\Gamma_{2}(v),\Gamma_{2}(v)\rangle_{2}\leq
q\sum_{i=1}^{p}\int_{U}\lvert v\rvert^{2}dA(z)=pq\langle
v,v\rangle_{1}<+\infty,$

that is

$\langle\Gamma_{2}(v),\Gamma_{2}(v)\rangle_{2}\leq pq\langle v,v\rangle_{1}.$

Thus, $\Gamma_{2}$ is a bounded linear operator from $L^{2}(D_{1})$ into
$L^{2}(D_{2})$. Now if $v\in A^{2}(D_{1})$, it is clear from the definition of
$\Gamma_{2}$ that $\Gamma_{2}(v)\in\mathcal{O}(D_{2}\setminus V_{2})$. Since
$V_{2}$ is a discrete subset of $D_{2}$, it follows by Result 2.1 that
$\Gamma_{2}(v)\in\mathcal{O}(D_{2})$, and therefore $\Gamma_{2}(v)\in
A^{2}(D_{2})$.

Now let $u\in L^{2}(D_{2})$ be a smooth function with very small support
$W\subset D_{2}\setminus V_{2}$ such that $\tilde{W}_{j}:=F_{j}(W)$ are
mutually disjoint for $1\leq j\leq q$, and let $f_{i_{j}}$ be the inverse of
$F_{j}$ on $\tilde{W}_{j}$. Observe that

$\displaystyle\langle\Gamma_{1}(u),\Gamma_{1}(u)\rangle_{1}$ $\displaystyle=$
$\displaystyle\int_{D_{1}}\lvert\Gamma_{1}(u)(z)\rvert^{2}\,dA(z)$
$\displaystyle\leq$ $\displaystyle p\int_{D_{1}}\sum_{i=1}^{p}\lvert
f_{i}^{\prime}(z)\rvert^{2}\lvert u(f_{i}(z))\rvert^{2}dA(z)$ $\displaystyle=$
$\displaystyle p\sum_{j=1}^{q}\int_{\tilde{W}_{j}}\sum_{i=1}^{p}\lvert
f_{i}^{\prime}(z)\rvert^{2}\lvert u(f_{i}(z))\rvert^{2}dA(z).$

Since $f_{i}(\tilde{W}_{j})\cap W\neq\emptyset$ if and only if $i=i_{j}$, set
$f_{i_{j}}(z)=w$ on $\tilde{W}_{j}$, we get

$\langle\Gamma_{1}(u),\Gamma_{1}(u)\rangle_{1}\leq
p\sum_{j=1}^{q}\int_{\tilde{W}_{j}}\lvert
f_{i_{j}}^{\prime}(z)\rvert^{2}\lvert u\circ
f_{i_{j}}(z)\rvert^{2}dA(z)=p\sum_{j=1}^{q}\int_{W}\lvert
u\rvert^{2}dA(w)=pq\langle u,u\rangle_{2},$

that is

$\langle\Gamma_{1}(u),\Gamma_{1}(u)\rangle_{1}\leq pq\langle u,u\rangle_{2}.$

Thus, $\Gamma_{1}$ is a bounded linear operator from $L^{2}(D_{2})$ into
$L^{2}(D_{1})$. Now if $u\in A^{2}(D_{2})$, clearly
$\Gamma_{1}(u)\in\mathcal{O}(D_{1}\setminus V_{1})$. Since $V_{1}$ is a
discrete subset of $D_{1}$, it follows by Result 2.1 that
$\Gamma_{1}(u)\in\mathcal{O}(D_{1})$, and therefore $\Gamma_{1}(u)\in
A^{2}(D_{1})$.

∎

######  Lemma 2.3.

If $f:D_{1}\multimap D_{2}$ is a proper holomorphic correspondence between
bounded planar domains, then

$\langle\Gamma_{1}u,v\rangle_{1}=\langle u,\Gamma_{2}v\rangle_{2}$

for all $u\in L^{2}(D_{2})$ and $v\in L^{2}(D_{1})$.

###### Proof.

We shall first prove the above result for all smooth $u\in L^{2}(D_{2})$
supported in small neighbourhoods of points in $D_{2}\setminus V_{2}$ and for
all $v\in L^{2}(D_{1})$.

Let $D_{w_{0}}$ be a neighbourhood of $w_{0}\in D_{2}\setminus V_{2}$ such
that $F_{j}$’s for $1\leq j\leq q$ are well defined and
$W_{j}:=F_{j}(D_{w_{0}})$ are pairwise disjoint sets. Let $f_{i_{j}}$ be the
inverse of $F_{j}$ on $W_{j}$. Let $u\in L^{2}(D_{2})$ be such that support of
$u$ is contained in $D_{w_{0}}$. Therefore, support of $\Gamma_{1}u$ is
contained in $\bigcup_{j=1}^{q}W_{j}$ where
$\Gamma_{1}u=\sum_{i=1}^{p}f_{i}^{\prime}(u\circ f_{i})$. Now see that

$\displaystyle\langle\Gamma_{1}u,v\rangle_{1}$ $\displaystyle=$
$\displaystyle\int_{D_{1}}(\Gamma_{1}u)\bar{v}dA(z)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{q}\int_{W_{j}}(\Gamma_{1}u)\bar{v}dA(z)$
$\displaystyle=$
$\displaystyle\sum_{j=1}^{q}\int_{W_{j}}\sum_{i=1}^{p}f_{i}^{\prime}(u\circ
f_{i})\bar{v}dA(z)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{q}\int_{W_{j}}f_{i_{j}}^{\prime}(u\circ
f_{i_{j}})\bar{v}dA(z)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{q}\int_{D_{w_{0}}}u\overline{F_{j}^{\prime}(v\circ
F_{j})}dA(w)\quad\text{(by change of variables)}$ $\displaystyle=$
$\displaystyle\int_{D_{w_{0}}}u\sum_{j=1}^{q}\overline{F_{j}^{\prime}(v\circ
F_{j})}dA(w)$ $\displaystyle=$
$\displaystyle\int_{D_{w_{0}}}u\,\overline{\Gamma_{2}v}\,dA(w)$
$\displaystyle=$ $\displaystyle\int_{D_{2}}u\,\overline{\Gamma_{2}v}\,dA(w)$
$\displaystyle=$ $\displaystyle\langle u,\Gamma_{2}v\rangle_{2}.$

Hence the result follows by density argument, as the linear span of all such
$u$’s is dense in $L^{2}(D_{2})$. ∎

We need the following lemma about the extension of holomorphic functions:

######  Lemma 2.4.

Let $D\subset\mathbb{C}$ be a domain and $V\subset D$ be a discrete set.
Suppose that $h$ is holomorphic on $D\setminus V$ and $h^{\prime}$ has
removable singularities at each point in $V$. Then $h$ also has removable
singularities at each point in $V$.

###### Proof.

Let $p\in D\setminus V$ be arbitrary. Choose $r>0$ such that the disc
$D(p,r)\subset D\setminus V$. We are given that
$h\in\mathcal{O}(D(p,r)\setminus\\{p\\})$ and
$h^{\prime}\in\mathcal{O}(D(p,r))$. Since $D(p,r)$ is simply connected, there
exists a primitive $g$ of $h^{\prime}$ on $D(p,r)$, i.e, there exists
$g\in\mathcal{O}(D(p,r))$ such that

$g^{\prime}(z)=h^{\prime}(z)\;\text{ for all }\,z\in D(p,r).$

Consider the holomorphic function on $D(p,r)\setminus\\{p\\}$ given by
$f(z):=h(z)-g(z)$. Observe that on $D(p,r)\setminus\\{p\\}$,

$f^{\prime}\equiv 0.$

Therefore $h-g\equiv c$ on $D(p,r)\setminus\\{p\\}$ for some constant
$c\in\mathbb{C}$. So $h(z)=g(z)+c$ for all $z\in D(p,r)\setminus\\{p\\}$. But
$g+c$ is holomorphic on $D(p,r)$. Thus we get that $p$ is a removable
singularity of $h$.

∎

Now we have the following proposition about reduced Bergman spaces:

######  Proposition 2.5.

If $f:D_{1}\multimap D_{2}$ is a proper holomorphic correspondence between
bounded planar domains, then

$\tilde{\Gamma}_{1}(u):=\Gamma_{1}|_{\mathcal{D}(D_{2})}(u)\in\mathcal{D}(D_{1})\;\;\text{and}\;\;\tilde{\Gamma}_{2}(v):=\Gamma_{2}|_{\mathcal{D}(D_{1})}(v)\in\mathcal{D}(D_{2})$

for all $u\in\mathcal{D}(D_{2})$ and $v\in\mathcal{D}(D_{1})$.

###### Proof.

Let $u\in\mathcal{D}(D_{2})$. Therefore $u=\tilde{u}^{\prime}$ for some
$\tilde{u}\in\mathcal{O}(D_{2})$. Observe that

$\tilde{\Gamma}_{1}(u)=\Gamma_{1}(u)=\sum_{i=1}^{p}f_{i}^{\prime}(u\circ
f_{i})=\sum_{i=1}^{p}\left(\tilde{u}\circ
f_{i}\right)^{\prime}=\left(\sum_{i=1}^{p}\tilde{u}\circ
f_{i}\right)^{\prime}.$

Consider the function $h_{1}=\sum_{i=1}^{p}\tilde{u}\circ f_{i}$. It is easy
to see that $h_{1}\in\mathcal{O}(D_{1}\setminus V_{1})$, and since $V_{1}$ is
a discrete set and $\tilde{\Gamma}_{1}(u)\in L^{2}(D_{1})$ i.e.
$h_{1}^{\prime}\in L^{2}(D_{1})$, it follows from Result 2.1 that
$h_{1}^{\prime}$ has removable singularities at points of $V_{1}$. Now by
Lemma 2.4, $h_{1}$ also has removable singularities at points of $V_{1}$.
Therefore $h_{1}\in\mathcal{O}(D_{1})$ and hence for all
$u\in\mathcal{D}(D_{2})$, we have

$\tilde{\Gamma}_{1}(u)\in\mathcal{D}(D_{1}).$

Now let $v\in\mathcal{D}(D_{1})$ be arbitrary. Therefore
$v=\tilde{v}^{\prime}$ for some $\tilde{v}\in\mathcal{O}(D_{1})$. Observe that

$\tilde{\Gamma}_{2}(v)=\Gamma_{2}(v)=\sum_{j=1}^{q}F_{j}^{\prime}(v\circ
F_{j})=\sum_{j=1}^{q}\left(\tilde{v}\circ
F_{j}\right)^{\prime}=\left(\sum_{j=1}^{q}\tilde{v}\circ
F_{j}\right)^{\prime}.$

Consider the function $h_{2}=\sum_{j=1}^{q}\tilde{v}\circ F_{j}$. It is easy
to see that $h_{2}\in\mathcal{O}(D_{2}\setminus V_{2})$. Since $V_{2}$ is a
discrete set and $\tilde{\Gamma}_{2}(v)\in L^{2}(D_{2})$, therefore with an
exactly similar argument as for $h_{1}$ above, we have
$h_{2}\in\mathcal{O}(D_{2})$. Thus we have

$\tilde{\Gamma}_{2}(v)\in\mathcal{D}(D_{2}).$

∎

######  Proposition 2.6.

If $f:D_{1}\multimap D_{2}$ is a proper holomorphic correspondence between
bounded planar domains, then

$\langle\tilde{\Gamma}_{1}u,v\rangle_{1}=\langle
u,\tilde{\Gamma}_{2}v\rangle_{2}$

for all $u\in\mathcal{D}(D_{2})$ and $v\in\mathcal{D}(D_{1})$.

###### Proof.

Since $\tilde{\Gamma}_{i}$ is just the restriction of $\Gamma_{i}$ on
$\mathcal{D}(D_{j})$ where $j\neq i$, this result holds trivially from Lemma
2.3 and Proposition 2.5. ∎

###### Proof of Theorem 1.2.

For a fixed $w\in D_{2}\setminus V_{2}$ , consider the function

$G(z)=\sum_{j=1}^{q}\tilde{K}_{1}(z,F_{j}(w))\overline{F_{j}^{\prime}(w)},\quad
z\in D_{1}.$

Since $\tilde{K_{1}}(\cdot,\zeta)\in\mathcal{D}(D_{1})$ for all $\zeta\in
D_{1}$, we observe that $G\in\mathcal{D}(D_{1})$. Now using the reproducing
property of the reduced Bergman kernel, we get for an arbitrarily chosen
$v\in\mathcal{D}(D_{1})$

$\displaystyle\langle v,G\rangle_{1}$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{q}F_{j}^{\prime}(w)\,\langle
v,\tilde{K}_{1}(\cdot,F_{j}(w))\rangle_{1}$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{q}F_{j}^{\prime}(w)\,v(F_{j}(w))$ $\displaystyle=$
$\displaystyle(\tilde{\Gamma}_{2}v)(w)$ $\displaystyle=$
$\displaystyle\langle\tilde{\Gamma}_{2}v,\tilde{K}_{2}(\cdot,w)\rangle_{2}$
$\displaystyle=$ $\displaystyle\langle
v,\tilde{\Gamma}_{1}u\rangle_{1}\quad\quad\quad\quad\text{(by Proposition
2.6)}$

where $u=\tilde{K}_{2}(\cdot,w)$. So, $\langle v,G\rangle_{1}=\langle
v,\tilde{\Gamma}_{1}u\rangle_{1}$ for all $v\in\mathcal{D}(D_{1})$. Therefore,
$\tilde{\Gamma}_{1}u=G$. Thus, we have proved that

$\sum_{i=1}^{p}f_{i}^{\prime}(z)\tilde{K}_{2}(f_{i}(z),w)=\sum_{j=1}^{q}\tilde{K}_{1}(z,F_{j}(w))\overline{F_{j}^{\prime}(w)}\quad\text{
for }z\in D_{1},\,w\in D_{2}\setminus V_{2}.$

Since LHS is anti-holomorphic in $w$ and $V_{2}$ is discrete, the points in
$V_{2}$ are removable singularities of RHS. Hence, the formula holds
everywhere by continuity. ∎

##  3\. Proof of Theorem 1.6

We will start by proving a weighted version of Result 2.1.

######  Lemma 3.1.

Let $D\subset\mathbb{C}$ be a domain and $V\subset D$ be a discrete subset.
Suppose that $h$ is holomorphic on $D\setminus V$ and $h\in
L^{2,\mu}(D\setminus V)$, i.e.,

$\int_{D\setminus V}|h|^{2}\mu dA<\infty$

where $\mu$ is a positive measurable function on $D$ such that $1/\mu\in
L^{\infty}_{loc}(D)$, then $h$ has a removable singularity at each point in
$V$ and $h\in L^{2,\mu}(D)$.

###### Proof.

Let $p\in V$ and $U$ be a relatively compact neighourhood of $p$ in $D$ such
that $U\cap V=\\{p\\}$. By definition, $\mu\geq c$ a.e. on $U$ for some $c>0$.
Observe that

$\int_{U\setminus\\{p\\}}\lvert
h\rvert^{2}dA=\frac{1}{c}\int_{U\setminus\\{p\\}}\lvert
h\rvert^{2}c\,dA\leq\frac{1}{c}\int_{U\setminus\\{p\\}}\lvert h\rvert^{2}\mu
dA\leq\frac{1}{c}\int_{D\setminus V}\lvert h\rvert^{2}\mu dA<\infty.$

Therefore, by Result $2.1$, $h$ has a removable singularity at $p$. Since
$p\in V$ is arbitrary, $h$ has a removable singularity at each point in $V$,
i.e., $h\in\mathcal{O}(D)\cap L^{2,\mu}(D)$. ∎

Let $f:D_{1}\longrightarrow D_{2}$ be a proper holomorphic map between two
bounded planar domains $D_{1}$ and $D_{2}$ and $\nu$ be a weight function on
$D_{2}$ as in Theorem $1.6$. As we did in the previous section, we will define
corresponding maps $\Lambda_{1}$ and $\Lambda_{2}$ by:

$\Lambda_{1}(u)=f^{\prime}(u\circ f)$
$\Lambda_{2}(v)=\sum_{k=1}^{m}F_{k}^{\prime}(v\circ F_{k}),$

where $u$ is function on $D_{2}$ and $v$ is a function on $D_{1}$, and
$F_{k}$’s are the local inverses of $f$ defined outside the set of critical
values $V\subset D_{2}$ with $m$ being the multiplicity of $f$.

######  Lemma 3.2.

Given $(D_{1},D_{2},f,\nu)$ as above, the $\Lambda_{i}$’s are bounded linear
maps that satisfy:

$\Lambda_{1}(A^{2,\nu}(D_{2}))\subset A^{2,\nu\circ
f}(D_{1})\quad\text{and}\quad\Lambda_{2}(A^{2,\nu\circ f}(D_{1}))\subset
A^{2,\nu}(D_{2}),$

where $A^{2,\mu}(D)=\\{f\in\mathcal{O}(D):\int_{D}\lvert f\rvert^{2}\mu
dA<\infty\\}$ denotes the weighted Bergman space of $D$ with weight $\mu$.

###### Proof.

The linearity of maps $\Lambda_{i}$’s is straightforward. First we will show
that $\Lambda_{1}:L^{2,\nu}(D_{2})\rightarrow L^{2,\nu\circ f}(D_{1})$ and
$\Lambda_{2}:L^{2,\nu\circ f}(D_{1})\rightarrow L^{2,\nu}(D_{2})$ are well
defined bounded operators. Since smooth functions with compact support in
$D_{1}$ and $D_{2}$ are dense in $L^{2,\nu\circ f}(D_{1})$ and
$L^{2,\nu}(D_{2})$ respectively, it is sufficient to work with these functions
and the final conclusion will follow by density property and using partition
of unity.

Let $v\in L^{2,\nu\circ f}(D_{1})$ with very small support $U\subset
D_{1}\setminus\tilde{V}$ such that $f$ is one-one on $U$, where
$\tilde{V}=f^{-1}(V)$. Set $\tilde{U}=f(U)$, and let $F_{k_{0}}$ be the
inverse of $f$ on $\tilde{U}$. Observe that

$\langle\Lambda_{2}(v),\Lambda_{2}(v)\rangle_{\nu}=\int_{D_{2}}\lvert\Lambda_{2}(v)(w)\rvert^{2}\nu(w)dA(w)\leq
m\int_{D_{2}}\sum_{k=1}^{m}\lvert F_{k}^{\prime}(w)\rvert^{2}\lvert v\circ
F_{k}(w)\rvert^{2}\nu(w)dA(w).$

Since $F_{k}(\tilde{U})\cap U\neq\emptyset$ if and only if $k=k_{0}$,
therefore

$\langle\Lambda_{2}(v),\Lambda_{2}(v)\rangle_{\nu}\leq m\int_{\tilde{U}}\lvert
F_{k_{0}}^{\prime}(w)\rvert^{2}\lvert v\circ
F_{k_{0}}(w)\rvert^{2}\nu(w)dA(w).$

Set $F_{k_{0}}(w)=z$ on $\tilde{U}$, we get $w=f(z)$ and

$\langle\Lambda_{2}(v),\Lambda_{2}(v)\rangle_{\nu}\leq m\int_{U}\lvert
v(z)\rvert^{2}(\nu\circ f)(z)dA(z)=m\langle v,v\rangle_{\nu\circ f}<+\infty.$

Thus, $\Lambda_{2}$ is a bounded operator from $L^{2,\nu\circ f}(D_{1})$ into
$L^{2,\nu}(D_{2})$. Now if $v\in A^{2,\nu\circ f}(D_{1})$, clearly
$\Lambda_{2}(v)\in\mathcal{O}(D_{2}\setminus V)$. Since $V$ is discrete, Lemma
3.1 implies that $\Lambda_{2}(v)\in\mathcal{O}(D_{2})$, and hence
$\Lambda_{2}(v)\in A^{2,\nu}(D_{2})$.

Now let $u\in L^{2,\nu}(D_{2})$ with very small support $W\subset
D_{2}\setminus V$ such that $U_{k}:=F_{k}(W)$ are pairwise disjoint and
therefore $f$ is one-one on each $U_{k}$. Observe that

$\langle\Lambda_{1}(u),\Lambda_{1}(u)\rangle_{\nu\circ f}=\int_{D_{1}}\lvert
f^{\prime}(z)\rvert^{2}\lvert u\circ f(z)\rvert^{2}(\nu\circ
f)(z)dA(z)=\sum_{k=1}^{m}\int_{U_{k}}\lvert f^{\prime}(z)\rvert^{2}\lvert
u\circ f(z)\rvert^{2}(\nu\circ f)(z)dA(z).$

Since $f$ is one-one on each $U_{k}$, set $w=f(z)$ and we get

$\langle\Lambda_{1}(u),\Lambda_{1}(u)\rangle_{\nu\circ
f}=\sum_{k=1}^{m}\int_{W}\lvert u(w)\rvert^{2}\nu(w)dA(w)=m\langle
u,u\rangle_{\nu}<+\infty.$

Therefore $\Lambda_{1}$ is a bounded operator from $L^{2,\nu}(D_{2})$ into
$L^{2,\nu\circ f}(D_{1})$. Now if $u\in A^{2,\nu}(D_{2})$, clearly
$\Lambda_{1}(u)\in\mathcal{O}(D_{1})$, and thus $\Lambda_{1}(u)\in
A^{2,\nu\circ f}(D_{1})$.

∎

######  Lemma 3.3.

If $f:D_{1}\longrightarrow D_{2}$ is a proper holomorphic mapping between
bounded planar domains and $\nu$ is a weight on $D_{2}$ as given in Theorem
$1.6$, then

$\langle\Lambda_{1}u,v\rangle_{\nu\circ f}=\langle
u,\Lambda_{2}v\rangle_{\nu}$

for all $u\in L^{2,\nu}(D_{2})$ and $v\in L^{2,\nu\circ f}(D_{1})$.

###### Proof.

We shall prove the above result for all $u\in L^{2,\nu}(D_{2})$ supported in
small neighbourhoods of points in $D_{2}\setminus V$ and for all $v\in
L^{2,\nu\circ f}(D_{1})$.

Let $D_{w_{0}}$ be a neighbourhood of $w_{0}\in D_{2}\setminus V$ such that
$F_{j}$’s for $1\leq j\leq m$ are well defined and $W_{j}:=F_{j}(D_{w_{0}})$
are pairwise disjoint sets. Let $u\in L^{2,\nu}(D_{2})$ be such that support
of $u$ is contained in $D_{w_{0}}$. Therefore, support of $\Lambda_{1}u$ is
contained in $\bigcup_{j=1}^{m}W_{j}$, where $\Lambda_{1}u=f^{\prime}(u\circ
f)$. Now see that

$\displaystyle\langle\Lambda_{1}u,v\rangle_{\nu\circ f}$ $\displaystyle=$
$\displaystyle\int_{D_{1}}(\Lambda_{1}u)\,\bar{v}\,(\nu\circ f)$
$\displaystyle=$
$\displaystyle\sum_{j=1}^{m}\int_{W_{j}}(\Lambda_{1}u)\,\bar{v}\,(\nu\circ f)$
$\displaystyle=$ $\displaystyle\sum_{j=1}^{m}\int_{W_{j}}f^{\prime}(u\circ
f)\,\bar{v}\,(\nu\circ f)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{m}\int_{D_{w_{0}}}u\,\overline{F_{j}^{\prime}(v\circ
F_{j})}\,\nu$ $\displaystyle=$
$\displaystyle\int_{D_{w_{0}}}u\sum_{j=1}^{m}\overline{F_{j}^{\prime}(v\circ
F_{j})}\,\nu$

Since $\sum_{j=1}^{m}\overline{F_{j}^{\prime}(v\circ
F_{j})}=\overline{\Lambda_{2}v}$, therefore

$\displaystyle\langle\Lambda_{1}u,v\rangle_{\nu\circ f}$ $\displaystyle=$
$\displaystyle\int_{D_{w_{0}}}u\sum_{j=1}^{m}\overline{F_{j}^{\prime}(v\circ
F_{j})}\,\nu$ $\displaystyle=$
$\displaystyle\int_{D_{w_{0}}}u\,\overline{\Lambda_{2}v}\,\nu$
$\displaystyle=$ $\displaystyle\int_{D_{2}}u\,\overline{\Lambda_{2}v}\,\nu$
$\displaystyle=$ $\displaystyle\langle u,\Lambda_{2}v\rangle_{\nu}.$

Hence the result follows by density argument as the linear span of all such
$u$’s is dense in $L^{2,\nu}(D_{2})$. ∎

Now we have the following proposition about weighted reduced Bergman spaces:

######  Proposition 3.4.

If $f:D_{1}\longrightarrow D_{2}$ is a proper holomorphic mapping between
bounded planar domains and $\nu$ is a weight on $D_{2}$ as in Theorem 1.6,
then

$\tilde{\Lambda}_{1}(u):=\Lambda_{1}|_{\mathcal{D}^{\nu}(D_{2})}(u)\in\mathcal{D}^{\nu\circ
f}(D_{1})\;\;\text{and}\;\;\tilde{\Lambda}_{2}(v):=\Lambda_{2}|_{\mathcal{D}^{\nu\circ
f}(D_{1})}(v)\in\mathcal{D}^{\nu}(D_{2})$

for all $u\in\mathcal{D}^{\nu}(D_{2})$ and $v\in\mathcal{D}^{\nu\circ
f}(D_{1})$.

###### Proof.

Let $u\in\mathcal{D}^{\nu}(D_{2})$. Therefore we have $u=\tilde{u}^{\prime}$
for some $\tilde{u}\in\mathcal{O}(D_{2})$. By definition,

$\tilde{\Lambda}_{1}(u)=\Lambda_{1}(u)=f^{\prime}(u\circ
f)=f^{\prime}(\tilde{u}^{\prime}\circ f)=(\tilde{u}\circ f)^{\prime}.$

Since $\tilde{u}\circ f\in\mathcal{O}(D_{1})$, we get that for all
$u\in\mathcal{D}^{\nu}(D_{2})$,

$\tilde{\Lambda}_{1}(u)\in\mathcal{D}^{\nu\circ f}(D_{1}).$

Now let $v\in\mathcal{D}^{\nu\circ f}(D_{1})$ be arbitrary. Therefore
$v=\tilde{v}^{\prime}$ for some $\tilde{v}\in\mathcal{O}(D_{1})$. Observe that

$\tilde{\Lambda}_{2}(v)=\Lambda_{2}(v)=\sum_{k=1}^{m}F_{k}^{\prime}(v\circ
F_{k})=\sum_{k=1}^{m}\left(\tilde{v}\circ
F_{k}\right)^{\prime}=\left(\sum_{k=1}^{m}\tilde{v}\circ
F_{k}\right)^{\prime}.$

Consider the function $h=\sum_{k=1}^{m}\tilde{v}\circ F_{k}$. It is easy to
see that $h\in\mathcal{O}(D_{2}\setminus V)$ where $V$ is the set of critical
values of $f$. Since $V$ is a discrete set and $\tilde{\Lambda}_{2}(v)\in
L^{2,\nu}(D_{2})$ i.e. $h^{\prime}\in L^{2,\nu}(D_{2})$, it follows from Lemma
$3.1$ that $h^{\prime}$ has removable singularities at all points in $V$. Now
by Lemma 2.4, $h$ also has removable singularities at all points of $V$.
Therefore $h\in\mathcal{O}(D_{2})$, and hence for all
$v\in\mathcal{D}^{\nu\circ f}(D_{1})$ we have

$\tilde{\Lambda}_{2}(v)=h^{\prime}\in\mathcal{D}^{\nu}(D_{2}).$

∎

######  Proposition 3.5.

If $f:D_{1}\longrightarrow D_{2}$ is a proper holomorphic mapping between
bounded planar domains and $\nu$ is a weight on $D_{2}$ as in Theorem 1.6,
then

$\langle\tilde{\Lambda}_{1}u,v\rangle_{\nu\circ f}=\langle
u,\tilde{\Lambda}_{2}v\rangle_{\nu}$

for all $u\in\mathcal{D}^{\nu}(D_{2})$ and $v\in\mathcal{D}^{\nu\circ
f}(D_{1})$.

###### Proof.

Since $\tilde{\Lambda}_{i}$ is just the restriction of $\Lambda_{i}$ on
$\mathcal{D}^{\nu_{j}}(D_{j})$, where $j\neq i$ and $\nu_{1}=\nu\circ f$,
$\nu_{2}=\nu$, this result holds trivially from Lemma 3.3 and Proposition 3.4.
∎

###### Proof of Theorem 1.6.

Let $V=\\{f(z):f^{\prime}(z)=0\\}$. Since f is proper, f can not be
identically constant on $D_{1}$. Therefore, $V$ is a discrete subset of
$D_{2}$. For a fixed $w\in D_{2}\setminus V$, consider the function

$G(z)=\sum_{k=1}^{m}\tilde{K}^{\nu\circ
f}_{1}(z,F_{k}(w))\overline{F_{k}^{\prime}(w)},\quad z\in D_{1}.$

Since $\tilde{K}^{\nu\circ f}_{1}(\cdot,\zeta)\in\mathcal{D}^{\nu\circ
f}(D_{1})$ for all $\zeta\in D_{1}$, we observe that
$G\in\mathcal{D}^{\nu\circ f}(D_{1})$. Now using the reproducing property of
the reduced Bergman kernel, we get for an arbitrarily chosen
$v\in\mathcal{D}^{\nu\circ f}(D_{1})$

$\displaystyle\langle v,G\rangle_{\nu\circ f}$ $\displaystyle=$
$\displaystyle\sum_{k=1}^{m}F_{k}^{\prime}(w)\,\langle v,\tilde{K}^{\nu\circ
f}_{1}(\cdot,F_{k}(w))\rangle_{\nu\circ f}$ $\displaystyle=$
$\displaystyle\sum_{k=1}^{m}F_{k}^{\prime}(w)\,v(F_{k}(w))$ $\displaystyle=$
$\displaystyle(\tilde{\Lambda}_{2}v)(w)$ $\displaystyle=$
$\displaystyle\langle\tilde{\Lambda}_{2}v,\tilde{K}^{\nu}_{2}(\cdot,w)\rangle_{\nu}$
$\displaystyle=$ $\displaystyle\langle v,\tilde{\Lambda}_{1}u\rangle_{\nu\circ
f}\quad\quad\quad\quad\text{(by Proposition 3.5)}$

where $u=\tilde{K}^{\nu}_{2}(\cdot,w)$. So, $\langle v,G\rangle_{\nu\circ
f}=\langle v,\tilde{\Lambda}_{1}u\rangle_{\nu\circ f}$ for all
$v\in\mathcal{D}^{\nu\circ f}(D_{1})$. Therefore, $\tilde{\Lambda}_{1}u=G$.
Thus, we have proved that

$f^{\prime}(z)\tilde{K}^{\nu}_{2}(f(z),w)=\sum_{k=1}^{m}\tilde{K}^{\nu\circ
f}_{1}(z,F_{k}(w))\overline{F_{k}^{\prime}(w)}\quad\quad\text{for }z\in
D_{1},\;w\in D_{2}\setminus V.$

Since LHS is anti-holomorphic in $w$ and $V$ is discrete, the points in $V$
are removable singularities of RHS. Hence, the formula holds everywhere by
continuity. ∎

##  4\. Proof of Theorem 1.7

Let $\tilde{K}(z,w)$ and $K(z,w)$ denote the reduced Bergman kernel functions
associated to $D$ and $\mathbb{D}$ respectively, where $D$ is a bounded planar
domain and $\mathbb{D}$ is the unit disc in $\mathbb{C}$. For a proper map
$f:D\longrightarrow\mathbb{D}$, let $\\{F_{k}\\}_{k=1}^{m}$ denote the local
inverses of $f$ defined on $\mathbb{D}\setminus V$ where $m$ is the
multiplicity of $f$ and $V$ is the set of all critical values of $f$.
According to Corollary 1.3, the kernel functions transform according to

$f^{\prime}(z)K(f(z),w)=\sum_{k=1}^{m}\tilde{K}(z,F_{k}(w))\overline{F_{k}^{\prime}(w)},\quad
z\in D,w\in\mathbb{D}.$

For fixed $w\in\mathbb{D}$ and $\alpha\in\mathbb{N}\cup\\{0\\}$, define a
linear functional $\Lambda$ on the Bergman space of $D$ i.e. $A^{2}(D)$ by

$\Lambda(h)=\partial^{\alpha}\bigg{(}\sum_{k=1}^{m}F^{\prime}_{k}(h\circ
F_{k})\bigg{)}(w),$

where $\partial^{\alpha}=\frac{\partial^{\alpha}}{\partial w^{\alpha}}$
denotes the standard holomorphic differential operator of order $\alpha$.
Following is a lemma by Bell about $\Lambda$:

######  Lemma 4.1 (Bell, [4]).

Let $\xi_{1},\xi_{2},\ldots,\xi_{q}$ denote the points in $f^{-1}(w)$. There
exist a positive integer $s$ and constants $c_{l,\beta}$ such that:

$\Lambda(h)=\sum_{l=1}^{q}\sum_{\beta\leq
s}c_{l,\beta}\partial^{\beta}h(\xi_{l})$

for all $h\in A^{2}(D)$.

We now prove Theorem $1.7$ using the above lemma.

###### Proof.

Let $f^{-1}(0)=\\{\zeta_{1},\cdots,\zeta_{q}\\}$. For the linear functional
$\Lambda$ on $A^{2}(D)$, as defined above, corresponding to
$\alpha=1\in\mathbb{N}\cup\\{0\\}$ and $0\in\mathbb{D}$, i.e.

$\Lambda(h)=\frac{\partial}{\partial
w}\left(\sum_{k=1}^{m}F^{\prime}_{k}(h\circ F_{k})\right)(0)$

the above lemma gives a positive integer $s>0$ and constants $c_{l,\beta}$
such that

$\frac{\partial}{\partial w}\left(\sum_{k=1}^{m}F^{\prime}_{k}(h\circ
F_{k})\right)(0)=\sum_{l=1}^{q}\sum_{\beta\leq
s}c_{l,\beta}\partial^{\beta}h(\zeta_{l})$

for all $h\in A^{2}(D)$.

Therefore, on differentiating the transformation formula for the reduced
Bergman kernels under $f$ with respect to $\bar{w}$ and setting $w=0$, we get

$\displaystyle f^{\prime}(z)\frac{\partial}{\partial\bar{w}}K(f(z),0)$
$\displaystyle=$
$\displaystyle\frac{\partial}{\partial\bar{w}}\left(\sum_{k=1}^{m}\tilde{K}(z,F_{k}(\cdot))\overline{F_{k}^{\prime}(\cdot)}\right)(0)$
$\displaystyle=$ $\displaystyle\overline{\frac{\partial}{\partial
w}\left(\sum_{k=1}^{m}\overline{\tilde{K}(z,F_{k}(\cdot))}F_{k}^{\prime}(\cdot)\right)}(0)$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{q}\sum_{\beta\leq
s}\bar{c}_{l,\beta}\overline{\partial^{\beta}\overline{\tilde{K}(z,\zeta_{l})}}$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{q}\sum_{\beta\leq
s}\bar{c}_{l,\beta}\,\overline{\partial}^{\beta}\tilde{K}(z,\zeta_{l}),$

where
$\bar{\partial}^{\beta}=\frac{\partial^{\beta}}{\partial\bar{w}^{\beta}}$.
Since $\tilde{K}$ is a rational function,
$f^{\prime}(z)\frac{\partial}{\partial\bar{w}}K(f(z),0)$ is therefore a
rational function in $z$. Similarly, taking $\alpha=0$ proves that
$f^{\prime}(z)K(f(z),0)$ is a rational function in $z$. Note that for disc
$\mathbb{D}$, the reduced Bergman kernel is equal to the Bergman kernel as
$\mathcal{D}(\mathbb{D})=A^{2}(\mathbb{D})$. Therefore,

$K(z,w)=\frac{1}{\pi}\frac{1}{(1-z\bar{w})^{2}}.$

So, $\frac{\partial}{\partial\bar{w}}K(z,0)=\frac{2z}{\pi}$ and
$K(z,0)=\frac{1}{\pi}$. Thus,

$\frac{f^{\prime}(z)\frac{\partial}{\partial\bar{w}}K(f(z),0)}{f^{\prime}(z)K(f(z),0)}=2f(z).$

Hence, $f$ is a rational function. ∎

## References

  * [1] Eric Bedford and Steven Bell, _Holomorphic correspondences of bounded domains in ${\bf C}^{n}$_, Complex analysis (Toulouse, 1983), Lecture Notes in Math., vol. 1094, Springer, Berlin, 1984, pp. 1–14. MR 773097
  * [2] Steven R. Bell, _Proper holomorphic mappings and the Bergman projection_ , Duke Math. J. 48 (1981), no. 1, 167–175. MR 610182
  * [3] by same author, _The Bergman kernel function and proper holomorphic mappings_ , Trans. Amer. Math. Soc. 270 (1982), no. 2, 685–691. MR 645338
  * [4] by same author, _Proper holomorphic mappings that must be rational_ , Trans. Amer. Math. Soc. 284 (1984), no. 1, 425–429. MR 742433
  * [5] by same author, _The Cauchy transform, potential theory, and conformal mapping_ , Studies in Advanced Mathematics, CRC Press, Boca Raton, FL, 1992. MR 1228442
  * [6] Makoto Sakai, _The sub-mean-value property of subharmonic functions and its application to the estimation of the Gaussian curvature of the span metric_ , Hiroshima Math. J. 9 (1979), no. 3, 555–593. MR 549663


# A Mixed Precision, Multi-GPU Design for Large-scale Top-K Sparse
Eigenproblems

Francesco Sgherzi1, Alberto Parravicini2, Marco D. Santambrogio2
<EMAIL_ADDRESS>2{alberto.parravicini,
<EMAIL_ADDRESS>Politecnico di Milano, DEIB, Milan, Italy

###### Abstract

Graph analytics techniques based on spectral methods process extremely large
sparse matrices with millions or even billions of non-zero values. Behind
these algorithms lies the Top-K sparse eigenproblem, the computation of the
largest eigenvalues and their associated eigenvectors. In this work, we
leverage GPUs to scale the Top-K sparse eigenproblem to bigger matrices than
previously achieved while also providing state-of-the-art execution times. We
can transparently partition the computation across multiple GPUs, process out-
of-core matrices, and tune precision and execution time using mixed-precision
floating-point arithmetic. Overall, we are 67$\times$ faster than the highly
optimized ARPACK library running on a 104-thread CPU and 1.9$\times$ than a
recent FPGA hardware design. We also determine how mixed-precision floating-
point arithmetic improves execution time by 50 % over double-precision, and is
12$\times$ more accurate than single-precision floating-point arithmetic.

##  I Introduction

Modern data science and numerical mathematics applications operate on larger
and larger data, often with strict requirements of minimizing execution time
and power consumption. For many of these applications, hardware accelerators
such as Graphics Processing Units (GPUs) and Field Programmable Gate Arrays
(FPGAs) are a highly-effective solution, especially when mixed-precision and
reduced-precision arithmetic come into play [1, 2, 3, 4, 5, 6]. As spectral
methods become ubiquitous in the large scale graph pipelines of Spectral
Clustering [7], Information Retrieval (IR) [8] and ranking [9], such
techniques require algorithms that can compute only a subset of the most
relevant eigenvalues (i.e. the largest in modulo) and their associated
eigenvectors while taking advantage of the sparsity of real-world graphs.
Graph analytics pipelines usually operate on graphs with millions or even
billions of edges, rendering traditional methods computing all eigenvalues
impractical, as their space and time cost scales at least quadratically with
the number of vertices. As the size of real-world graphs exceeds the device
memory size of modern GPUs, an eigensolver must be able to handle out-of-core
matrices as well as be capable of distributing the computation to multiple
GPUs. Moreover, they need to support different numerical data types for
storage and computation to meet storage and accuracy requirements.

In this work, we introduce a novel Top-K GPU eigensolver for real-valued,
sparse matrices capable of handling matrices with billion of non-zero entries,
of partitioning the computation across multiple GPUs and leveraging mixed-
precision arithmetic to optimize accuracy and execution time. We compare how
our eigensolver against state-of-the-art CPU and FPGA implementations and
investigate how mixed-precision enables intermediate operations with higher
precision while results are stored with space-efficient representations.

In summary, we present the following contributions:

  * •

A multi-GPU, mixed precision, Top-K eigensolver that can process out-of-core
sparse matrices with billions of non-zeros. To the best of our knowledge, this
is the largest amount reported in the literature (Section III).

  * •

A performance evaluation of our GPU eigensolver against state-of-the-art Top-K
eigensolvers on multiple architectures. We are on average 67$\times$ faster
than a multi-core CPU implementation and 1.9$\times$ faster than an FPGA
hardware design, with average reconstruction error below $10^{-5}$ (Section
IV-B).

  * •

A characterization of mixed-precision arithmetic in terms of accuracy versus
execution time. We prove how mixed-precision is 50 % faster than double-
precision floating-point arithmetic, and 12$\times$ more accurate than single-
precision (Section IV-D).

Figure 1: High-level design of our Top-K sparse eigensolver, divided into the
Lanczos and Jacobi algorithms.

##  II Related Work

Although solving Top-K sparse eigenproblems is computationally demanding and
has strong practical applications, little prior research optimizes them with
hardware accelerators. On GPUs, most Top-K eigensolvers are domain-specific,
do not support large-scale inputs and multiple devices, or are outright not
supported on modern GPUs architectures [10, 11, 12]. The nvGRAPH library [13]
by Nvidia uses internally the Lanczos algorithm, whose implementation is,
however, not user-accessible. Mixed precision arithmetic on numerical
algorithms on GPUs has been evaluated on multiple algorithms from the QUDA
library. However, the effectiveness of mixed and reduced precision on the
numerically unstable Lanczos algorithm is still unknown [14, 3].

Custom hardware designs for Top-K sparse eigensolvers have been recently
investigated by Sgherzi et al. [6], who prototyped their work on high-end
FPGAs equipped with High Bandwidth Memory (HBM). They investigates the role of
mixed-precision arithmetic for Top-K sparse eigensolvers, but the proposed
design does not scale to multiple devices or large out-of-core matrices.
Moreover, limitations of the FPGA’s HBM controller force unnecessary data
replication and allow achieving only a fraction of the maximum HBM bandwidth.
To the best of our knowledge, no other work optimizes large Top-K sparse
eigencomputations using FPGAs or Domain Specific Architectures (DSAs). There
are numerous implementations of large-scale Top-K sparse eigenproblem solver
for CPUs [15, 16, 17]. However, none is as well-known as ARPACK [18], a multi-
core Fortran library that implements the Implicitly Restarted Arnoldi Method
(IRAM), a variation of the Lanczos algorithm with support for non-Hermitian
matrices.

##  III Implementation

Our sparse eigensolver employs a two-phase algorithm, as in Figure 1. The
first step is based on the Lanczos [19] algorithm, which takes as input the
original matrix $\mathbf{M}$, the number of desired eigencomponents $K$ and a
L2-normalized random vector $v_{1}\in\mathbb{R}^{n}$. The algorithm proceeds
by incrementally building a Kyrlov subspace (the Lanczos vectors
$\mathcal{V}$) of $\mathbf{M}$, through vector projections (Algorithm 1 line
9) and orthogonalizations (lines 11–21). A tridiagonal matrix $\mathcal{T}$
stores the residuals of the previous operations (lines 6, 10) and reduces the
problem from size $n\times n$ to one of size $K\times K$ ($K\ll n$).

Algorithm 1 Top-K eigenvalues/vectors Lanczos algorithm

1:Input Matrix $\mathbf{M}$, partitioned in
$\mathbf{M_{1}}\ldots\mathbf{M_{G}}$

2:$K$, number of output eigenvectors

3:$L2$-normalized input vectors $v_{1}:=\\{v_{1}^{1},\ldots,v_{1}^{G}\\}$

4:Temporary vectors $v_{tmp}$ and next vectors $v_{nxt}$

5:function Lanczos($\mathbf{M},K,v_{1},v_{tmp},v_{nxt}$)

6: $\alpha_{1}\leftarrow 0;\quad\beta_{1}\leftarrow 0$ $\triangleright$
Initialization

7: for $i$ in $1,K$ do

8: $\triangleright$ Normalize and compute new Lanczos vector

9: if $i\neq 1$ then

10: $\beta_{i}\leftarrow\sqrt{\sum_{j=1}^{G}{(v_{nxt}^{j})^{2}}}$
$\triangleright$ $\beta_{i}\leftarrow$ $\|v_{nxt}\|_{2}$

11: $v_{i}^{[1\ldots G]}\leftarrow\forall_{j\in[1\ldots
G]}(v_{nxt}^{j}/\beta_{i})$ $\triangleright$ $v_{i}\leftarrow
v_{nxt}/\beta_{i}$

12: $\triangleright$ Compute the next projection

13: $v_{t}^{[1\ldots G]}\leftarrow
SpMV(\mathbf{M_{1}}\ldots\mathbf{M_{G}},v_{i}^{[1\ldots G]})$

14: $\alpha_{i}\leftarrow\sum_{j=1}^{G}v_{i}^{j}\cdot v_{t}^{j}$

15: $v_{nxt}^{[1\ldots G]}\leftarrow v_{tmp}^{[1\ldots
G]}-\alpha_{i}v_{i}^{[1\ldots G]}-\beta_{i}v_{i-1}^{[1\ldots G]}$

16: for $j\in[1,i]$ do $\triangleright$ Orthogonalization

17: if $j\%2\neq 0$ then

18: $o\leftarrow\sum_{k=1}^{G}v_{j,k}\cdot v_{t,k}$

19: $v_{t}^{[1\ldots G]}\leftarrow v_{t}^{[1\ldots G]}-o\cdot v_{j}^{[1\ldots
G]}$

20: else

21: $o\leftarrow\sum_{k=1}^{G}v_{j,k}\cdot v_{n,k}$

22: $v_{n}^{[1\ldots G]}\leftarrow v_{n}^{[1\ldots G]}-o\cdot v_{j}^{[1\ldots
G]}$

23: if $i==j$ then $\triangleright$ Copy to temporary vector

24: $v_{t}^{[1\ldots G]}\leftarrow v_{n}^{[1\ldots G]}$

25: $\triangleright$ Tridiagonal matrix $\mathcal{T}$ and Lanczos vectors
$\mathcal{V}$

26: return
$\\{\mathcal{T}=[\alpha_{1},\ldots,\alpha_{K}],[\beta_{1},\ldots,\beta_{K-1}]\\}$

27: return $\mathcal{V}=[v_{1},\ldots,v_{K}]$

The second phase employs the Jacobi algorithm [20] to solve the eigenproblem
on the much smaller matrix $\mathcal{T}$. The Jacobi algorithm stores the
eigenvalues of $\mathbf{M}$ in the main diagonal of $\mathcal{T}$, and the
eigenvectors of $\mathcal{T}$ in $\mathbf{V}$. The eigenvectors of
$\mathbf{M}$ are given by $\mathcal{V}\mathbf{V}$.

###  III-A Optimizing sparse eigensolvers for GPUs

We desire to render our GPU sparse eigensolver scalable to real-world matrices
with billions of non-zero values, often encountered in graph analytics. To do
so, we devise a workload partition scheme that distributes the computation
across multiple GPUs while minimizing unnecessary data movements and
synchronization events.

The Lanczos algorithm has two synchronization points (Algorithm 1, lines 6 and
10), corresponding to the computation of $\alpha$ and $\beta$ (Figure 1 A B).
Another optional synchronization point occurs if reorthogonalization of the
Lanczos vectors is needed (lines 15–18, Figure 1 C). All other operations
operate linearly on the input arrays and can be computed across multiple GPUs
independently. The input matrix is partitioned by balancing the number of non-
zero elements in each partition. All vectors, except for $v_{i}$, are
partitioned according to the same partition scheme as the input matrix. As the
Sparse Matrix-Vector Multiplication (SpMV) performs indirect accesses to the
vector $v_{i}$, we replicate it to all GPU instead of partitioning it. There
is an additional synchronization at each iteration when the previous Lanczos
vector becomes the input of the SpMV. We prevent this synchronization by
having each GPU copy, in a round-robin fashion, a single partition to a single
replica of $v_{i}$ (Figure 1 C). When all GPUs have completed a cycle, $v_{i}$
has been fully copied, and the computation can proceed to a new iteration.

The orthogonality of the Lanczos vectors and the quality of the final
eigencomponents produced crucially depends on the output of the scalar product
($\alpha$, line 10) and the L2-norm ($\beta$, line 6). For this reason, our
eigensolver can perform the intermediate operations of each kernel in double-
precision floating-point arithmetic to ensure maximum accuracy. However,
vectors can still be stored in single-precision floating-point arithmetic, to
consume less device memory and better use the available memory bandwidth. In
our experiments, other data types (half-precision FP16, BFLOAT16) resulted in
numerical instability, and have been omitted from Section IV.

Figure 2: Speedup (log-scale, higher is better) of our GPU Top-K sparse
eigensolver versus the ARPACK multi-core CPU library, and the FPGA
implementation in Sgherzi et al. [6]. Our GPU implementations runs on a single
GPU. The two largest matrices (KRON and URAND) are not supported by the FPGA
implementation.

###  III-B Implementation details

We implemented our eigensolver using the GrCUDA API [21] and GraalVM [22] to
support several high-level programming languages automatically, while we wrote
the core GPU computational kernels in CUDA. Since GrCUDA internally leverages
CUDA unified memory, our eigensolver can scale to out-of-core computations on
sparse matrices that would not otherwise fit in the GPU memory. We also
modified the internal GrCUDA runtime to schedule GPU kernels across multiple
devices, using a round-robin device selection policy for kernels operating on
disjoint data. Through the partition swapping presented in Section III-A, we
minimize unnecessary memory transfers between devices and out-of-core memory
pages, guaranteeing scalability in what would be an otherwise memory and
transfer-bound computation (Section IV-C). The small tridiagonal matrices that
the Lanczos algorithm outputs ($\approx 24\times 24$) cannot saturate the
stream processors of a modern GPU [23]. Instead, we achieve better execution
time by performing this step on a CPU (Figure 1 D).

TABLE I: Sparse matrices used in our evaluation, by increasing number of non-
zero entries (in millions). We also report the memory footprint in
$\text{\,}\mathrm{GB}$ of each matrix, stored as COO.

ID |  Name |  Rows (M) |  Non-zeros (M) |  Sparsity (%) |  Size (GB)
---|---|---|---|---|---
WB-TA | wiki-Talk | 2.39 | 5.02 | $8.79\text{\times}{10}^{-04}\text{\,}$ | $0.06\text{\,}\mathrm{GB}$
WB-GO | web-Google | 0.91 | 5.11 | $6.17\text{\times}{10}^{-04}\text{\,}$ | $0.07\text{\,}\mathrm{GB}$
WB-BE | web-Berkstan | 0.69 | 7.60 | $1.60\text{\times}{10}^{-03}\text{\,}$ | $0.10\text{\,}\mathrm{GB}$
FL | Flickr | 0.82 | 9.84 | $1.46\text{\times}{10}^{-03}\text{\,}$ | $0.13\text{\,}\mathrm{GB}$
IT | italy_osm | 6.69 | 14.02 | $3.13\text{\times}{10}^{-05}\text{\,}$ | $0.18\text{\,}\mathrm{GB}$
PA | patents | 3.77 | 14.97 | $1.05\text{\times}{10}^{-04}\text{\,}$ | $0.19\text{\,}\mathrm{GB}$
VL3 | venturiLevel3 | 4.02 | 16.10 | $9.96\text{\times}{10}^{-05}\text{\,}$ | $0.21\text{\,}\mathrm{GB}$
DE | germany_osm | 11.54 | 24.73 | $1.86\text{\times}{10}^{-05}\text{\,}$ | $0.32\text{\,}\mathrm{GB}$
ASIA | asia_osm | 11.95 | 25.42 | $1.78\text{\times}{10}^{-05}\text{\,}$ | $0.33\text{\,}\mathrm{GB}$
RC | road_central | 14.08 | 33.87 | $1.71\text{\times}{10}^{-05}\text{\,}$ | $0.43\text{\,}\mathrm{GB}$
WK | Wikipedia | 3.56 | 45.00 | $3.55\text{\times}{10}^{-4}\text{\,}$ | $0.60\text{\,}\mathrm{GB}$
HT | hugetrace-00020 | 16.00 | 47.80 | $1.87\text{\times}{10}^{-05}\text{\,}$ | $0.61\text{\,}\mathrm{GB}$
WB | wb-edu | 9.84 | 57.15 | $5.90\text{\times}{10}^{-05}\text{\,}$ | $0.73\text{\,}\mathrm{GB}$
KRON | GAP-kron | 134.21 | 4223.26 | $2.34\text{\times}{10}^{-05}\text{\,}$ | $50.67\text{\,}\mathrm{GB}$
URAND | GAP-urand | 134.21 | 4294.96 | $2.39\text{\times}{10}^{-05}\text{\,}$ | $51.54\text{\,}\mathrm{GB}$

##  IV Experimental Evaluation

We evaluate the quality of our sparse eigensolver in terms of execution time
and results’ quality, and provide a performance characterization against
state-of-the-art sparse eigensolvers running on different hardware
architectures. We provide an in-depth evaluation over a single GPU, and
validate the scalability of our algorithm over multiple GPUs (up to 8). Most
importantly, we assess the impact of mixed-precision arithmetic and prove how
reduced precision results in faster execution time with no meaningful
detriment to accuracy.

###  IV-A Experimental Setup

We measure results for our Top-K sparse eigensolver using up to 8 Nvidia Tesla
V100s ($16\text{\,}\mathrm{GB}$ of HBM2 for each GPU). As baselines, we employ
the multi-threaded ARPACK library [18], a Top-K sparse eigensolver that uses
the IRAM algorithm, running on two Intel Xeon Platinum 8167M (104 threads in
total) and $755\text{\,}\mathrm{GB}$ of DDR4 memory, with single-precision
floating point arithmetic. We also compare against the recent FPGA
implementation by Sgherzi et al. [6], running on a Xilinx Alveo U280
accelerator card equipped with $8\text{\,}\mathrm{GB}$ of HBM2 memory. We
repeat measurements 20 times, using random initialization for the Lanczos
vectors $v_{1}$.

To provide a fair comparison, we use the same collection of sparse matrices in
Sgherzi et al. [6], enriched with two extremely large matrices (billions of
non-zero entries) that do not fit in the FPGA’s and GPU’s device memory, and
allows us to test the out-of-core performance of our GPU implementation. All
matrices come from the SuiteSparse collection [24] and represent graph
topologies, although the eigensolvers in our analysis can be applied to other
domains as well [25].

###  IV-B Execution Time Comparison

We first compare the speed of our GPU eigensolver, when running on a single
GPU, against the CPU and FPGA baselines, on matrices of increasing size
(Figure 2). Results have been aggregated over an increasing amount of
eigenvectors $K$, from 8 to 24, as the execution time scales linearly with
$K$. For the FPGA implementation, we use the values reported by the authors.
Results of the two largest matrices (KRON and URAND) have been omitted for the
FPGA hardware design as it does not support out-of-core computations.

CPU and GPU use single-precision floating point arithmetic, while the FPGA
hardware design uses 32-bit signed fixed point arithmetic with one bit of
integer part (S1.1.30) for Lanczos, and half-precision floating point
arithmetic for Jacobi.

Our GPU eigensolver is always faster than both the CPU and FPGA baselines (on
the RC matrix the difference is not statistically significant). On average,
the GPU eigensolver is 67$\times$ faster than the CPU implementation and
1.9$\times$ faster than the FPGA hardware design. The FPGA hardware design is
still competitive in terms of Performance/Watt, as the FPGA design consumes
38W [6], versus the 300W of the GPU [26].

As our partitioning minimizes inter-GPU data-transfer (Section III-A), we are
$\approx$ 180$\times$ faster than the CPU on very large out-of-core matrices
despite storing only a small fraction of the input data on the GPU at any
given time.

Figure 3: A Relative execution time for increasing number of GPUs. Two outlier
matrices (plotted separately) perform worse with more GPUs due to the larger
inter-GPU communication. B Accuracy of our eigensolver, in terms of
orthogonality of the eigenvectors and L2 error, for increasing $K$.

###  IV-C Multi-GPU Performance

Scaling the computation of the Top-K eigenvectors on sparse matrices to
multiple GPUs is far from trivial, as explained in Section III-A. From Figure
3, we observe how our partitioning scheme improves the execution time when
using multiple GPUs, with somewhat diminishing returns. On average, two GPUs
provide a 50 % speedup, while eight GPUs are close to a 100 % speedup. Only on
two very small matrices we observe a loss of performance on systems with 4 or
8 GPUs. This phenomenon is explained by the heterogeneous NVLink
interconnection found in V100-based systems like ours [27]. Some GPU pairs are
not directly connected with NVLink, and data transfer has to go through the
CPU and PCIe, which has $\approx$ 10$\times$ lower bandwidth than NVLink.

Figure 4: L2 reconstruction error vs. execution time, divided by graph and
data-type configuration. The mixed-precision float-double-float configuration
is 50 % faster than double-precision and has 12$\times$ lower error than pure
single-precision.

###  IV-D Impact of Reorthogonalization and Mixed-precision

To measure the quality of our eigensolver, we measure the average angle that
occurs between every pair of eigenvectors. Eigenvectors are by definition
pairwise orthogonal, i.e. their angle is $\pi/2$, and their dot product must
be 0. Figure 3 provides, for increasing $K$, the average orthogonality and the
L2 norm of $\mathbf{M}v-\lambda v$, the reconstruction error computed using
the definition of eigenvalues. Both results are aggregated for all matrices
due to space limitations. We observe how reorthogonalization improves the
results’ quality, with $\approx$ 2 degrees of difference compared to the
implementation without reorthogonalization. Choosing whether
reorthogonalization is suitable or not depends on the application. Spectral
methods in machine learning often do not demand the same numerical accuracy as
engineering applications, and reorthogonalization increases the algorithmic
complexity by an $\mathcal{O}(nK^{2}/2)$ factor.

Employing mixed-precision arithmetic in numerical algorithms is usually a
matter of trade-offs, with better precision translating to higher execution
time. We visualize this behavior in Figure 4, showing for each matrix the L2
reconstruction error and the relative execution time, and a linear regression
to capture the general trend. In all cases, increasing precision reduces the
error and increases the execution time. The float-double-float (FDF)
configuration (Section III-A) is 50 % faster than a pure double-precision
implementation (DDD). Its error is only 40 % higher, and 12$\times$ lower than
the floating-point implementation (FFF), showing how mixed-precision
arithmetic is a great compromise in Top-K sparse eigensolvers.

##  V Conclusion

As graph analytics and spectral methods deal with larger and larger sparse
matrices, it is critical to have high-performance Top-K sparse eigensolvers to
extract low-dimensional representations of sparse datasets. We provide a novel
GPU Top-K sparse eigensolver that can scale to out-of-core matrices with
billion of non-zero entries, partition the computation over multiple GPUs, and
leverage mixed-precision floating-point arithmetic. We are on average
67$\times$ faster than the multi-core ARPACK CPU library implementation and
1.9$\times$ faster than a state-of-the-art FPGA hardware design. As future
work, we will extend our implementation to fixed-point arithmetic and validate
if novel interconnection technologies such as NVSwitch can improve even
further multi-GPU scaling.

## References

  * [1] M. A. Clark, R. Babich, K. Barros, R. C. Brower, and C. Rebbi, “Solving lattice qcd systems of equations using mixed precision solvers on gpus,” _Computer Physics Communications_ , vol. 181, no. 9, pp. 1517–1528, 2010\.
  * [2] S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, “Deep learning with limited numerical precision,” in _International conference on machine learning_. PMLR, 2015, pp. 1737–1746.
  * [3] M. A. Clark, A. Strelchenko, A. Vaquero, M. Wagner, and E. Weinberg, “Pushing memory bandwidth limitations through efficient implementations of block-krylov space solvers on gpus,” _Computer Physics Communications_ , vol. 233, pp. 29–40, 2018.
  * [4] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient processing of deep neural networks,” _Synthesis Lectures on Computer Architecture_ , vol. 15, no. 2, pp. 1–341, 2020.
  * [5] A. Parravicini, F. Sgherzi, and M. D. Santambrogio, “A reduced-precision streaming spmv architecture for personalized pagerank on fpga,” _arXiv preprint arXiv:2009.10443_ , 2020.
  * [6] F. Sgherzi, A. Parravicini, M. Siracusa, and M. D. Santambrogio, “Solving large top-k graph eigenproblems with a memory and compute-optimized fpga design,” in _2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)_. IEEE, 2021, pp. 78–87.
  * [7] A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in _Advances in neural information processing systems_ , 2002, pp. 849–856.
  * [8] A. N. Langville and C. D. Meyer, “A survey of eigenvector methods for web information retrieval,” _SIAM review_ , vol. 47, no. 1, pp. 135–161, 2005\.
  * [9] G. Dawson and R. Polikar, “Opinionrank: Extracting ground truth labels from unreliable expert opinions with graph-based spectral ranking,” _arXiv preprint arXiv:2102.05884_ , 2021.
  * [10] J. L. Aurentz, V. Kalantzis, and Y. Saad, “A gpu implementation of the filtered lanczos procedure,” Tech. Report ys-2015-4, Dept. Computer Science and Engineering, University, Tech. Rep., 2015.
  * [11] N. M. Evstigneev, “Implementation of implicitly restarted arnoldi method on multigpu architecture with application to fluid dynamics problems,” in _International Conference on Parallel Computational Technologies_. Springer, 2017, pp. 301–316.
  * [12] J. Dubois, C. Calvin, and S. Petiton, “Accelerating the explicitly restarted arnoldi method with gpus using an autotuned matrix vector product,” _SIAM Journal on Scientific Computing_ , 2011.
  * [13] Nvidia, “nvgraph,” 2019. [Online]. Available: docs.nvidia.com/cuda/nvgraph/index.html
  * [14] R. Babich, M. A. Clark, and B. Joó, “Parallelizing the quda library for multi-gpu calculations in lattice quantum chromodynamics,” in _SC’10: Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis_. IEEE, 2010, pp. 1–11.
  * [15] H. M. Aktulga, C. Yang, E. G. Ng, P. Maris, and J. P. Vary, “Topology-aware mappings for large-scale eigenvalue problems,” in _European Conference on Parallel Processing_. Springer, 2012, pp. 830–842.
  * [16] V. Hernandez, J. Roman, A. Tomas, and V. Vidal, “A survey of software for sparse eigenvalue problems,” _Universitat Politecnica De Valencia, SLEPs technical report STR-6_ , 2009.
  * [17] D. Lee, T. Hoshi, T. Sogabe, Y. Miyatake, and S.-L. Zhang, “Solution of the k-th eigenvalue problem in large-scale electronic structure calculations,” _Journal of Computational Physics_ , vol. 371, 2018.
  * [18] R. B. Lehoucq, D. C. Sorensen, and C. Yang, _ARPACK users’ guide: solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods_. SIAM, 1998.
  * [19] C. Lanczos, _An iteration method for the solution of the eigenvalue problem of linear differential and integral operators_ , 1950.
  * [20] H. Rutishauser, “The jacobi method for real symmetric matrices,” _Numerische Mathematik_ , vol. 9, no. 1, pp. 1–10, Nov 1966. [Online]. Available: https://doi.org/10.1007/BF02165223
  * [21] A. Parravicini, A. Delamare, M. Arnaboldi, and M. D. Santambrogio, “Dag-based scheduling with resource sharing for multi-task applications in a polyglot gpu runtime,” in _2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS)_. IEEE, 2021, pp. 111–120.
  * [22] T. Würthinger, C. Wimmer, A. Wöß, L. Stadler, G. Duboscq, C. Humer, G. Richards, D. Simon, and M. Wolczko, “One vm to rule them all,” in _Proceedings of the 2013 ACM international symposium on New ideas, new paradigms, and reflections on programming & software_, 2013, pp. 187–204.
  * [23] A. Cosnuau, “Computation on gpu of eigenvalues and eigenvectors of a large number of small hermitian matrices,” _Procedia Computer Science_ , vol. 29, pp. 800–810, 12 2014.
  * [24] T. A. Davis and Y. Hu, “The university of florida sparse matrix collection,” _ACM Transactions on Mathematical Software (TOMS)_ , vol. 38, no. 1, pp. 1–25, 2011. [Online]. Available: sparse.tamu.edu
  * [25] F. Tung, A. Wong, and D. A. Clausi, “Enabling scalable spectral clustering for image segmentation,” _Pattern Recognition_ , vol. 43, no. 12, pp. 4069–4076, 2010.
  * [26] Nvidia, “Nvidia tesla v100 gpu architecture,” https://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf, 2017, retrieved on 2021-10-25.
  * [27] A. Li, S. L. Song, J. Chen, J. Li, X. Liu, N. R. Tallent, and K. J. Barker, “Evaluating modern gpu interconnect: Pcie, nvlink, nv-sli, nvswitch and gpudirect,” _IEEE Transactions on Parallel and Distributed Systems_ , vol. 31, no. 1, pp. 94–110, 2019.


11institutetext: Department of Electrical Engineering, V.J.T.I, Mumbai, India.
22institutetext: 22email: {oabhilare_b19,rsingh_b18, vvparanjape_b18
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
# DEEPFAKE CLI: Accelerated Deepfake Detection using FPGAs

Omkar Bhilare These Authors contributed equally to this work1122 Rahul Singh0
1122 Vedant Paranjape0 1122 Sravan Chittupalli0 1122 Shraddha Suratkar 1122
Faruk Kazi 1122

###### Abstract

Because of the availability of larger datasets and recent improvements in the
generative model, more realistic Deepfake videos are being produced each day.
People consume around one billion hours of video on social media platforms
every day, and that’s why it is very important to stop the spread of fake
videos as they can be damaging, dangerous, and malicious. There has been a
significant improvement in the field of deepfake classification, but deepfake
detection and inference have remained a difficult task. To solve this problem
in this paper, we propose a novel DEEPFAKE C-L-I (Classification –
Localization – Inference) in which we have explored the idea of accelerating
Quantized Deepfake Detection Models using FPGAs due to their ability of
maximum parallelism and energy efficiency compared to generalized GPUs. In
this paper, we have used light MesoNet with EFF-YNet structure and accelerated
it on VCK5000 FPGA, powered by state-of-the-art VC1902 Versal Architecture
which uses AI, DSP, and Adaptable Engines for acceleration. We have
benchmarked our inference speed with other state-of-the-art inference nodes,
got 316.8 FPS on VCK5000 while maintaining 93% Accuracy.

###### Keywords:

Generative Models Deepfake Detection Deepfake Classification Machine Learning
Quantized FPGAs MesoNet EFF-YNet VCK5000 VC1902 Versal Architecture AI DSP
Adaptable Engines

##  1 Introduction

Deepfake is artificially created media in which a frame is created
synthetically using someone else’s features like face, structure, lip
movements ,etc. They are usually created by leveraging a Generative
Adversarial Network to create a picture or video which looks realistic enough
to deceive any person. While Deepfakes were initially created to prank
individuals, they started getting attention due to their use in illegal
activities like celebrity pornographic videos, fake news, and bullying. Hence,
detecting deepfakes has become a very important issue in recent years.
Upon going through relevant literature and existing methods of deepfake
classification, we observed that it is essentially an image classification
problem, but the catch here is that pathological differences between the real
and fake images are quite small, as a result existing CNN models need to be
modified and tuned to detect these minute differences.
MesoNet is a CNN architecture which is used to detect Face2Face and Deepfakes
manipulations accurately[1]. It is designed in such a way that it uses cropped
faces from videos and analyses mid-level features. The model focuses on the
right amount of details by using an architecture with small number of layers.
XceptionNet is a complex model originally designed for working with 2D Images
which uses depthwise separable convolutional layers with residual
connections[2], as a result it gives higher performance than MesoNet[1].
However, this model takes a lot of time to train. EfficientNets which are
relatively a newer family of CNN models aimed at providing efficient resource
management by balancing model parameters like width, depth, and resolution,
outperform models which have a similar number of parameters[3]. Researchers
have taken a step further and proposed classifying each pixel of the image as
real or fake. The U-Net architecture addresses this issue by employing an
encoder-decoder based network with skip connections[4]. To improve
classification accuracy, Eff-Ynet describes a novel architecture which
combines EfficientNet encoder with a classification and segmentation
branch[5]. It is designed to classify an image and also find regions where the
image is real and where it is fake. The job of segmentation helps train the
classifier, and at the same time it also produces useful segmentation masks.
The inference of neural networks is usually slow on general-purpose GPUs[6].
One way to accelerate the inference of these networks is to use soft cores
emulating on the FPGAs, which allows the user to utilize the task and data
level parallelism to reach the performance of ASIC implementations while
taking reduced design time[7]. It is also possible to make RNN or CNN specific
hardware architectures. In this paper, they have accelerated Deep Recurrent
Neural Network (DRNN) on a hardware accelerator running on XILINX ZYNQ
FPGA[8]. These ZYNQ boards are heterogeneous in nature, which means it has a
hardcore CPU besides FPGA. In one of the paper, researchers have mapped BNNs
(Binarized neural networks) onto an FPGA device while FP32 networks are mapped
to the CPU, this hybrid mapping increases overall neural network efficiency
while maintaining inference speed[9]. Researchers have found these ZYNQ boards
perform better than CPUs and GPUs[10].
One of the paper, transforms the model into FP8111FP8 means 8-bit floating
point representation format for speed up[11]. Upon evaluating this, we decided
to go with INT8222INT8 means 8-bit integer representation in our
implementation, thus quantized our model to INT8 precision. This means that
our model, which was originally trained on FP32333FP32 means 32-bit floating
point representation precision, was converted to INT811footnotemark: 1
precision. By reducing the precision, the number of bits required to store the
model parameters are reduced, which means it requires less amount of memory to
store and reduces the number of clock cycles to transfer data between memory
and the accelerator over the PCIe bus.
We propose U-YNet, a combined segmentation, and classification model which
consists of UNet Encoder and Decoder responsible for producing a segmentation
map to show the altered regions in the deepfake content and a classification
branch present at the end of the UNet Encoder branch responsible for
classifying if the media content is real or deepfake origin. The segmentation
map is a novel way to identify the regions of a face that have been mangled to
create the deepfake, giving an insight into the construction of a deepfake and
paving the way for creating models to reverse a deepfake as well. This model
runs on a Deep Learning Processing Unit(DPU)666DPU stands for Deep Learning
Processor Unit. Read more: https://www.xilinx.com/products/intellectual-
property/dpu.html present in the AMD-XILINX VCK 5000 Versal FPGA device. It
consists of dedicated processing units like hardware accelerated convolution
engine which enables convolution based model like UNet (and U-YNet) to run
with higher inference speeds, enabling real-time classification of deepfake
content.

##  2 Motivation and Background

Improvement in generative models and abundance of datasets has led to
evolution of models that can generate realistic looking deepfake videos,
deceiving the human eye and machines as well (Fig. 1). There is a huge
potential to spread deepfaked videos by malicious actors for their gains, as
more than 100 million hours of video content is watched every day on social
media.

Figure 1: Completely believable deepfakes can be generated with ease nowadays.

Looking at this with the perspective of computation, faster computation speeds
and easily available compute resources means that deepfake videos can be
generated with ease. This makes fast and effective detection of these videos
very crucial to stop the malicious use of deepfake videos.
On March 16, 2022, a video claiming to show Ukraine President Volodymyr
Zelensky calling for the Ukrainian people to surrender to Russia was aired on
news channel Ukraine 24, and circulated on social media. Such events clearly
show how potent of a tool, deepfakes are for creating social chaos. Thus,
there is a need for faster detection of such deepfake videos so that they can
be taken down even before reaching the social media users. This paper aims to
leverage the massively parallel nature of FPGAs to efficiently run Deep Neural
networks that power the deepfake detection algorithms.

##  3 Implementation

Deepfake C-L-I (Classification – Localization – Inference) is a novel system
that we are proposing in this paper, which consists of an optimum combination
of software and hardware tools to accelerate deepfake detection on FPGAs. The
block diagram of Deepfake CLI is shown in Fig. 2.

Figure 2: Flowchart of Deepfake CLI

The software part of the acceleration system consists of a deep learning model
comprised of a UNet Segmentation and a Classification Model. As for the
hardware part, these models are first quantized to INT8 for DPU Architecture
supported by VCK5000 to achieve maximum performance, then later on it is run
on the FPGA.
The implementation of Deepfake C-L-I is divided into two parts as software and
hardware as follows:

###  3.1 Software Implementation

We implemented a multitask model called U-YNet, that can produce
classification (real or fake) and segmentation results on deepfake content.
The backbone which is the UNet Encoder takes a frame as the input and then the
encoded features are passed to the classification branch and the UNet decoder
simultaneously. The reason for doing this is that the combination of loss of
classification and segmentation tasks may help the classification branch to
learn from the segmentation branch features during training.

####  3.1.1 Dataset

Currently, there are many deepfake detection datasets available, which include
both real and manipulated videos and images. To name a few, we have
FaceForensics++[12], UADFV, Google DFD, CelebDF[13] and the DFDC dataset[14].
The source of videos for all datasets is different, and the fake videos are
generated from the real frames using various deepfake generation models.

Figure 3: Comparison of current deepfakes datasets [15]

DFDC is currently the largest available deepfakes dataset (Fig. 3) with the
most number of videos and faces. So we trained the model using DFDC. The
dataset does not ship with the segmentation masks that we need for training
the segmentation branch of the U-YNet model. To create the mask, we find the
absolute difference between the real frame and the fake frame, which encodes
one for a manipulated pixel and a zero for the real one.
Data Leakage challenge: Initially, while training we were getting 99% accuracy
on the DFDC dataset, which meant that the model was overfitting. After
insinuating on this problem, we found that DFDC has 1:124 videos per subject.
So, while randomly splitting the dataset into testing, training, and
validation datasets, the training dataset had examples of all the faces in the
dataset. So the model was basically learning the faces and not classifying the
deepfake features, giving high accuracy on the test dataset. This problem is
called data leakage.
Solution: We solved this by clustering similar faces together and then
segregating the dataset such that a face in the training dataset does not
repeat in the testing dataset. This prevented overfitting, and the model
focused on learning features rather than faces.

####  3.1.2 Loss Function and Training

For the segmentation task we used Cross Entropy Loss and for the
classification task we used Binary Cross Entropy Loss (BCE).
Cross-entropy is used to measure the difference between two probability
distributions, and it is used for both classification and segmentation tasks.
We use the average of both losses as the final loss.
The model was trained using the final loss. Finally, Adam Optimiser was used
for converging the model. The model was trained for 40 epochs on the created
DFDC dataset on NVIDIA DGX-1.

###  3.2 Hardware Implementation

The hardware setup of Deepfake CLI consists of Dell R740 Server which is a two
socket machine with two Intel Xeon processor installed and paired with 32 GB
DDR4 RAM running Ubuntu Server 20.04 LTS, kernel version 5.4.0. The server was
installed with VCK5000444Read more: https://www.xilinx.com/products/boards-
and-kits/vck5000.html, an FPGA development card built on XILINX 7 nm Versal
ACAP555ACAP stands for Adaptive Compute Acceleration Platform. Read more:
https://www.xilinx.com/products/silicon-devices/acap/versal.html Architecture
which is ideal for acceleration of AI models for cloud and edge application.

(a) Vitis Flow

(b) Step 1: Quantization

Figure 4: Vitis Flow & Step 1: Quantization

In the previous chapter, we have described the changes that had to be done in
the dataset to overcome the challenges. For efficient usage of FPGA resources,
it is crucial to train a hardware centric model with ideal weights for the
model. Since earlier the model was in FP32, and it is not optimal to run this
on the FPGA [11], we used the Vitis-AI toolchain to convert the FP32 model to
INT8. The entire vitis flow is show in Fig. 4(a) which consists of three steps
which are Quantization, Compilation, and Inference explained as follows:

####  3.2.1 Quantization

Neural nets after being trained on some dataset store all the weights and
biases in the FP32 format in the model. If this model is directly used for
inference, it will be compute and memory intensive and might not be able to
serve the real-time processing demands. Since detecting deepfake videos in
real-time is a crucial goal of this project, to get better inference speed we
have quantized our model to be more hardware-centric using Vitis AI Quantizer.
This used the Post Training Quantization method to quantize Deepfake CLI model
suitable for our real-time needs. Post Training model is less demanding in
terms of runtime, being done by running few inference cycles on a dataset
known as the calibration dataset. This calibration dataset can be from a
training or testing set, and it has to be around 100-1000 images. As shown in
Fig. 4(b), we were able to quantize the Deepfake CLI model from FP32 format to
INT8 with Quantize parameters and Quantize activations modules.

####  3.2.2 Compilation

Once the model quantization step has been completed which generates a .xmodel
file, we are ready for the final steps to deploy our model onto VCK5000. We
will need to compile our model weights file by using the compiler provided by
Vitis-AI. Vitis-AI provides a compiler tool that acts as an interface for
optimization of deep neural networks for specific DPUs666DPU stands for Deep
Learning Processor Unit. Read more:
https://www.xilinx.com/products/intellectual-property/dpu.html. With a range
of compiler option present due to different DPUs666DPU stands for Deep
Learning Processor Unit. Read more:
https://www.xilinx.com/products/intellectual-property/dpu.html available by
Xilinx, it will transform the neural network to the targeted platform DPU
instruction set for efficient optimization.

(a) Step 2: Compilation

(b) Step 3: Inference

Figure 5: Hardware Implementation flow

Every DPU variant designed by Xilinx to be run on their FPGA board has its own
set of optimized instruction to ensure efficient execution of the neural
network on hardware. To adhere to this demand, we need to compile our
quantized model for the DPU variant it will run on. An overview of Vitis AI
compiler framework is described in Fig. 5(a). After it has completed topology
parsing of the neural network obtained by quantization, the Vitis AI compiler
will construct a computational graph as Intermediate Representation. The
compiler will also be responsible for further optimization such as fusion of
computational nodes which provides higher performance and reduces memory
footprint, using the existing parallelism to make sure the instructions are
scheduled efficiently.

####  3.2.3 Inference

Now that compilation of our model weights has been completed, we are ready to
use it for Inference. We use the Vitis AI Runtime Library(VART) available in
Python and C++. We pre-process the input image by resizing the image to
224×224, and then scale the values from 0-255 to 0-1. The images also have a
scaling operation applied to them to convert them to INT8 format as required
by the model. We then append the images to a list and pass it to the DPU
running on the VCK5000 loaded with our quantized model.
VCK5000 takes the images in batches of eight and gives us two outputs, the
segmented mask, and a classification label that tells us if the image or video
is a deepfake content or not. The segmented output is stored in a .npy format
for post-processing, so we could visualize the output in a Jupyter notebook.
The classification output is given at the end of the inference code, which
tells us how well the model performed on unseen data. A JSON file contains the
label for unseen data, we compare it with the model output and print the
accuracy of how well it performs. The inference process was explained in the
Fig. 5(b).

##  4 Results

###  4.1 Software Results

The Deepfake CLI model was evaluated on a testing dataset which had around
2,00,000 frames. These frames have faces that were never seen by the model
before. Our main task was classification, hence segmentation results are not
presented here. As we can see from the Table 1, we were able to increase the
accuracy of the model by almost 6% using a multitask learning model.

Model | Accuracy
---|---
U-YNet model | 94.12%
U-YNet model without segmentation branch | 88.27%

Table 1: Module Accuracy Comparison Figure 6: (Left to Right) Input Image,
Ground Truth, Predicted Mask

###  4.2 Hardware Results

The Deepfake CLI model was first quantized, compiled and then inferenced on
the VCK5000 FPGA card. We have tested the model on the FPGA with various
deepfake videos and images. In the Fig. 6, the predicted mask by Deepfake CLI
model matches with the Ground Truth Mask for the given input image. Now for
the hardware results, as shown in Fig. 7 the segmentation output of VCK5000
also matches with the Ground Truth Mask.

Figure 7: (Left to Right) Input Image, Ground Truth, FPGA Output

Deepfake CLI’s main aim was to increase the inference speed of the deepfake
detection. Here are the results of the classification branch, shown in the
Table 2.

Parameter | Value
---|---
Total Images | 5001
Correct Predictions | 4851
Wrong Predictions | 350
Accuracy | 93.001 %
FPS | 316.80

Table 2: Classification Results

We have obtained 316.80 FPS for the classification model of deepfake detection
on VCK5000. Because of quantization of the model to INT8, there was around 1%
loss in accuracy, but we got better inference speeds than other inference
nodes.

###  4.3 Benchmarking Deepfake CLI with other nodes

We had created a dedicated hardware centric model and inferenced it on
VCK5000. We had compared the results of the segmentation model on different
state-of-the-art inference nodes available in the market.

Figure 8: FPS vs Inference Nodes 777All nodes were running INT8 Deepfake CLI
Unet model, TensorRT SDK was used to quantize INT8 model for Tesla T4 and V100
while for VCK5000 we have used Vitis AI Flow.888In this benchmarking test, we
have compared the inference speed of simpler Deepfake CLI model which only
consists of Unet model (segmentation branch). This was done due to the
TensorRT quantization limitation for Tesla T4 and V100 nodes. We have also
obtained the complete Y-UNet Inference speed for VCK5000, which was 316.80 FPS
(Table 2). When both branches are included in Deepfake CLI, the layers in the
network gets approximately doubled and thus reduces inference speed by half.

As shown in Fig. 8, we have obtained 775 FPS for the segmentation model of
deepfake detection on VCK5000. For the same model, we had got a much lower FPS
on other inference nodes, like Nvidia’s Tesla T4 and V100. This result shows
that with proper model preprocessing, quantization, and compilation for FPGA
architecture will give better inference speeds than its counterpart GPUs.

##  5 Conclusion

Because of the rapid increase in computation speeds, deepfake detection will
be one of the concerns in the upcoming era. In this paper, we implemented a
novel architecture that combined segmentation and classification models, which
were trained on DGX A100 and inferred on VCK5000. We used a modified DFDC
dataset to reduce data leakage and solve the over fitting problem. The
Deepfake CLI software model was designed from a hardware-centric point of
view, and it was quantized and compiled for the VCK5000 AI inference versal
card. We were able to classify the deepfake images as well as videos on the
Accelerated FPGA fabric. We have obtained a 120% gain on the inference speed
on the VCK5000 FPGA fabric compared to state-of-the-art Tesla T4 inference
node.

##  6 Future Work

The ever-increasing research in deepfake generation demands better and faster
techniques to decode and detect these deepfake images and videos in real-time.
The future goals of Deepfake C-L-I are as follows:

  * •

Modeling and Training more generalized model for deepfake detection.

  * •

Restoring the deepfaked region of the frame.

  * •

Adding AWS FPGA support for easy access.

  * •

Adding accelerators for pre and post-processing of images and videos for ML
model.

##  7 Acknowledgment

We are grateful to the Centre of Excellence in Complex and Non-Linear
Dynamical Systems, V.J.T.I. (_CoE-CNDS_) for support in the project. Also, we
would like to thank the entire hackster.io and AMD-Xilinx Team for shipping us
the VCK5000 board, and the technical support to get it up and running.

## References

  * [1] D. Afchar, V. Nozick, J. Yamagishi and I. Echizen, ”MesoNet: a Compact Facial Video Forgery Detection Network,” 2018 IEEE International Workshop on Information Forensics and Security (WIFS), 2018, pp. 1-7 10.1109/WIFS.2018.8630761
  * [2] F. Chollet, ”Xception: Deep Learning with Depthwise Separable Convolutions,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1800-1807 10.1109/CVPR.2017.195
  * [3] M. Tan and Q. V. Le, “Efficient Net: Rethinking model scaling for convolutional neural networks,” ArXiv, vol. abs/1905.11946, 2019
  * [4] Ronneberger, O., Fischer, P., Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W., Frangi, A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science(), vol 9351. Springer, Cham. 10.1007/978-3-319-24574-4-28
  * [5] E. Tjon, M. Moh and T. -S. Moh, ”Eff-YNet: A Dual Task Network for DeepFake Detection and Segmentation,” 2021 15th International Conference on Ubiquitous Information Management and Communication (IMCOM), 2021, pp. 1-8, 10.1109/IMCOM51814.2021.9377373
  * [6] A. Boutros et al., ”Beyond Peak Performance: Comparing the Real Performance of AI-Optimized FPGAs and GPUs,” 2020 International Conference on Field-Programmable Technology (ICFPT), 2020, pp. 10-19, doi: 10.1109/ICFPT51103.2020.00011.
  * [7] Amiri, M., Siddiqui, F.M., Kelly, C. et al. FPGA-Based Soft-Core Processors for Image Processing Applications. J Sign Process Syst 87, 139–156 (2017) 10.1007/s11265-016-1185-7
  * [8] Hao, Y., & Quigley, S.F. (2017). The implementation of a Deep Recurrent Neural Network Language Model on a Xilinx FPGA. ArXiv, abs/1710.10296
  * [9] S. Amiri, M. Hosseinabady, S. McIntosh-Smith and J. Nunez-Yanez, ”Multi-precision convolutional neural networks on heterogeneous hardware,” 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), 2018, pp. 419-424, 10.23919/DATE.2018.8342046
  * [10] A. B. Mahjoub and M. Atri, ”Implementation of convolutional-LSTM network based on CPU, GPU and pynq-zl board,” 2019 IEEE International Conference on Design & Test of Integrated Micro & Nano-Systems (DTS), 2019, pp. 1-6, 10.1109/DTSS.2019.8915287
  * [11] Sun, Xiao et al. “Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks.” NeurIPS (2019). https://proceedings.neurips.cc/paper/2019/file/65fc9fb4897a89789352e211ca2d398f-Paper.pdf
  * [12] Rössler, Andreas & Cozzolino, Davide & Verdoliva, Luisa & Riess, Christian & Thies, Justus & Nießner, Matthias. (2019). FaceForensics++: Learning to Detect Manipulated Facial Images. arXiv:1901.08971
  * [13] Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-df: A largescale challenging dataset for deepfake forensics,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3207–3216
  * [14] B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, and C. C. Ferrer, “The deepfake detection challenge dataset,” arXiv preprint arXiv:2006.07397, 2020
  * [15] S. Das, S. Seferbekov, A. Datta, M. S. Islam, M. R. Amin, ‘Towards Solving the DeepFake Problem: An Analysis on Improving DeepFake Detection using Dynamic Face Augmentation’. arXiv, 2021