# FAS-assisted Wireless Powered Communication Systems

Xiazhi Lai, Kangda Zhi, Wanyi Li, Tuo Wu, Cunhua Pan, _Senior Member, IEEE_ ,
and
Maged Elkashlan , _Senior Member, IEEE_ _(Corresponding author: Tuo Wu and
Cunhua Pan.)_ X. Lai and W. Li are with the School of Computer Science,
Guangdong University of Education, Guangzhou, Guangdong, China (E-mail:
<EMAIL_ADDRESS>wanyili@gdei.edu.cn). T. Wu, K. Zhi and M. Elkashlan are
with the School of Electronic Engineering and Computer Science at Queen Mary
University of London, London E1 4NS, U.K. (Email:{tuo.wu,k.zhi,
maged.elkashlan}@qmul.ac.uk).C. Pan is with the National Mobile Communications
Research Laboratory, Southeast University, Nanjing 210096, China. (e-mail:
cpan@seu.edu.cn).

###### Abstract

Fluid Antenna System (FAS) is recognized as a promising technology for
enhancing communication performance. In this context, we explored the
potential of FAS-assisted wireless powered communication systems.
Specifically, the transmitter, equipped with FAS, harvests the radio frequency
(RF) signal from a power beacon and utilizes the harvested energy for data
transmission to the receiver. To evaluate the performance of the considered
systems, we derive both the analytical and asymptotic expressions of the
outage probability. Simulation results indicate that the diversity of the
considered network closely aligns with the number of ports. Besides, it is
also revealed that the port selection criteria based solely on single-hop
configurations yield a diversity order of only one.

###### Index Terms:

Fluid antenna system (FAS), wireless powered communications, outage
probability.

##  I Introduction

Fluid antenna systems (FASs) have emerged as a promising technique for
advancing future sixth generation (6G) communications [1, 2]. FASs leverage
the unique capability of switching the antenna element to an optimal position,
referred to as “ports”, with the help of either liquid metal [3, 5] or pixel-
based antennas [4, 6], and therefore significantly improve the system
performance.

Recognizing the significant potential of FAS, the foundational work by Wong
_et al._ pioneered analysis in FAS-enabled communication networks, developing
the probability density function (PDF) and cumulative distribution function
(CDF) for the signal-to-noise ratio (SNR) in point-to-point systems [7, 8, 9].
Building on this work, subsequent studies have expanded the application of FAS
into multiple access networks [10, 11, 12, 13, 14, 15]. For instance, [10]
delves into channel estimation challenges within a multiuser millimeter-wave
(mmWave) time-division duplexing (TDD) system. Following this, Xu _et al._ in
[11] explored capacity maximization in mmWave systems through the joint
optimization of transmit covariance matrices and user antenna position
vectors. Then, Xu _et al._ progressed to examining the outage probability in
two-user fluid antenna multiple access systems, as presented in [12].
Additionally, recent advances have seen the integration of machine learning
techniques into FAS, particularly for the optimization of port selection [16].

Besides, the wireless powered communications (WPC) [18] have also been
regarded as a powerful technology, promising to enhance the functionality and
efficiency of battery-powered devices. Specifically, the power beacons are
employed to charge the battery of the Internet-of-Things (IoT) devices with
radio frequency (RF) signal, thereby supporting the signal transmission of the
IoT devices which are placed in hazard environment or inconvenient positions
[19].

Utilizing FAS at mobile devices even with limited space offers a great spatial
diversity gain, which matches perfectly with the principles of WPC systems.
Specifically, the spatial diversity gain from FAS can significantly enhance
the capability of IoT devices to utilize charging power more efficiently for
data transmission. This strategy not only boosts efficiency but also promotes
more sustainable ways of communication. However, the integration of FAS into
WPC systems remains unexplored. To fill this research gap, this paper delves
into exploring the potential of the FAS-assisted WPC systems. To evaluate the
impact of network parameters on performance, we have derived both closed-form
and asymptotic expressions for outage probability. Additionally, we present
several simulation results, including comparisons of FAS port selection
strategies in single hop scenarios.

Specifically, our contributions are summarized as follows:

  * •

First, we consider a wireless powered system wherein the transmitter is
equipped with a fluid antenna, which can switch the antenna fluid to the
optimal port, out of the total $\displaystyle M$ ports. Besides, the
transmitter receives the wireless power from a power beacon, then uses this
energy to transmit signal to the receiver at a data rate $\displaystyle R$.

  * •

We theoretically derive the analytical and asymptotic expressions of outage
probability. Besides, the asymptotic analysis of the outage probability
demonstrates that the diversity order of the system under consideration is
approximately $\displaystyle M$. Consequently, larger values of $\displaystyle
M$ can significantly improve the performance of the FAS-assisted WPC systems.

  * •

The simulation results substantiate the effectiveness of the proposed
analytical approach, thereby confirming and validating our insights and
discussions.

##  II System Model

In this study, we examine a wireless powered system wherein the transmitter,
receiving wireless power from a power beacon, utilizes this energy to transmit
information to the receiver at a data rate $\displaystyle R$. To enhance the
performance, the transmitter is equipped with a fluid antenna, which can
flexibly adjust the antenna fluid to the optimal port, out of the total
$\displaystyle M$ switchable ports. In the following, we will detail the
procedure of communications and introduce the FAS channel model, respectively.

###  II-A Communications Model

To clarify, we define the channel parameters between the power beacon and the
$\displaystyle m$-th port of the transmitter as $\displaystyle
h_{1,m}\sim\mathcal{CN}(0,\psi_{1})$, and those between the $\displaystyle
m$-th port of the transmitter and the receiver as $\displaystyle
h_{2,m}\sim\mathcal{CN}(0,\psi_{2})$. Here, $\displaystyle
m\in\mathcal{M}=\\{1,2,\cdots,M\\}$, $\displaystyle\psi_{1}$ represents the
average channel gain from the power beacon to the transmitter, and
$\displaystyle\psi_{2}$ denotes the average channel gain from the transmitter
to the receiver. Assuming that the $\displaystyle m$-th port is selected for
both wireless power harvesting and subsequent data transmission, the power
signal received at the transmitter is given as

$\displaystyle\displaystyle g_{1}=h_{1,m}\sqrt{P_{S}}b+v_{1},$ (1)

where $\displaystyle P_{S}$ denotes the transmit power of the power beacon,
$\displaystyle b\sim\mathcal{CN}(0,1)$ represents the wireless power signal,
and $\displaystyle v_{1}\sim\mathcal{CN}(0,\sigma^{2})$ denotes the zero-mean
complex Gaussian noise. Therefore, the total harvested energy at the
transmitter is $\displaystyle|h_{1,m}|^{2}P_{S}T_{1}$, where $\displaystyle
T_{1}$ is the time duration of the first hop.

Then, the transmitter utilizes the harvested wireless energy to transmit data
$\displaystyle s\sim\mathcal{CN}(0,1)$. Consequently, the signal received at
the receiver is written as

$\displaystyle\displaystyle
g_{2}=h_{2,m}|h_{1,m}|\sqrt{P_{S}\rho\theta}s+v_{2},$ (2)

where $\displaystyle v_{2}\sim\mathcal{CN}(0,\sigma^{2})$ represents the zero-
mean complex Gaussian noise, $\displaystyle\rho$ denotes the energy conversion
efficiency, $\displaystyle\theta=T_{1}/T_{2}$, and $\displaystyle T_{2}$ is
the time duration of the second hop.

Drawing on the analysis from Eq. (1) and Eq. (2), the received SNR at the
receiver utilizing the $\displaystyle m$-th port can be formulated as

$\displaystyle\displaystyle\Gamma_{m}=\frac{P_{S}\rho\theta}{\sigma^{2}}\gamma_{1,m}\gamma_{2,m},$
(3)

where $\displaystyle\gamma_{i,m}=|h_{i,m}|^{2}$ for $\displaystyle
i\in\\{1,2\\}$ represent the channel gains of the first and second hops,
respectively.

###  II-B FAS Channel Model

In this subsection, we present the details of the channel model of FAS under
the Rayleigh fading environments [9], which accounts for the combined impact
of port correlation. It is important to note that while some more intricate
models are available, such as those described in [17], they often yield
results that are impractical for widespread applications due to their
complexity. Building on the Rayleigh fading channel model, the expression for
$\displaystyle h_{i,m}$ is expressed as

$\displaystyle\displaystyle h_{i,m}=\sqrt{\mu}h_{i,0}+\sqrt{1-\mu}e_{i,m},$
(4)

where $\displaystyle e_{i,m}\sim\mathcal{CN}(0,\psi_{i})$ for $\displaystyle
m\in\mathcal{M}$ are independently and identically distributed (i.i.d.) random
variables (RVs). Here, $\displaystyle\psi_{i}$ denotes the average channel
gain both from the power beacon to the transmitter and from the transmitter to
the receiver. Moreover, $\displaystyle h_{1,0}\sim\mathcal{CN}(0,\psi_{1})$
and $\displaystyle h_{2,0}\sim\mathcal{CN}(0,\psi_{2})$ serve as virtual
reference channels for these respective links. Consequently, the PDFs of
$\displaystyle\gamma_{i,0}=|h_{i,0}|^{2}$ for $\displaystyle i\in\\{1,2\\}$
can be expressed as

$\displaystyle\displaystyle
f_{\gamma_{i,0}}(x)=\frac{1}{\psi_{i}}e^{-\frac{x}{\psi_{i}}}.$ (5)

Additionally, $\displaystyle\mu$ represnts the correlation factor, which is
given by [21]

$\displaystyle\displaystyle\mu=$
$\displaystyle\displaystyle\sqrt{2}\sqrt{{}_{1}F_{2}\Big{(}\frac{1}{2};1;\frac{3}{2};-\pi^{2}W^{2}\Big{)}-\frac{J_{1}(2\pi
W)}{2\pi W}},$ (6)

where $\displaystyle{}_{a}F_{b}$ denotes the generalized hypergeometric
function and $\displaystyle J_{1}(\cdot)$ is the first kind Bessel function
with order 1.

Given a fixed channel parameter $\displaystyle h_{i,0}$, and in accordance
with $\displaystyle\gamma_{i,0}$, the associated channel gain for
$\displaystyle h_{i,m}$, expressed as
$\displaystyle\gamma_{i,m}=|h_{i,m}|^{2}$, follows a non-central chi-square
distribution. Consequently, the conditional PDF can be expressed as

$\displaystyle\displaystyle f_{\gamma_{1,m}|\gamma_{1,0}=x_{0}}(x)=$
$\displaystyle\displaystyle\omega_{1}e^{-\omega_{1}(x+\mu
x_{0})}I_{0}\big{(}2\omega_{1}\sqrt{\mu x_{0}x}\big{)},$ (7)
$\displaystyle\displaystyle f_{\gamma_{2,m}|\gamma_{2,0}=y_{0}}(y)=$
$\displaystyle\displaystyle\omega_{2}e^{-\omega_{2}(y+\mu
y_{0})}I_{0}\big{(}2\omega_{2}\sqrt{\mu y_{0}y}\big{)},$ (8)

where $\displaystyle\omega_{i}=\big{(}\psi_{i}(1-\mu)\big{)}^{-1}$. Besides,
$\displaystyle I_{0}(u)$ is the modified Bessel function of the first kind
with order $\displaystyle 0$, which can be expressed in series representation
as [22]

$\displaystyle\displaystyle
I_{0}(z)=\sum_{k=0}^{\infty}\frac{z^{2k}}{2^{2k}k!\Gamma(k+1)}.$ (9)

Combining (7)–(9), we further derive $\displaystyle
f_{\gamma_{1,m}|\gamma_{1,0}=x_{0}}(x)$ and $\displaystyle
f_{\gamma_{2,m}|\gamma_{2,0}=y_{0}}(y)$ as

$\displaystyle\displaystyle f_{\gamma_{1,m}|\gamma_{1,0}=x_{0}}(x)=$
$\displaystyle\displaystyle\sum_{k=0}^{\infty}a_{k}x_{0}^{k}e^{-\omega_{1}\mu
x_{0}}x^{k}e^{-\omega_{1}x},$ (10) $\displaystyle\displaystyle
f_{\gamma_{2,m}|\gamma_{2,0}=y_{0}}(y)=$
$\displaystyle\displaystyle\sum_{n=0}^{\infty}b_{n}y_{0}^{n}e^{-\omega_{2}\mu
y_{0}}y^{n}e^{-\omega_{2}y},$ (11)

where

$\displaystyle\displaystyle a_{k}=$
$\displaystyle\displaystyle\frac{\omega_{1}^{2k+1}\mu^{k}}{(k!)^{2}},$
$\displaystyle\displaystyle b_{n}=$
$\displaystyle\displaystyle\frac{\omega_{2}^{2n+1}\mu^{n}}{(n!)^{2}}.$ (12)

In communication networks equipped with FAS, the transmitter possesses the
capability to switch the antenna fluid to the optimal port, thereby having
additional degree-of-freedom to improve the system performance. To maximize
the received SNR at the receiver, the selection of the port for transmission
is governed by the following criterion:

$\displaystyle\displaystyle
m^{*}=\arg\max_{m\in\mathcal{M}}(\gamma_{1,m}\gamma_{2,m}).$ (13)

Hence, the received SNR at the receiver is given by

$\displaystyle\displaystyle\Gamma_{m^{*}}=\frac{P_{S}\rho\theta}{\sigma^{2}}\max_{m\in\mathcal{M}}(\gamma_{1,m}\gamma_{2,m}).$
(14)

##  III Outage Analysis

In this section, we derive the closed-form expression of the outage
probability. Subsequently, the asymptotic expression of the outage probability
is derived. These derivations offer valuable insights for the practical
application of the FAS-enabled WPC system.

###  III-A Closed-Form Outage Probability

The outage of communications happens when the received SNR falls below the
given threshold $\displaystyle\Gamma_{\rm{th}}$, i.e.,
$\displaystyle\Gamma_{\rm{th}}=2^{R}-1$. Consequently, the outage probability
is given by

$\displaystyle\displaystyle
P_{\mathrm{out}}=\Pr\left(\max_{m\in\mathcal{M}}(\gamma_{1,m}\gamma_{2,m})\leq
z\right),$ (15)

where $\displaystyle z=\frac{\Gamma_{\rm{th}}\sigma^{2}}{P_{S}\rho\theta}$.

To derive the analytical expression for the outage probability $\displaystyle
P_{\mathrm{out}}$, we reformulate $\displaystyle P_{\mathrm{out}}$ as

$\displaystyle\displaystyle P_{\mathrm{out}}$
$\displaystyle\displaystyle=\mathbf{E}_{\gamma_{1,0},\gamma_{2,0}}\left[F(z\big{|}\gamma_{1,0},\gamma_{2,0})\right]$
$\displaystyle\displaystyle=\int_{0}^{\infty}\int_{0}^{\infty}F(z\big{|}x_{0},y_{0})f_{\gamma_{1,0}}(x_{0})f_{\gamma_{2,0}}(y_{0})dx_{0}dy_{0},$
(16)

where $\displaystyle\mathbf{E}_{X}\left[\cdot\right]$ denotes the expectation
over RV $\displaystyle X$. Besides, $\displaystyle F(z\big{|}x_{0},y_{0})$
represents the conditional CDF with given $\displaystyle\gamma_{1,0}=x_{0}$
and $\displaystyle\gamma_{2,0}=y_{0}$, which is defined as

$\displaystyle\displaystyle F(z\big{|}x_{0},y_{0})$
$\displaystyle\displaystyle=\Pr\big{(}\max_{m\in\mathcal{M}}(\gamma_{1,m}\gamma_{2,m})\leq
z\big{|}\gamma_{1,0}=x_{0},\gamma_{2,0}=y_{0}\big{)}.$ (17)

Expanding on this, and considering that
$\displaystyle\gamma_{1,m}\gamma_{2,m}$ for $\displaystyle m\in\cal{M}$ are
independent with each other, the conditional CDF $\displaystyle
F(z\big{|}x_{0},y_{0})$, when conditioned on
$\displaystyle\gamma_{1,0}=x_{0},\gamma_{2,0}=y_{0}$, can be further rewritten
as

$\displaystyle\displaystyle
F(z\big{|}x_{0},y_{0})=\big{(}\Phi(z\big{|}x_{0},y_{0})\big{)}^{M},$ (18)

where $\displaystyle\Phi(z\big{|}x_{0},y_{0})$ is defined as

$\displaystyle\displaystyle\Phi(z\big{|}x_{0},y_{0})$
$\displaystyle\displaystyle=\Pr\left(\gamma_{1,m}\gamma_{2,m}\leq
z\big{|}\gamma_{1,0}=x_{0},\gamma_{2,0}=y_{0}\right)$
$\displaystyle\displaystyle=\int_{0}^{\infty}\int_{0}^{z/y}f_{\gamma_{1,m}|\gamma_{1,0}=x_{0}}(x)f_{\gamma_{2,m}|\gamma_{2,0}=y_{0}}(y)dxdy$
(19)

Building upon this, further mathematical manipulations are applied to
calculate this function. The detailed derivation leads to

$\displaystyle\displaystyle\Phi(z\big{|}x_{0},y_{0})$
$\displaystyle\displaystyle=\sum_{k=0}^{\infty}\sum_{n=0}^{\infty}a_{k}b_{n}x_{0}^{k}e^{-\omega_{1}\mu
x_{0}}y_{0}^{n}e^{-\omega_{2}\mu y_{0}}$
$\displaystyle\displaystyle\quad\times\int_{0}^{\infty}\int_{0}^{\frac{z\omega_{1}}{y}}x^{k}e^{-\omega_{1}x}y^{n}e^{-\omega_{2}y}dxdy$
$\displaystyle\displaystyle=\sum_{k=0}^{\infty}\sum_{n=0}^{\infty}\frac{a_{k}b_{n}x_{0}^{k}e^{-\omega_{1}\mu
x_{0}}y_{0}^{n}e^{-\omega_{2}\mu y_{0}}}{\omega_{1}^{k+1}}$
$\displaystyle\displaystyle\quad\times\int_{0}^{\infty}\gamma\Big{(}k+1,\frac{z\omega_{1}}{y}\Big{)}y^{n}e^{-\omega_{2}y}dy$
(20)

where $\displaystyle\gamma(\alpha,x)$ is the upper Gamma function, which can
be expressed as

$\displaystyle\displaystyle\gamma(\alpha,x)$
$\displaystyle\displaystyle=\int_{0}^{x}e^{-t}t^{\alpha-1}dt$
$\displaystyle\displaystyle=(\alpha-1)!\bigg{(}1-e^{x}\sum_{d=0}^{\alpha-1}\frac{x^{d}}{d!}\bigg{)}.$
(21)

Applying (III-A) in (III-A), $\displaystyle\Phi(z\big{|}x_{0},y_{0})$ can be
expressed in an series representation as

$\displaystyle\displaystyle\Phi(z\big{|}x_{0},y_{0})$
$\displaystyle\displaystyle=\sum_{k=0}^{\infty}\sum_{n=0}^{\infty}\frac{a_{k}b_{n}x_{0}^{k}e^{-\omega_{1}\mu
x_{0}}k!y_{0}^{n}e^{-\omega_{2}\mu y_{0}}}{\omega_{1}^{k+1}}$
$\displaystyle\displaystyle\quad\times\bigg{(}\int_{0}^{\infty}y^{n}e^{-\omega_{2}y}dy-\sum_{l=0}^{k}\int_{0}^{\infty}e^{-\frac{z\omega_{1}}{y}-\omega_{2}y}\frac{(z\omega_{1})^{l}}{l!y^{l}}dy\bigg{)}$
$\displaystyle\displaystyle=\sum_{k=0}^{\infty}\sum_{n=0}^{\infty}\frac{a_{k}b_{n}x_{0}^{k}e^{-\omega_{1}\mu
x_{0}}k!y_{0}^{n}e^{-\omega_{2}\mu y_{0}}}{\omega_{1}^{k+1}\omega_{2}^{n+1}}$
$\displaystyle\displaystyle\quad\times\big{(}n!-\sum_{l=0}^{k}\frac{2(z\omega_{1}\omega_{2})^{\frac{n+l+1}{2}}}{l!}K_{n-l+1}(2\sqrt{z\omega_{1}\omega_{2}})\big{)},$
(22)

where $\displaystyle K_{\nu}(\cdot)$ denotes the second kind modified Bessel
function with order $\displaystyle\nu$. The final expression in (III-A) is
derived using the result from [22, 3.471.9], that is

$\displaystyle\displaystyle\int_{o}^{\infty}x^{a-1}e^{-\frac{b}{x}-cx}dx=2\Big{(}\frac{b}{c}\Big{)}^{\frac{a}{2}}K_{a}(2\sqrt{bc}).$
(23)

Then, by substituting (III-A) into (18), we can achieve

$\displaystyle\displaystyle F(z\big{|}x_{0},y_{0})$
$\displaystyle\displaystyle=\sum_{k_{m},m\in\mathcal{M}}^{\infty}\sum_{n_{m},n\in\mathcal{M}}^{\infty}D_{k,n}x_{0}^{\epsilon_{k}}y_{0}^{\lambda_{n}}$
$\displaystyle\displaystyle\quad\times e^{-M\omega_{1}\mu
x_{0}}e^{-M\omega_{2}\mu y_{0}},$ (24)

where

$\displaystyle\displaystyle D_{k,n}$
$\displaystyle\displaystyle=\prod_{m=1}^{M}\big{(}n_{m}!-\sum_{l=0}^{k_{m}}\frac{2(z\omega_{1}\omega_{2})^{\frac{n_{m}+l+1}{2}}}{l!}K_{n_{m}-l+1}(2\sqrt{z\omega_{1}\omega_{2}})\big{)}$
$\displaystyle\displaystyle\quad\times\frac{\omega_{1}^{k_{m}}\omega_{2}^{n_{m}}\mu^{k_{m}+n_{m}}}{k_{m}!(n_{m}!)^{2}},$
$\displaystyle\displaystyle\epsilon_{k}=\sum_{m=1}^{M}k_{m},$
$\displaystyle\displaystyle\lambda_{n}=\sum_{m=1}^{M}n_{m}.$ (25)

To derive the analytical expression of the outage probability $\displaystyle
P_{\rm{out}}$, we substitute the expressions for $\displaystyle
F(z\big{|}x_{0},y_{0})$ from (III-A) and $\displaystyle f_{\gamma_{i,0}}(x)$
from (5) into (III-A). This process yields

$\displaystyle\displaystyle P_{\rm{out}}$
$\displaystyle\displaystyle=\sum_{k_{m},m\in\mathcal{M}}^{\infty}\sum_{n_{m},n\in\mathcal{M}}^{\infty}\frac{D_{k,n}}{\psi_{1}\psi_{2}}$
$\displaystyle\displaystyle\quad\times\int_{0}^{\infty}x_{0}^{\epsilon_{k}}e^{-M\omega_{1}\mu\frac{x_{0}}{\psi_{1}}}dx_{0}\times\int_{0}^{\infty}y_{0}^{\lambda_{n}}e^{-M\omega_{2}\mu\frac{y_{0}}{\psi_{2}}}dy_{0}$
$\displaystyle\displaystyle=\sum_{k_{m},m\in\mathcal{M}}^{\infty}\sum_{n_{m},n\in\mathcal{M}}^{\infty}D_{k,n}\frac{\epsilon_{k}!\lambda_{n}!}{q_{1}^{\epsilon_{k}+1}q_{2}^{\lambda_{n}+1}\psi_{1}\psi_{2}}$
$\displaystyle\displaystyle\approx\sum_{t=0}^{U_{k}\times
M}\sum_{u=0}^{U_{n}\times
M}\frac{t!u!}{q_{1}^{t+1}q_{2}^{u+1}\psi_{1}\psi_{2}}\sum_{\mathcal{P}_{k}(t,U_{k})}\
\sum_{\mathcal{P}_{n}(u,U_{n})}D_{k,n},$ (26)

where $\displaystyle\mathcal{P}_{k}(t,U_{k})$ denotes the set including all
compositions of $\displaystyle k_{m}\in\cal{M}$, and in each composition
$\displaystyle\sum_{m=1}^{M}k_{m}=t$ and $\displaystyle\forall k_{m}\leq
U_{k}$ hold. $\displaystyle\mathcal{P}_{n}(u,U_{n})$ denotes the set including
all compositions of $\displaystyle n_{m}\in\cal{M}$, and in each composition
$\displaystyle\sum_{m=1}^{M}n_{m}=u$ and $\displaystyle\forall n_{m}\leq
U_{n}$ hold. In addition, the approximation in (III-A) involves truncation in
practice, with truncation numbers $\displaystyle U_{1}$ and $\displaystyle
U_{2}$. The corresponding parameters are defined as

$\displaystyle\displaystyle q_{1}=$ $\displaystyle\displaystyle
M\omega_{1}\mu+\frac{1}{\psi_{1}},$ $\displaystyle\displaystyle q_{2}=$
$\displaystyle\displaystyle M\omega_{2}\mu+\frac{1}{\psi_{2}}.$ (27)

###  III-B Asymptotic Analysis

To more effectively assess the outage performance of the system under
consideration, a theorem is proposed as following.

###### Theorem 1

In the high SNR region, particularly for small values of $\displaystyle z$,
the asymptotic expression of outage probability is

$\displaystyle\displaystyle
P{\rm{out}}\approx\chi\big{(}\beta\ln(\beta)\big{)}^{M},$ (28)

where

$\displaystyle\displaystyle\chi=\Big{(}\frac{1-\mu}{M\mu+1-\mu}\Big{)}^{2},$
$\displaystyle\displaystyle\beta=\frac{\Gamma_{\rm{th}}\sigma^{2}}{P_{S}\rho\phi_{1}\phi_{2}(1-\mu)^{2}}.$
(29)

_Proof 1:_ By setting $\displaystyle U_{1}$ and $\displaystyle U_{2}$ to 0,
and then applying the approximations $\displaystyle e^{-x}=1-x$ and
$\displaystyle K_{0}(x)\approx-\ln(x)$ for tiny value of $\displaystyle|x|$
[23], we can easily obtain the asymptotic expression.
$\displaystyle\hfill\blacksquare$

Accordingly, the following remark can be made concerning the proposed FAS-
assisted WPC systems.

###### Remark 1

It is demonstrated that the diversity order of the system under consideration
is approximately $\displaystyle M$. Consequently, larger values of
$\displaystyle M$ can significantly enhance the performance of the FAS-
assisted WPC systems. Furthermore, as evident from equation (III-A), the
outage probability $\displaystyle P{\rm{out}}$ inversely correlates with the
parameter $\displaystyle\mu$. Therefore, with an increased value of
$\displaystyle W$, the efficiency of the FAS-assisted WPC systems can be
further improved.

##  IV Numerical Results

In this section, simulation results are presented to validate the proposed
analysis. For the simulations, the parameters are set as follows:
$\displaystyle\rho=0.5$, $\displaystyle\theta=1$. Unless otherwise specified,
the data rate $\displaystyle R$ is fixed at $\displaystyle 1$ bit/s/Hz. The
average channel gains for each hop are normalized to unity, implying that
$\displaystyle\psi_{1}=\psi_{2}=1$.

Figure 1: Outage probability versus transmit SNR $\displaystyle
P_{S}/\sigma^{2}$.

Fig. 1 illustrates the performance of the considered systems, across various
transmit SNR, denoted as $\displaystyle P_{S}/\sigma^{2}$. In these
simulations, $\displaystyle M$ is set to either $\displaystyle 5$ or
$\displaystyle 7$, and $\displaystyle W=0.6$ is considered. For comparative
purposes, the results for a single-hop FAS network, labeled ‘SF’ in the
legend, are also included. In the single-hop scenario, the FAS at the
transmitter switches the port based solely on either the first or the second
hop, which means that the received SNR at the receiver is given by

$\displaystyle\displaystyle\Gamma_{\tilde{m}}=\frac{P_{S}\rho\theta}{\sigma^{2}}\max_{m\in\mathcal{M}}(\gamma_{1,m})\gamma_{2,m},$
(30)

or

$\displaystyle\displaystyle\Gamma_{\tilde{m}}=\frac{P_{S}\rho\theta}{\sigma^{2}}\gamma_{1,m}\max_{m\in\mathcal{M}}(\gamma_{2,m}).$
(31)

As observed, the analytical results align perfectly with the simulation
results, and the asymptotic results converge with the simulation outcomes at
higher transmit SNR levels, which validate the correctness of the derivations.

Moreover, Fig. 1 not only confirms the expected trend of decreasing outage
probability with increasing transmit SNR or the number of ports $\displaystyle
M$ but also reveals the slight difference in the performance of the single-hop
FAS-assisted systems. Notably, while an increase in $\displaystyle M$
generally enhances system performance, the improvement becomes marginally less
significant in single-hop FAS-assisted systems as $\displaystyle M$ grows
larger. This observation suggests diminishing returns on the performance gains
from increasing the number of ports beyond a certain point in single-hop
scenarios. Such insights could be pivotal for optimizing resource allocation
and system design in FAS-assisted systems.

Figure 2: Outage probability versus different number of ports $\displaystyle
M$.

Fig. 2 demonstrates the change in outage probability across different port
numbers, $\displaystyle M$, ranging from 5 to 40, under a constant transmit
SNR of $\displaystyle P_{S}/\sigma^{2}=10$ dB. This figure distinctly shows
that within the proposed FAS network, the outage probability diminishes as the
number of ports, $\displaystyle M$, increases. Notably, this trend is more
pronounced with larger values of $\displaystyle W$, aligning with the
predictions made in Theorem 1. These results underscore the importance of port
number in enhancing network performance, particularly in scenarios with higher
values of $\displaystyle W$.

Figure 3: Outage probability versus different data rate $\displaystyle R$.

Fig. 3 showcases the variation in outage probability with different data rate
values, i.e., $\displaystyle R$, within a range from 0.5 to 4 bit/s/Hz. This
is observed in a transmission environment with an SNR of $\displaystyle
P_{S}/\sigma^{2}=10$ dB and $\displaystyle M=20$. As depicted in Fig. 3, there
is a notable increase in the outage probability in the proposed FAS-assisted
communication systems as the data rate $\displaystyle R$ escalates. This
increase is particularly significant for larger values of $\displaystyle W$,
affirming the theoretical predictions presented in Theorem 1. These results
demonstrate the impact of varying data rates on network performance,
especially in scenarios with higher values of $\displaystyle W$.

##  V Conclusions

In this work, we proposed a performance analysis of the FAS-assisted WPC
systems. We derived both exact and asymptotic expressions for the outage
probability of the proposed system with Rayleigh fading channels. Our analysis
revealed that the diversity order of the system is equal to the number of
ports. Numerical results were provided to demonstrate the correctness of our
analysis.

## References

  * [1] K. K. Wong, K. F. Tong, Y. Shen, Y. Chen, and Y. Zhang, “Bruce Lee-inspired fluid antenna system: Six research topics and the potentials for 6G,” Frontiers Commun. Netw., section Wireless Commun., 3:853416, Mar. 2022.
  * [2] K.-K. Wong, K.-F. Tong, and C.-B. Chae, “Fluid antenna system-part II: Research opportunities,” _IEEE Commun. Lett._ , vol. 27, no. 8, pp. 1924–1928, Aug. 2023.
  * [3] Y. Huang, L. Xing, C. Song, S. Wang, and F. Elhouni, “Liquid antennas: Past, present and future,” _IEEE Open J. Antennas Propag._ , vol. 2, pp. 473–487, 2021.
  * [4] D. Rodrigo, B. A. Cetiner, and L. Jofre, “Frequency, radiation pattern and polarization reconfigurable antenna using a parasitic pixel layer,” _IEEE Trans. Antennas Propag._ , vol. 62, no. 6, pp. 3422–3427, Jun. 2014.
  * [5] J. Zheng, et. al. “FAS-assisted NOMA Short-Packet Communication Systems.” _arXiv preprint_ , arXiv:2310.14251.
  * [6] J. Yao, et. al. “Proactive Monitoring via Jamming in Fluid Antenna Systems.” _arXiv preprint_ , arXiv:2310.07550.
  * [7] K.-K. Wong, A. Shojaeifard, K.-F. Tong and Y. Zhang, “Performance limits of fluid antenna systems,” _IEEE Commun. Letters_ , vol. 24, no. 11, pp. 2469–2472, Nov. 2020.
  * [8] K.-K. Wong, A. Shojaeifard, K.-F. Tong, and Y. Zhang, “Fluid antenna systems,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 3, pp. 1950–1962, March 2021.
  * [9] X. Lai, T. Wu, J. Yao, C. Pan, M. Elkashlan, and K. -K. Wong, “On performance of fluid antenna system using maximum ratio combining,” _IEEE Commun. Lett._ , doi: 10.1109/LCOMM.2023.3348028.
  * [10] H. Xu, et. al. “Channel estimation for FAS-assisted multiuser mmWave systems,” _IEEE Commun. Lett._ , 2023.
  * [11] H. Xu, et. al. “Capacity maximization for FAS-assisted multiple access channels,” _arXiv preprint_ , arXiv:2311.11037.
  * [12] H. Xu, et. al. “On outage probability for two-user fluid antenna multiple access,” _IEEE International Conference on Communications_ , IEEE, 2023.
  * [13] K.-K. Wong and K.-F. Tong, “Fluid antenna multiple access,” _IEEE Trans. Wireless Commun._ , vol. 21, no. 7, pp. 4801-C4815, Jul. 2022.
  * [14] K.-K. Wong, D. Morales-Jimenez, K.-F. Tong, and C.-B. Chae, “Slow fluid antenna multiple access,” _IEEE Trans. Commun._ , vol. 71, no. 5, pp. 2831–2846, May 2023.
  * [15] N. Waqar, K.-K. Wong, K.-F. Tong, A. Sharples, and Y. Zhang, “Deep learning enabled slow fluid antenna multiple access,” _IEEE Commun. Lett._ , vol. 27, no. 3, pp. 861–865, March 2023.
  * [16] Z. Chai, K.-K. Wong, K.-F. Tong, Y. Chen, and Y. Zhang, “Port selection for fluid antenna systems,” _IEEE Commun. Lett._ , vol. 26, no. 5, pp. 1180–1184, May 2022.
  * [17] M. Khammassi, A. Kammoun and M. -S. Alouini, “A new analytical approximation of the fluid antenna system channel,” IEEE Trans. Wireless Commun., vol. 22, no. 12, pp. 8843–8858, Dec. 2023.
  * [18] K. Liang, L. Zhao, G. Zheng, and H. -H. Chen, “Non-uniform deployment of power beacons in wireless powered communication networks,” _IEEE Trans. Wireless Commun._ , vol. 18, no. 3, pp. 1887-1899, March 2019.
  * [19] D. M. Mughal, D. Munir, and M. Y. Chung, “Outage analysis of IRS-assisted RF powered networks for energy-constrained IoT devices,” _IEEE Trans. Wireless Commun._ , vol. 22, no. 11, pp. 7805–7818, Nov. 2023.
  * [20] K. Xie, G. Cai, G. Kaddoum, and J. He, “Performance analysis and resource allocation of STAR-RIS-aided wireless-powered NOMA system,” _IEEE Trans. Wireless Commun._ , vol. 71, no. 10, pp. 5740–5755, Oct. 2023.
  * [21] K.-K. Wong, K. F. Tong, Y. Chen, and Y. Zhang, “Closed-form expressions for spatial correlation parameters for performance analysis of fluid antenna systems,” _IET Electron. Lett._ , vol. 58, no. 11, pp. 454–457, May 2022.
  * [22] I. S. Gradshteyn and I. M. Ryzhik, _Table of Integrals, Series, and Products, 7th ed_. San Diego, CA: Academic, 2007.
  * [23] M. Abramowitz, I. A. Stegun, _Handbook of mathematical functions: With formulas, graphs, and mathematical tables, 1st ed_. Nassau County, NY: Dover, 1965.


# Towards End-to-End Neural Face Authentication in the Wild – Quantifying and
Compensating for Directional Lighting Effects.

Viktor Varkarakis  Wang Yao  Peter Corcoran

###### Abstract

The recent availability of low-power neural accelerator hardware, combined
with improvements in end-to-end neural facial recognition algorithms provides
enabling technology for on-device facial authentication. The present research
work examines the effects of directional lighting on a State-of-Art (SoA)
neural face recognizer. A synthetic re-lighting technique is used to augment
data samples due to the lack of public data-sets with sufficient directional
lighting variations. Top lighting and its variants (top-left, top-right) are
found to have minimal effect on accuracy, while bottom-left or bottom-right
directional lighting have the most pronounced effects. Following the fine-
tuning of network weights, the face recognition model is shown to achieve
close to the original Receiver Operating Characteristic curve (ROC)
performance across all lighting conditions, and demonstrates an ability to
generalize beyond the lighting augmentations used in the fine-tuning data-set.
This work shows that a SoA neural face recognition models can be tuned to
compensate for directional lighting effects, removing the need for a pre-
processing step prior to applying facial recognition.

###### keywords:

Directional Lighting, Face Illumination, Face Recognition, Face Re-Lighting
Method

††journal: Pattern Recognition Letters

[inst1]organization=Department of Electronic Engineering, College of
Engineering, National University of Ireland Galway, addressline=University
Road, city=Galway, country=Ireland

##  1 Introduction

Human Face Recognition (FR) has been an active research field in computer
vision since the early 1990’s [38] and early Convolutional Neural Network
(CNN) based approaches were in evidence before the end of that decade [19].
Over the last two decades FR has been well-studied in the literature with the
most recent advances being driven by advances in CNN and deep learning [32,
36, 7, 41, 26]. In much of the literature the test samples for FR are assumed
to be normalized in terms of pose, facial expression and illumination to
simplify the challenge of accurately distinguishing an individual identity
among a very large population. But, as it is not always feasible to capture
optimal facial samples, some studies have explored the effects of different
factors on the accuracy of State-of-Art (SoA) FR systems.

The main factors that affect FR include (i) pose [37, 47, 3, 18], (ii)
illumination [2, 49, 42], (iii) facial expression [29, 30, 28], (iv) age [31,
6] and, (v) gender variations [44, 27].

In this work, the focus is on the latest end-to-end fully neural FR techniques
[9] as these represent current SoA in terms of accuracy and have the potential
for implementation in the latest neural accelerators [4, 12]. The initial
focus for implementation of neural algorithms in embedded devices was on
network optimizations such as parameter quantization and pruning, compressed
convolutional filters and matrix factorizations [12]. However, the attention
has recently shifted towards specialized neural topologies [22, 16] and ultra-
low power realizations in hardware [10, 14]. Such optimizations enable a SoA
neural FR architectures to be implemented in a low-power consumer appliance,
enabling a new generation of devices capable of identifying their owners,
provide access control and personalize the device’s responses and behaviour.
However, this introduces new challenges as such FR embodiment can no longer
rely on pre-processing of input facial samples to optimize power consumption.
Thus, all image processing must be achieved in a fully neural implementation,
requiring a neural FR to be robust to factors such as pose and illumination.
Here our goal is to determine the feasibility of modifying a high accuracy SoA
neural FR architecture to demonstrate robustness to uncompensated input image
samples.

This leads to the research questions posed in this work: (i) can we better
quantify the effects of the external factors that affect fully neural FR and
develop metrics to evaluate these; and (ii) can a fully neural FR architecture
be modified through tuning and/or re-training to compensate internally for
such external factors? As human interaction with a consumer device will
typically lead the user to look directly at a screen, or similar user-
interface, this work has a focus on the effects of lighting/illumination
scenarios on FR. A consideration of the effects of facial pose and other
external factors is left to future studies. Specifically for the lighting
variation, numerous image pre-processing methods exist to improve the
performance of the FR model [35, 50, 34], but studies exploring the tuning or
training of FR models to be robust to lighting variations are relatively rare
[5, 21].

As a first step towards answering these research questions, this work employs
a SoA re-lighting methodology to augment a set of high-quality facial images
with directional lighting effects. The effect of these augmentations on the
performance of a SoA FR method is quantified using Receiver Operating
Characteristic curve (ROC) techniques. A similar approach was used recently to
validate synthetic facial identities [40]. Note that a re-lighting
augmentation approach was adopted as existing public datasets do not provide
sufficient lighting variability. This is discussed in the sections 4 and 6.
Finally, this works studies the feasibility of handling lighting variations by
fine-tuning the neural FR network. The results for directional lighting are
promising and indicate the potential for an end-to-end neural face
authentication solution for in-the-wild faces.

##  2 Related Works

The advances in computational resources and with a surge in access to very
large datasets, deep learning architectures have been developed and pushed the
SoA in the FR task achieving exceptional accuracy results [26]. Famous deep
neural approaches include DeepFace, FaceNet and ArcFace [32, 36, 7, 41]. Each
work advanced the accuracy on the benchmarks of FR and new loss functions,
pre-processing techniques and deep neural architectures were introduced. More
information regarding the SoA of deep neural FR approaches as well as the
entire pipeline of the FR and the methods used are given in [9, 1].

Despite the improvements, the FR task remained challenging in several cases.
Studies revealed that many factors can have a negative impact on the FR
performance, with the main factors being pose, illumination and others [2, 26,
18, 23]. Specifically, on face samples with lighting variation, techniques
were proposed that compliment both the traditional and deep learning FR
methods reporting improved performance [26, 15]. The approaches include pre-
processing of the facial samples to normalise any variation, before feeding it
to the face recognition algorithm [33, 17, 50, 34, 35].

Databases have been introduced to facilitate the development of FR models with
light, pose, expression and other variations. The Yale b, CMU Multi-PIE, AR,
Postech01 and UHDB31 [13, 25, 11, 8, 20] are a few of the datasets that have
incorporate face variations and either focus on a single variation or a
combination of different variations. Despite the developments of such
datasets, the number of the variations and human subjects remain limited along
with the fact that these datasets are usually acquired in controlled
environments and therefore not being able to represent in-the-wild conditions.

Relighting techniques have been introduced in the literature with impressive
results and able to introduce lighting into the face images without the
degradation of the image by artifacts [48, 43] giving the ability to augment
in-the-wild face datasets. Thus, providing a solution to the limited variation
of lighting and human subjects in face datasets, which is discussed in section
4.

##  3 Methodology

In this section the various techniques and methodologies used in this work are
detailed.

###  3.1 Face Re-Lighting Method

In this work, the lighting variation is applied to the CelebA-HQ dataset using
the Deep Single Image Portrait Relighting (DPR) technique [48]. In this method
a CNN is trained to generate a relighted image based on a Spherical Harmonics
(SH) description of a lighting source. The method achieves SoA results, and in
particular avoids introducing artifacts to the relighted samples - a drawback
of other re-lighting methods that were considered for use in this study. The
selected DPR method is trained on the well-known CelebA-HQ dataset which
provides good variability in term of subject identity, combined with
consistent face image quality. This makes the combination of the DPR re-
lighting methodology and CelebA-HQ ideal for this work as side-effects are
eliminated due to either variable facial sample quality or re-lighting
artifacts, either of which could distort our experimental outcomes.

In this work we restricted our experiments to a select set of directional
lighting components in order to gain a better understanding of the overall
effect of directional lighting. The selected scenarios that are examined
include lighting from 4 main directions: right, left, top and bottom of the
face image. This has the added benefit of keeping the computation requirements
for experiments bounded to reasonable time-frame, with most individual
experiments completing in less than a 48 hour period.

The representative Spherical Harmonic (SH) lighting sources used are shown in
Fig.1. More SH lighting scenarios can be found in
111https://zhhoper.github.io/dpr.html . The illumination variations (right,
left, top and bottom) are introduced to each sample of the CelebA-HQ dataset,
resolving with 4 new sets of the CelebA-HQ, each containing of one
illumination variation (CelebA-HQ-right, CelebA-HQ-left, CelebA-HQ-top and
CelebA-HQ-bottom). Examples of the CelebA-HQ samples after introducing the
illumination variations are illustrated in Fig.1. It can be seen from Fig.1
that the DPR method has high quality outputs incorporating the target SH
lighting to the images realistically and without generating any artifacts to
the face images. Instructions on how to generate the sets of CelebA-HQ with
the different illumination scenarios are given in the Github repository of
this work 222https://github.com/C3Imaging/Deep-Learning-
Techniques/tree/Quantify-Retrain-FR-for-Light .

Figure 1: The target SH lighting that is examined in this work is presented on
the top row. The original images of CelebA-HQ are on the left column. Examples
of lighting injected in the original images using the DPR method [48] are
shown for each examined illumination scenario (right, left, top & bottom).

###  3.2 Face Recognition Model

A public reference implementation of the ArcFace [7] model is available, as
the authors have released optimized, pre-trained, weights for the model.This
reference ArcFace model has high performance on the dataset used in this work
and provides a useful public baseline for future performance comparisons. This
has motivated our use of ArcFace throughout this study. Other SoA FR models
such as FaceNet [32] or CosFace [41] do not provide reference implementations
and thus restrict direct experimental comparisons. An unofficial, but public,
implementation of FaceNet 333https://github.com/davidsandberg/facenet  was
also tested but could not provide a similar level of performance on the
baseline or test datasets used in this work.

In this work, the recommended workflow, by the authors of ArcFace is followed
before the face samples are fed to the network. The MTCNN [45] is used to
detect and crop the face samples. The detected area is cropped and resized to
$112\times 112$, using bilinear interpolation, before passing to the ArcFace
network which calculates the 512-embedding corresponding to a facial sample.
Finally, the cosine similarity is used to compute the score representing the
identity similarity when comparing two embeddings. The pretrained network used
in this work is provided by the authors of ArcFace and can be found in
444https://www.dropbox.com/s/ou8v3c307vyzawc/model-r50-arcface-
ms1m-refine-v1.zip?dl=0 .

Due to the introduction of the lighting variation and other factors, the face
detection is not able to process all the face images from the CelebA-HQ sets.
In the experiments only the images which the face detection network was able
to process in all the illumination scenarios along with the original image,
are used in order to keep the consistency in the experiments. Therefore, from
the initial 30,000 images, in this work 28,222 are used from each set (CelebA-
HQ-left, right, top, bottom and original). For the requirements of this study
the dataset is divided into a train and test set with 19,570 images from 4k
identities and 8,654 images from 2k identities, respectively. This is applied
to each CelebA-HQ-set. A list of the images used in the experiments can be
found in 2.

###  3.3 Using ROC Curves as a Metric

The ROC curve is created by plotting the true positive rate (TPR) against the
false positive rate (FPR) at various threshold settings [24]. More
specifically, to compute an ROC curve, an equal number of positive-identity
and negative-identity pairs are created. Using the corresponding embeddings
(extracted from the FR model for each image) of the pairs, similarity scores
are calculated and used to plot the ROC curve. The closer an ROC curve is to
unity, the better the performance of the FR model on the selected samples.
More information regarding the ROC and its interpretation and use can be found
in [40].

##  4 Quantifying the Effects of Illumination on the Face Recognition

In this section, the experiments conducted to quantify the effect of different
lighting conditions on the FR’s performance are presented with a discussion of
the findings.

###  4.1 Experiments on Initial Lighting Variations

The effects of the 4 directional lighting scenarios shown in Fig.1 on the FR’s
performance are examined. Initially an ROC curve is calculated using only the
samples from the test set of the original CelebA-HQ (ROC-Original) following
the procedure described in section 3.3. All possible positive pairs from the
test set of the original CelebA-HQ are used, in total 31k image pairs and an
equal number of negative pairs are created randomly. Using the corresponding
FR embeddings, the similarity scores are calculated and used to plot the ROC
curve. The ROCs corresponding to re-lighting augmented scenarios are
calculated following a similar procedure. The same positive and negative
identity pairs as in ROC-Original are used but one of the samples from each
pair has a re-lighting augmentation applied. Thus resulting in 4 main ROC
curves (ROC-Left, ROC-Right, ROC-Top, ROC-Bottom) representing the FR’s
performance in each illumination scenario. The positive and negative pairs
used to compute each ROC can be found in 2. The resulting ROCs enable a direct
comparison of the effects of different types of directional illumination with
the original set of test image pairs and between them. This is presented in
Fig.2.

From Fig.2 the initial experimental results are largely self consistent and
show well-defined performance degradation of the FR which is largely
consistent with what might be expected. The ROC-Original curve illustrates
that the FR model has a SoA performance on the non-augmented test dataset
approaching close to unity, of 0.99 TPR on the corresponding to $10^{-4}$ FPR
value. The re-lighting augmented ROC curves show significant deviations from
this baseline performance and are largely consistent with what might be
expected. Thus, the smallest deviation is for the ROC-Top, which starts at
0.925 TPR, followed by the ROC-Right and ROC-Left curves at 0.86 and 0.85
respectively. The worst performing ROC is that of the bottom light, starting a
TPR of only 0.725. Looking at the examples shown in Fig.1 these results make
sense - the top lighting augmentation causes the least distortion to the
facial image from a human perspective, whereas the bottom-lighting creates
more obvious distortions in the facial features. Finally, the left/right
lighting augmentations would be expected to have similar effects due to the
symmetry of a human face. Note that the slight variation between ROC-Left and
ROC-Right is most likely due to slight left-right pose variations in some
facial samples leading to eccentricities in the corresponding lighting
augmentations.

Figure 2: ROC curves, representing the performance of the FR model 4 on the
original images and the 4 initial directional illumination scenarios (left,
right, top, bottom) examined in this work. Figure 3: The additional SH
lighting that is examined in section 4.2 is presented on the top row. The
original images of CelebA-HQ are on the left column. Examples of lighting
injected in the original images using the DPR method [48] are shown for each
examined illumination scenario (top-left, top-right, bottom-left, bottom-
right).

###  4.2 Experiments on Additional Lighting Variations

The initial results shown in Fig.2 encouraged a more extensive set of
experiments to include additional top-right, top-left and bottom-right,
bottom-left lighting augmentations, to further improve our understanding of
mixed directional lighting modes. The new SH lighting used and examples of the
CelebA-HQ samples after introducing these illumination variations are
illustrated in Fig.3. The goals of this additional set of experiments were to
provide a second validation of our results, and in addition to explore the
effects of more varied re-lighting augmentations.

Due to the introduction of the new lighting variations, the face detection is
not able to process all the face images from the test set of the CelebA-HQ.
Similarly as in section 3.2, only the images which the face detection network
was able to process in all 8 illumination scenarios and the original images
are used in order to keep the consistency in the experiments. Therefore, the
initial test set of 8,654 images from 2k identities is reduced to 8,552 images
from 1,979 identities. In order to calculate the ROCs, corresponding to the
original images and the 8 illuminations scenarios the procedure described in
3.3 and 4.1, is followed, using the new tests. As the size of the test set is
reduced, so is the number of all possible positive image pairs used to compute
the ROCs. For this set of experiments 30k positive pairs and an equal number
of negative pairs are used. These pairs can be found in 2.

The primary directional ROCs curves (top, bottom, left & right) presented in
Fig.4, differ slightly from those of Fig.2, as the image pairs used in these
experiments are different. However their behaviour has a broadly similar
characteristic. The bottom, bottom-left and bottom-right lighting
augmentations are seen to be the most challenging for the FR task, while the
top-left and top-right illuminations have the least effect on the FR’s
performance. There is a small drop and a small increase on the FR’s
performance from the top and right light respectively, compared to the Fig.2.
These are attributed to the use of different pairs. These results are useful
as a baseline for the section 5, as they help demonstrate that fine-tuning on
the primary set of directional lighting augmentations can generalize across a
broader range of directional lighting effects.

Figure 4: Additional ROC curves, representing the performance of the FR model
4 on the original images and the 8 directional illumination scenarios (left,
top-left, bottom-left, right, top-right, bottom-right, top and bottom)
examined in this work.

###  4.3 Experiments on Public Face Illumination Data-sets

Similar experiments as in 4.1 and 4.2, were conducted using face datasets
which include illumination variation in their samples, to measure the effect
of their lighting scenarios on the FR’s performance. More specifically the AR
and Postech01 [25, 8] face datasets were used. These experiments did not show
any measurable degradation of the FR’s performance across the illumination
conditions provided in these data-sets. This is attributed to the relatively
limited variation of the illumination scenarios, the numbers of human
subjects, the total number of images and the controlled environments in which
these face datasets were acquired. This is also indicated through the
information regarding these datasets, provided in Table 1. Thus, revealing the
need for the development of a face dataset that resembles in-the-wild
conditions with illumination variation, which is further discussed in section
6.

Table 1: Summary Information of Face Data-sets with Illumination Variations. (The number of Pose Variations in this table is referred with regards to the Illumination Variation) Face Data-sets |  #IDs |  #Illumination Variations |  #Pose Variations
---|---|---|---
AR [25] |  126 |  3 |  1
Postech01 [8] |  103 |  4 |  1

##  5 Fine-Tuning the Face Recognition model on Directional Illuminations

In the first part of this work the potential effects of in-the-wild lighting
conditions, in particular directional lighting effects on a SoA neural face
recognition method have been demonstrated and quantified. The next step is to
determine whether the FR can be tuned to compensate for these effects. In this
section the selected FR model is fine-tuned, using a similar approach to [39]
with samples augmented with directional lighting.

###  5.1 Fine-tuning Process & ROCs Computation

The initial pretrained network provided by the authors of Arcface 4, is fine-
tuned using a training set comprising samples from the original CelebA-HQ
dataset and samples with all 4 primary directional lighting augmentations
(CelebA-HQ-Left, Right, Top and Bottom). In total 97,850 high-quality facial
samples were used for fine-tuning, or 19,750 from the original data and each
of the four primary lighting sub-category.

For the re-training process the standard Arcface loss function is used, with
the learning rate set to 0.005 and a batch size of 128, following the
instructions from the authors of ArcFace. The network is fine-tuned for 40
epochs, as the number of images used is relatively large and all network
layers are unfrozen for the fine-tuning process. After 40 epochs the network
showed satisfactory results on the training data used and therefore stopped.
Longer training could result in over-fitting to the training data and thus not
being able to generalise. More details regarding the fine-tuning process and
the corresponding training code can be found at
555https://github.com/deepinsight/insightface . The fine-tuned network
resulting from this re-training process is released at 2.

The fine-tuned network, is used to calculate the embeddings of the test
samples. The same procedure as described in 3.3 and 4.1 is followed to
calculate the ROC-Original-finetuned (ROC-Original-FT) and the ROCs
corresponding to the 8 different illumination scenarios (ROC-Left-FT, Right-
FT, Top-FT, Bottom-FT, Top-Left-FT, Top-Right-FT, Bottom-Left-FT, Bottom-
Right-FT,), using the positive and negative pairs from section 4.2. These ROCs
are compared with the ROC-Original-FT and between them to explore whether the
fine-tuned FR model is able to handle the variation in illumination as well as
whether can generalise across the illuminations that were not used for the
fine-tuning process.

Figure 5: ROC curves, representing the performance of the finetuned FR network
2 on the original images and the 8 directional illumination scenarios (left,
top-left, bottom-left, right, top-right, bottom-right, top and bottom)
examined in this work.

###  5.2 ROCs Comparison

The ROCs representing the performance of the fine-tuned FR model on the
original images and on the 8 directional lightings are presented in the Fig.5.
From Fig.5 it is illustrated that the ROCs corresponding to the fine-tuned FR
model on the 4 main illuminations (left, right, top, bottom) used in the fine-
tuning process (Fig.5) are at higher levels compared to the ROCs corresponding
to the performance of the initial network (Fig.4) on these illumination
scenarios. More importantly, the ROCs corresponding to the fine-tuned model on
the 4 illuminations scenarios that are not used during the fine-tuning (top-
left, top-right, bottom-left, bottom-right), are also at higher levels, thus
showing that the network is able to generalise to other variations of
illumination that it was not trained on. Overall, the performance of the fine-
tuned FR model on any given illumination scenario has increased and its above
0.95 TPR, even on the lower FPR values. Notably, the ROCs are very close to
the performance of the FR on the original images. Therefore, concluding that
the FR model when trained with lighting variation is able to adopt and handle
face samples that include illumination and achieve high accuracy results and
also generalise across different illumination variations that are not used
during fine-tuning. Thus, showing that the illumination can be compensated
through training methods and augmentation techniques eliminating the need for
pre-processing methods correcting the lighting, which are not optimal for use
in neural accelerators.

##  6 Conclusion & Future Work

It is clear from the results of the experiments, illustrated in Fig. 2 & 4
that fully end-to-end neural FR solutions will be challenged by in-the-wild
lighting conditions. As was indicated in [26] this problem is typically solved
by additional pre-processing of image samples to correct for lighting
conditions. In section 5.2 the practicality of fine-tuning a high-performing
neural FR model has been demonstrated, recovering performance levels close to
the original baseline for such lighting conditions. The fine-turning process
also indicated that generalization from the primary directions to combinations
of directional lighting is achieved - a promising result given the non-linear
nature of lighting conditions. Note that providing a broader and more varied
range of re-lighting samples and refining the training methodology to identify
the more sensitive network layers in the ArcFace model should further improve
these results, but even as they stand it is clear that a full end-to-end
neural FR can be realized.

These initial results, especially the effectiveness of the fine-tuning process
are very promising. They suggest that SoA neural FR algorithms can be fine-
tuned to handle difficult in-the-wild acquisition conditions such as
directional lighting. However there are other challenges for FR algorithms in-
the-wild, including those listed in the introduction. A broader study on
factors that can affect FR is indicated. In this regard the availability of
several large 3D facial model datasets [46] could provide sufficient
individual identities and support more complex data variations to support such
a study.

## Acknowledgments

This research is funded by (i) the Science Foundation Ireland Strategic
Partnership Program (Project ID: 13/SPP/I2868), (ii) Irish Research Council
Enterprise Partnership Ph.D. Scheme (Project ID: EPSPG/2020/40) and, (iii)
Xperi Corporation, Ireland.

## References

  * Adjabi et al. [2020] Adjabi, I., Ouahabi, A., Benzaoui, A., Taleb-Ahmed, A., 2020\.  Past, present, and future of face recognition: A review.  Electronics 9, 1188\.
  * Beveridge et al. [2010] Beveridge, J.R., Bolme, D.S., Draper, B.A., Givens, G.H., Lui, Y.M., Phillips, P.J., 2010\.  Quantifying how lighting and focus affect face recognition performance, IEEE. pp. 74–81.
  * Choi et al. [2011] Choi, I.S., Choi, C.H., Kwak, N., 2011.  Face recognition based on 2d images under illumination and pose variations.  Pattern Recognition Letters 32, 561–571.  doi:10.1016/j.patrec.2010.11.021.
  * Corcoran et al. [2019] Corcoran, P., Lemley, J., Costache, C., Varkarakis, V., 2019\.  Deep learning for consumer devices and services 2—ai gets embedded at the edge.  IEEE Consumer Electronics Magazine 8, 10–19.  doi:10.1109/mce.2019.2923042.
  * Crispell et al. [2017] Crispell, D., Biris, O., Crosswhite, N., Byrne, J., Mundy, J.L., 2017.  Dataset augmentation for pose and lighting invariant face recognition.  arXiv preprint arXiv:1704.04326 .
  * Deb et al. [2017] Deb, D., Best-Rowden, L., Jain, A.K., 2017.  Face recognition performance under aging, pp. 548–556.  doi:10.1109/CVPRW.2017.82.
  * Deng et al. [2019] Deng, J., Guo, J., Xue, N., Zafeiriou, S., 2019.  Arcface: Additive angular margin loss for deep face recognition, pp. 4690–4699.
  * Dong and Gu [2001] Dong, H., Gu, N., 2001.  Asian face image database pf01.  2006-01-05](2014-11). http://nava. postech. ac. kr/archives/irndb. html .
  * Du et al. [2020] Du, H., Shi, H., Zeng, D., Mei, T., 2020.  The elements of end-to-end deep face recognition: A survey of recent advances.  arXiv preprint arXiv:2009.13290 .
  * Fleischer et al. [2018] Fleischer, B., Shukla, S., Ziegler, M., Silberman, J., Oh, J., Srinivasan, V., Choi, J., Mueller, S., Agrawal, A., Babinsky, T., Cao, N., Chen, C.Y., Chuang, P., Fox, T., Gristede, G., Guillorn, M., Haynie, H., Klaiber, M., Lee, D., Lo, S.H., Maier, G., Scheuermann, M., Venkataramani, S., Vezyrtzis, C., Wang, N., Yee, F., Zhou, C., Lu, P.F., Curran, B., Chang, L., Gopalakrishnan, K., 2018\.  A scalable multi-teraops deep learning processor core for ai trainina and inference, pp. 35–36.  doi:10.1109/VLSIC.2018.8502276.
  * Georghiades et al. [2001] Georghiades, A.S., Belhumeur, P.N., Kriegman, D.J., 2001.  From few to many: Illumination cone models for face recognition under variable lighting and pose.  IEEE transactions on pattern analysis and machine intelligence 23, 643–660.
  * Goel et al. [2020] Goel, A., Tung, C., Lu, Y.H., Thiruvathukal, G.K., 2020.  A survey of methods for low-power deep learning and computer vision, Institute of Electrical and Electronics Engineers Inc.  doi:10.1109/WF-IoT48130.2020.9221198.
  * Gross et al. [2010] Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S., 2010.  Multi-pie.  Image and Vision Computing 28, 807–813.
  * Guicquero and Verdant [2020] Guicquero, W., Verdant, A., 2020\.  Algorithmic enablers for compact neural network topology hardware design: Review and trends, pp. 1–5.  doi:10.1109/iscas45731.2020.9181005.
  * Han et al. [2013] Han, H., Shan, S., Chen, X., Gao, W., 2013.  A comparative study on illumination preprocessing in face recognition.  Pattern Recognition 46, 1691–1699.
  * He et al. [2018] He, Z., Gong, B., Fan, D., 2018.  Optimize deep convolutional neural network with ternarized weights and high accuracy.  arXiv .
  * Hussain Shah et al. [2015] Hussain Shah, J., Sharif, M., Raza, M., Murtaza, M., Ur-Rehman, S., 2015.  Robust face recognition technique under varying illumination.  Journal of applied research and technology 13, 97–105.
  * Lanitis et al. [1997] Lanitis, A., Taylor, C., Cootes, T., 1997.  Automatic interpretation and coding of face images using flexible models.  Pattern analysis and machine intelligence, IEEE Transactions on 19, 743–756.  doi:10.1109/34.598231.
  * Lawrence et al. [1997] Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D., 1997\.  Face recognition: A convolutional neural-network approach.  IEEE Transactions on Neural Networks 8, 98–113.  doi:10.1109/72.554195.
  * Le and Kakadiaris [2017] Le, H.A., Kakadiaris, I.A., 2017\.  Uhdb31: A dataset for better understanding face recognition across pose and illumination variation, pp. 2555–2563.
  * Le and Kakadiaris [2019] Le, H.A., Kakadiaris, I.A., 2019\.  Illumination-invariant face recognition with deep relit face images, in: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE. pp. 2146–2155.
  * Li et al. [2016] Li, F., Zhang, B., Liu, B., 2016.  Ternary weight networks .
  * Mahmood et al. [2016] Mahmood, Z., Ali, T., Khan, S.U., 2016.  Effects of pose and image resolution on automatic face recognition.  IET biometrics 5, 111–119.
  * Mansfield [2006] Mansfield, A., 2006.  Information technology–biometric performance testing and reporting–part 1: Principles and framework.  ISO/IEC , 19791–19795.
  * Martinez [1998] Martinez, A.M., 1998.  The ar face database.  CVC Technical Report24 .
  * Mehdipour Ghazi and Kemal Ekenel [2016] Mehdipour Ghazi, M., Kemal Ekenel, H., 2016\.  A comprehensive analysis of deep learning based representation for face recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 34–41.
  * Narang and Bourlai [2016] Narang, N., Bourlai, T., 2016\.  Gender and ethnicity classification using deep learning in heterogeneous face recognition.  doi:10.1109/ICB.2016.7550082.
  * Pavlović et al. [2018] Pavlović, M., Petrović, R., Stojanović, B., Stanković, S., 2018\.  Facial expression and lighting conditions influence on face recognition performance.  Technical Report.
  * Peña et al. [2020] Peña, A., Serna, I., Morales, A., Fierrez, J., Lapedriza, A., 2020.  Facial expressions as a vulnerability in face recognition.  arXiv preprint arXiv:2011.08809 .
  * Peng and Yin [2018] Peng, Y., Yin, H., 2018.  Facial expression analysis and expression-invariant face recognition by manifold-based synthesis.  Machine Vision and Applications 29, 263–284.  doi:10.1007/s00138-017-0895-6.
  * Riaz et al. [2019] Riaz, S., Ali, Z., Park, U., Choi, J., Masi, I., Natarajan, P., 2019.  Age-invariant face recognition using gender specific 3d aging modeling.  Multimedia Tools and Applications doi:10.1007/s11042-019-7694-1.
  * Schroff et al. [2015] Schroff, F., Kalenichenko, D., Philbin, J., 2015.  Facenet: A unified embedding for face recognition and clustering, pp. 815–823.
  * Shan et al. [2003] Shan, S., Gao, W., Cao, B., Zhao, D., 2003.  Illumination normalization for robust face recognition against varying lighting conditions, IEEE. pp. 157–164.
  * Shinwari et al. [2019] Shinwari, A.R., Balooch, A.J., Alariki, A.A., Abdulhak, S.A., 2019\.  A comparative study of face recognition algorithms under facial expression and illumination, IEEE. pp. 390–394.
  * Ruiz-del Solar and Quinteros [2008] Ruiz-del Solar, J., Quinteros, J., 2008\.  Illumination compensation and normalization in eigenspace-based face recognition: A comparative study of different pre-processing approaches.  Pattern Recognition Letters 29, 1966–1979.
  * Taigman et al. [2014] Taigman, Y., Yang, M., Ranzato, M., Wolf, L., 2014\.  Deepface: Closing the gap to human-level performance in face verification, pp. 1701–1708.
  * Tran et al. [2017] Tran, L., Yin, X., Liu, X., 2017.  Disentangled representation learning gan for pose-invariant face recognition, pp. 1283–1292.  doi:10.1109/CVPR.2017.141.
  * Turk and Pentland [1991] Turk, M., Pentland, A., 1991\.  Eigenfaces for recognition.  Journal of cognitive neuroscience doi:10.1162/jocn.1991.3.1.71.
  * Varkarakis et al. [2020a] Varkarakis, V., Bazrafkan, S., Corcoran, P., 2020a.  Deep neural network and data augmentation methodology for off-axis iris segmentation in wearable headsets.  Neural Networks 121, 101–121.
  * Varkarakis et al. [2020b] Varkarakis, V., Bazrafkan, S., Costache, G., Corcoran, P., 2020b.  Validating seed data samples for synthetic identities–methodology and uniqueness metrics.  IEEE Access 8, 152532–152550.
  * Wang et al. [2018a] Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., Liu, W., 2018a.  Cosface: Large margin cosine loss for deep face recognition, pp. 5265–5274.
  * Wang et al. [2018b] Wang, J.W., Le, N.T., Lee, J.S., Wang, C.C., 2018b.  Illumination compensation for face recognition using adaptive singular value decomposition in the wavelet domain.  Information Sciences 435, 69–93.  doi:10.1016/j.ins.2017.12.057.
  * Wang et al. [2020] Wang, Z., Yu, X., Lu, M., Wang, Q., Qian, C., Xu, F., 2020.  Single image portrait relighting via explicit multiple reflectance channel modeling.  ACM Transactions on Graphics (TOG) 39, 1–13.
  * Werther et al. [2018] Werther, C., Ferguson, M., Park, K., Kling, T., Chen, C., Wang, Y., 2018.  Gender effect on face recognition for a large longitudinal database.  arXiv .
  * Xiang and Zhu [2017] Xiang, J., Zhu, G., 2017.  Joint face detection and facial expression recognition with mtcnn, IEEE. pp. 424–427.
  * Yang et al. [2020] Yang, H., Zhu, H., Wang, Y., Huang, M., Shen, Q., Yang, R., Cao, X., 2020.  Facescape: a large-scale high quality 3d face dataset and detailed riggable 3d face prediction, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 601–610.
  * Zhao et al. [2018] Zhao, J., Cheng, Y., Xu, Y., Xiong, L., Li, J., Zhao, F., Jayashree, K., Pranata, S., Shen, S., Xing, J., Yan, S., Feng, J., 2018.  Towards pose invariant face recognition in the wild, pp. 2207–2216.  doi:10.1109/CVPR.2018.00235.
  * Zhou et al. [2019] Zhou, H., Hadap, S., Sunkavalli, K., Jacobs, D.W., 2019\.  Deep single-image portrait relighting, pp. 7194–7202.
  * Zhu et al. [2017] Zhu, J.Y., Zheng, W.S., Lu, F., Lai, J.H., 2017.  Illumination invariant single face image recognition under heterogeneous lighting condition.  Pattern Recognition 66, 313–327.  doi:10.1016/j.patcog.2016.12.029.
  * Zou et al. [2007] Zou, X., Kittler, J., Messer, K., 2007.  Illumination invariant face recognition: A survey, IEEE. pp. 1–8.