# Floquet engineering of the Lifshitz phase transition in the Hubbard model

I. V. Iorsh1,2,3<EMAIL_ADDRESS>D. D. Sedov2 S. A. Kolodny2,3 R. E.
Sinitskiy3 O. V. Kibis3 1Abrikosov Center for Theoretical Physics, MIPT
University, Dolgoprudny 141701, Russia  2Department of Physics and
Engineering, ITMO University, Saint-Petersburg 197101, Russia  3Department of
Applied and Theoretical Physics, Novosibirsk State Technical University, Karl
Marx Avenue 20, Novosibirsk 630073, Russia

###### Abstract

Within the Floquet theory of periodically driven quantum systems, we
demonstrate that an off-resonant high-frequency electromagnetic field can
induce the Lifshitz phase transition in periodical structures described by the
one-dimensional repulsive Hubbard model with the nearest and next-nearest-
neighbor hopping. The transition changes the topology of electron energy
spectrum at the Fermi level, transforming it from the two Fermi points to the
four Fermi points, which facilitates the emergence of the superconducting
fluctuations in the structure. Possible manifestations of the effect and
conditions of its experimental observability are discussed.

##  I Introduction

Controlling electronic properties by an off-resonant high-frequency
electromagnetic field, which is based on the Floquet theory of periodically
driven quantum system (“Floquet engineering”), has been thoroughly explored in
last decades and by now is an established research area Casas  _et al._
(2001); Eckardt and Anisimovas (2015); Bukov  _et al._ (2015); Goldman and
Dalibard (2014); Basov  _et al._ (2017); Oka and Kitamura (2019) with numerous
predicted and experimentally observed phenomena in atomic systems Sørensen
_et al._ (2005); Eckardt  _et al._ (2005); Bermudez  _et al._ (2011); Kolovsky
(2011); Hauke  _et al._ (2012); Kibis_2022 ; Delone_book , quantum circuits
Son  _et al._ (2009); Kyriienko and Sørensen (2018); Deng  _et al._ (2015);
Ilin  _et al._ (2023), solid state systems Oka and Aoki (2009); Kibis (2010);
McIver  _et al._ (2020); Kibis  _et al._ (2017) and nanostructures Kibis
(2011); Yin  _et al._ (2011); Glazov and Ganichev (2014); Wang  _et al._
(2013); Park  _et al._ (2022); Iorsh_2022 ; Iurov_2022 . Since the off-
resonant field cannot be absorbed by electrons, it only dresses them,
modifying all electronic properties. Such a dressing leads both to the
renormalization of the existing terms in the electron Hamiltonian and to the
emergence of new terms (e.g. the spin-orbit coupling Ostermann  _et al._
(2019)), what drastically changes band structure and electronic transport.
Particularly, the electromagnetic dressing results in the substantial
modification of the electron interactions, inducing the electron states bound
at repulsive potentials Kibis  _et al._ (2020), the electron pairing in
systems containing charge carriers with different effective masses Kibis_2019
; Kibis  _et al._ (2021) and the new interactions (such as pair-hoppings and
density dependent tunneling) in the electronic systems with the non-parabolic
dispersion (e.g., within the simplest one-dimensional single-band Hubbard
model) Itin and Katsnelson (2015a). The interplay of modified electron
dispersion and electron interactions leads to the emergence of the many-body
phase transitions in the driven systems, including such correlated phases as
the Kitaev spin liquids Kumar  _et al._ (2022); Banerjee and Lin (2023); Sun
_et al._ (2023) and the superconducting phases arisen from the modification of
interaction potential Dehghani  _et al._ (2021) or the suppression of
competing correlated phases (e.g., charge density waves Dehghani  _et al._
(2020); Raines  _et al._ (2015)).

In the present article, we demonstrate theoretically that a dressing
electromagnetic field induces changing the topology of the Fermi surface (the
Lifshitz phase transition) for a periodical structure described by the one-
dimensional single-band Hubbard model with the nearest and next-nearest
neighbor hopping. As a consequence of the phase transition, the optically
induced superconductivity can appear. The article is organized as follows. In
Sec. II, the effective stationary Hamiltonian of the considered structure is
derived within the conventional Floquet theory of periodically driven quantum
systems. In Sec. III, the optically induced Lifshitz transition and the
associated superconductivity are discussed. The last two sections contain
conclusion and acknowledgements.

##  II Model

Figure 1: Sketch of the system under consideration: A one-dimensional periodic
structure consisting of identical unit cells marked by circles and irradiated
by a linearly polarized electromagnetic wave (EMW) with the electric field
amplitude $E$ and the frequency $\omega$, where $t$ and $t^{\prime}$ are the
nearest and next-nearest neighbor hopping coefficients, respectively,
describing the electron transitions between the unit cells, and $d$ is the
period of the structure.

Let us consider a one-dimensional infinite periodical structure irradiated by
a linearly polarized off-resonant electromagnetic field within the
conventional Hubbard model with the next and next-nearest neighbor hoppings
(see Fig. 1). The electron Hamiltonian of the system reads

$\displaystyle\hat{\cal H}(\tau)$ $\displaystyle=$
$\displaystyle\sum_{k,\sigma}\varepsilon(k-eA(\tau)/\hbar)\hat{c}_{k\sigma}^{\dagger}\hat{c}_{k\sigma}$
(1) $\displaystyle+$
$\displaystyle\sum_{k,k^{\prime},q}V_{0}\hat{c}_{k-q,\uparrow}^{\dagger}\hat{c}_{k^{\prime}+q,\downarrow}^{\dagger}\hat{c}_{k^{\prime}\downarrow}\hat{c}_{k\uparrow},$

where $\hat{c}_{k\sigma}^{\dagger}(\hat{c}_{k\sigma})$ are the creation
(annihilation) operators,

$\varepsilon(k)=-t\cos(kd)-t^{\prime}\cos(2kd)$ (2)

is the electron energy spectrum in the field’s absence, $k$ is the continuous
electron wave vector defined within the first Brillouin zone,
$A(\tau)=E\sin(\omega\tau)/\omega$ is the vector potential of the field,
$\tau$ is the time, $E$ is the electric field amplitude, $\omega$ is the field
frequency assumed to be far from all resonant frequencies of the electron
system, $\sigma=\uparrow,\downarrow$ is the spin quantum number, $d$ is the
period of the structure, $t$ and $t^{\prime}$ are the nearest and next-nearest
neighbor hopping coefficients, $V_{0}$ is the potential of the on-site Coulomb
repulsion of electrons, and $q$ is the wavevector transferred in the process
of electron-electron interaction. It should be noted that we consider an off-
resonant field, assuming the field frequency to be far from resonant
frequencies of the electronic system. As a consequence, the field absorption
is suppressed and, correspondingly, the heating of the system can be
neglected.

Within the conventional Floquet theory of periodically driven quantum systems,
one can introduce the unitary transformation $\hat{\cal
U}(\tau)=e^{i\hat{S}(\tau)}$, which transforms the periodically time-dependent
Hamiltonian (1) into the effective stationary Hamiltonian

$\hat{\cal H}_{\mathrm{eff}}=\hat{\cal U}(\tau)^{\dagger}\hat{\cal H}\hat{\cal
U}(\tau)-i\hbar\hat{\cal U}^{\dagger}(\tau)\partial_{\tau}\hat{\cal U}(\tau).$
(3)

There is the regular method to find the transformation operator
$\hat{S}(\tau)$ in the case of high-frequency field which satisfies the
condition $\hat{\cal H}_{n}/\hbar\omega\ll 1$, where $\hat{\cal H}_{n}$ are
the components of the Fourier expansion of the time-dependent Hamiltonian (1),

$\hat{\cal H}(\tau)=\sum_{n=-\infty}^{\infty}\hat{\cal
H}_{n}e^{in\omega\tau}.$ (4)

Namely, both the operator $\hat{S}(t)$ and the stationary Hamiltonian (3) can
be found as an $1/\omega$-expansion (the Floquet-Magnus expansion) Casas  _et
al._ (2001); Eckardt and Anisimovas (2015); Bukov  _et al._ (2015); Goldman
and Dalibard (2014), which leads to the effective stationary Hamiltonian

$\hat{\cal H}_{\mathrm{eff}}=\hat{\cal
H}_{0}+\sum_{n=1}^{\infty}\frac{[\hat{\cal H}_{n},\hat{\cal
H}_{-n}]}{n\hbar\omega}+{\it o}\left(\frac{\hat{\cal
H}_{n}}{\hbar\omega}\right),$ (5)

In the following, we will restrict the analysis by the high-frequency limit,

$t/\hbar\omega\ll 1.$ (6)

Under this condition, one can take into account only the main term of the
expansion (5), which reads

$\hat{\cal
H}_{0}=\sum_{k,\sigma}\tilde{\varepsilon}(k)\hat{c}_{k\sigma}^{\dagger}\hat{c}_{k\sigma}+\sum_{k,k^{\prime},q}V_{0}\hat{c}_{k-q,\uparrow}^{\dagger}\hat{c}_{k^{\prime}+q,\downarrow}^{\dagger}\hat{c}_{k^{\prime}\downarrow}\hat{c}_{k\uparrow},$
(7)

where

$\tilde{\varepsilon}(k)=-\tilde{t}\cos(kd)-\tilde{t}^{\prime}\cos(2kd)$ (8)

is the electron energy spectrum modified by the field,
$\tilde{t}=tJ_{0}(\eta)$ and $\tilde{t}^{\prime}=t^{\prime}J_{0}(2\eta)$ are
the hopping coefficients modified by the field (dressed hopping coefficients),
$J_{0}(\eta)$ is the Bessel function of the first kind, and
$\eta=eEd/\hbar\omega$ is the dimensionless parameter describing the strength
of electron-field interaction in the considered system. The effective
Hamiltonian (7) derived under the condition (6) has the clear physical
meaning. Namely, the condition (6) can be rewritten as $\tau_{0}/T\gg 1$,
where $\tau_{0}\sim h/t$ is the characteristic lifetime of an electron in an
unit cell of the periodic structure pictured in Fig. 1, and $T=2\pi/\omega$ is
the field period. Evidently, if the lifetime $\tau_{0}$ much exceeds the field
period $T$, the electron “feels” only the time-averaged Hamiltonian (1), which
is described by Eq. (7). It follows from the Hamiltonian (7), particularly,
that the high-frequency field modifies only the hopping coefficients, whereas
the interaction part of the Hamiltonian remains the same in the main order of
the Floquet-Magnus expansion. As to neglected terms of the Hamiltonian (5),
they contribute with the smallness $\sim(t/\hbar\omega)^{2}$ and can be
omitted under the condition (6) (see Appendix for more details).

##  III Results and discussion

Figure 2: Electron energy spectrum near the Fermi level (dashed horizontal
line) for the half-filled system with $t^{\prime}=0.2t$: (a) in the absence of
irradiation; (b) under irradiation with $\eta=eEd/\hbar\omega=2.2$.

The structure of the electron energy spectrum (8) at the Fermi level depends
on the ratio of the dressed hopping coefficients,

$\frac{\tilde{t}^{\prime}}{\tilde{t}}=\frac{{t}^{\prime}}{{t}}\frac{J_{0}(2\eta)}{J_{0}(\eta)}.$
(9)

Normally, the hopping coefficient $t$ much exceeds the coefficient
$t^{\prime}$. As a consequence, the Fermi surface in the field’s absence
consists of the two Fermi points with the wave vectors $\pm k_{F0}$ (see Fig.
2b). It follows from Eq. (9) that the dressing field changes the hopping
coefficients in the broad range. Particularly, the ratio (9) can be very large
near $\eta=\eta_{0}$, where $\eta_{0}\approx 2.409$ is the first root of the
Bessel function, $J_{0}(\eta_{0})=0$. Therefore, the field crucially changes
the Fermi surface, inducing the four Fermi-points with the wave vectors $\pm
k_{F1}$ and $\pm k_{F2}$ (see Fig. 2b). As a consequence, the Lifshitz
transition with the phase diagram plotted in Fig. 3 appears. It should be
noted that the Fermi surface has four points even at very low filling factors
near the root $\eta=\eta_{0}$.

Figure 3: Phase diagram of the Lifshitz transition from the two-point Fermi
surface to the four-point Fermi surface at $t/t^{\prime}=0.2$, where the
filling factor $f\in[0,\,2]$ is the number of electrons per unit cell.

To describe the effect of the complex structure of the Fermi surface on the
emergence of superconducting instability, one has account for the dynamical
screening of the repulsion potential $V_{0}$ by the electron-hole fluctuations
in the vicinity of the Fermi surface. Within the conventional random phase
approximation (RPA), the screened potential of electron-electron interaction
reads Mahan (2000)

$\displaystyle
V(q)=\frac{V_{0}}{1+V_{0}\Pi_{q}},\quad\Pi_{q}=\sum_{k}\frac{f_{0}(\tilde{\varepsilon}_{k})-f_{0}(\tilde{\varepsilon}_{k+q})}{\tilde{\varepsilon}_{k+q}-\tilde{\varepsilon}_{k}},$
(10)

where $\Pi_{q}$ is the static polarizability of the electrons, and
$f_{0}(\varepsilon)$ is the Fermi-Dirac distribution function. In the case of
complex Fermi contour, polarizability $\Pi_{q}$ has peaks at the wave vectors
$\pm 2k_{F1}$, $\pm 2k_{F2}$ and $\pm(k_{F1}-k_{F2})$. This leads to the
strong non-monotonic dependence of the screened potential $V(q)$ plotted in
Fig. 4a as a function of the wavevector $q$ transferred in the interaction
process.

Figure 4: The interaction potential for the half-filled system with
$t^{\prime}=0.2t$ under irradiation with $\eta=eEd/\hbar\omega=2.2$ in the
reciprocal space (a) and the coordinate space (b).

In the coordinate space, this non-monotonic profile projects to the negative
value of the interaction potential as shown in Fig. 4b. The attraction at
finite distance, which corresponds to these negative values, can favour the
Cooper instability with the formation of electron pairs. Following Kohn and
Luttinger Kohn and Luttinger (1965), such a mechanism results in the formation
of pairs with finite angular momentum, what leads to superconductivity. Within
the mean field approach Coleman (2015), the order parameter of the
superconducting phase, $\Delta(q)$, is defined by the self-consistent equation

$\displaystyle\Delta(k)=-\sum_{k^{\prime}}V(k-k^{\prime})\frac{\tanh(\mathcal{E}_{k^{\prime}}/(2T))\Delta(k^{\prime})}{2\mathcal{E}_{k^{\prime}}},$
(11)

where $T$ is the temperature,
$\mathcal{E}_{k}=\sqrt{\Delta^{2}(k)+(\tilde{\varepsilon}_{k}-\varepsilon_{F})^{2}}$
is the excitation spectrum, and $\varepsilon_{F}$ is the Fermi energy. To
estimate the critical temperature $T_{c}$ and the order parameter near this
temperature, it should be noted that $|\Delta(k)|\rightarrow 0$ at
$T\rightarrow T_{c}$ and, therefore, one can linearize Eq. (11) near the
temperature $T_{c}$ by setting
$\mathcal{E}_{k}\approx|\tilde{\varepsilon}_{k}-\varepsilon_{F}|$. Then the
linearized Eq. (11) reads

$\displaystyle\Delta(k)=-\sum_{k^{\prime}}V(k-k^{\prime})\frac{\tanh(|\tilde{\varepsilon}_{k}-\varepsilon_{F}|/2T)}{2|\tilde{\varepsilon}_{k}-\varepsilon_{F}|}\Delta(k^{\prime}).$
(12)

It can be seen that the right-hand side of Eq. (12) acts on the function
$\Delta(k)$ as a linear integral operator with the eigenvalues
$\lambda=\lambda(T)$. The maximal temperature $T$ which satisfies the self-
consistent equation (11) is the critical temperature $T_{c}$, what corresponds
to $\lambda(T_{c})=1$. As a result of solving this equation, we arrive at the
profile of the order parameter slightly below the temperature $T_{c}$, which
is plotted in Fig. 5. It follows from the plots that the order parameter has
nodes and is symmetric with respect to the replacement $q\rightarrow-q$.
Following the conventional terminology, such a behavior of the order parameter
corresponds to the spin-singlet nodal superconductivity. It should be noted
that the appearance of the nodes in the profile of the superconducting gap is
typical feature of the Kohn-Luttinger mechanism Kohn and Luttinger (1965).
Within this model, the interaction potential is non-monotonic in the momentum
space and changes its sign in the real space. Thus, the potential is repulsive
at short distances and attractive at larger distances. Such an interaction
potential structure favors the Cooper pair wave function with a complex
profile in both the momentum space and the coordinate space. In turn, the
profile of the Cooper wave function defines the order parameter profile.

Figure 5: Order parameter (in arbitrary units) for the half-filled system with
$t^{\prime}=0.2t$ under irradiation with $\eta=eEd/\hbar\omega=2.2$ in the
reciprocal space (a) and the coordinate space (b).

Certainly, the RPA approach should be used carefully in the strong coupling
regime. Therefore, further analysis of the strong coupling regime may be
required for specific materials. However, the superconducting instability
emerges at arbitrarily small values of interaction (at least, in the zero
temperature limit) provided that the dynamically screened interaction becomes
attractive. It should be noted also that the mean-field results should be
treated with care in the one-dimensional system under consideration since
fluctuations may play crucial role even in the finite systems. Particularly,
the superconducting correlations are $\langle\Delta(x)\Delta(0)\rangle\sim
x^{-\gamma}$, where $\gamma$ is the critical exponent. It is well-known that
there are multiple instabilities coexisting with superconducting ones (e.g.
ferromagnetic or charge density wave instabilities). In a sufficiently long
structure, an instability with the smallest value of $\gamma$, which decays
most slowly, is dominant. To find the dominant instability, one has to
calculate the critical exponent $\gamma$, what is very complicated problem in
the most general case. However, in the particular case of the weak coupling
regime, $V_{0}/\tilde{t}\lesssim 1$, the calculation of $\gamma$ within the
Hubbard model can be made with the help of renormalization group technique
Balents and Fisher (1996) or the Hartree-Fock approximation Sano (2000). As a
result of the calculation, it has been shown that the superconducting
fluctuations are dominant in a wide range of parameters in the phase with the
four points at the Fermi surface within the discussed $t-t^{\prime}$ one-
dimensional Hubbard model Nishimoto  _et al._ (2008). It should be noted that
the strong coupling regime, $V_{0}/\tilde{t}\gg 1$, can also be realized in
our set-up since the dressed hopping coefficient, $\tilde{t}=tJ_{0}(\eta)$, is
very small near a zero of the Bessel function. The discussed one-dimensional
Hubbard model with the next-nearest hopping in the strong coupling regime has
been analyzed numerically by the density matrix renormalization group methods
Nishimoto  _et al._ (2008). It follows from this analysis that the spin-
singlet superconductive fluctuations remain dominant also in the strong
coupling regime if the ratio (9) satisfies the condition
$|\tilde{t}^{\prime}/\tilde{t}|>1$. Since the order parameter behavior plotted
in Fig. 5 and discussed above corresponds to the spin-singlet
superconductivity, the superconductivity can take place in the considered one-
dimensional Hubbard chain of finite length.

It should be noted that the discussed phenomena can also occur in coupled
Hubbard chains — so-called two-leg Hubbard ladders Lin_1997  — which are used,
particularly, to describe the wide class of novel unconventional
superconductors based on the copper oxides Matsukawa  _et al._ (2004). The
two-leg Hubbard ladders comprise two parallel chains of atoms with the intra-
chain hopping $t$ and the inter-chain hopping $t^{\prime}$. In such systems,
the ratio (9) can be controlled by changing the angle between the field
polarization and the system axis, what will result in the same optically
induced Lifshitz transition and the corresponding superconductivity.
Completing the discussion, it should be stressed that the Lifshitz transitions
can be achieved also in a few-layer graphene via the static gate voltage or
the electromagnetic dressing Iorsh  _et al._ (2017). Since the emergence of
unconventional superconductivity was observed recently in the gating trilayer
graphene Zhou  _et al._ (2021), one can expect the experimentally observable
similar effects in the systems described by the Hubbard model.

To estimate the experimental feasibility of the effect, one can apply the
dressing fields which were recently used to observe the Floquet gap opening in
graphene McIver  _et al._ (2020). There the graphene sample was irradiated by
short (500 fs) pulses with central frequency $\sim 10$ THz, the mean fluence
of $1~{}\mathrm{mJcm^{-2}}$ and the peak intensity of $1\mathrm{GWcm^{-2}}$.
For these field parameters and the chain period $d\approx 1$nm, the electron-
field interaction parameter is $\eta\approx 0.7$. Thus, the currently
available THz fields are appropriate for observation of the discussed effects.

It should be noted that the considered system may support many correlated
phases, including the Mott insulating phases and the ferromagnetic phases.
However, the question of the dominant instability (charge waves or spin
density waves, superconducting phase or ferromagnetic phase, etc) is the
numerically challenging problem which cannot be solved in a general form.
Therefore, this problem should be analyzed carefully for specific structures
planned to be studied experimentally.

##  IV Conclusion

We showed theoretically that an off-resonant high-frequency electromagnetic
field can induce the Lifshitz phase transition in periodical structures
described by the one-dimensional repulsive Hubbard model with the nearest and
next-nearest neighbor hopping. The transition changes the topology of electron
energy spectrum at the Fermi level, transforming it from the two Fermi-points
to the four Fermi-points. This facilitates the emergence of the spin-singlet
superconducting fluctuations in the structure, which can be experimentally
observable for the currently available THz fields.

###### Acknowledgements.

The reported study was funded by the Russian Science Foundation (project
20-12-00001).

##  Appendix A The effective Hamiltonian

To describe the electron-field interaction with the Hamiltonian (1), it is
convenient to introduce the two field-dependent dimensionless parameters,
$\eta=eEd/\hbar\omega$ and $\xi=t/\hbar\omega$, which can be varied
independently by varying the field amplitude $E$ and the field frequency
$\omega$. In the present theory, we assume the field frequency to be high
enough to satisfy the condition $\xi\ll 1$. This allows to describe the
dependence of electronic properties on the parameter $\xi$ within the
conventional perturbation theory, expanding the Hamiltonian into the
$\xi$-power series. As to the field amplitude $E$, it is not assumed to be
small. Correspondingly, the parameter $\eta$ can be arbitrary large and will
be taken into account accurately.

To proceed, let us expand the periodically time-dependent electron energy
$\varepsilon(k-eA(\tau)/\hbar)$ in the Hamiltonian (1) into the Fourier series

$\varepsilon(k-eA(\tau)/\hbar)=\tilde{\varepsilon}(k)+\sum_{m}\varepsilon_{m}(k)e^{itm\tau/\hbar\xi},$
(13)

where

$\tilde{\varepsilon}_{m}(k)=-tJ_{m}(\eta)\cos kd-t^{\prime}J_{m}(2\eta)\cos
2kd,$ (14)

$J_{m}(z)$ is the Bessel function of the first kind, $m=\pm 1,\pm 2,...$ is
the order of the Bessel function, and
$\tilde{\varepsilon}(k)=\varepsilon_{0}(k)$. Then the time-independent part of
the Hamiltonian (1) reduces to the unperturbed Hubbard Hamiltonian with the
renormalized tunnelling coefficients, $\tilde{t}=tJ_{0}(\eta)$ and
$\tilde{t}^{\prime}=t^{\prime}J_{0}(2\eta)$. To simplify the following
analysis, let us apply the time-dependent unitary transformation $\hat{\cal
U}_{0}=\exp(i\hat{S}_{0})$, where

$\displaystyle\hat{S}_{0}=-i\xi\sum_{k}\sum_{m}\frac{\varepsilon_{m}(k)}{mt}e^{itm\tau/\hbar\xi}\hat{c}_{k\sigma}^{\dagger}\hat{c}_{k\sigma}$
(15)

is the transformation operator. Then the transformed Hamiltonian (1) reads

$\displaystyle\hat{\cal H}^{\prime}(\tau)=\hat{\cal U}_{0}\hat{\cal
H}\hat{\cal U}_{0}^{\dagger}-i\hat{\cal U}_{0}\partial_{\tau}\hat{\cal
U}_{0}^{\dagger}$
$\displaystyle=\sum_{k}\tilde{\varepsilon}(k)\hat{c}_{k\sigma}^{\dagger}\hat{c}_{k\sigma}+\sum_{k,k^{\prime},q}\tilde{V}(\tau)\hat{c}_{k-q,\uparrow}^{\dagger}\hat{c}_{k^{\prime}+q,\downarrow}^{\dagger}\hat{c}_{k^{\prime}\downarrow}\hat{c}_{k\uparrow},$
(16)

where

$\displaystyle\tilde{V}(\tau)=V_{0}\exp\left[\xi\sum_{m}\frac{\varepsilon_{m}(k)+\varepsilon_{m}(k^{\prime})}{mt}\exp(itm\tau/\hbar\xi)\right]$
$\displaystyle\times\exp\left[-\xi\sum_{m}\frac{\varepsilon_{m}(k+q)+\varepsilon_{m}(k-q)}{mt}\exp(itm\tau/\hbar\xi)\right].$
(17)

Comparing Eqs. (1) and (A), one can see that the Hamiltonian (1) in the
field’s absence ($A(\tau)=0$) and the Hamiltonian (A) have the same form with
the replacements $\varepsilon(k)\rightarrow\tilde{\varepsilon}(k)$ and
$V_{0}\rightarrow\tilde{V}(\tau)$. Therefore, the function
$\tilde{\varepsilon}(k)$ defined by Eq. (8) should be treated as the electron
energy spectrum dressed by the field, whereas the function $\tilde{V}(\tau)$
defined by Eq. (A) is the dressed interaction potential discussed below.

Since the interaction part of the Hamiltonian (A) becomes periodically time-
dependent, it can be expanded into the Fourier series,
$V(\tau)=V_{0}\sum_{l}V^{(l)}e^{ilt\tau/\hbar\xi}$, and results for $\xi\ll 1$
in $V^{(0)}=V_{0}+\mathcal{O}(\xi^{2})$ and $V^{(j\neq 0)}=\mathcal{O}(\xi)$.
Applying the Floquet-Magnus expansion (5) to the Hamiltonian (A), one can find
that the leading correction to the effective static Hamiltonian is
proportional to $\xi\sum_{j\neq 0}[V^{(j)},V^{(-j)}]\sim\mathcal{O}(\xi^{3})$.
Thus, up to the linear order in $\xi$, the effective static interaction term
remains unchanged, $V(\tau)\approx V_{0}+\mathcal{O}(\xi^{2})$. As to the non-
linear terms, they have a complicated structure describing the field-induced
corrections to the interaction between electrons localized at different atoms,
including such processes as the electron pair hopping, the exchange
interaction, the dissociation of electron pairs localized at a single atom and
the density dependent tunnelling (see, e.g., Ref. Itin and Katsnelson, 2015a).
However, the field-induced corrections to the electron interactions decays
exponentially as $\xi^{2n}$, where $nd$ is the distance between the
interacting electrons ($n=1,2,3,...$). Therefore, these corrections can be
neglected as a first approximation under the condition $\xi\ll 1$. As a
result, we arrive at the effective static Hamiltonian (7).

## References

  * Casas  _et al._ (2001) F. Casas, J. Oteo, and J. Ros, Floquet theory: exponential perturbative treatment, J. Phys. A: Mathematical and General 34, 3379 (2001).
  * Eckardt and Anisimovas (2015) A. Eckardt and E. Anisimovas, High-frequency approximation for periodically driven quantum systems from a Floquet-space perspective, New J. Phys. 17, 093039 (2015).
  * Bukov  _et al._ (2015) M. Bukov, L. D’Alessio, and A. Polkovnikov, Universal high-frequency behavior of periodically driven systems: From dynamical stabilization to Floquet engineering, Adv. Phys. 64, 139 (2015).
  * Goldman and Dalibard (2014) N. Goldman and J. Dalibard, Periodically driven quantum systems: Effective Hamiltonians and engineered gauge fields, Phys. Rev. X 4, 031027 (2014).
  * Basov  _et al._ (2017) D. Basov, R. Averitt, and D. Hsieh, Towards properties on demand in quantum materials, Nat. Mater. 16, 1077 (2017).
  * Oka and Kitamura (2019) T. Oka and S. Kitamura, Floquet engineering of quantum materials, Ann. Rev. Cond. Matter Phys. 10, 387 (2019).
  * Sørensen  _et al._ (2005) A. S. Sørensen, E. Demler, and M. D. Lukin, Fractional quantum Hall states of atoms in optical lattices, Phys. Rev. Lett. 94, 086803 (2005).
  * Eckardt  _et al._ (2005) A. Eckardt, T. Jinasundera, C. Weiss, and M. Holthaus, Analog of photon-assisted tunneling in a Bose-Einstein condensate, Phys. Rev. Lett. 95, 200401 (2005).
  * Bermudez  _et al._ (2011) A. Bermudez, T. Schaetz, and D. Porras, Synthetic gauge fields for vibrational excitations of trapped ions, Phys. Rev. Lett. 107, 150501 (2011).
  * Kolovsky (2011) A. R. Kolovsky, Creating artificial magnetic fields for cold atoms by photon-assisted tunneling, Europhys. Lett. 93, 20003 (2011).
  * Hauke  _et al._ (2012) P. Hauke, O. Tieleman, A. Celi, C. Ölschläger, J. Simonet, J. Struck, M. Weinberg, P. Windpassinger, K. Sengstock, M. Lewenstein, _et al._ , Non-abelian gauge fields and topological insulators in shaken optical lattices, Phys. Rev. Lett. 109, 145301 (2012).
  * (12) O. V. Kibis, Floquet theory of spin dynamics under circularly polarized light pulses, Phys. Rev. A 105, 043106 (2022).
  * (13) N. B. Delone and V. P. Krainov, Multiphoton Processes in Atoms (Spinger-Verlag, Berlin, 2000).
  * Son  _et al._ (2009) S.-K. Son, S. Han, S.-I. Chu, _et al._ , Floquet formulation for the investigation of multiphoton quantum interference in a superconducting qubit driven by a strong ac field, Phys. Rev. A 79, 032301 (2009).
  * Kyriienko and Sørensen (2018) O. Kyriienko and A. S. Sørensen, Floquet quantum simulation with superconducting qubits, Phys. Rev. Appl. 9, 064029 (2018).
  * Deng  _et al._ (2015) C. Deng, J.-L. Orgiazzi, F. Shen, S. Ashhab, and A. Lupascu, Observation of Floquet states in a strongly driven artificial atom, Phys. Rev. Lett. 115, 133601 (2015).
  * Ilin  _et al._ (2023) D. Ilin, A. V. Poshakinskiy, A. N. Poddubny, and I. Iorsh, Frequency combs with parity-protected cross-correlations and entanglement from dynamically modulated qubit arrays, Phys. Rev. Lett. 130, 023601 (2023).
  * Oka and Aoki (2009) T. Oka and H. Aoki, Photovoltaic Hall effect in graphene, Phys. Rev. B 79, 081406 (2009).
  * Kibis (2010) O. Kibis, Metal-insulator transition in graphene induced by circularly polarized photons, Phys. Rev. B 81, 165433 (2010).
  * McIver  _et al._ (2020) J. W. McIver, B. Schulte, F.-U. Stein, T. Matsuyama, G. Jotzu, G. Meier, and A. Cavalleri, Light-induced anomalous Hall effect in graphene, Nat. Phys. 16, 38 (2020).
  * Kibis  _et al._ (2017) O. Kibis, K. Dini, I. Iorsh, and I. Shelykh, All-optical band engineering of gapped Dirac materials, Phys. Rev. B 95, 125401 (2017).
  * Kibis (2011) O. Kibis, Dissipationless electron transport in photon-dressed nanostructures, Phys. Rev. Lett. 107, 106802 (2011).
  * Yin  _et al._ (2011) C. Yin, N. Tang, S. Zhang, J. Duan, F. Xu, J. Song, F. Mei, X. Wang, B. Shen, Y. Chen, _et al._ , Observation of the photoinduced anomalous Hall effect in GaN-based heterostructures, Appl. Phys. Lett. 98, 122104 (2011).
  * Glazov and Ganichev (2014) M. Glazov and S. Ganichev, High frequency electric field induced nonlinear effects in graphene, Phys. Rep. 535, 101 (2014).
  * Wang  _et al._ (2013) Y. Wang, H. Steinberg, P. Jarillo-Herrero, and N. Gedik, Observation of Floquet-Bloch states on the surface of a topological insulator, Science 342, 453 (2013).
  * Park  _et al._ (2022) S. Park, W. Lee, S. Jang, Y.-B. Choi, J. Park, W. Jung, K. Watanabe, T. Taniguchi, G. Y. Cho, and G.-H. Lee, Steady Floquet-Andreev states in graphene Josephson junctions, Nature 603, 421 (2022).
  * (27) I. V. Iorsh, D. A. Zezyulin, S. A. Kolodny, R. E. Sinitskiy, and O. V. Kibis, Floquet engineering of excitons in semiconductor quantum dots, Phys. Rev. B 105, 165414 (2022).
  * (28) A. Iurov, L. Zhemchuzhna, G. Gumbs, D. Huang, W.-K. Tse, K. Blaise and C. Ejiogu, Floquet engineering of tilted and gapped Dirac bandstructure in 1T′-MoS2, Sci. Rep. 12, 21348 (2022).
  * Ostermann  _et al._ (2019) S. Ostermann, H.-W. Lau, H. Ritsch, and F. Mivehvar, Cavity-induced emergent topological spin textures in a Bose-Einstein condensate, New J. Phys. 21, 013029 (2019).
  * Kibis  _et al._ (2020) O. V. Kibis, M. V. Boev, and V. M. Kovalev, Light-induced bound electron states in two-dimensional systems: Contribution to electron transport, Phys. Rev. B 102, 075412 (2020).
  * (31) O. V. Kibis, Electron pairing in nanostructures driven by an oscillating field, Phys. Rev. B 99, 235416 (2019).
  * Kibis  _et al._ (2021) O. Kibis, M. Boev, and V. Kovalev, Optically induced hybrid Bose-Fermi system in quantum wells with different charge carriers, Opt. Lett. 46, 5316 (2021).
  * Itin and Katsnelson (2015a) A. Itin and M. Katsnelson, Effective Hamiltonians for rapidly driven many-body lattice systems: Induced exchange interactions and density-dependent hoppings, Phys. Rev. Lett. 115, 075301 (2015a).
  * Kumar  _et al._ (2022) U. Kumar, S. Banerjee, and S.-Z. Lin, Floquet engineering of Kitaev quantum magnets, Commun. Phys. 5, 157 (2022).
  * Banerjee and Lin (2023) S. Banerjee and S.-Z. Lin, Emergent orbital magnetization in Kitaev quantum magnets, SciPost Physics 14, 127 (2023).
  * Sun  _et al._ (2023) B.-Y. Sun, N. Goldman, M. Aidelsburger, and M. Bukov, Engineering and probing non-abelian chiral spin liquids using periodically driven ultracold atoms, PRX Quantum 4, 020329 (2023).
  * Dehghani  _et al._ (2021) H. Dehghani, M. Hafezi, and P. Ghaemi, Light-induced topological superconductivity via Floquet interaction engineering, Phys. Rev. Res. 3, 023039 (2021).
  * Dehghani  _et al._ (2020) H. Dehghani, Z. M. Raines, V. M. Galitski, and M. Hafezi, Optical enhancement of superconductivity via targeted destruction of charge density waves, Phys. Rev. B 101, 224506 (2020).
  * Raines  _et al._ (2015) Z. M. Raines, V. Stanev, and V. M. Galitski, Enhancement of superconductivity via periodic modulation in a three-dimensional model of cuprates, Phys. Rev. B 91, 184506 (2015).
  * Mahan (2000) G. D. Mahan,  _Many-Particle Physics_ (Springer Science & Business Media, Berlin, 2000).
  * Kohn and Luttinger (1965) W. Kohn and J. M. Luttinger, New mechanism for superconductivity, Phys. Rev. Lett. 15, 524 (1965).
  * Coleman (2015) P. Coleman,  _Introduction to Many-Body Physics_ (Cambridge University Press, Cambridge, 2015).
  * Balents and Fisher (1996) L. Balents and M. P. A. Fisher, Weak-coupling phase diagram of the two-chain Hubbard model, Phys. Rev. B 53, 12133 (1996).
  * Sano (2000) K. Sano, Luttinger-liquid parameter of Hubbard chain and Hubbard ladder, J. Phys. Soc. Japan 69, 1000 (2000).
  * Nishimoto  _et al._ (2008) S. Nishimoto, K. Sano, and Y. Ohta, Phase diagram of the one-dimensional Hubbard model with next-nearest-neighbor hopping, Phys. Rev. B 77, 085119 (2008).
  * (46) H.-H. Lin, L. Balents, and M. P. A. Fisher, N-chain Hubbard model in weak coupling, Phys. Rev. B 56, 6569 (1997).
  * Matsukawa  _et al._ (2004) M. Matsukawa, Y. Yamada, M. Chiba, H. Ogasawara, T. Shibata, A. Matsushita, and Y. Takano, Superconductivity in Pr2Ba4Cu7O15-δ with metallic double chains, Phys. C: Superconductivity and its applications 411, 101 (2004).
  * Iorsh  _et al._ (2017) I. V. Iorsh, K. Dini, O. V. Kibis, and I. A. Shelykh, Optically induced Lifshitz transition in bilayer graphene, Phys. Rev. B 96, 155432 (2017).
  * Zhou  _et al._ (2021) H. Zhou, T. Xie, T. Taniguchi, K. Watanabe, and A. F. Young, Superconductivity in rhombohedral trilayer graphene, Nature 598, 434 (2021).


11institutetext: Machine Learning Team, Pioneer Alpha Ltd
11email<EMAIL_ADDRESS>
11email<EMAIL_ADDRESS>
11email<EMAIL_ADDRESS>

# Real-Time Face Recognition System for Remote Employee Tracking

Mohammad Sabik Irbaz  MD Abdullah Al Nasim  Refat E Ferdous

###### Abstract

During the COVID-19 pandemic, most of the human-to-human interactions have
been stopped. To mitigate the spread of deadly coronavirus, many offices took
the initiative so that the employees can work from home. But, tracking the
employees and finding out if they are really performing what they were
supposed to turn out to be a serious challenge for all the companies and
organizations who are facilitating ”Work From Home”. To deal with the
challenge effectively, we came up with a solution to track the employees with
face recognition. We have been testing this system experimentally for our
office. To train the face recognition module, we used FaceNet with KNN using
the Labeled Faces in the Wild (LFW) dataset and achieved 97.8% accuracy. We
integrated the trained model into our central system, where the employees log
their time. In this paper, we discuss in brief the system we have been
experimenting with and the pros and cons of the system.

###### keywords:

Face Recognition, Face Detection, Computer Vision

##  1 Introduction

Face recognition is an innovative identifying system that works on recognizing
the face of a potential individual after comparing it with a source image
stored in a database by a vision system. This technology extracts features
from an input image and recognizes them by detecting the specific and
particular information about an individual’s face. We can see many
applications of this, for example, access and security, criminal
identification, online payments, design of a smart home, etc. In the face
recognition and detection process, no human cooperation is needed, which makes
it the best method for person identification [1]. There are various biometric
authentication systems available, face recognition is one of them. In
biometric-based strategies, one’s physiological and behavioural attributes
have been analyzed with a specified end goal to determine their uniformity
[2]. Hence compared to other biometric-based systems, face recognition has
been considered as one of the most significant biometric authentication
techniques among the most constitutive applications.

Starting from 2020, during the COVID-19 pandemic, many offices took the
initiative so that the employees can work from home. But, tracking the
employees and finding out if they are really there to work had been a serious
challenge for the administration. Existing time tracker solutions can be
easily fooled, which we have already seen in our company. To deal with the
challenge and build a system that cannot be fooled, we came up with a solution
to track the employees with face recognition. We have been testing this system
internally. To train the face recognition module, we used FaceNet with KNN
using the Labeled Faces in the Wild (LFW) dataset [3].

We organized our paper as follows: In section II, we presented the existing
literature methods that helped us to reach our goal. In section III, we gave a
description of the dataset we used in our model. Section IV provides a summary
of our methodology. Experimental evaluations have been discussed in section V.
Finally, in section VI, we give a brief discussion on future work and
conclusion.

##  2 Related Works

In a review till 2006 on face recognition [4][5], we can find various
databases for face recognition. This is the start of face recognition
revolution. We find more interesting works in 2011 by Deniz et. al. [6]. As
HOG operates on local cells, it is invariant to geometric transformations and
photometric transformations, except for object orientation. This experiment
has used single angle orientation to allow more differentiation between
patterns. Overlapping in HOG significantly improves the performance of
detection and identification which would have been otherwise quite difficult
in presence of poor lighting conditions. To remove redundancy in the data and
avoid over fitting, they proposed to use, dimensionality reduction in the HOG
representation. To provide robustness to facial feature detection, they
propose uniform sampling of the HOG features.

Fast forward to very recent works we can find Prasad et. al. [7] using Deep
Learning based representations. They input raw data in convolve filters in
different levels that automatically detect underlying high levels from
labelled or unlabeled data. Face verification approaches classified into three
sets. First one is used to extract face feature vectors and process them by
classifiers. Second one directly enhances the proof loss for matching and non-
matching pairs. Third one combined identification and verification constraints
to improve the deep face model. For identifying faces two approaches are
explained here. First one is effective Convolutional Neural Network designs
for biometric face recognition. Second one is, explanation of a face plan
which is based on two models (VGG-Face and Lightened CNN). This study has
discovered that Deep Learning based face recognition is more robust to
misalignment facial images.

##  3 Literature Review

###  3.1 FaceNet

FaceNet model was developed by Google researchers Schroff et al., which can
reduce the difficulties in face detection and verification process and achieve
the desired result [8]. The FaceNet algorithm works by taking an input of an
image of a person’s face and converting the high dimensional vectors of that
image into a 128 low dimensional Euclidean Space. This conversion process is
also known as embedding. The FaceNet model uses the Deep Convolutional
Networks, which makes the most effective use of its embedding, compared to
using intermediate bottleneck layers as a test of previous deep learning
approaches [9]. This method is known as one-shot learning. In this way,
triplet loss has been trained over the produced FaceNet model, which can
achieve a result of uniformity and distinction over the given collections of
images. After collecting the result of similarities between the faces we can
consider the FaceNet embedding as feature vectors. After creating the vector
space, operations like face recognition, classification, verification, and
clustering could be implemented more spontaneously. Additionally, FaceNet is
able to train any difficult learning system of any single model that
illustrates an entire goal system by collecting all of the factors at the same
time and this is the most significant part of the FaceNet model.

###  3.2 Triplet Loss

In Deep Neural Networks, the triplet-based loss function used to learn the
mapping is an adaption of Kilian Weinberger’s Large Margin Nearest Neighbor
(LMNN) classifier [10]. Triplet loss is one of the best ways to learn good
128-dimensional embedding for each individual face. [11] As it is an
equivalent of the loss function, a comparison between the baseline input with
positive input and negative input has been observed here. That’s why to
compare to other loss functions, triplet loss is more compatible with face
verification.
The triplet loss function has been categorized into three main parts. They
are:

  * •

Anchor

  * •

Positive

  * •

Negative

Here anchor is the starting input point, which is used for comparison.
Positive denote similar identity like the anchor and negative denotes the
different identity from the anchor. The distance between the anchor and a
chosen image will be minimized if that image is positive, which denotes that
the compared two images have a similar identity. In contrast, the distance
will be maximized for different images.

###  3.3 Triplet Selection

Triplet selection is a core part of improving the performance of triplet loss.
So, it is essential to select all hard triplets. Training with randomly hard
triplets almost does not converge whereas training with the hardest triplets
often leads to a bad local solution. There are two types of triplet loss
methods. They are online and offline triplet mining methods. Offline triplet
mining worked with a full pass on the training set to generate triplets. Since
it needs to update regularly, this method is not efficient. So, we used online
triplet mining.
First of all, we picked all the necessary triplets. Depending on the loss we
classify the triplets in three categories: easy, semi-hard, hard. As easy
triplets are with loss 0 or very small loss so, we intended not to take them.
Next, the semi-hard triplets are also far away from the anchor than the ideal
model. In a hard batch, we select a set of hard anchor-positive pairs and then
select the hardest negatives within the mini-batch. Here small mini-batches
are used since they improve the process of converging.

###  3.4 MTCNN

MTCNN or multi cascade convolutional neural network is an improved neural
network algorithm in the detection of faces and facial landmarks on images.
The purpose of the proposed MTCNN is to form an avalanched structure and use
it as material for multi-task knowledge to forecast the location of the face
and it is marked in a coarse-to-fine method. And in its application, MTCNN is
able to detect real-time with fairly high accuracy [13]. The MTCNN
architecture is made up of three convolutional networks, which are connected
in a cascade. The first one is, Proposal Network(P-Net), which main task is
giving a boundary box for each face bounds. In this process, a large number of
face detection and false detection has been taken place. The second one is,
Refine Network(R-Net), which is slightly similar to P-Net. But R-Net includes
more appropriate bounding boxes compare to P-Net. Thus, making a refinement of
the result by eliminating most of the false detection and aggregate bounding
boxes [12]. The last one is, Output Network (O-Net), which produces an output
different from P-Net and R-Net. There are three types of output that O-Net
produces. The output of the first layer is used for measuring face
probabilities in the box. The second layer is used to give the boundary
coordinates in the box. And the last layer is used for the coordinates of the
five landmarks of the faces [9].

##  4 Dataset

We used Labeled Faces in the wild (LFW) [3] here to uphold the effectiveness
of the face recognition process. It has been said that the LFW dataset is a
standard for the face verification process which is mainly developed to
perusing the unconstrained problem of face recognition. This dataset consists
of around 13K images collected from the web. Also, each image has been labeled
with the name of that specific person. There are four different sets of LFW
images including the original and three different types of ‘aligned’ images.
Among them, deep funneled images produce superior results for most face
verification algorithms over the original and funneled images. Hence, the
dataset we used here is the deep funneled version.

Figure 1: Full Pipeline overview of our methodology

##  5 Methodology

To track the presence of the employees, we initially put all the information
and images of the employees in our central database. Then we used those images
during the face recognition period. Later, we took the history and kept that
in a separate relational database. So, in short, our system has three stages.

###  5.1 Adding, Updating and Deleting Employee Details

An admin can add new users when a new employee joins the company, update user
details if anything changes related to any employee, and delete employee
details when an employee leaves the company. While doing so, the face
recognition system does not need to change anything. The recognition system
can adapt with the user changes dynamically.

###  5.2 Face Recognition

We used the module in Figure 2. We trained the MTCNN with triplet loss to
train the module to detect face encodings from faces. After that, we compared
the employee images using the KNN classifier. This returned us with the class
i.e name of the person and bounding box identifying where the faces are.

###  5.3 Tracking and Storing Face Recognition History

Every time any employee starts their work session. The video feed is taken
from the webcam randomly from different portions of the daily sessions. If an
employee who started his session is not found in video feeds consecutively
then an automatic alert goes to the admin and the employee. All the face
recognition history is stored as archive with employee details. Admins are not
allowed to access the archive to keep the trasparency between employees.

Figure 2: Face Recognition Module

##  6 Experiment Analysis

Experimental Setup

We did all our experiments using Google Colab which is a hosted Jupyter
notebook service. We used it because Google Colab provides free GPU for 12
hours a day. In our experiments we used Numpy, Pandas, etc. for data
processing and Tensor for training and testing. PyTorch [12] is an open source
machine learning framework. We chose 10% randomly as validation set from the
LFW dataset. We trained all the models for 10 epochs with Adam Optimizer [13].
After taking a pair of two face images, we measured a squared L2 distance
threshold, which is mainly used for classifying the similar and different
images. The optimal threshold value we got here is 1.24. That means this
threshold value can accurately classify all pairs of images.

Result Analysis

Here, 128-dimensional float vector was used throughout the training season.
This 128-byte dimensional vector tightly illustrate each face, which is most
suitable for recognition and clustering. Here we took input images of size
220*220 pixels and trained our model through this. But still, it works good on
80*80 images, which is admissible. Table 1 shows the size of training images
and their corresponding accuracies.

Table 1: Effect of training samples on performance No. of Training Images | Accuracy
---|---
70 | 90.5
700 | 92.3
7,000 | 95.6
Full Dataset | 97.8

Finally, face recognition task was completed by our pre-trained model based on
FaceNet model. A number of facial encoding was generated by our trained model
and by using Euclidean distance, which can show a comparison of facial
coordinates for different samples. This results in a distance, which directly
assembles to the measure of resemblance. This acquainted our model whether the
face of input image matches with anyone or not. And that’s how we trained our
model. For LFW dataset, we achieved a classification accuracy of 97.8% . So,
we can say that, in face recognition method, FaceNet with KNN shows up the
accurate results with confident.

##  7 Qualitative Evaluation

In Figure 1, we get a full overview of our working prototype. Our prototype
takes the webcam video feed given by users which passes through the Flask [14]
backend to the Face Recognition module. Later, we get the face encoding and
bounding box from the module. We store detected faces in the database as
history. We used this system experimentally in our company on remote
employees. In short, they also acted as evaluators. Our backend can
effectively extract faces and recognize them in a dynamic way. The evaluators
were mainly interested in how well the software can perform rather than the
user interface. The software was very much self-explanatory and most of our
evaluators were expert enough. We asked them to look through the eyes of
general people. They did not face any confusion while evaluating. Some
evaluators complained that if they were looking somewhere else during random
time video feed extraction, their faces were not detected. This is a limit of
the dataset also which we plan to extend in the future. However, in most
cases, even with blurry cameras our system could identify the faces. We did
not find any case where our model predicted the wrong evaluator. We found out
that even when our evaluators added 10 years old images, our system could find
them.

##  8 Applications

Even though we used this whole pipeline for remote employee tracking. This can
be used in various other cases:

  * •

Access and Security:
a) Instead of using passcode, mobile phones, laptops and other consumer
electronics will be accessed via the owner’s facial feature. Apple, Samsung,
Xiaomi already installed FaceTech in their phones.
b) In the near future, consumers will get into their cars, houses and other
physical locations simply by looking at them.
c) Innovative facial security could be helpful for any company or organization
where sensitive data needs to keep tight control on who enters their
facilities.

  * •

Payments:
a) In 2016, Mastercard launched a new selfie pay app called “Mastercard
Identity” check. Customers open the app to confirm a payment using the camera.
b) With FaceTech, customers wouldn’t even need their cards.
c) In the future, we can do the same for online payments.
d) Automatic face recognition to prohibit deduplication of identity to
authentication of mobile payment. This is mainly used for face spoof attacks
also known as biometric sensor presentation attacks, where a photo or video of
an authorized person’s face could be used to gain access.

  * •

Criminal Identification: FaceTech can be used to keep unauthorized people out
of facilities.

  * •

Smart Home: The design of smart homes or cities has become one of the things
that many researchers have focused on. Especially people with special needs or
patients.

  * •

Video Surveillance:
a) Surveillance used for protection, intelligent gathering, searching for drug
offenders, CCTV control, power grid surveillance.
b) CCTV cameras can be used to monitor any well-known criminals and
authorities are notified if one is located. But this is quite challenging for
light illumination, pose variation and facial expression.

  * •

Video Indexing: Labeling faces in video.[1]

##  9 Conclusion

Nowadays, face recognition is an unbounded exploration and development subject
for research. We can use face recognition technology for remote employee
tracking. Some extreme secure applications like person authentication or
controlling entrance at certain areas are carried out using this technology.
In our research, we try to represent a practical approach for recognition
systems using the FaceNet model. Unlike the other existing methods, for
example, principal component analysis (PCA) or support vector machine (SVM),
our model doesn’t require any extra operations like classifying and grouping
different images or creating a decision surface. Instead, our model shows
improved performance on the recognition process as we used an end-to-end
learning approach. Our method is more capable of improving the percentage of
collecting valuable information and produce more distinct feature information,
which basically increases the face recognition rate.
In our future works, we will try to improve our model by diminishing existing
limitations and try to gain more efficiency. Again, a face spoofing attack is
nowadays a big issue in security and authentication systems. So, we will make
an attempt to develop an anti-spoofing method, which will help to improve the
security of a biometric system.

## References

  * [1] Marcialis, G.L., Roli, F.: Fusion of face recognition algorithms for video-based surveillance systems. In: Multisensor Surveillance Systems, pp. 235–249. Springer (2003)
  * [2] Kak, S.F., Mustafa, F.M., Valente, P.: A review of person recognition based on face model. Eurasian Journal of Science & Engineering 4(1), 157–168 (2018)
  * [3] Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H., Hua, G.: Labeled faces in the wild: A survey. In: Advances in face detection and facial image analysis, pp. 189–248. Springer (2016)
  * [4] Tolba, A., El-Baz, A., El-Harby, A.: Face recognition: A literature review. International Journal of Signal Processing 2(2), 88–103 (2006)
  * [5] Schreiber, A.C., Rousset, S., Tiberghlen, G.: Facenet: A connectionist model of face identification in context. European Journal of Cognitive Psychology 3(1), 177–198 (1991)
  * [6] Déniz, O., Bueno, G., Salido, J., De la Torre, F.: Face recognition using histograms of oriented gradients. Pattern recognition letters 32(12), 1598–1603 (2011)
  * [7] Prasad, P.S., Pathak, R., Gunjan, V.K., Rao, H.R.: Deep learning based representation for face recognition. In: ICCCE 2019, pp. 419–424. Springer (2020)
  * [8] Jose, E., Greeshma, M., Haridas, M.T., Supriya, M.: Face recognition based surveillance system using facenet and mtcnn on jetson tx2. In: 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS). pp. 608–613. IEEE (2019)
  * [9] William, I., Rachmawanto, E.H., Santoso, H.A., Sari, C.A., et al.: Face recognition using facenet (survey, performance test, and comparison). In: 2019 Fourth International Conference on Informatics and Computing (ICIC). pp. 1–6. IEEE (2019)
  * [10] Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 815–823 (2015)
  * [11] Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research 10(2) (2009)
  * [12] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)
  * [13] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [14] Grinberg, M.: Flask web development: developing web applications with python. ” O’Reilly Media, Inc.” (2018)