# DFT based investigation of the structural, magnetic, electronic, and half-
metallic properties of solid In1-xTixSb solutions

S. Amrani  $|$ ${}^{\textsf{\footnotesize{\mbox{?}}}}$, M. Berber  $|$
${}^{\textsf{\footnotesize{\mbox{?},\penalty 1000\mbox{?}}}}$ , M. Mebrek  $|$
${}^{\textsf{\footnotesize{\mbox{?},\penalty 1000\mbox{?}}}}$ Corresponding
author<EMAIL_ADDRESS>
(Received Septmeber 02, 2021, in final form October 30, 2021)

###### Abstract

With the intention to reveal the effect of the substitution, Ti-doped InSb
alloy, we accomplished a first-principles prediction within the FPLAPW+lo
method. We used GGA-PBEsol scheme attached with the improved TB-mBJ approach
to predict structural, electronic, and magnetic properties of In1-xTixSb with
concentration $x=0$, $0.125$, $0.25$, $0.50$, $0.75$, $0.875$, and $1$. Our
lattice parameters are found in favorable agreement with the available
theoretical and experimental data. The calculation shows that all structures
are energetically stable. The substitutional doping transforms the ionic
character of the InSb compound in half-metallic ferromagnetic comportment for
concentration $x=0$, $0.125$, $0.25$, and $0.50$, with a spin polarization of
$100$% at the Fermi level, and metallic nature for In0.25Ti0.75Sb and
In0.125Ti0.875Sb. The total magnetic moments are also estimated at about 1
$\mu_{\text{B}}$. In0.875Ti0.125Sb, In0.75Ti0.25Sb, and In0.50Ti0.50Sb have
half-metallic ferromagnets comportment and they can be upcoming applicants for
spintronics applications.

Key words: DFT, TB-mBJ, electronic structures, half-metallic, ferromagnetic,
spintronic, FPLAW

###### Abstract

Ç ìåòîþ ç’ÿñóâàííÿ âïëèâó çàìùåííÿ â ëåãîâàíîìó àòîìàìè Ti ñïëàâ InSb
ïðîâåäåíî ïåðøîïðèíöèïí ðîçðàõóíêè â ðàìêàõ ìåòîäó ëíåàðèçîâàíèõ ðîçøèðåíèõ
ïëîñêèõ õâèëü ç ïîâíèì ïîòåíöàëîì (ËÐÏÕçÏÏ). Ç âèêîðèñòàííÿì óçàãàëüíåíîãî
ãðàäíòíîãî íàáëèæåííÿ Ïåðäüþ-Áóðêå-Åðíöåðãîôà äëÿ òâåðäèõ òë (ÓÃÍ-ÏÁÅòò) â
ïîêðàùåíîìó ïäõîä Òðàí-Áëàõà ç ìîäèôêàöþ Áåêà-Äæîíñîíà (ÒÁ-ìÁÄ) ïåðåäáàåíî
ñòðóêòóðí, åëåêòðîíí ìàãíòí âëàñòèâîñò ñïîëóê In1-xTixSb ç êîíöåíòðàöÿìè $x=0$
, $0.125$, $0.25$, $0.50$, $0.75$, $0.875$ òà $1$. Ðîçðàõîâàí çíàåííÿ ñòàëèõ
ðàòêè äîáðå óçãîäæóþòüñÿ ç íàÿâíèìè òåîðåòèíèìè òà åêñïåðèìåíòàëüíèìè äàíèìè.
ßê ïîêàçóþòü ðîçðàõóíêè, óñ ñòðóêòóðè åíåðãåòèíî ñòéêèìè. Ëåãóâàííÿ çàìùåííÿ
ïåðåòâîðþ îííèé õàðàêòåð âèõäíî¿ ñïîëóêè InSb íà íàïâìåòàëíó ôåðîìàãíòíó
ïîâåäíêó äëÿ êîíöåíòðàöé $x=0$, $0.125$, $0.25$ $0.50$, ç $100$% ñïíîâîþ
ïîëÿðèçàöþ íà ðâí Ôåðì òà íà ìåòàëíó äëÿ In0.25Ti0.75Sb In0.125Ti0.875Sb.
Îöíîí çíàåííÿ ïîâíèõ ìàãíòíèõ ìîìåíòâ ñòàíîâëÿòü ïðèáëèçíî 1 $\mu_{\text{B}}$.
$In_{0.875}$Ti0.125Sb, In0.75Ti0.25Sb, òà In0.50Ti0.50Sb äåìîíñòðóþòü
íàïâìåòàëíó ôåðîìàãíòíó ïîâåäíêó òà ïîòåíöéíî ìîæóòü âèêîðèñòîâóâàòèñÿ â
ñïíòðîíö.

Ключов слова: ìåòîä ôóíêöîíàëó ãóñòèíè, ÒÁ-ìÁÄ, åëåêòðîíí ñòðóêòóðè,
íàïâìåòàëè, ôåðîìàãíåòèê, ñïíòðîíêà, ËÐÏÕçÏÏ

##  1 Introduction

The possibility to dope the semiconductors with magnetic elements has created
a new category of materials called diluted magnetic semiconductors (DMS). One
of the peculiarities of DMS is the appearance of localized magnetic moments
generated by electron-hole coupling [1]. This leads to interesting properties,
for example, increasing the $g$ factor of the charge carriers or new
excitements, such as magnetic polarons [2]. However, ferromagnetism is rarely
observed in semiconductors due to the low density of carriers and because of
the predominance of superexchange between local magnetic moments. In most
cases, magnetic semiconductors have low Curie temperatures. Interest in DMSs
was recently aroused by theoretical and experimental work on semiconductors
[4, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]. DMS based on
the transition element doped and on semiconductors has two important
characteristics. The first one, the transition concentration of the elements
can be up to ten percent (using molecular beam epitaxy techniques, MBE) [20].
The second one, the concentration of carriers can be controlled in a wide
range and permits to choose between $n$-type and $p$-type doping. These two
aspects offer the possibility of varying the magnetic behavior by changing the
concentration of the carriers. Indium Antimonide is one of the best frequently
exploited semiconductors, which is used in Hall effect sensors, thermoelectric
generators, infrared detectors, high-speed photodetectors, and transistors
[21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]. This extensive utilization
follows from its distinct properties, such as a narrow band gap, small
effective electron mass, and high electron mobility. Doping different
materials into the InSb structure fascinates several scientists. Consequently,
the category of dopant strongly influences InSb characteristics and should be
precisely chosen according to suitable properties and future applications. The
optimization of pulse electrodeposition method for the synthesis of Te-doped
InSb films was performed by Rajska et al. [32]. They show that the bandgap
values affirm an impact of a dopant content on the optical properties of thin
Te-doped InSb films. The maximum figure of merit (ZT) was found to be about
0.6 and 673 K for Te-doped InSb [33]. They conclude that this material is a
very promising candidate for thermoelectric devices. The transport properties
of Sn-doped InSb, were experimentally studied by Okamoto and Shibasaki [34]
and confirm its use in Hall sensors applications. InSb doped with magnetic
elements for spintronics applications has been the patience of several
researchers. Theoretical and experimental studies have been carried out to
describe the magnetic phenomenon and the half-metallicity comportment. Mn-
doped InSb was studied by Novotortsev et al. [35], (Fe, Mn, and Cr) doped InSb
[36, 37, 38, 39, 40, 41, 42, 43]. The aim of our study is to predict
structural, electronic, and magnetic properties of Ti-doped InSb.

Figure 1: (Colour online) Crystal supercell structure of InSb.

##  2 Computational method

With the intention of calculating appropriate properties of Ti-doped InSb, we
performed calculations within the full-potential linearized augmented plane
wave (FP-LAPW) method [44, 45, 46, 47] based on the density functional theory
(DFT) [47, 48] implemented in WIEN2k code [48]. We employed
Perdew–Burke–Ernzerhof GGA-PBEsol approximation [49], combined with Tran–Blaha
modified Becke–Johnson TB-mBJ approximation [50, 51, 52].In our calculation,
we used the following parameters:

  * •

$K_{mesh}$ 200 k-points in Brillion zone (BZ).

  * •

The muffin-tin radii for Ti, In, and Sb are chosen as 1.65, 1.85, and 1.96
a.u., respectively.

  * •

$RMT\cdot K_{\rm{max}}=7$, where $RMT$ is the smallest value of muffin-tin
sphere radius, and $K_{\rm{max}}$ indicates the largest vector in-plane wave
expansion.

  * •

$l_{\rm{max}}=6$, where $l_{\rm{max}}$ is the wave function inside the muffin-
tin spheres.

  * •

$G_{\rm{max}}=14$, where $G_{\rm{max}}$ is the magnitude largest vector in the
charge density Fourier expansion.

  * •

The cut-off energy selected as $-6$ Ry (the separation of valence and core
states ).

  * •

The self-consistent calculations are converged when the condition is set to
0.0001 Ry.

  * •

Employing configuration $2\times 2\times 2$, a supercell of In8Sb8 with
(Zincblende, F4 3m, space group N:216) was constructed, which, consists of 16
atoms in which 8 atoms belong to Indium and 8 atoms belong to Antimonide. The
doping of Titanium cation was done on an In1-xTixSb structure.
In0.875Ti0.125Sb, In0.75Ti0.25Sb, In0.50Ti0.50Sb, In0.25Ti0.75Sb, and
In0.125Ti0.875Sb structures were created by Ti atom on substitution sites with
Indium. The relaxation maintained the F4 3m symmetry for the structure
investigated.

Table 1: Calculated lattice constant ($a$), bulk modulus ($B$), and its pressure derivative ($B^{\prime}$) for for In1-xTixSb at concentrations $x=0$, $0.125$, $0.25$, $0.50$, $0.75$, $0.875$ and $1$. Compound |  $a$ (Å) |  $B$ (GPa) | $B^{\prime}$ | $E_{\rm{Form}}$ | $E_{\rm{coh}}$ | Method
---|---|---|---|---|---|---
This Work |  |  |  |  |  | GGA-WC
InSb | 6.5554 | 42.4612 | 4.0525 |  |  |
In0.875Ti0.125Sb | 6.4876, 6.5069 [53] | 39.7953 | 5.0000 | $-0.341$ | 0.941 |
In0.75Ti0.25Sb | 6.4455, 6.4585 [53] | 54.2042 | 5.0000 | $-0.522$ | 0.852 |
In0.50Ti0.50Sb | 6.3878, 6.3616 [53] | 52.0305 | 6.7370 | $-0.435$ | 0.902 |
In0.25Ti0.75Sb | 6.3087, 6.2648 [53] | 53.8414 | 2.9134 | $-0.476$ | 0.871 |
In0.125Ti0.875Sb | 6.2315, 6.2164 [53] | 62.1805 | 4.7886 | $-0.393$ | 0.932 |
TiSb | 6.1697 | 58.0755 | 4.3008 |  |  |
Other calculations |  |  |  |  |  |
InSb | 6.464, 6.36 [28] |  |  |  |  |
| 6.479 [29] |  |  |  |  |
| 6.643 [30] |  |  |  |  |
| 6.629 [31] | 37.368 [31] | 4.452 [31] |  |  | GGA-PBE
| 6.47937 [32], 6.4791 [54] |  |  |  |  | Experimental
| 6.49 [34], 6.47937 [55] |  |  |  |  | Experimental

Figure 2: (Colour online) Energy unit cell versus volume with FM and NFM phase
of (a) InSb, (b) In0.875Ti0.125Sb, (c) In0.75Ti0.25Sb, (d) In0.50Ti0.50Sb, (e)
In0.25Ti0.75Sb, (f) In0.125Ti0.875Sb, and (g) TiSb.

##  3 Results and discussion

###  3.1 Structural properties

Indium Antimonide (InSb) crystallizes in the zincblende structure (No. 216).
Figure 1 exhibits the crystal supercell structure of InSb. Its conventional
structure has two types of atoms, In and Sb, located at (0, 0, 0) and (0.25,
0.25, 0.25) positions, respectively [56, 57]. The In0.875Ti0.125Sb,
In0.75Ti0.25Sb, In0.50Ti0.50Sb, In0.25Ti0.75Sb, and In0.125Ti0.875Sb compounds
are constructed by substitutional doping of one ($1/8=0.125$), two
($2/8=0.25$), four ($4/8=0.50$), six ($6/8=0.75$) and seven ($7/8=0.875$)
atoms of Titanium (Ti) at Indium (In) sites, respectively. The structures of
In1-xTixSb with concentration $x=0$ (InSb), 0.125, 0.25, 0.50, 0.75, 0.875,
and 1 (TiSb) are completely optimized and relaxed. For both ferromagnetic (FM)
and non-ferromagnetic (NFM) phases, the energy curves as a function of the
volume of each structure (see figure 2) are calculated and fitted by the
suitableness of Birch–Murnaghan’s equation of state (EOS) [58]. According to
figure 2, all the structures are stable in the FM phase except the binary
InSb. In this respect, we estimated the structural properties including the
equilibrium lattice constant “$a$”, bulk modulus “$B_{0}$”, and its derivative
$B^{\prime}$. The results are collected in table 1 accompanied by the
available theoretical and experimental data. The thermodynamic stability and
the probability of synthesizing these alloys [59, 60] are determined by the
formation and cohesive energies. To confirm the phase stability of the
In0.875Ti0.125Sb, In0.75Ti0.25Sb, In0.50Ti0.50Sb, In0.25Ti0.75Sb, and
In0.125Ti0.875Sb structures, we calculated the formation and cohesive energies
using the following expressions [43, 44]:

Figure 3: (Colour online) The calculated lattice constant $a$ of In1-xTixSb as
a function of concentration ($x$).

$\displaystyle
E^{\rm{In}_{1-x}\rm{Ti}_{x}\rm{Sb}}_{\rm{form}}=E^{\rm{In}_{1-x}\rm{Ti}_{x}\rm{Sb}}_{\rm{\rm{total}}}-\left[(1-x)E^{\rm{In}}_{\rm{solid}}+xE^{\rm{Ti}}_{\rm{solid}}+8E^{\rm{Sb}}_{\rm{solid}}\right],$
(3.1) $\displaystyle
E^{\rm{In}_{1-x}\rm{Ti}_{x}\rm{Sb}}_{\rm{coh}}=\left[(1-x)E^{\rm{In}}_{\rm{atom}}+xE^{\rm{Ti}}_{\rm{atom}}+8E^{\rm{Sb}}_{\rm{atom}}\right]-E^{\rm{In}_{1-x}\rm{Ti}_{x}\rm{Sb}}_{\rm{total}}.$
(3.2)

Here $E^{\rm{In}_{1-x}\rm{Ti}_{x}\rm{Sb}}_{\rm{total}}$ is the total energy
per atom of In1-xTixSb and $E^{\rm{In}}_{\rm{solid}}$,
$E^{\rm{Ti}}_{\rm{solid}}$, and $E^{\rm{Sb}}_{\rm{solid}}$ are the total
energies per atom of In, Ti, and Sb solids, respectively.
$E^{\rm{In}}_{\rm{atom}}$, $E^{\rm{Ti}}_{\rm{atom}}$, and
$E^{\rm{Sb}}_{\rm{atom}}$ are the isolated atomic energies of the pure
constituents of In, Ti, and Sb, respectively. Our computed values of formation
and cohesive energies for In0.875Ti0.125Sb, In0.75Ti0.25Sb, In0.50Ti0.50Sb,
In0.25Ti0.75Sb, and In0.125Ti0.875Sb compounds are collected in table 1.
Therefore, the negative values of the formation energy and the positive values
of cohesive energy, imply that the ternary alloys In1-xTixSb with
concentration $x=0$ (InSb), 0.125, 0.25, 0.50, 0.75, 0.875, and 1 (TiSb) are
energetically and thermodynamically stable and can be synthesized under normal
conditions. The deviation of a lattice constant $a$ as a function of the
composition ($x$) for In1-xTixSb is shown in figure 3. The linear relationship
between the lattice constant and the dopant concentration $x$ [53] is
predicted by Vegard’s law as follows:

$\displaystyle a(x)=6.5554-0.3875x.$ (3.3)

However, the quadratic relationship is widely used for both experimental and
theoretical purposes. We obtained the fitting equation for In1-xTixSb:

$\displaystyle a(x)=6.53614-0.26049x-0.09602x^{2}.$ (3.4)

Figure 4: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for InSb: (a) majority spin (up) and (b) minority spin (dn). The Fermi level
is set to zero (horizontal dotted red line).

Figure 5: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for In0.875Ti0.125Sb: (a) majority spin (up) and (b) minority spin (dn). The
Fermi level is set to zero (horizontal dotted red line).

Figure 6: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for In0.75Ti0.25Sb: (a) majority spin (up) and (b) minority spin (dn). The
Fermi level is set to zero (horizontal dotted red line).

Figure 7: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for In0.50Ti0.50Sb: (a) majority spin (up) and (b) minority spin (dn). The
Fermi level is set to zero (horizontal dotted red line).

Figure 8: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for In0.25Ti0.75Sb: (a) majority spin (up) and (b) minority spin (dn). The
Fermi level is set to zero (horizontal dotted red line).

###  3.2 Electronic properties

In this section, we calculated the electronic properties result in the band
structure, total and partial densities of state schemes. The calculations were
performed for In1-xTixSb with different composition $x$ ($x=0$, $0.125$,
$0.25$, $0.50$, $0.75$, $0.875$ and $1$), employing GGA-PBEsol associated with
TB-mBJ of Tran–Blaha modified Becke–Johnson approximation [50, 51, 52, 61].
Figure 4 (a) and figure 4 (b) exhibit the spin-polarized band structure of the
binary InSb ($x=0$), and display that the two spin channels of InSb have
identical semiconducting band structures with a direct bandgap located between
and high-symmetry points. The band structures of In0.875Ti0.125Sb,
In0.75Ti0.25Sb, and In0.50Ti0.50Sb are presented in figures 5, 6, and 7. We
notice that the majority-spin bands are metallic with respect to the gap
located at the Fermi level for the minority-spin channel.

Figure 9: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for In0.125Ti0.875Sb: (a) majority spin (up) and (b) minority spin (dn). The
Fermi level is set to zero (horizontal dotted red line).

Figure 10: (Colour online) Spin-polarized band structure obtained with TB-mBJ
for TiSb: (a) majority spin (up) and (b) minority spin (dn). The Fermi level
is set to zero (horizontal dotted red line).

Figure 11: (Colour online) Spin-polarized total densities of states of InSb.
The Fermi level is set to zero (vertical dotted red line).

Figure 12: (Colour online) Spin-polarized (a) total and (b) partial densities
of states of In0.875Ti0.125Sb. The Fermi level is set to zero (vertical dotted
red line).

Figure 13: (Colour online) Spin-polarized (a) total and (b) partial densities of states of In0.75Ti0.25Sb. The Fermi level is set to zero (vertical dotted red line). Table 2: The calculated indirect bandgap for InSb, half-metallic ferromagnetic gap $G_{\rm{HMF}}$ and half-metallic gap $G_{\rm{HM}}$ of minority-spin bands for In1-xTixSb at concentrations $x=0$, $0.125$, $0.25$, $0.50$, $0.75$, $0.875$ and $1$. Compound |  $G_{\rm{HMF}}$ (eV) |  $G_{\rm{HM}}$ (eV) |  $E^{\Gamma-\Gamma}$(eV) | Method | Behavior
---|---|---|---|---|---
This work |  |  |  | GGA-WC |
InSb |  |  | 0.3216 |  |
In0.875Ti0.125Sb | 0.8423 | 0.3989 |  |  | HMF
In0.75Ti0.25Sb | 1.3635 | 0.5182 |  |  | HMF
In0.50Ti0.50Sb | 1.3194 | 0.3072 |  |  | HMF
In0.25Ti0.75Sb | - | - |  |  | Metallic
In0.125Ti0.875Sb | - | - |  |  | Metallic
TiSb | - | - |  |  | Metallic
Other calculations |  |  |  |  |
InSb |  |  | 0.24 [29] | GGA-WC |
|  |  | 0.23, 0.2169, 0.1947, 0.17 [40] | Experimental |

Concerning the minority spin, we notice two gaps. The half-metallic
ferromagnetic gap $G_{\rm{HMF}}$ and the half-metallic gap (i.e., spin-flip
excitation) $G_{\rm{HM}}$. $G_{\rm{HMF}}$ is the energy difference between the
valence-band maximum (VBM) and conduction-band minimum (CBM) while
$G_{\rm{HM}}$ is the smaller energy between the minimum of the conduction band
energy compared to the Fermi level and the maximum value of the valence band
energy [61]. Figures 8, 9, and 10 shows the spin-polarized band structure of
In0.25Ti0.75Sb, In0.125Ti0.875Sb, and TiSb, respectively. It is clear that the
structures In1-xTixSb for $x=0.75$, 0.875, and 1 (TiSb) have a metallic
behavior for the two channels (spin up and spin down). The calculated TB-mBJ
values of Eg, HMF $G_{\rm{HMF}}$, and HM $G_{\rm{HM}}$ gaps of In1-xTixSb for
different concentrations are exposed in table 2 along with other experimental
and theoretical data. In order to know the comportment of these alloys around
the Fermi level (Fermi energy) $E_{\rm{F}}$, we have investigated the total
density of states (TDOS) and partial densities of states (PDOS) of the
In1-xTixSb material. Such results are illustrated in figures 11–17. From these
figures, it is found that binary InSb displays a semiconductor alloy, and for
In0.875Ti0.125Sb, In0.75Ti0.25Sb, and In0.50Ti0.50Sb systems exhibit a half-
metallic ferromagnetic character with spin polarization of 100% by using the
following formula given by [62]:

$\displaystyle
P=\frac{N\uparrow(E_{\rm{F}})-N\downarrow(E_{\rm{F}})}{N\uparrow(E_{\rm{F}})+N\downarrow(E_{\rm{F}})}.$
(3.5)

Here $P$ is the spin polarization, $N\uparrow(E_{\rm{F}})$ and
$N\downarrow(E_{\rm{F}})$ are the densities of states of the majority spin and
minority spin around the Fermi level, respectively. In accordance with the
calculated $P$ and the total DOS plot and owing to the performance that
In0.875Ti0.125Sb, In0.75Ti0.25Sb, and In0.50Ti0.50Sb structures are equal to
zero, the spin polarization is 100%, while In0.25Ti0.75Sb, In0.125Ti0.875Sb,
and TiSb demonstrates metallic comportment. It evidently appears that the
titanium bands are situated at the Fermi level (plot in green). Figures 11
(b), 12 (b), 13 (b), 14 (b), 15 (b), 16 (b), and 17 (b), depict the spin-
polarized partial electronic density of states for titanium atoms in
In1-xTixSb structures. We can observe that the high section of valence bands
is formed by the $3d$ (Ti) partially occupied states, and the lowest part of
the conduction bands is dominated by the $3d$ (Ti) unoccupied states higher
than Fermi level $E_{\rm{F}}$.

Figure 14: (Colour online) Spin-polarized (a) total and (b) partial densities
of states of In0.50Ti0.50Sb. The Fermi level is set to zero (vertical dotted
red line).

Figure 15: (Colour online) Spin-polarized (a) total and (b) partial densities
of states of In0.25Ti0.75Sb. The Fermi level is set to zero (vertical dotted
red line).

Figure 16: (Colour online) Spin-polarized (a) total and (b) partial densities
of states of In0.125Ti0.875Sb. The Fermi level is set to zero (vertical dotted
red line).

Figure 17: (Colour online) Spin-polarized (a) total and (b) partial densities
of states of TiSb. The Fermi level is set to zero (vertical dotted red line).

###  3.3 Magnetic and half-metallicity properties

To explicate the specificity of the effects of the $p-d$ exchange aspect on
the magnetic comportment, we calculated the total and local magnetic moments
of the Ti, In, and Sb atoms as well as the interstitial sites in the
In1-xTixSb structures. Table 3 exposes the total and local magnetic moments of
In1-xTixSb for different concentration $x$. We can note that the total
magnetic moment for each alloy was around 1 $\mu_{\text{B}}$, usually overdue
to the local magnetic moment of the Ti atom, and could be mainly attributed to
the strong $p-d$ exchange interaction between $3d$ $t_{\text{2g}}$ (Ti) and
$p$ (Sb) states. The calculated magnetic moment in interstitial sites in the
In1-xTixSb alloys increases with the concentration of Ti. The results also
indicate that for In1-xTixSb at all compositions except $x=0$ (i.e., InSb),
the magnetic spins of the atoms between Ti and Sb and between Sb and In have
opposite signs, signifying the antiparallel induced ferromagnetic interaction
between Ti and Sb and between Sb and In.

Table 3: Calculated total and local magnetic moments per Ti atom of the relevant Ti, In, and Sb atoms and in the interstitial sites (in Bohr magneton $B$) for In1-xTixSb at concentrations $x=0$, $0.125$, $0.25$, $0.50$, $0.75$, $0.875$ and $1$. Compound | Total ( $\mu_{\text{B}}$) | Ti ( $\mu_{\text{B}}$) | In ( $\mu_{\text{B}}$) | Sb ( $\mu_{\text{B}}$) | Interstitial ( $\mu_{\text{B}}$)
---|---|---|---|---|---
InSb | 0 | - | 0 | 0 | 0
In0.875Ti0.125Sb | 1 | 0.8584 | 0.0023 | $-0.0164$ | 0.1925
In0.75Ti0.25Sb | 1 | 0.8462 | 0.0046 | $-0.0266$ | 0.3864
In0.50Ti0.50Sb | 1 | 0.8641 | 0.0151 | $-0.1164$ | 0.8054
In0.25Ti0.75Sb | 1 | 0.8063 | 0.0343 | $-0.0635$ | 1.4459
In0.125Ti0.875Sb | 1 | 0.8477 | 0.0416 | $-0.1295$ | 1.6442
TiSb | 1 | 0.70123 | - | $-0.04942$ | 0.1800

##  4 Conclusion

In this paper, we used the Full-Potential Linearized Augmented Plane Wave (FP-
LAPW+lo) method to predict the structural, electronic, and magnetic properties
of the substitutional doping of Titanium on the binary InSb alloy.
In0.875Ti0.125Sb, In0.75Ti0.25Sb, In0.50Ti0.50Sb, In0.25Ti0.75Sb, and
In0.125Ti0.875Sb structures were evaluated. The calculated formation and
cohesive energy demonstrates that In0.875Ti0.125Sb, In0.75Ti0.25Sb,
In0.50Ti0.50Sb, In0.25Ti0.75Sb, and In0.125Ti0.875Sb alloys are stable and can
be synthesized. We used GGA-PBEsol approximation to predict the ground-state
properties. The calculated structural parameters of InSb are in good agreement
with the available theoretical and experimental data. GGA-PBEsol coupled with
(TB-mBJ) of Tran-Blaha modified Becke-Johnson approximation was used to
estimate electronic and magnetic properties. We demonstrated that
In0.875Ti0.125Sb, In0.75Ti0.25Sb, and In0.50Ti0.50Sb are half-metallic
ferromagnetic structures with a spin polarization of 100% at the Fermi level,
while In0.25Ti0.75Sb, and In0.125Ti0.875Sb present a metallic comportment. The
total magnetic moments are 1 $\mu_{\text{B}}$ for all compounds. The
investigation of magnetic properties demonstrates that the $p-d$ exchange
systems operate in stabilizing ferromagnetic state configuration, and are the
origin of a low magnetic moment in the investigated alloys. According to our
bibliographic research, there are no earlier theoretical or experimental
studies on the In1-xTixSb materials. Thus, we hope that our results serve as a
reference for future theoretical and experimental research.

## References

  * [1] Volkov B. A., Mnatsakanov T. T., Zh. Eksp. Teor. Fiz., 1978, 75, 563.
  * [2] Yakovlev D. R., Ossau W., Introduction to the Physics of Diluted Magnetic Semiconductors, Springer, Berlin, 2010.
  * [3] Laroussi A., Berber M., Mokaddem A., Doumi B., Abid H., Boudali A., Acta Phys. Pol. A, 2019, 135, 451,
doi:10.12693/APhysPolA.135.451.

  * [4] Laroussi A., Berber M., Doumi B., Mokaddem A., Abid H., Boudali A., Bahloul H., Moujri H., Appl. Phys. A, 2019, 125, No. 10, 1–13, doi:10.1007/s00339-019-2973-2.
  * [5] Berber M., Doumi B., Mokaddem A., Mogulkoc Y., Sayede A., Tadjer A., J. Comput. Electron., 2017, 16, No. 3, 542–547, doi:10.1007/s10825-017-1038-z.
  * [6] Berber M., Doumi B., Mokaddem A., Mogulkoc Y., Sayede A., Tadjer A., J. Electron. Mater., 2018, 47, No. 1, 449–456, doi:10.1007/s11664-017-5793-1.
  * [7] Bahloul H., Mokaddem A., Doumi B., Berber M., Boudali A., J. Supercond. Novel Magn., 2019, 32, No. 7, 2185–2192, doi:10.1007/s10948-018-4948-5.
  * [8] Boutaleb M., Hsini M., Zemouli M., Berber M., Tadjer A., Int. J. Comput. Mater. Sci. Eng., 2021, 10, No. 1, 2150006, doi:10.1142/S2047684121500068.
  * [9] Bouziani I., Benhouria Y., Essaoudi I., Ainane A., Ahuja R., Mater. Sci. Eng., B, 2020, 253, 114484,
doi:10.1016/j.mseb.2019.114484.

  * [10] Khan M. S., Shi L., Zou B., Ali S., Comput. Mater. Sci., 2020, 174, 109491,
doi:10.1016/j.commatsci.2019.109491.

  * [11] Marquina J., Quintero E., Ruette F., Bentarcurt Y., Chin. J. Phys., 2020, 63, 63–69,
doi:10.1016/j.cjph.2019.09.037.

  * [12] Monir M. E. A., Bahnes A., Boukortt A., Reguig A. B., Mouchaal Y., J. Magn. Magn. Mater., 2020, 497, 166067,
doi:10.1016/j.jmmm.2019.166067.

  * [13] Korichi K., Doumi B., Mokaddem A., Sayede A., Tadjer A., Philos. Mag., 2020, 100, No. 9, 1172–1190,
doi:10.1080/14786435.2020.1723812.

  * [14] Hamidane N., Baaziz H., Ocak H. Y., Baddari K., Uğur Ş., Uğur G., Charifi Z., J. Supercond. Novel Magn., 2020, 33, No. 10, 3263–3272, doi:10.1007/s10948-020-05527-w.
  * [15] Maftouh A., Rami R., Drissi L. B., El Fatni O., Ahl Laamara R., Solid State Commun., 2021, 339, 114482,
doi:10.1016/j.ssc.2021.114482.

  * [16] Kumar A., Kumar M., Sati P. C., Srivastava M. K., Ghosh S., Kumar S., Curr. Appl Phys., 2021, 32, 24–35,
doi:10.1016/j.cap.2021.09.002.

  * [17] Abo-Naf S. M., Ibrahim S., Marzouk M. A., Appl. Phys. A, 2021, 127, No. 6, 1–11, doi:10.1007/s00339-021-04583-w.
  * [18] Yehia M., Labib Sh., Ismail S. M., J. Inorg. Organomet. Polym. Mater., 2021, 31, 4491–4503, doi:10.1007/s10904-021-02051-9.
  * [19] Zhao X., Liang H., Chen Y., Chen X., Zhang W., Wang J., Zhang G., Belotelov V. I., Song Y., J. Mater. Chem. C, 2021, 9, No. 13, 4619–4627, doi:10.1039/D0TC06026G.
  * [20] Cho A. Y., Arthur J. R., Prog. Solid. State Ch., 1975, 10, 157–191, doi:10.1016/0079-6786(75)90005-9.
  * [21] Sandhu A., Sanbonsugi H., Shibasaki I., Abe M., Handa H., Jpn. J. Appl. Phys., 2004, 43, No. 7A, L868,
doi:10.1143/JJAP.43.L868.

  * [22] Nilsson H. A., Caroff P., Thelander C., Larsson M., Wagner J. B., Wernersson L. E., Samuelson L., Xu H. Q., Nano Lett., 2009, 9, No. 9, 3151–3156, doi:10.1021/nl901333a.
  * [23] Kuo C. H., Wu J. M., Lin S. J., Chang W. C., Nanoscale Res. Lett., 2013, 8, No. 1, 327, doi:10.1186/1556-276X-8-327.
  * [24] Minamoto M. T., Allen C. M., Solid-State Electron., 1962, 5, No. 4, 263–266, doi:10.1016/0038-1101(62)90107-7.
  * [25] Maffitt K. N., Mueller R. K., J. Appl. Phys., 1964, 35, No. 5, 1563–1564, doi:10.1063/1.1713667.
  * [26] Hattori K., Yuito M., Amakusa T., Phys. Status Solidi A, 1982, 73, No. 1, 157–164,
doi:10.1002/pssa.2210730120.

  * [27] Kimukin I., Biyikli N., Ozbay E., J. Appl. Phys., 2003, 94, No. 8, 5414–5416, doi:10.1063/1.1611286.
  * [28] Ashley T., Buckle L., Datta S., Emeny M. T., Hayes D. G., Hilton K. P., Jefferies R., Martin T., Phillips T. J., Wallis D. J., Wilding P. J., Chau R., Electron. Lett., 2007, 43, No. 14, doi:10.1049/el:20071335.
  * [29] Singh J., Poolla R., J. Mater. Sci.: Mater. Electron., 2017, 28, No. 18, 13716–13726, doi:10.1007/s10854-017-7216-8.
  * [30] Fawcett W., Ruch J. G., Appl. Phys. Lett., 1969, 15, No. 11, 368–370, doi:10.1063/1.1652863.
  * [31] Ashley T., Dean A. B., Elliott C. T., Pryce G. J., Johnson A. D., Willis H., Appl. Phys. Lett., 1995, 66, No. 4, 481–483, doi:10.1063/1.114063.
  * [32] Rajska D., Brzózka A., Marciszko-Wiąckowska M., Marzec M. M., Chlebda D., Hnida-Gut K. E., Sulka G. D., Appl. Surf. Sci., 2021, 537, 147715, doi:10.1016/j.apsusc.2020.147715.
  * [33] Yamaguchi S., Matsumoto T., Yamazaki J., Kaiwa N., Yamamoto A., Appl. Phys. Lett., 2005, 87, No. 20, 201902, doi:10.1063/1.2130390.
  * [34] Okamoto A., Shibasaki I., J. Cryst. Growth, 2003, 251, No. 1–4, 560–564, doi:10.1016/s0022-0248(02)02448-x.
  * [35] Kochura A., Marenkin S., Novodvorsky O., Mikhalevsky V., Davidov A., Shakhov M., Lahderanta E., Zakhvalinskii V., Kuzmenko A., EPJ Web of Conferences, 2018, 185, 06003, doi:10.1051/epjconf/201818506003.
  * [36] You J. Y., Gu B., Maekawa S., Su G., Phys. Rev. B, 2020, 102, No. 9, 094432,
doi:10.1103/PhysRevB.102.094432.

  * [37] Mir S. A., Gupta D. C., J. Magn. Magn. Mater., 2020, 493, 165722, doi:10.1016/j.jmmm.2019.165722.
  * [38] Berber M., Bouzouira N. E., Abid H., Boudali A., Moujri H., Rev. Mex. de Fis., 66, No. 5, 604–609,
doi:10.31349/RevMexFis.66.604.

  * [39] Pappert K., Schmidt M. J., Hümpfner S., Rüster C., Schott G. M., Brunner K., Gould C., Schmidt G., Molenkamp L. W., Phys. Rev. Lett., 2006, 97, No. 18, 186402, doi:10.1103/PhysRevLett.97.186402.
  * [40] Ohno H., Shen A., Matsukura F., Oiwa A., Endo A., Katsumoto S., Iye Y., Appl. Phys. Lett., 1996, 69, No. 3, 363–365, doi:10.1063/1.118061.
  * [41] Sato K., Katayama-Yoshida H., Jpn. J. Appl. Phys., 2000, 39, No. 6B, L555, doi:10.1143/JJAP.39.L555.
  * [42] Ruderman M. A., Kittel C., Phys. Rev., 1954, 96, No. 1, 99, doi:10.1103/PhysRev.96.99.
  * [43] Yosida K., Phys. Rev., 1957, 106, No. 5, 893, doi:10.1103/PhysRev.106.893.
  * [44] Wong K. M., Alay-e-Abbas S. M., Shaukat A., Fang Y., Lei Y., J. Appl. Phys., 2013, 113, No. 1, 014304, doi:10.1063/1.4772647.
  * [45] Khalil B., Naji S., Labrim H., Bhihi M., El Hachimi A. G., Lakhal M., Belhaj A., Benyoussef A., El Kenz A., J. Supercond. Novel Magn., 2014, 27, No. 1, 203–208, doi:10.1007/s10948-013-2241-1.
  * [46] Berri S., Kouriche A., Maouche D., Zerarga F., Attallah M., Mater. Sci. Semicond. Process., 2015, 38, 101–106, doi:10.1016/j.mssp.2015.04.016.
  * [47] Sjöstedt E., Nordström L., Singh D. J., Solid State Commun., 2000, 114, No. 1, 15–20, doi:10.1016/S0038-1098(99)00577-3.
  * [48] Blaha P., Schwarz K., Madsen G. K. H., Kvasnicka D., Luitz J., Laskowski R., Tran F., Marks L. D., WIEN2k: An Augmented Plane Wave Plus Local Orbitals Program for Calculating Crystal Properties, Vienna University of Technology, Austria, 2018.
  * [49] Perdew J. P., Burke K., Ernzerhof M., Phys. Rev. Lett., 1996, 77, No. 18, 3865, doi:10.1103/PhysRevLett.77.3865.
  * [50] Koller D., Tran F., Blaha P., Phys. Rev. B, 2011, 83, No. 19, 195134, doi:10.1103/PhysRevB.83.195134.
  * [51] Koller D., Tran F., Blaha P., Phys. Rev. B, 2012, 85, No. 15, 155109, doi:10.1103/PhysRevB.85.155109.
  * [52] Tran F., Blaha P., Phys. Rev. Lett., 2009, 102, No. 22, 226401, doi:10.1103/PhysRevLett.102.226401.
  * [53] Vegard L., J. Phys., 1921, 5, No. 5, 393–395.
  * [54] Breivik M., Nilsen T. A., Fimland B. O., J. Cryst. Growth, 2013, 381, 165–168,
doi:10.1016/j.jcrysgro.2013.06.034.

  * [55] Adachi S., In: Optical Constants of Crystalline and Amorphous Semiconductors, Springer, Boston, MA, 1999, 268–278, doi:10.1007/978-1-4615-5247-5_27.
  * [56] Liu T. S., Peretti E. A., JOM, 1951, 3, No. 9, 791–791, doi:10.1007/BF03397373.
  * [57] Orton J. W., Semiconductors and the Information Revolution: Magic Crystals That Made IT Happen, Academic Press, 2009.
  * [58] Birch F., Phys. Rev., 1947, 71, No. 11, 809, doi:10.1103/PhysRev.71.809.
  * [59] Reunchan P., Umezawa N., Ouyang S., Ye J., Phys. Chem. Chem. Phys., 2012, 14, No. 6, 1876–1880,
doi:10.1039/C2CP23348G.

  * [60] Liu Y., Zhou W., Wu P., J. Appl. Phys., 2017, 121, No. 7, 075102, doi:10.1063/1.4976812.
  * [61] Gao G. Y., Yao K. L., Şaşıoğlu E., Sandratskii L. M., Liu Z. L., Jiang J. L., Phys. Rev. B, 2007, 75, No. 17, 174442, doi:10.1103/PhysRevB.75.174442.
  * [62] Bai J., Raulot J. M., Zhang Y., Esling C., Zhao X., Zuo L., Appl. Phys. Lett., 2011, 98, No. 16, 164103,
doi:10.1063/1.3582239.

Ukrainian 0

Äîñëäæåííÿ ñòðóêòóðíèõ, ìàãíòíèõ åëåêòðîííèõ âëàñòèâîñòåé òà íàïâìåòàëíîñò
òâåðäèõ ðîçèíâ In1-xTixSb íà îñíîâ ìåòîäó ôóíêöîíàëó ãóñòèíè

Ñ. Àìðàí  $|$ ${}^{\textsf{\footnotesize{\mbox{?}}}}$, Ì. Áåðáåð  $|$
${}^{\textsf{\footnotesize{\mbox{?},\penalty 1000\mbox{?}}}}$, Ì. Ìåáðåê  $|$
${}^{\textsf{\footnotesize{\mbox{?},\penalty 1000\mbox{?}}}}$

${}^{\textsf{\footnotesize 1}}$ Ëàáîðàòîðÿ ôçèêî-õìíèõ äîñëäæåíü, Óíâåðñèòåò
Ñà¿äè, 20000 Ñà¿äà, Àëæèð ${}^{\textsf{\footnotesize 2}}$ Ëàáîðàòîðÿ
ïåðñïåêòèâíèõ ìàòåðàëâ ïðèëàäîáóäóâàííÿ, Óíâåðñèòåòñüêèé öåíòð Íóð Åëü-Áàøèð
Åëü-Áàÿäõ, Àëæèð ${}^{\textsf{\footnotesize 3}}$ Óíâåðñèòåòñüêèé öåíòð Íóð
Åëü-Áàøèð Åëü-Áàÿäõ, Åëü-Áàÿäõ, 32000, Àëæèð


[table]capposition=top

# Continual Learning of Hand Gestures for Human-Robot Interaction

Xavier Cucurull1 and Anaís Garrell1

###### Abstract

In this paper, we present an efficient method to incrementally learn to
classify static hand gestures. This method allows users to teach a robot to
recognize new symbols in an incremental manner. Contrary to other works which
use special sensors or external devices such as color or data gloves, our
proposed approach makes use of a single RGB camera to perform static hand
gesture recognition from 2D images. Furthermore, our system is able to
incrementally learn up to 38 new symbols using only 5 samples for each old
class, achieving a final average accuracy of over 90%. In addition to that,
the incremental training time can be reduced to a 10% of the time required
when using all data available.

##  I Introduction

When we think about a natural form of communication among humans, the use of
speech is one of the first things that comes to mind. However, human
communication is multimodal, with speech and gestures coexisting and having
both great importance [1, 2]. In infants, global motor development occurs
earlier than oral language and as a result, the first form of communication
appears in the form of gestures [3], such as finger pointing, and can be used
as an effective form of early communication.

In the case of human-robot interaction (HRI), during the past decades we have
evolved from simple teleoperation using remote controls, keyboards and
joysticks, to a relationship that is increasingly becoming more natural and
similar to the communication that takes place among humans. With the
appearance of collaborative [4] and assistive robots [5], using natural forms
of interaction is becoming more and more important. This work focuses on the
use of hand gesture recognition by making use of an RGB camera, to simplify
the needed equipment, and to offer a cost-effective solution.

If hand gestures are used to communicate with a robot with the goal of
providing a natural and easy way of interaction, it might be desirable to
teach the robot to recognize new symbols. Continual Learning (CL), which is a
machine learning paradigm by which a system can incrementally learn new
information without forgetting what it has previously learned [6], can be used
to provide a system with a more natural way of learning. To this end, we
propose an approach based on deep learning, called HAGIL (HAnd Gesture
Incremental Learning), a system that can incrementally learn to classify
static hand gestures from RGB images and that is targeted at human robot
interaction.

Figure 1: The proposed approach is used to teach a robot to incrementally
recognize new hand gestures.

An overview of this approach is shown in Fig. 2. The proposed method can be
divided into two modules. The first one is dedicated to the extraction of
visual features, more particularly hand landmarks (joints), from the input
image. In this work, we rely on the open-source Mediapipe framework [7], which
offers a lightweight and reliable hand landmark detector. The second module
encodes the extracted landmarks into a feature representation, which is fed
into a neural network (NN) in order to predict static hand gestures. In order
to implement continual learning capabilities, the neural network classifier is
encapsulated into a modified version of the FACIL [8] incremental learning
framework, which implements various incremental learning (IL) approaches.

Figure 2: Overview of the proposed approach for hand gesture incremental
learning. Given an input image (a), the proposed approach uses the MediPipe
Hands pose detector (b) to extract the landmarks (c) of the hands present in
the input. Next, the detected landmarks are encoded into a feature vector
representation, which is then used to perform hand gesture recognition through
the use of a NN (d). To incorporate CL capabilities, the NN classifier is
encapsulated into a modified version of the FACIL [8] incremental learning
framework.

Finally, our method is thoroughly evaluated on a novel static hand gestures
dataset that combines the existing UniPd datasets [9, 10, 11, 12] with a
custom-made dataset, accounting for a total of 38 unique gestures performed by
different subjects. In addition, two American Sign Language (ASL) alphabet
datasets are also used to evaluate the performance of the proposed HAGIL
system.

The contribution of this paper is threefold: first, a deep learning approach
for continual learning of static hand gestures called HAGIL is proposed.
Second, we evaluate and study different incremental learning strategies to
determine which components contribute the most to avoiding catastrophic
forgetting. An extensive evaluation of the IL system is performed under a
variety of parameters and datasets, to find a balance between accuracy and
resource consumption, and to demonstrate its applicability and efficacy.
Finally, this work introduces a new dataset that consists of 21 hand gestures
that are performed by 4 distinct subjects, and which can be used as a
benchmark in hand gesture continual learning.

The remainder of this paper is organized as follows. Section II presents the
related work of hand gesture recognition, and continual learning. Section III
describes the components of the proposed method.In Section IV, the method is
evaluated in a state-of-the-art dataset. Finally, some conclusions and future
work are provided in Section V.

##  II Related Work

In this section, we present some related work in the fields of static hand
gesture recognition, and continual learning methods applied in the field of
robotics.

###  II-A Hand Gesture Recognition

As mentioned above, the use of hand gestures constitutes a natural form of
communication among humans and can therefore be an effective method for
natural and accessible HRI. In addition to that, hand gestures might offer a
better alternative to speech recognition to overcome challenges such as noise,
reverberation and distant speech [13, 14]. As a result, hand gesture
recognition has been thoroughly studied in the field of HRI.

In the late 1980s and 1990s, several authors proposed the use of data gloves
equipped with sensors [15, 16] to recognize hand gestures. Later, [17]
introduced the use of colored gloves to detect hand regions. Following the
appearance of the Microsoft Kinect, many authors proposed the use of depth
sensors and RGB-D images [18, 19]. Other sensor-based approaches include the
use of radar [20] and reflected impulses [21]. Regarding hand gesture
recognition from RGB images, the extraction of features was initially
performed using techniques such as histogram of oriented gradients (HOG) [22,
23, 24] and later by applying CNNs [25, 26, 27].

Lately, pose estimation algorithms such as OpenPose [28], Mediapipe Holistic
[7] and OpenMMLab MMpose [29] have allowed skeleton-based hand gesture
recognition [30, 31, 32] from RGB images. With respect to the classification
methods employed, the most common algorithms used in the literature are SVM
[9, 10], kNN [33, 34] and CNN [35, 36].

Most of the presented approaches require external devices such as sensors,
depth cameras or color gloves. Our proposal uses a skeleton-based approach
with just an RGB camera. This approach allows the user to interact with a
robot in a natural way, without being required to wear additional equipment,
in a cost-effective manner.

###  II-B Continual Learning in Robotics

In continual learning, the system must continuously learn new tasks without
forgetting what it has previously learned. This type of learning is especially
interesting for robotics, where an autonomous agent might need to constantly
learn from its interaction with the environment [6]. Below, we review various
examples of continual learning with neural networks applied to visual tasks in
robotics.

In [37], the Hello iCubWorld dataset, consisting of images of 7 objects
captured with a humanoid robot, was introduced. Using the iCubWorld28 dataset,
[38] trained a humanoid robot to incrementally classify objects using a CNN
trained on ImageNet [39] as a fixed feature extractor and a Regularized Least
Squares (RLS) [40] classifier on top of it. Other works also make use of
ImageNet pre-trained networks as fixed feature extractors and incrementally
learn to classify images using nearest-centroid-based classifiers using t-SNE
[41] or Agg-Var [42] clustering. To address the lack of benchmark datasets
specifically designed for continuous object recognition, [43] introduced
CORe50, a collection of 50 domestic objects captured in a sequence where the
objects move slightly in front of the camera, and presented CWR (Copy Weights
with Re-init), an architectural CL strategy based on a VGG [44] CNN. In [45],
a visual system with incremental learning for the detection of actions from
child-robot interaction is proposed. In their work, a Temporal Segment Network
(TSN) [46] is used coupled with iCaRL [47] for continual learning of actions.

Concerning continual learning for static hand gesture classification, [48]
also used iCaRL to incrementally learn hand symbols captured by event-based
cameras, learning up to 16 symbols are learned with a final classification
accuracy of 80%. In this paper, we propose a framework to learn a larger
number of symbols with a higher classification accuracy by using RGB still
images captured by a standard CMOS camera, a more cost-effective option
compared to event-based systems.

##  III Proposed Method

In this section, we describe the proposed method, HAGIL, which is depicted in
Fig. 2. It consists of two main modules, one to detect hand landmarks, and the
second one dedicated to perform continual learning of hand gestures by means
of a neural network. In the following, these modules are explained in more
detail.

###  III-A Hand Landmarks Detection

As seen above, our project uses the open-source MediaPipe framework for real-
time extraction of hand landmarks. The use of a skeleton-based approach,
compared to using an image-based gesture recognition method, allows the system
to be more robust to variations in terms of illumination, subject
characteristics, and background. Consequently, a robot can be trained with
more limited data in a room with artificial lighting and a clear background
and eventually be used in a different setting such as outdoors, with natural
lighting and a cluttered background, and still perform with great accuracy.

###  III-B Hand Gesture Recognition

The recognition of static hand gestures is performed by a neural network model
that uses the hand landmarks provided by the MediaPipe detector. The input to
this NN can simply be the raw landmark coordinates $(x,y,z)$ of the detected
joints. However, in this section, we describe the feature engineering process
used to select a robust feature representation that leads to a better
classification performance, and we present the neural network architecture for
hand gesture recognition. In these initial tests, the UniPd dataset, which
consists of 17 unique static hand gestures, is used for evaluation. For these
experiments, only the training and validation splits are used, since the test
split is kept apart for the final experiments.

####  III-B1 Feature Representation

As seen in Figure 2, each hand consists of 21 landmarks. MediaPipe returns
2.5D landmarks (the term 3D is not used because the depth is relative to the
wrist), resulting in 21 (x, y, z) coordinates that can be used to represent
each hand pose. A naive feature representation can be obtained by flattening
the coordinates into a $21\times 3=63$ values array. Alternatively, if only
the 2D coordinates are used, this results in a $21\times 2=42$ values array.
In our experiments, using only the 2D coordinates gave the best results, so
the third axis is discarded. This first feature representations can be
referred to as Raw, and can work really well when the hands in the training
and validation images are located in similar positions (e.g. centered on the
frame). However, if the hand moves around, for example to a corner of the
image, the values of the joint coordinates will drastically change and model
performance will decrease. For that reason, a translation invariant feature
representation shall be used in order to cope with this problem.

The first method to achieve a translation invariant feature vector is to
calculate the distance between each landmark and the wrist, which is the most
invariable joint of the hand. Equation 1 shows how the distance between a
joint j and the wrist w is calculated.

$d(j,w)=(x_{j}-x_{w},y_{j}-y_{w})$ (1)

This feature representation, referred to as WristDiff results in a
$(21-1)\times 2=40$ values array. The second method studied, called
WristEuclidean2D, computes the Euclidean distance from each landmark to the
wrist, resulting in a feature vector of $21-1=20$ values. Another feature
representation alternative that was explored is based on calculating the
distances among each of the hand landmarks. In the first approach, AllDiff
computes the difference between each of the landmarks to each of the others,
resulting in a feature vector of $(21^{2}-21)\times 2=420$. Alternatively,
AllEuclidean, comptues the Euclidean distance within all landmarks, resulting
in a feature vector of $21^{2}-21=210$ values.

The classification accuracy for each feature representation method, averaged
on 10 runs, can be seen in Table I. These results display that the three best
feature representations, according to the classification accuracy achieved,
are AllDiff, WristDiff, and AllEuclidean, with an average accuracy above
98.10%. With the goal of obtaining a robust feature representation that is
translation invariant but rotation dependent, the three best representations
are combined, resulting in the best overall classification accuracy. After
concatenating AllEuclidean + WristDiff + AllDiff, a final feature vector of
$210+40+420=670$ values is obtained.

Features | Accuracy (%)
---|---
Raw | 94.4 ± 1.49
WristDiff | 98.27 ± 0.41
WristEuclidean | 84.87 ± 0.62
AllEuclidean | 98.12 ± 0.49
AllDiff | 98.31 ± 0.46
AllEuclidean+WristDiff+AllDiff | 98.57 ± 0.48
TABLE I: Average classification accuracy in % (± standard deviation) for
different feature representations evaluated on the UniPd dataset.

####  III-B2 Network Architecture

After selecting the best feature representation, the next step was to choose
the right neural network architecture. In this process, the UniPd daset was
used once again for evaluation. Initially, the width of the network (i.e. the
number of units of the hidden layer) was increased by a factor of 2. Next, the
length (i.e. the number of layers) was incremented successively by adding one
layer. Adam [49] was used as the optimizer. The initial learning rate was set
to 0.001, and was decreased by a factor of 3 when the loss did not decrease
for 5 consecutive epochs. In addition, various dropout probabilities were
evaluated, obtaining the best results for (p=0.35). Figure 3 depicts the
chosen network architecture.

Figure 3: Layers diagram of the HagilNet network architecture.

Finally, an ablation test was made to evaluate the contribution of batch
normalization and dropout to the model accuracy. As observed in Table II,
batch normalization is the regularization technique that contributes the most
to the model performance.

Dropout | Batch Normalization | Accuracy (%)
---|---|---
✓ | ✓ | 99.26 ± 0.26
✗ | ✓ | 99.02 ± 0.39
✗ | ✗ | 97.12 ± 0.89
TABLE II: Ablation study on the regularization methods applied. Average
classification accuracy in % (± standard deviation), evaluated on the UniPd
dataset.

###  III-C Continual Learning Framework

In [8], the Framework for Analysis of Class-Incremental Learning (FACIL) was
introduced. FACIL implements many state-of-the-art regularization and
rehearsal incremental learning strategies focused on the task of class-
incremental image recognition. In order to adapt the framework to the use of
skeleton-based features, some modifications to the code were made. Since FACIL
was designed to analyze and compare various incremental learning approaches,
the entire dataset used for the evaluation should be provided at the beginning
of the training phase. In order to be able to successively train the model
with new data, HAGIL extends the existing framework by storing a copy of the
model that can be subsequently re-trained incrementally. In addition to that,
more changes were made to the original code to be able to use multiple
datasets and different number of epochs for the initial and incremental tasks.

###  III-D Custom HandGest Dataset

In addition to the proposed framework, in this paper we present the HAGIL
HandGest dataset 111https://xaviercucurull.ml/HAGIL, a custom hand gesture
dataset that was built with the goal of expanding the existing UniPd dataset,
by adding new gestures that can be used in an incremental learning scenario.
To construct this dataset, some signs from the ASL alphabet were used together
with other easy to remember hand gestures. In order to employ different
subjects for the train, validation, and test splits, four subjects were used.
Figure 4 shows some example images from the HandGest dataset. Note that to
preserve the privacy of the subjects, their faces were automatically pixelated
with the help of a simple Haar cascade [50] face detector.

##  IV Experiments

In this section, first the datasets used for the state-of-the-art experiments
are described. Next, the incremental learning experiments, which compare
different CL methods and evaluate different parameters, are presented.
Finally, results regarding execution times are provided. All the experimental
results are averaged over 10 runs, and the order in which new gestures are
learned is shuffled at each run

###  IV-A Datasets

In order to evaluate the applicability and performance of the HAGIL framework,
two group of tests are made. First, the UniPd dataset is extended with the
previously presented HAGIL HandGest to compose a dataset consisting of 38
unique symbols. This dataset is referred to as UniPd+HandGest. For this
combined dataset, the number of training samples is low, ranging from 25 to 50
examples for each unique gesture. Although this amount of data may not be
suitable for training a large neural network, good results are obtained using
HagilNet. Moreover, this short number of image samples reflects what can be a
real use-case scenario, where a robot needs to learn from the interaction with
a human user and time and memory constraints limit the number of samples that
can be obtained.

Figure 4: Example images of the HandGest custom dataset. Four static gestures
performed by four subjects.

Furthermore, two ASL Alphabet datasets are used to evaluate the performance of
the proposed framework. The ASL Finger Spelling dataset [51] contains 24
static hand gestures that correspond to the static signs of the ASL alphabet.
The Kaggle ASL Alphabet dataset [52] [53], which contains 28 static gestures,
includes all the ASL alphabet (with a static version of the j and z letters)
and two additional signs.

For all the experiments, different subjects were used for the train,
validation, and test splits. This split by subject is really important as to
report accurate results close to real-world performance of the model in the
wild.

###  IV-B Incremental Learning Experiments

This section presents the incremental learning tests that were done in order
to evaluate the performance of two rehearsal IL approaches: IL2M [54] and
iCaRL. In addition to that, a regularization approach whicch does not use an
exemplar memory (LwF [55]) is used as a lower baseline and Joint training, in
which the model has access to all the training data from previous classes
throughout all the IL process, is used as an upper baseline. The IL
experiments consist of initially training the model with two classes and then
learning the rest of the classes incrementally one by one.

In the following, two main groups of experiments are presented. First, various
tests are performed using different numbers of exemplars per class. Then, a
second group of experiments evaluates different numbers of epochs used in the
incremental tasks. Last, an ablation study is conducted to evaluate the
contribution of the different components of the finally selected IL approach.
For these experiments, the UniPd+HandGest dataset is used.

####  IV-B1 Number of Exemplars

In these experiments, the performance of using an increasing number of
exemplars per class is evaluated, with the maximum number of training epochs
set to 50. The goal is to find the lowest rehearsal memory size that leads to
a good accuracy, as one of the main goals of continual learning is to avoid
storing many data examples from the classes already learned.

((a)) iCaRL

((b)) IL2M

Figure 5: Experimental results for different number of exemplars per class,
evaluated on the UniPd+HandGest dataset. Accuracy at each task. For clarity,
standard deviation is not plotted.

In Figure 5, the average task accuracy for different numbers of exemplars is
illustrated for iCaRL and IL2M. In addition, results for the Joint learning
scenario are also provided. It can be observed that for the initial tasks,
when only a reduced number of classes have been learned, the system lacks
stability and exhibits a high variance in the classification accuracy. As more
classes are learned, the accuracy results become stable. This instability is
higher for a rehearsal memory size of 1 exemplar per class and for IL2M. As
iCaRL uses a nearest-exemplar-mean (NEM) classifier, it shows that it can
handle better the situation where only a few number of classes are learned
with a limited rehearsal memory. It is remarkable to note that iCaRL with 5
exemplars achieves a final accuracy higher than that obtained with Joint
training. Therefore, it seems that the NEM classifier of iCaRL provides a
robust method to classify the static hand gestures of the UniPd+HandGest
dataset.

####  IV-B2 Number of Incremental Epochs

Being able to incrementally train a system with a low number of epochs, while
achieving a good accuracy, is something desirable in a real-world scenario, as
it reduces training time. For these experiments, the initial 2 classes are
trained for 50 epochs. Next, for the rest of the CL episodes, different number
of epochs are tested. In this case, a rehearsal memory of 5 exemplars per
class selected by herding [56] is chosen.

((a)) iCaRL

((b)) IL2M

Figure 6: Experimental results for different number of incremental epochs.
Accuracy at each task, evaluated on the UniPd+HandGest dataset. Colored
regions show standard deviation.

The results (see Figure 6) show that iCaRL does not suffer from the cold start
problem as much as IL2M, as it obtains good accuracy results from the
beginning. In addition, using just 10 incremental epochs gives a performance
comparable to using 40 epochs, which can be used to significantly reduce
training time.

####  IV-B3 Ablation Study

Method | Accuracy (%)
---|---
Joint | 94.29 ± 4.48
HAGIL | 93.04 ± 5.25
iCaRL -kdl | 92.52 ± 4.85
iCaRL -kdl-NEM | 91.23 ± 5.87
iCaRL Raw | 86.05 ± 7.04
LwF | 16.09 ± 19.59
TABLE III: HAGIL ablation study experimental results, evaluated on the
UniPd+HandGest dataset.

Table III provides the average accuracy results for the ablation study, as
well as for the upper baseline Joint and the lower baseline LwF. The HAGIL
method, which refers to iCaRL with the AllEuclidean+WristDiff+AllDiff feature
representation, a rehearsal memory of 5 exemplars per class selected by
herding, and 15 incremental epochs, is the IL method that gets the best
accuracy. Not using the knowledge distillation loss (iCaRL -kdl), slightly
degrades the final accuracy, although it obtains good results for the initial
learned classes. Removing the NEM classifier (iCaRL -kdl-NEM), which is a main
component of the iCaRL approach, also impacts the final average classification
accuracy, and particularly the accuracy results when only few classes are
learned are negatively affected. This effect was also observed in Figure 6
when comparing iCaRL to IL2M. Finally, using the Raw feature representation
causes the average accuracy to drop. This indicates that selecting a good
feature representation is a key aspect of obtaining a good classifier.

####  IV-B4 ASL Alphabet Experiments

In the final experiments, two datasets based on the ASL datasets are
evaluated: the Kaggle ASL Alphabet and the ASL Finger Spelling datasets.
Although these datasets have less classes than the previously used UniPd+
HandGest dataset, they can be more difficult to learn as the inter-class
similarity is higher. For these experiments, HAGIL is used with the
configuration found on the previous experiments: iCaRL, 5 exemplars per class
with herding, and 15 incremental epochs. The experiment scenario is defined as
before, initially learning 2 classes, and learning the rest one by one. It is
important to note that the ASL datasets, in contrast to the UniPd and Handgest
databases, have hundreds of instances per class, so using a rehearsal memory
of 5 exemplars per class represents only a small fraction of the whole
training data.

((a)) Kaggle ASL Alphabet

((b)) ASL Finger Spelling

Figure 7: Average accuracy for different number of incremental epochs, using 5
exemplars per class, evaluated on the Kaggle ASL Alphabet and ASL Finger
Spelling datasets. For clarity, standard deviation is not plotted.

As seen in Figure 7, HAGIL obtains reasonably good accuracy results in the ASL
datasets. However, since the amount of training data available for these
datasets is greater than for the previously evaluated UniPd+Handgest dataset,
increasing the number of incremental epochs significantly increases the model
average accuracy. Comparing the incremental learning with the Joint baseline,
we can see that Joint obtains better overall results for both datasets. For
the Kaggle ASL Alphabet, however, the IL approach stabilizes after learning
the initial 7 classes and maintains a similiar accuracy for the subsequent IL
episodes, while the accuracy of the Joint baseliene keeps decreasing. The ASL
Finger Spelling dataset, on the other hand, experiences more forgetting and
its accuracy performance degrades as more classes are learned. Nevertheless,
the obtained results are remarkably good, given the fact that only 5 exemplars
are used, compared to the hundreds of training instances. In addition to that,
this dataset is more challenging because most of the signs are presented in
different rotations and orientations, making it more difficult to learn them
with a small rehearsal memory.

###  IV-C Execution Times

An important part of the deployment of an AI system to be used in a real-life
scenario is the computation time required by each of the components, as this
has a direct impact on the performance of the model in real-time settings.

As seen in Table IV, the landmarks extraction is one of the steps which takes
more time. Once the landmarks are extracted, the rest of the operations run in
a shorter time. HAGIL adds a latency of $34.2+2.73+0.61=37.5ms$, which
theoretically would allow an application to run at a maximum rate of around
$1/37.5\times 1000=26$ fps.

In order to evaluate the time to train the model, a new experiment is
conducted with the Kaggle ASL Alphabet dataset. In this experiment, a model is
initially trained with 27 classes, and then it is trained to learn a new
class. For both the HAGIL and Joint approaches, the number of epochs used to
learn the new symbol is set to 15. As observed in the results, HAGIL takes
advantage of the rehearsal memory and obtains a reduced training time of less
than 10% of the time required to learn a new class using all the data from the
previously learned classes.

Operation | Execution time
---|---
Mediapipe Landmark Extraction | 34.2 ms
Feature Representation | 2.73 ms
Model Inference | 608 us
Model Training (HAGIL) | 7.76 s
Model Training (Joint) | 79.73 s
TABLE IV: Analysis of the execution time of the different components of the
HAGIL framework. Training time evaluated by initially training a model with
the first 27 classes of the Kaggle ASL Alphabet dataset, and then learning a
new class.

##  V Conclusions

In this work we have presented a system that is capable of incrementally
learning static hand gestures from RGB images, by using hand landmarks
extraction and the iCaRL class-incremental CL strategy.

To the best of our knowledge, this is the only work that addresses incremental
learning of hand gestures from images, and together with the introduced
dataset, establishes a benchmark that can benefit both the robotics and
continual learning community. HAGIL has proven to be a high performance method
that has obtained very competitive results in a variety of datasets.

Moreover, in this work, we have evaluated two popular rehearsal CL techniques
(iCaRL and IL2M) in a task other than image classification, as is done in most
of the related work. It has been observed that the use of an exemplar memory
represents a key component to obtain good accuracy results in an incremental
learning scenario. However, it has been possible to keep this memory to a
reduced set of exemplars, representing just a small fraction of the original
training data, thus reducing the training time and the memory requirements of
the system. Thus, the nearest exemplar mean classifier used in iCaRL has
proven to be more efficient than the softmax classifier of IL2M when few
exemplars and a reduced number of training epochs are used.

Furthermore, we are currently working in the application of this method in the
communication between human and robot in agriculture applications,
specifically in harvesting and pruning vineyards.

## References

  * [1] M. Tomasello, _Origins of human communication_. MIT press, 2010.
  * [2] S. C. Levinson and J. Holler, “The origin of human multi-modal communication,” _Philosophical Transactions of the Royal Society B: Biological Sciences_ , vol. 369, no. 1651, p. 20130302, 2014.
  * [3] C. D. Vallotton, “Signs of emotion: What can preverbal children “say” about internal states?” _Infant Mental Health Journal: Official Publication of The World Association for Infant Mental Health_ , vol. 29, no. 3, pp. 234–258, 2008.
  * [4] E. Repiso, A. Garrell, and A. Sanfeliu, “Robot approaching and engaging people in a human-robot companion framework,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 8200–8205.
  * [5] M. A. Peshkin, J. E. Colgate, W. Wannasuphoprasit, C. A. Moore, R. B. Gillespie, and P. Akella, “Cobot architecture,” _IEEE Transactions on Robotics and Automation_ , vol. 17, no. 4, pp. 377–390, 2001.
  * [6] T. Lesort, V. Lomonaco, A. Stoian, D. Maltoni, D. Filliat, and N. Díaz-Rodríguez, “Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges,” _Information Fusion_ , vol. 58, pp. 52–68, June 2020.
  * [7] A. Vakunov, C.-L. Chang, F. Zhang, G. Sung, M. Grundmann, and V. Bazarevsky, “Mediapipe hands: On-device real-time hand tracking,” 2020.
  * [8] M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer, “Class-incremental learning: survey and performance evaluation on image classification,” _arXiv preprint arXiv:2010.15277_ , 2020.
  * [9] G. Marin, F. Dominio, and P. Zanuttigh, “Hand gesture recognition with leap motion and kinect devices,” in _2014 IEEE International conference on image processing (ICIP)_. IEEE, 2014, pp. 1565–1569.
  * [10] A. Memo, L. Minto, and P. Zanuttigh, “Exploiting silhouette descriptors and synthetic data for hand gesture recognition,” 2015.
  * [11] G. Marin, F. Dominio, and P. Zanuttigh, “Hand gesture recognition with jointly calibrated leap motion and depth sensor,” _Multimedia Tools and Applications_ , vol. 75, no. 22, pp. 14 991–15 015, 2016.
  * [12] A. Memo and P. Zanuttigh, “Head-mounted gesture controlled interface for human-computer interaction,” _Multimedia Tools and Applications_ , vol. 77, no. 1, pp. 27–53, 2018.
  * [13] M. Ravanelli, J. Zhong, S. Pascual, P. Swietojanski, J. Monteiro, J. Trmal, and Y. Bengio, “Multi-task self-supervised learning for robust speech recognition,” in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 6989–6993.
  * [14] J. Laplaza, J. J. Oliver, R. Romero, A. Sanfeliu, and A. Garrell, “Body gesture recognition to control a social robot,” _arXiv preprint arXiv:2206.07538_ , 2022.
  * [15] T. G. Zimmerman, J. Lanier, C. Blanchard, S. Bryson, and Y. Harvill, “A hand gesture interface device,” _ACM Sigchi Bulletin_ , vol. 18, no. 4, pp. 189–192, 1986.
  * [16] R.-H. Liang and M. Ouhyoung, “A real-time continuous alphabetic sign language to speech conversion vr system,” in _Computer Graphics Forum_ , vol. 14, no. 3. Wiley Online Library, 1995, pp. 67–76.
  * [17] Y. Iwai, K. Watanabe, Y. Yagi, and M. Yachida, “Gesture recognition by using colored gloves,” in _1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No. 96CH35929)_ , vol. 1. IEEE, 1996, pp. 76–81.
  * [18] Z. Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti, “American sign language recognition with the kinect,” in _Proceedings of the 13th international conference on multimodal interfaces_ , 2011, pp. 279–286.
  * [19] W. Tao, M. C. Leu, and Z. Yin, “American sign language alphabet recognition using convolutional neural networks with multiview augmentation and inference fusion,” _Engineering Applications of Artificial Intelligence_ , vol. 76, pp. 202–213, 2018.
  * [20] P. Molchanov, S. Gupta, K. Kim, and K. Pulli, “Multi-sensor system for driver’s hand-gesture recognition,” in _2015 11th IEEE international conference and workshops on automatic face and gesture recognition (FG)_ , vol. 1. IEEE, 2015, pp. 1–8.
  * [21] S. Y. Kim, H. G. Han, J. W. Kim, S. Lee, and T. W. Kim, “A hand gesture recognition sensor using reflected impulses,” _IEEE Sensors Journal_ , vol. 17, no. 10, pp. 2975–2976, 2017.
  * [22] W. T. Freeman and M. Roth, “Orientation histograms for hand gesture recognition,” in _International workshop on automatic face and gesture recognition_ , vol. 12. Zurich, Switzerland, 1995, pp. 296–301.
  * [23] K.-p. Feng and F. Yuan, “Static hand gesture recognition based on hog characters and support vector machines,” in _2013 2nd international symposium on instrumentation and measurement, sensor network and automation (IMSNA)_. IEEE, 2013, pp. 936–938.
  * [24] L. Prasuhn, Y. Oyamada, Y. Mochizuki, and H. Ishikawa, “A hog-based hand gesture recognition system on a mobile device,” in _2014 IEEE international conference on image processing (ICIP)_. IEEE, 2014, pp. 3973–3977.
  * [25] R. Ahuja, D. Jain, D. Sachdeva, A. Garg, and C. Rajput, “Convolutional neural network based american sign language static hand gesture recognition,” _International Journal of Ambient Computing and Intelligence (IJACI)_ , vol. 10, no. 3, pp. 60–73, 2019.
  * [26] M. Z. Islam, M. S. Hossain, R. ul Islam, and K. Andersson, “Static hand gesture recognition using convolutional neural network with data augmentation,” in _2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR)_. IEEE, 2019, pp. 324–329.
  * [27] O. Mazhar, B. Navarro, S. Ramdani, R. Passama, and A. Cherubini, “A real-time human-robot interaction framework with robust background invariant hand gesture detection,” _Robotics and Computer-Integrated Manufacturing_ , vol. 60, pp. 34–48, 2019.
  * [28] Z. Cao, T. Simon, S.-E. Wei, and Y. Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 7291–7299.
  * [29] M. Contributors, “Openmmlab pose estimation toolbox and benchmark,” https://github.com/open-mmlab/mmpose, 2020.
  * [30] Q. De Smedt, H. Wannous, and J.-P. Vandeborre, “Skeleton-based dynamic hand gesture recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2016, pp. 1–9.
  * [31] A. Moryossef, I. Tsochantaridis, J. Dinn, N. C. Camgoz, R. Bowden, T. Jiang, A. Rios, M. Muller, and S. Ebling, “Evaluating the immediate applicability of pose estimation for sign language recognition,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 3434–3440.
  * [32] M. Peral, A. Sanfeliu, and A. Garrell, “Efficient hand gesture recognition for human-robot interaction,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 4, pp. 10 272–10 279, 2022.
  * [33] C.-H. Chuan, E. Regina, and C. Guardino, “American sign language recognition using leap motion sensor,” in _2014 13th International Conference on Machine Learning and Applications_. IEEE, 2014, pp. 541–544.
  * [34] D. Aryanie and Y. Heryadi, “American sign language-based finger-spelling recognition using k-nearest neighbors classifier,” in _2015 3rd International Conference on Information and Communication Technology (ICoICT)_. IEEE, 2015, pp. 533–536.
  * [35] B. Garcia and S. A. Viesca, “Real-time american sign language recognition with convolutional neural networks,” _Convolutional Neural Networks for Visual Recognition_ , vol. 2, pp. 225–232, 2016.
  * [36] A. A. Abdulhussein and F. A. Raheem, “Hand gesture recognition of static letters american sign language (asl) using deep learning,” _Engineering and Technology Journal_ , vol. 38, no. 6, pp. 926–937, 2020.
  * [37] S. Fanello, C. Ciliberto, M. Santoro, L. Natale, G. Metta, L. Rosasco, and F. Odone, “icub world: Friendly robots help building good vision data-sets,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2013, pp. 700–705.
  * [38] G. Pasquale, C. Ciliberto, F. Odone, L. Rosasco, and L. Natale, “Teaching icub to recognize objects using deep convolutional neural networks,” in _Machine Learning for Interactive Systems_. PMLR, 2015, pp. 21–25.
  * [39] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, _et al._ , “Imagenet large scale visual recognition challenge,” _International journal of computer vision_ , vol. 115, no. 3, pp. 211–252, 2015.
  * [40] R. Rifkin, G. Yeo, T. Poggio, _et al._ , “Regularized least-squares classification,” _Nato Science Series Sub Series III Computer and Systems Sciences_ , vol. 190, pp. 131–154, 2003.
  * [41] G. Pasquale, C. Ciliberto, L. Rosasco, and L. Natale, “Object identification from few examples by improving the invariance of a deep convolutional neural network,” in _2016 IEEE/RSJ international conference on intelligent robots and systems (IROS)_. IEEE, 2016, pp. 4904–4911.
  * [42] A. Ayub and A. R. Wagner, “Tell me what this is: few-shot incremental object learning by a robot,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020, pp. 8344–8350.
  * [43] V. Lomonaco and D. Maltoni, “Core50: a new dataset and benchmark for continuous object recognition,” in _Conference on Robot Learning_. PMLR, 2017, pp. 17–26.
  * [44] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _arXiv preprint arXiv:1409.1556_ , 2014.
  * [45] N. Efthymiou, P. P. Filntisis, G. Potamianos, and P. Maragos, “Visual robotic perception system with incremental learning for child–robot interaction scenarios,” _Technologies_ , vol. 9, no. 4, p. 86, 2021.
  * [46] L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. V. Gool, “Temporal segment networks: Towards good practices for deep action recognition,” in _European conference on computer vision_. Springer, 2016, pp. 20–36.
  * [47] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2017, pp. 2001–2010.
  * [48] I. A. Lungu, S.-C. Liu, and T. Delbruck, “Incremental learning of hand symbols using event-based cameras,” _IEEE Journal on Emerging and Selected Topics in Circuits and Systems_ , vol. 9, no. 4, pp. 690–696, 2019.
  * [49] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [50] M. J. Jones, P. Viola, _et al._ , “Robust real-time object detection,” in _Workshop on statistical and computational theories of vision_ , vol. 266, 2001, p. 56.
  * [51] N. Pugeault and R. Bowden, “Spelling it out: Real-time asl fingerspelling recognition,” in _2011 IEEE International conference on computer vision workshops (ICCV workshops)_. IEEE, 2011, pp. 1114–1119.
  * [52] A. Nagaraj. (2018) Asl alphabet dataset — Kaggle. [Online]. Available: https://www.kaggle.com/datasets/grassknoted/asl-alphabet/versions/1
  * [53] D. Rasband. (2018) Asl alphabet dataset — Kaggle. [Online]. Available: https://www.kaggle.com/datasets/danrasband/asl-alphabet-test/asl-alphabet/versions/1
  * [54] E. Belouadah and A. Popescu, “Il2m: Class incremental learning with dual memory,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 583–592.
  * [55] Z. Li and D. Hoiem, “Learning without forgetting,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 40, no. 12, pp. 2935–2947, 2017\.
  * [56] M. Welling, “Herding dynamical weights to learn,” in _Proceedings of the 26th Annual International Conference on Machine Learning_ , 2009, pp. 1121–1128.