ɔ ɖ

# AfriVEC: Word Embedding Models for African Languages. Case Study of Fon and
Nobiin

Bonaventure F. P. Dossou
Jacobs University Bremen
<EMAIL_ADDRESS>
&Mohammed Sabry
University of Khartoum
<EMAIL_ADDRESS>

###### Abstract

From Word2Vec to GloVe, word embedding models have played key roles in the
current state-of-the-art results achieved in Natural Language Processing.
Designed to give significant and unique vectorized representations of words
and entities, those models have proven to efficiently extract similarities and
establish relationships reflecting semantic and contextual meaning among words
and entities. African Languages, representing more than 31% of the worldwide
spoken languages, have recently been subject to lots of research. However, to
the best of our knowledge, there are currently very few to none word embedding
models for those languages words and entities, and none for the languages
under study in this paper. After describing Glove, Word2Vec, and Poincaré
embeddings functionalities, we build Word2Vec and Poincaré word embedding
models for Fon and Nobiin, which show promising results. We test the
applicability of transfer learning between these models as a landmark for
African Languages to jointly involve in mitigating the scarcity of their
resources, and attempt to provide linguistic and social interpretations of our
results. Our main contribution is to arouse more interest in creating word
embedding models proper to African Languages, ready for use, and that can
significantly improve the performances of Natural Language Processing
downstream tasks on them. The official repository and implementation is at:
https://github.com/bonaventuredossou/afrivec

##  1 Introduction

Word Embedding models are very useful in Natural Language Processing
downstream tasks and got modernized usage with learning paradigms like zero-
shot learning (Xian et al., 2017), addressing labels representation problems
in both image and text classification tasks (Norouzi et al., 2014; Sappadla et
al., 2016). This makes it for the sake of scientific research on African
Languages very important to consider, because of their scarse data resources.
Throughout this paper, our main contribution is to provide standardization and
evaluation guidelines to any research on the space, through our methods and
experiments.

Fon, and Nobiin are the two African Indigenous Languages (ALs) chosen for this
study. Fon is a native language of Benin Republic, spoken in average by more
than 2.2 million people in Benin, in Nigeria, and Togo. Nobiin is native to
Northern Sudan and Southern Egypt, spoken in average by a million of people.
Both languages cover a wide differential range of cultures as the speakers are
from Western and Northern Africa.

Fon alphabet is based on the latin alphabet, with the addition of the letters
so o, d, e, and the digraphs gb, hw, kp, ny, and xw. There are 10 vowel
phonemes in Fon: 6 said to be $closed$ [i, u, ĩ, ũ], and 4 said to be $opened$
[(e, o, a, ã]. There are 22 consonants (m, b, n, d, p, t, d, c, j, k, g, kp,
gb, f, v, s, z, x, h, xw, hw, w).

Nobiin alphabet is primarily based on greek alphabet with some meroitic
characters, but in most of the resources and modern usage of the language, it
uses the following schema of 28 letters: there are 10 vowels: 5 are $opened$:
(a, e, i, o, u) and 5 are $closed$: (â, ê, î, ô, û). There are 18 consonants:
(b, d, f, g, h, j, k, m, n, r, s, t, w, y, sh, ch, gn, ģ).

Word Embedding (WE) modeling is an approach that provides a dense vector
representation of words and captures something about their meaning. The goal
of embedding methods is to organize symbolic objects (words, entities,
concepts etc.) in a way such that their similarities in the embedding space
reflects their semantic or functional similarities. WEs models are
improvements of naive bag-of-word (BOW) modeling that relies on statistics
like word counts and frequencies to create large sparse vectors, describing
documents but not the meaning of the words.

WEs work by using an algorithm to train a set of fixed-length dense and
continuous-valued vectors based on a large corpus of text. Each word is
represented by a point in the embedding space and these points are learned and
moved around based on the words that surround the target word. Their massive
use in text representations is one of the key methods that led to breakthrough
performances in many fields of natural language processing like machine
translation, and named entity recognition, just to mention a few. The most
famous WEs are GloVe Pennington et al. (2014), Word2Vec Mikolov et al. (2013),
and Poincaré Embeddings Nickel and Kiela (2017).

Additionally, we would also like to highlight BERT embeddings (Devlin et al.,
2018) from Transformers, which are ubiquitous nowadays in Natural Language
Processing, and have improved systems performances. However, transformer-based
architectures require a lot of computing power and data, and as such they may
not be suitable for small datasets (which is the case in the current paper) or
to researchers that do not have access to GPUs, whereas Word2vec and Poincare
and Glove are not computationally expensive.

##  2 Word Embeddings: Related Works

###  2.1 GloVe

GloVe, or Global Vectors for Word Representation, is an approach to capture
the meaning of one word embedding towards the corpus (set of documents - a
document is a sentence). The GloVe model trains on global co-occurrence counts
of words and makes a sufficient use of statistics by minimizing least-squares
error. This produces a word vector space with meaningful substructure, that
sufficiently preserves words similarities with vector distance. The
probability that word at a given index $t^{\prime}$ occurs in the context of a
word $t$ is defined as:

$P_{tt^{\prime}}=\frac{A_{tt^{\prime}}}{A_{t}}$ (1)

where $A$ is the words co-occurrence matrix. Each entry of $A$, is the number
of times word $t^{\prime}$ occurs in the context of word $t$. The function F,
encoding the information about the ratio co-occurrence between two vectors is
defined as:

$F(v_{t}-v_{t^{\prime}};\hat{v_{k}})=\frac{P_{tk}}{P_{t^{\prime}k}}$ (2)

where $v_{t}$, $v_{t^{\prime}}$ are word vectors with indices $t$ and
$t^{\prime}$, and $\hat{v_{k}}$ a context vector with index $k$. The left
arguments of F are vectors and the right side is a scalar. Since F could be
taken as a complicated parameterized function like a neural network for
instance, doing so would obfuscate the linear structure we are trying to
capture. To avoid this issue, we can first take the dot product of the
arguments, preventing F from mixing the vector dimensions in undesirable ways.
The equation (2) becomes then:

$F((v_{t}-v_{t^{\prime}})^{T}\hat{v_{k}})=\frac{P_{tk}}{P_{t^{\prime}k}}$ (3)

Since in the word-word co-occurrence matrix, the distinction between context
words and standard words is arbitrary, probabilities ratio are replaced and
the equation (3) becomes:

$F((v_{t}-v_{t^{\prime}})^{T}\hat{v_{k}})=\frac{F(v_{t}^{T}\hat{v_{k}})}{F(v_{t^{\prime}}^{T}\hat{v_{k}})}$
(4)

In the original paper, Pennington et al. (2014) set the equation (5) and
solved it for $F$ being the exponential function:

$F(v_{t}^{T}\hat{v_{k}})=P_{tk}=\frac{A_{tk}}{A_{t}}$ (5)

The final solution of the equation (5) is:

$v_{t}^{T}\hat{v_{k}}+b_{t}+\hat{b_{k}}=log(A_{tk})$ (6)

where $b_{t}$ and $\hat{b_{k}}$ are respective bias for $v_{t}$ and
$\hat{v_{k}}$, added to restore symmetry.

Finally the loss function to minimize is hence a linear regression function
defined as:

$J=\sum_{t,t^{\prime}=1}^{V}(f(A_{tt^{\prime}})v_{t}^{T}\hat{v_{t^{\prime}}}+b_{i}+\hat{b_{j}}-log(A_{tt^{\prime}}))^{2},$

with $V$ being the size of the WE’s vocabulary. $f(A_{tt^{\prime}})$ is a pre-
defined weighting function, that should be continuous, non-decreasing and
relatively small for large values of the argument. Obviously, there are
infinite functions that could be constructed to satisfy these criterias. The
authors Pennington et al. (2014) used the function $f$ defined as:

$f(x)=\begin{cases}(\frac{x}{x_{max}})^{\alpha},x<x_{max}\\\ 1,x\geq
x_{max}\end{cases}$

where $\alpha\in(0;1)$

###  2.2 Word2Vec

There are 3 different types of Word2Vec parameter learning, and all of them
are based on neural network models Mikolov et al. (2013).

####  2.2.1 One-Word Context

This approach is known as Continuous Bag-Of-Word (CBOW). The main idea is the
consideration of a single word per context i.e. we have to predict one word
given only one word. The input of the neural network in this context is a one-
hot encoded vector of size (V,) followed by a hidden layer of size N with an
input hidden layer weights matrix W of size V×N, and an output layer weights
matrix W’ of size N×V, with softmax activation function. The objective here is
to compute as probability, the vector representation of the word with index
$i$:

$p(w_{j}|w_{i})$

Let $a$ be our input vector filled with zeros, and a single 1 at the position
t. The hidden vector $h$ is computed with the formula below:

$h=W^{T}a=z^{T}_{w_{i}}$

where $z$ is the output vector of the word $w_{i}$. We can look at $h$ as the
«input vector» of the word $a$. At the next step, we take the vector $h$ and
apply a matrix multiplication similar to the previous one:

$o_{j}=z^{\prime T}_{w_{j}}h$

where $z\textquoteright$ is the output vector of the word $w_{j}$ with. This
multiplication is performed for every entry $o$ with index $j$. The $softmax$
activation is defined as followed:

$p(w_{j}|w_{i})=y_{j}=\frac{e^{o_{j}}}{\sum_{j^{\prime}=1}^{V}e^{o_{j^{\prime}}}}$

####  2.2.2 Multi-Word Context

The concept of Multi-word Context is very similar to the concept of CBOW. The
only difference is that we want to capture the relationship between our target
word and other words from the corpus. The probability distribution is then
defined as:

$p(w_{i}|w_{1,1},...,w_{1,c})$

obtained by changing the hidden layer function to:

$h=\frac{1}{C}(\sum_{i=1}^{C}a_{i})$

The optimization function becomes then:

$-log(p(w_{i}|w_{1,1},...,w_{1,c}))$

####  2.2.3 Skip-Gram Model

The concept of Skip-Gram Model is opposite to the Multi-Word Model: the task
is to predict $c$ context words having one target word on the input. The
process of Skip-Gram is the reverse procedure of the Multi-Word Context
Mikolov et al. (2013). The optimization function is then defined as followed:

$\frac{1}{T}\sum_{t=1}^{T}\sum_{-c\leq i\leq c}log(p(w_{t+i}|w_{t}))$

$-c$ and $c$ are limits of the context window and $w_{t}$ is a word at index
$t$. $T$ is the total number number of words in the vocabulary. The hidden
vector $h$ is computed the same way as in the case of CBOW and Multi-Word
Context. The output layer is computed with:

$o_{c,j}=o_{j}=z^{\prime T}_{w_{j}}h$

and the activation function is defined as followed:

$p(w_{c,j}=w_{j,c}|w_{i})=y_{c,j}=\frac{e^{o_{c,j}}}{\sum_{j^{\prime}=1}^{V}e^{o_{j^{\prime}}}}$

###  2.3 Poincaré Embeddings

The concept of Poincaré Embeddings uses hyperbolic geometry to capture
hierarchical complexities (Ravasz and Barabási, 2003) and properties of the
words that can not be captured directly in Euclidean space. There is a need to
use such kind of geometry together with Poincaré ball to capture the fact that
distance from the root of the tree to its leaves grows exponentially with
every new child, and hyperbolic geometry is able to represent this property.
Hyperbolic geometry studies non-Euclidean spaces of constant negative
curvature. Its main 2 axioms and theorems are:

  * •

$\forall$ line $a$ and $\forall$ point $p$ $\not\in$ a, there are at least two
distinct parallels passing through $p$.

  * •

all triangles have angles sum less than 180 degrees.

For 2-dimensional hyperbolic space, both the area $s$ and length $l$ of a
circle, grow exponentially. The are defined with the following formulas:

$l=2\pi sinh(r)$ and $s=2\pi(cosh(r)-1)$

where $r$ denotes the radius. The Poincaré Ball is defined then as:

$B^{d}=\\{x\in R^{d}|||x||<1\\}$

The distance measure between 2 WEs $t$ and $t^{\prime}$ is defined as:

$d(t,t^{\prime})=arcosh(1+2\frac{||t-t^{\prime}||^{2}}{(1-||t||^{2})((1-||t^{\prime}||^{2})})$.

Nickel and Kiela (2017) argued that, this measure allows not only to capture
effectively the similarity between the two WEs but also preserves their
hierarchy (through their norm).

##### Relevant word embeddings works on African Languages:

In this regard, very few explorations have been done, to the best of our
knowledge. However it worths mentioning the work of Alabi et al. (2019) which
introduced massive and curative embeddings for Yoruba and Twi, two other low-
resourced African Languages. The contextual word embeddings obtained have been
used to evaluate multilingual BERT on a named entity recognition task.
Alternatively, van der Westhuizen and Niesler (2017) albeit on a very small
monolingual English web text corpus, explored the use of word embeddings in
the synthesis of isiZulu-to-English code-switch bigrams used to augment sparse
language model training data.

##  3 Creation of our Word Embedding: Case Study of Fon and Nobiin Languages

###  3.1 Contextualization

A lookup at Google’s Word2Vec model, can allow us to define queen as followed:
$queen=(king-man)+woman$. $Woman$ and $Man$ are genders, while $queen$ and
$king$ could be referred as titles. However, this equation, gives a meaningful
and unique representation to the word $queen$, and its similarities with the
words $king$, $man$ and $woman$.

In the context of ALs, to the best of our knowledge, there are no such models
capable of capturing high-level relation between words and entities. Alabi et
al. (2019) also highlighted the difficulty of the evaluation on low-resourced
languages, of several architectures that are capable to learn semantic
representations from unannotated data, more suitable to high-resourced
languages, that have a smorgasbord of tasks and test sets to evaluate on.

In general, this lack of research is coupled with the scarcity of the data and
the morphological richness of ALs. Many studies including Orife (2018); Dossou
and Emezue (2020), regarding the way to handle the textual data of African
languages, have showed the importance of diacritics in the challenging task of
creating effective and robust neural machine translation systems, and Natural
Language Processing (NLP) tools for ALs. Therefore, during the preprocessing
of the data used for this study, we made sure to keep the words diacritics, in
order to not lose any meaningful information.

We aim to investigate the possibilities of creating proper word embedding
models for a better and meaningful representations of ALs words and entities;
models that could capture exclusive relation between words of the same
language. Moreover, the results of our work, could be an effort to creating or
improving Named Entity Recognition (NER) models for ALs. For each of the
languages selected (Fon and Nobiin), we tried the following:

  * •

create a word2vec embedding model, and test its capacity to establish
relationships or similarities among words.

  * •

build a Poincaré WE model and use it to predict the types of entities.

To promote reproducibility, and further improvements as well as the use of our
findings in NLP tasks on ALs. The datasets, and models source code will be
open-sourced, and contributions are welcome.

###  3.2 Fon and Nobiin Word2Vec Embedding Models

For investigating how well a Word2Vec model architecture could give provide
vectorized representations of the words of our dataset, as well as establish
the relationship between them, we created a basic Word2Vec embedding model
using the library $gensim$
111https://radimrehurek.com/gensim/models/word2vec.html. In our context we
used the CBOW as training algorithm, which is a feed-forward Neural Network
Language Model, where the non-linear hidden layer is removed and the
projection layer is shared for all words. This ensures that the continuous
distributed representations of words in their respective contexts are used.

Figure 1:  Visualization of Fon Word2Vec Embedding Space Figure 2:
Visualization of Nobiin Word2Vec Embedding Space

####  3.2.1  Word2Vec for Fon

For Fon, as a starting point, we chose to focus on the family domain (context)
where words are dad, mum, sister, brother, son, daughter. From the FFR
parallel dataset (Dossou and Emezue, 2020; Dossou et al., 2021), we filtered
and extracted, and manually cleaned Fon sentences containing the keywords
mentioned above. The resultant dataset contains 739 sentences, with an average
of 8 words per sentence. The following parameters have been used to create and
train the Word2Vec model for Fon:

  * •

size: the number of dimensions of the embedding, set to $100$.

  * •

min_count: the minimum count of words to consider when training the model, set
to $5$.

  * •

$\alpha$: the learning rate set to $0.5$.

  * •

window: the maximum distance between a target word and words around the target
word, set to $5$.

  * •

workers: the number of threads to use while training, set to $3$.

  * •

sg: the training algorithm; $0$ for $CBOW$.

The source code pipeline has been inspired from «Chapter 11: The Word
Embedding Model» from Brownlee (2017).

Figure 1, shows the visualization of Fon Word Embedding Space. We can see that
words like to (father) clusters close to fofó (big brother). We can also see
the word fofó (big brother) clusters close to sunnu (boy, man), while the word
yonnúvi (girl, little girl) clusters close to no (mother). We used the Fon
Word2Vec to get the most similar words given positve references (see Table 1).

Figure 3:  Fon Poincare Hierarchy Graph with constant negative curvature c =
10 Figure 4:  Fon Poincare Hierarchy Graph with constant negative curvature c
= 15

One limitation of the Word2Vec is its restriction to the input corpora
vocabulary. This makes the model, in case of very small corpora, very
sensitive to out of vocabulary words, as similar representations can not be
found or derived.

However, these examples show that the model is able to correlate words among
themselves taking into account their contexts. This proves the concept and
importance of building WEs models for ALs words and entities, that could make
easier NLP tasks on them.

####  3.2.2 Word2Vec for Nobiin

For Nobiin, we focus on the context of daily life style of family members,
this to some extend reflects the culture in the geographic areas of this
language. Our dataset contains nearly 40 sentences, with 7-50 words per
sentence, reflecting the daily lifestyle contexts of Nobians. We use word2Vec
with the following hyperparameters:

  * •

size: the number of dimensions of the embedding, set to $200$.

  * •

min_count: the minimum count of words to consider when training the model, set
to $1$.

  * •

$\alpha$: the learning rate set to $0.025$.

  * •

window: the maximum distance between a target word and words around the target
word, set to $15$.

  * •

workers: the number of threads to use while training, set to $3$.

  * •

sg: the training algorithm; $0$ for $CBOW$.

Figure 2, shows how the model represents relationships between different words
along with their contexts. As we can see, the words (abô, anên, annenga,
annessi) which are respectively defined as ($father$, $mother$, $brother$,
$sister$) are clustered close to one another with the words ($Hassan$, $Umar$,
$Nabra$, Munâ) which are also personal names. We see also that words like
(shây, fatûr) translated respectively as ($tea$, $breakfast$), very close to
each other, and words like (tâjir, $girish$, dukkân) standing for ($merchant$,
$money$, $supermarket$) clusters are close as well. The same remark is also
applicable to other words like (semâ, $masha$, $unatti$, $winji$) meaning
respectively ($sky$, $sun$, $moon$, $stars$).

References |  Most similar representation |  Similarity score
---|---|---
noví (sister, brother) |  dadá (big sister) |  0.7928
noví + sunnu (brother) |  asi (wife) |  0.6626
to + ce (my father) |  no (mother) |  0.6901
no (mother) |  gle (crop field) |  0.6933
nyonuvi (little girl) |  kpevi (small, little) |  0.8746
no + to (mother + father) |  kplon (education) |  0.7558

Table 1:  Examples of most similiar representations in the Fon word2vec
embedding space, given as input, positive references.

##  4 Fon and Nobiin Poincaré Embedding Models

We also used $genism$ 222https://radimrehurek.com/gensim/models/poincare.html
to implement Poincaré Embedding. To evaluate the model, we used the following
criteria: the $mean$_$rank$ $(MR)$, and the $Mean$ $Average$ $Precision$
$(MAP)$. The evaluation is done at two levels:

  * •

$reconstruction$ which is defined as the capability of the observed data to
reconstruct from the embedding. Nickel and Kiela (2017) defined it as a
measure to evaluate the model’s representation capacity.

  * •

$link$ $prediction$ to test generalization performance.

Also the following modifications have been made to the source code of the
model333https://github.com/alex-
tifrea/poincare˙glove/blob/master/gensim/models/poincare.py:

  * •

the parameter $encoding$ = "$utf$-$8$" has been added as parameter to the csv
reader object, used for the Link Prediction and Reconstruction file reading,
to help them handle the diacritics (non-ascii characters).

  * •

the function $find$_$matching$_$terms()$ to bring best possibilities in the
model’s vocabulary close to the input word, instead of considering only
vocabulary words starting by the input word (as in the original code source),
leading to $KeyError$ error.

Pull requests have been made to the official repository, and a repository
containing the new version has been created too. The updated Poincaré model is
available at:
https://github.com/bonaventuredossou/poincare_glove/blob/patch-3/gensim/models/poincare.py.

###  4.1 Poincaré Embedding Model for Fon

We implement Fon Poincaré Embedding on a dataset of names (boys, and girls or
mixed names), benin cities, body parts, and date components (months of the
year, days of the week). $Mixed$ names stand for names that could be
attributed to both boys and girls. The dataset consists of a single unique
relation among two different entities on each line following the HyperLex
(Vulić et al., 2016) format, and contains 642 data samples. The dataset has
been splitted into train (572 samples), validation (25 samples) and test (45
samples). Along the training dataset, we have 218 boy names, 192 girl names,
67 mixed names, 43 benin cities, 12 months of the year and 7 days of the week.
All information and data entry of each entity, has been scraped respectively
from external websites, and from crowd-sourcing through Google Form Surveys.
For a better, understandable visualization and interpretation, we trained
first the Fon Poincaré models with embedding space size set to 2. We believe
that the concept could still be applied to higher dimensions that would
however offer less understanding and interpretability. We tried also different
constant negative curvatures ($10$, $15$). All the models have been trained on
2000 epochs. The figures 3, and 4 show different embedding spaces depending of
the value of the constant negative curvature. We can notice clearly three
different types of entities: $boy$_$name$, $girl$_$name$ and $mixed$_$name$.
The $benin$_$city$ and $body$_$part$ entities components are all clustered
together and really closed: elements of each group are, based on the graphs
practically not distinguishable. Considering the figures 3, and 4 we can see
that there are normal connections between $boy$_$name$ and $mixed$_$name$, as
well as between $girl$_$name$ and $mixed$_$name$. However, on each figure,
there exist $incorrect$ or $not$ $normal$ connections between $boy$_$name$ and
$girl$_$name$ because both are not supposed to tie together, unless in case of
mixed_names. We conclude that the constant negative curvature does not impact
how groups intersect together but instead, impacts on the distance between
elements among and across groups.

Dimensionality | 2 | 5 | 10 | 15
---|---|---|---|---
Reconstruction (MR/MAP) | 2.37/0.44 | 2.00/0.50 | 2.10/0.51 | 1.99/0.52
Link Prediction (MR/MAP) | 2.42/0.43 | 2.05/0.50 | 2.04/0.50 | 2.00/0.51

Table 2: Mean Rank and Mean Average Precision for Reconstruction and Link
Prediction for Fon

Dimensionality | 2 | 5 | 10 | 15 | 20
---|---|---|---|---|---
Reconstruction (MR/MAP) | 3.03/0.35 | 2.00/0.50 | 2.00/0.50 | 2.00/0.50 | 2.0/0.5
Link Prediction (MR/MAP) | 2.75/0.39 | 2.00/0.50 | 2.00/0.50 | 2.00/0.50 | 2.00/0.50

Table 3: Mean Rank and Mean Average Precision for Reconstruction and Link
Prediction for Nobiin

We choose the best model among the two described above, which is the one with
constant negative curvature c = 15. We also tried many other dimension size
(5, 10, 15) and evaluated them. The results of the evaluations can be seen in
table 2. From these results, we can conclude that the higher the dimension,
the better are the model reproducibility and generalization capacities.

| Precision | Recall | F1-Score | Support
---|---|---|---|---
$boy$_$name$ | 57 | 63 | 60 | 27
$girl$_$name$ | 33 | 28 | 30 | 18
accuracy |  |  | 49 | 45
macro avg | 45 | 45 | 45 | 45
weighted avg | 47 | 49 | 48 | 45

Table 4: Classification Report of the Predictions of Fon model on the Fon
Testing Dataset

| Precision | Recall | F1-Score | Support
---|---|---|---|---
$boy$_$name$ | 50 | 86 | 63 | 7
$girl$_$name$ | 50 | 14 | 22 | 7
accuracy |  |  | 50 | 14
macro avg | 50 | 50 | 43 | 14
weighted avg | 50 | 50 | 43 | 14

Table 5: Classification Report of the Predictions of Nobiin model on the
Nobiin Testing Dataset

| Precision | Recall | F1-Score | Support
---|---|---|---|---
$boy$_$name$ | 62 | 96 | 75 | 27
$girl$_$name$ | 67 | 11 | 19 | 18
accuracy |  |  | 62 | 45
macro avg | 64 | 54 | 43 | 45
weighted avg | 64 | 62 | 53 | 45

Table 6: Classification Report of the Predictions of Nobiin model on the Fon
Testing Dataset

| Precision | Recall | F1-Score | Support
---|---|---|---|---
$boy$_$name$ | 51 | 91 | 66 | 23
$girl$_$name$ | 50 | 9 | 15 | 22
accuracy |  |  | 51 | 45
macro avg | 51 | 50 | 41 | 45
weighted avg | 51 | 51 | 41 | 45

Table 7: Classification Report of the Predictions of Fon model on the Nobiin
Testing Dataset

We continued the experiments on with Fon using as dimension size d = 15. Next,
we used the model to predict entities types using the $score$_$function()$
function, that computes the predicted score; extent to which a word $a$ is of
entity type $b$. For the predictions, we chose to focus on the entities:
$boy$_$name$, $girl$_$name$, $body$_$part$, $benin$_$city$. The entities
$body$_$part$ and $benin$_$city$ ended up being predicted only either as
$boy$_$name$ or $girl$_$name$, but more as $boy$_$name$ than $girl$_$name$.
Therefore, we decided to drop them and only consider names ($boy$_$name$ and
$girl$_$name$) entities. The model achieved an accuracy of 49%. A closer look
at the classification report (see table 4), shows that the current model is
better at predicting boy names than girl ones.

Collecting more data on various entities types, training on more epochs, with
higher constant negative curvature, or on higher dimensions could improve the
model’s performance and prediction capacity. Nickel and Kiela (2017) also
showed that Poincaré embeddings are very successful in the embedding of large
taxonomies, with regard to their representation capacity and their
generalization performance.

###  4.2 Poincaré Embedding Model for Nobiin

For Nobiin, we implement a Poincaré embedding model with different constant
negative curvatures (10, 15, 20, 25, 30) on a small dataset of nobian names
(boys and girls). The dataset is a single level relationship among entities in
each line following the HyperLex format (Vulić et al., 2016). It consists of
108 data samples (training samples: 84 and test samples: 24). All the models
have been trained on 2000 epochs.

We evaluated these models using the criteria described at the beginning of
this section 4. We found that the best representations have been obtained for
the models with constant negative curvature 15 and 20 (see figures 5 and 6).
To continue the experiments with Nobiin, we used the model with constant
negative curvature c = 20.

The Table 3 shows the results of link prediction and reconstruction for the
all trained models. The best results are already achieved from dimension size
d = 5. For all further experiments with Nobiin, we choose as dimension size d
= 10.

The Table 5 shows the classification report, with a global accuracy of 50% of
the best model built with constant negative curvature c = 20, and dimension
size d = 10.

Figure 5:  Nobiin Poincare Hierarchy Graph with constant negative curvature c
= 15 Figure 6:  Nobiin Poincare Hierarchy Graph with constant negative
curvature c = 20

###  4.3 Transfer Learning of Poincaré Embedding Models

Incentivized by the performance of transfer learning and its contribution to
the state-of-the-art on a wide range of NLP tasks, we tested the Fon Poincaré
Embedding model on 45 Nobian names, and the Nobiin Poincaré Embedding model on
45 Fon names.

In another words, we want to investigate how good, the information learned by
the Fon and Nobiin Poincaré models, would be to improve the prediction of the
Fon and Nobiin entities. To that extend, we evaluated the Nobiin Poincaré
model on the Fon testing dataset: we got 62% as global accuracy: the model
predicted well most of $boy$_$name$ and some of $girl$_$name$ (see Table 6).

Next, we evaluated the Fon Poincaré model on the Nobiin testing dataset. As
reported on Table 7, the Fon Poincaré model achieved an overall accuracy of
51.11%. We can also see from that the Fon model predicted well the Nobian
$boy$_$name$ and some of $girl$_$name$.

Despite the very small size of the datasets, one important remark here is the
improvement of the classification accuracy after transfer learning: the
accuracy of classification of the Fon Poincaré model improved on the Nobiin
test dataset, and likewise for the Nobiin Poincaré model on the Fon dataset.
However, there is a need of building larger and contextualized corpora, to
check on a more extended scale, the veracity of these promising results, for
the African and low-resourced languages NLP research communities.

Thinking about reproducibility, transfer learning, and accompanied by will and
hope, that the findings of this study could be extended to as many African
Languages as possible, we can say that the information the models respectively
and solely learned about the Fon and Nobiin data, helped them to predict
averagely well $boy$_$name$ and $girl$_$name$ in the other language: both
models used properties learned about the data they have been trained on, and
extended it to the other language, to get the entities types right.

##  5 An Approach to Linguistic and Social Interpretations

Unlike the $boy$_$name$, we noticed a low $Recall$ of $girl$_$name$ in the
Poincaré models. As those results could infer some disparities, we tried to
look for some linguistic, social interpretations. We found a plausible
explanation could be that indigenous girl names share a lot similarities with
indigenous boy names (mainly derivated from them), like the names ahoví
(prince) and ahossí (princesse), sharing the same root.

Figure 7: Visualization of Biased in one of Nobiin Word2Vec Embedding Models

WEs models can illustrate semantic and syntactic relationships between words,
but they are not without flaws. The Figure 7 shows one of our models
clustering words of housekeeping activities: (floor-sweeping: tôyi), (washing:
shûkki), (utensils: dâddi), (cooking: okke) close to the word mother (anên).
We mitigated this bias by fine-tuning the maximum distance between the current
and predicted words within a sentence, in the model of section 3.2.2, which
clustered words $mother$ with father (abô), sister (annessi), brother
(annenga), and housekeeping activities words with the word house (nôg).

Bolukbasi et al. (2016), also demonstrated in their study how WEs reinforced
gender stereotypes at an alarming rate. They tested how Google would translate
sentences from Turkish, which uses gender neutral pronouns, to English.
Mostly, when a sentence contained descriptors stereotypically attributed to
women (cook, teacher, nurse), the Turkish gender-neutral pronoun $o$ was
translated to $she$. Conversely, sentences with terms such as hard working,
lawyer, and $engineer$ saw the pronoun translated as $he$.

Bolukbasi et al. (2016) and Manzini et al. (2019) (inspired by (Bolukbasi et
al., 2016)), proposed efficient debiasing methods (hard and soft debiasing)
respectively for binary class (eg. male, female) and for multiclass (gender,
race, religion etc.) settings.

We want to precise that our point is not to blame Google but instead to
emphasize that algorithms are based on a corpus of human words containing
billions of data points. So WEs are merely reflecting already existing biases
in societies. As our African open source WE models evolve, working on reducing
disparities and inequalities would be a great future work pathway.

##  6 Conclusion and Future Work

In this paper we showed the possibilities and importance of building proper
word embedding models for African Languages words and entities. We created
Word2Vec and Poincaré Embedding models, for Fon and Nobiin, and showed that
they can successfully represent similarities and relationships among those
words and entities. We also presented promising transfer learning results,
from the models accross the languages. The models we proposed, albeit on very
small datasets, can further highly be improved, since some of the
relationships have not been well generalized. Nevertheless, we believe that
our findings are headrooms to developing better words embedding models, which
could highly ease downstream NLP tasks and challenges on African Languages.

##  7 Acknowledgments

Authors would like to thank all contributors that helped gathering the data
needed for the current study, especially Fabroni Bill Yoclunon, Ricardo
Ahounvlame, and Nerry Koukoui.

## References

  * Alabi et al. (2019) Jesujoba O. Alabi, Kwabena Amponsah-Kaakyire, David I. Adelani, and Cristina España-Bonet. 2019.  Massive vs. curated word embeddings for low-resourced languages. the case of yorùbá and twi.
  * Bolukbasi et al. (2016) Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016\.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.
  * Brownlee (2017) Jason Brownlee. 2017.  _Deep Learning for Natural Language Processing_.  Machine Learning Mastery.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.
  * Dossou and Emezue (2020) Bonaventure F. P. Dossou and Chris C. Emezue. 2020.  Ffr v1.1: Fon-french neural machine translation.
  * Dossou et al. (2021) Bonaventure F. P. Dossou, Fabroni Yoclounon, Ricardo Ahounvlamè, and Chris Emezue. 2021.  Fon french daily dialogues parallel data.
  * Manzini et al. (2019) Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. 2019.  Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.
  * Nickel and Kiela (2017) Maximilian Nickel and Douwe Kiela. 2017.  Poincaré embeddings for learning hierarchical representations.
  * Norouzi et al. (2014) Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S. Corrado, and Jeffrey Dean. 2014.  Zero-shot learning by convex combination of semantic embeddings.
  * Orife (2018) Iroro Orife. 2018.  Attentive sequence-to-sequence learning for diacritic restoration of yorùbá language text.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Ravasz and Barabási (2003) Erzsébet Ravasz and Albert-László Barabási. 2003.  Hierarchical organization in complex networks.  _Phys. Rev. E_ , 67:026112.
  * Sappadla et al. (2016) Prateek Veeranna Sappadla, Jinseok Nam, Eneldo Loza Mencía, and Johannes Fürnkranz. 2016.  Using semantic similarity for multi-label zero-shot classification of text documents.  In _Proceedings of the 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN-16)_ , Bruges, Belgium. d-side publications.
  * Vulić et al. (2016) Ivan Vulić, Daniela Gerz, Douwe Kiela, Felix Hill, and Anna Korhonen. 2016.  Hyperlex: A large-scale evaluation of graded lexical entailment.
  * van der Westhuizen and Niesler (2017) Ewald van der Westhuizen and Thomas Niesler. 2017.  Synthesising isizulu-english code-switch bigrams using word embeddings.
  * Xian et al. (2017) Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata. 2017.  Zero-shot learning – a comprehensive evaluation of the good, the bad and the ugly.


# Human-Robot Handovers using Task-Space Quadratic Programming

Mohamed Djeha, Antonin Dallard, Ahmed Zermane, Pierre Gergondet
and Abderrahmane Kheddar The authors are with the CNRS-University of
Montpellier LIRMM Interactive Digital Humans group, Montpellier, France.
<EMAIL_ADDRESS>Kheddar and P. Gergondet are also with the CNRS-AIST
Joint Robotics Laboratory, IRL, Tsukuba, Japan.

###### Abstract

Bidirectional object handover between a human and a robot enables an important
functionality skill in robotic human-centered manufacturing or services. The
problem in achieving this skill lies in the capacity of any solution to deal
with three important aspects: (i) synchronized timing for the handing over
phases; (ii) the handling of object pose constraints; and (iii) understanding
the haptic exchanging to seamlessly achieve some steps of the (i). We propose
a new approach for (i) and (ii) consisting in explicitly formulating the
handover process as constraints in a task-space quadratic programming control
framework to achieve implicit time and trajectory encounters. Our method is
implemented on Panda robotic arm taking objects from a human operator.

##  I Introduction

Human-centered robotics trends aim at close contact interactions in different
areas: hospitals, industry, malls, homes, restaurants, etc. This cohabitation
implies a large taxonomy of human-robot interactions. Among the most
challenging one, seamlessly exchanging objects between a human and robot as
_giver_ and _receiver_ agents through (bi-directional) handovers. Indeed,
handing objects between humans is a very frequent daily form of interaction in
almost all domains [1]. Therefore, embedding human-centered robotic systems in
automation and services with handovers capabilities, is a key enabler for rich
cognitive interaction [2, 3]. Although human-human handover is an intuitive
behavior, it is a result of complex and sophisticated learned social-cognitive
communication channels [4]. Grasping such a complexity in an equivalently
robot control formulation strategy is not a trivial problem.

Indeed, how to _codify_ the human-robot handover process has been thoroughly
addressed in the research community. The state-of-the-art studies split the
handover process in two main phases: the _pre-grasping_ phase [5, 6, 7, 8, 9,
10, 11, 12] and the _grasping_ phase [13, 14, 15, 16, 17]. Such a breakdown is
simply chronological: the former focuses on detecting the object,
estimating/predicting the human movement and planning the robot reaching
motion toward the meeting point, whereas the latter consists of understanding
the interaction forces (haptics) applied at the object by the two handing over
agents during the exchange such that to ensure a stable grasping. In [15] a
_retraction_ phase is considered, which describes how the giver and receiver
move away after the latter gets the full control of the object. Yet, for an
exhaustive handover survey, please refer to the excellent recent review in
[1].

Despite the huge amount of works addressing human-robot handover, one issue
remains open: how to ensure/codify an adaptive behavior of the robot w.r.t to
a human versatile intention on _where_ to exchange the object, aka _HandOver
Location_ (HOL), with the robot, and in which configuration (_orientation_). A
large amount of existing works required the knowledge of the HOL as a
precondition either for motion planning or control formulations of the pre-
grasping phase. In particular, the HOL is often considered either as fixed
(the robot moves systematically to a fixed spot to pick-up the object) or pre-
planned (real-time detection and estimation of the object fixed-location) [1,
Tables 1-2]. There are works that provide HOL estimation and on-line
prediction methods [18, 15, 19, 20, 21, 22]. However, such existing approaches
have at least one of the following shortcomings:

  * •

Collecting data sets to train motion prediction models;

  * •

Time-consuming computations leading to a hashed or a slow handover;

  * •

Magic-numbers of meta-parameters to be tuned;

  * •

Conservatism in the HOL prediction policy;

  * •

Non-systematic success of the handover performance;

  * •

The object orientation at the HOL is often kept the same during the
experiments;

  * •

Knowledge of the HOL is very often required.

Figure 1: Proposed seamless handover process executed without any prior
knowledge about the HOL and final object orientation.

In this work, we formulate the premises of a generic robot control-strategy
that ensures high fidelity and reliability of reaching motion for bi-
directional handover scenarios and guarantees adaptability w.r.t the HOL and
object orientation (Fig. 1). Particularly, our control formulation gathers the
advantages from both methods in [17, 15]. Indeed, our method is HOL knowledge
free. We only assume that the object structural properties and dimensions are
known, and there exists a sensor that provides the pose (Cartesian position
and orientation) of the object of interest, e.g. [23]. Conversely to [17], our
method ensures/codifies a real-time proactive and adaptive robot motion w.r.t
to the human versatile intention on _where_ to exchange the object with the
robot and in which configuration (_orientation_) without the need of offline
acquired data. Our aim is to build on our existing multi-objective task-space
optimization Quadratic-Programming (QP) framework [24, 25] that prove to be
very powerful in an industrial context [26] and a human-assistance context
[27]. In particular, our proposed idea relies on the concept of _coupled
tasks_ where the state of one task is forwarded as the reference for another
task. More precisely, we formulate an _observation task_ that estimates either
human hand or object full-state in terms of pose, velocity and acceleration
which are then forwarded as references for a _trajectory-tracking task_ for
the robot end-effector to track. Leveraging multi-objective QP control
paradigm, these two tasks run concurrently in a _leader-follower_ fashion: the
object (leader) moves and converges to the HOL while the robot end-effector
(follower) moves proactively toward the object while adjusting its orientation
accordingly. From this standpoint, our approach is similar to the coupled
Dynamical Systems (DS) in [15]. Except that our approach is computationally
cheap and does not require the estimation of the HOL nor to separately learn
human and robot DS models. To sum-up, our contributions to coupled task-space
observer-controller are as follows:

  * •

Systematic approach (no conservative policy of HOL prediction) with less
parameter to tune (task gains only);

  * •

No need for planning and time consuming processes of collecting data and model
training;

  * •

Requires no time-consuming computations resulting in a smooth (fluid) and
seamless handover motion;

  * •

Codifies rich behaviors with multi-objective QP control paradigm: proactive
position and orientation control while accounting for safety constraints
(collision avoidance, joint and actuation limits, etc.);

  * •

Generalizes to bi-directional human-robot handovers;

  * •

Needs only a sensor or estimator that provides the object pose, e.g. [28, 29,
23].

Notations: $x^{\star}\in\mathbb{R}^{3}$ is expressed in frame ${\cal
R}_{\star}$. $R_{\star\star}^{\star}\in SO(3)$ denotes the rotation matrix of
the frame ${\cal R}_{\star\star}$ w.r.t ${\cal R}_{\star}$. If ${\cal
R}_{\star}={\cal R}_{\rm w}$, the world-frame, then
$R_{\star\star}^{\star}=R_{\star\star}$.
$\begin{bmatrix}\omega\times\end{bmatrix}={\rm skew}(\omega)$ denotes the
skew-symmetric matrix associated with the vector $\omega\in\mathbb{R}^{3}$.
$\alpha_{x}$ denotes the velocity of $x\in{\cal X}$. If $\cal X$ is an
Euclidean space then $\alpha_{x}=\dot{x}$.
$\mathbf{q}_{\star}^{T}=\begin{bmatrix}\bar{\mathbf{q}}_{\star}&\hat{\mathbf{q}}_{\star}\end{bmatrix}\in\mathbb{R}^{4}$
is the unit quaternion representation of $R_{\star}$ where
$\bar{\mathbf{q}}\in\mathbb{R}$ and $\hat{\mathbf{q}}\in\mathbb{R}^{3}$ with
$\begin{Vmatrix}\mathbf{q}_{\star}\end{Vmatrix}=1$, and
${\mathbf{q}_{\star}^{-1}}^{T}=\begin{bmatrix}\bar{\mathbf{q}}_{\star}&-\hat{\mathbf{q}}_{\star}\end{bmatrix}$.
The vector part of the quaternion product
$\mathbf{q}_{\star}\otimes\mathbf{q}_{\star\star}$ is denoted as
$\hat{\mathbf{q}}_{\star}\ominus\hat{\mathbf{q}}_{\star\star}=\bar{\mathbf{q}}_{\star\star}\hat{\mathbf{q}}_{\star}+\bar{\mathbf{q}}_{\star}\hat{\mathbf{q}}_{\star\star}+\begin{bmatrix}\hat{\mathbf{q}}_{\star}\times\end{bmatrix}\hat{\mathbf{q}}_{\star\star}\in\mathbb{R}^{3}$.
If $\mathbf{q}_{\star\star}=\mathbf{q}_{\star}^{-1}$ then
$\hat{\mathbf{q}}_{\star}\ominus\hat{\mathbf{q}}_{\star\star}=0$.
$P_{\star}\in\mathbb{R}^{3}$ denotes the Cartesian coordinates of ${\cal
R}_{\star}$ origin. $\dot{P}_{\star},\ddot{P}_{\star}\in\mathbb{R}^{3}$ are is
linear velocity and acceleration. The angular velocity and acceleration of
${\cal R}_{\star}$ frame are
$\omega_{\star},\dot{\omega}_{\star}\in\mathbb{R}^{3}$, respectively.

##  II Handover Paradigm

###  II-A Handover Control Problem

In order to perform object handover efficiently, the robot reaching-motion
toward the HOL shall be synchronized with the object’s motion to achieve a
_one-shot continuous and smooth motion_. If the HOL is known in advance, the
control formulation resumes to perform simply a set-point reaching task for
the end-effector. Nevertheless, the HOL and its timing as well as the object
orientation are highly versatile: they may change during the handover motion
or between two successive processes depending on the human intention and
her/his posture. Handing over an object can even be aborted on the human
decision while moving. Hence, planning the robot motion is likely to fail in
such conditions. Alternatively, a reactive control method is more suitable for
its ability to adjust the robot motion in real-time. We denote by _end-
effector_ the robot terminal link used to pick-up the object. It encompasses
two-finger grippers, multi-finger hand and even a non-human like devices as a
suction-cup or an electromagnetic gripper [30].

Our approach is to tackle the human-robot handover problem from a reactive
closed-loop control perspective. We shall explain our approach when the human
is the giver, then we show how it applies when s/he is the receiver. The HOL
can be seen as an attractor toward which the object is converging (steered by
the human). Since the HOL is not known a priori, the robot end-effector can
track the object trajectory leading to a proactive motion such that when the
object converges to the HOL, its trajectory becomes a point to which the end-
effector converges. The same reasoning can be applied for the orientation. The
remaining question is how to obtain the object trajectory in terms of pose
$X_{\rm obj}\in\mathbb{R}^{7}$, linear and angular velocity $\alpha_{X_{\rm
obj}}\in\mathbb{R}^{6}$ and linear and angular acceleration
$\dot{\alpha}_{X_{\rm obj}}\in\mathbb{R}^{6}$? These terms are encompassed as
the _object full-state_

$s_{\rm obj}=\begin{bmatrix}X_{\rm obj}\\\ \alpha_{X_{\rm obj}}\\\
\dot{\alpha}_{X_{\rm obj}}\end{bmatrix}\in\mathbb{R}^{19}.$ (1)

Often, $s_{\rm obj}$ cannot be directly obtained by the sensors as generally
they provide only $X_{\rm obj}$. Hence, an estimation of $s_{\rm obj}$ needs
to be constructed from $X_{\rm obj}$ and it is denoted as

$s_{\rm obs}=\begin{bmatrix}X_{\rm obs}\\\ \alpha_{X_{\rm obs}}\\\
\dot{\alpha}_{X_{\rm obs}}\end{bmatrix}\in\mathbb{R}^{19}.$ (2)

To this end, the observation task is formulated as a PD controller that drives
$X_{\rm obs}$ toward $X_{\rm obj}$, e.g., [23]. While converging, the
observation task outputs also $\alpha_{X_{\rm obs}}$ and $\dot{\alpha}_{X_{\rm
obs}}$, e.g. [31]. $s_{\rm obs}$ terms are then used as references for the
trajectory-tracking task formulated for the end-effector. Note that the
observation task allows to generate trajectories for the orientation tracking
without the need for offline planning methods [32]. The same approach can be
adapted if the human is a receiver. In such case, the end-effector already
holds the object and its full-state known by forward kinematics. Instead, the
observation task constructs the full-state of the human hand [31].

Multi-objective task-space QP control has been extensively used as it enables
to specify various tasks objectives while explicitly accounting for a set of
convex constraints [33]. In [25], one single QP controller can be formulated
for multi-robots systems either decoupled or interacting with each other using
contact forces. In particular, the pre-grasping handover phase can be suitably
formulated using the multi-robots QP by considering the robotic arm and the
object as two decoupled robots where the former is a redundant multi-body
system where all the joints are actuated, and the latter as a floating-base
rigid-body system. Moreover, the observation task is formulated for the object
to compute $s_{\rm obs}$ and the trajectory-tracking task is formulated for
the end-effector. The following subsections explicit in details our approach.

###  II-B Background

Consider a $n$ degree-of-freedom redundant robotic arm such that its joint
configuration is defined by $q\in\mathbb{R}^{n}$. The robot is governed by the
following equation of motion:

$M(q)\ddot{q}+N(q,\dot{q})=\tau+J_{\rm c}^{T}f,$ (3)

where $M(q)\in\mathbb{R}^{n\times n}$ is the inertia matrix,
$N(q,\dot{q})\in\mathbb{R}^{n}$ encompasses Coriolis, centrifugal and gravity
torques, $\tau\in\mathbb{R}^{n}$ is the joint-torque, $J_{\rm
c}\in\mathbb{R}^{6\times n}$ is the Jacobian at the contact point and
$f\in\mathbb{R}^{6}$ is the contact wrench. Let us consider the frame ${\cal
R}_{\rm ee}$ rigidly attached to the end-effector and which pose is

$X_{\rm ee}=\begin{bmatrix}P_{\rm ee}\\\ \mathbf{q}_{\rm
ee}\end{bmatrix}\in\mathbb{R}^{7}.$ (4)

Its velocity and acceleration are given as

$\displaystyle\alpha_{X_{\rm ee}}$ $\displaystyle=\begin{bmatrix}\dot{P}_{\rm
ee}\\\ \omega_{\rm ee}\end{bmatrix}=J_{\rm ee}\dot{q}\in\mathbb{R}^{6},$ (5)
$\displaystyle\dot{\alpha}_{X_{\rm ee}}$
$\displaystyle=\begin{bmatrix}\ddot{P}_{\rm ee}\\\ \dot{\omega}_{\rm
ee}\end{bmatrix}=J_{\rm ee}\ddot{q}+\dot{J}_{\rm ee}\dot{q}\in\mathbb{R}^{6},$
(6)

where $J_{\rm ee}\in\mathbb{R}^{6\times n}$ is the end-effector Jacobian.

Let us consider the object as a one-rigid-link robot with 6 DoF to which a
frame ${\cal R}_{\rm obj}$ is rigidly attached and which pose is

$X_{\rm obj}=\begin{bmatrix}P_{\rm obj}\\\ \mathbf{q}_{\rm
obj}\end{bmatrix}\in\mathbb{R}^{7},$ (7)

where $X_{\rm obj}$ is assumed to be provided by a sensor. Its velocity and
acceleration are given as

$\displaystyle\alpha_{X_{\rm obj}}$ $\displaystyle=\begin{bmatrix}\dot{P}_{\rm
obj}\\\ \omega_{\rm obj}\end{bmatrix}\in\mathbb{R}^{6},\ \dot{\alpha}_{X_{\rm
obj}}=\begin{bmatrix}\ddot{P}_{\rm obj}\\\ \dot{\omega}_{\rm
obj}\end{bmatrix}\in\mathbb{R}^{6}.$ (8)

The object structural and dimension properties are known.

###  II-C Observation Task

Let us consider an observed object to which a frame ${\cal R}_{\rm obs}$ is
rigidly attached which pose is

$X_{\rm obs}=\begin{bmatrix}P_{\rm obs}\\\ \mathbf{q}_{\rm
obs}\end{bmatrix}\in\mathbb{R}^{7}.$ (9)

Assuming that the object (linear and angular) velocity and acceleration (8)
are not provided by the sensor (which is likely the case), the observation
task aims at constructing these non-measured states by estimating $s_{\rm
obs}$ in (2).

This is achieved by the observation task that steers $X_{\rm obs}$ toward
$X_{\rm obj}$ by keeping the observation error $e_{\rm obs}$ as small as
possible such that

$\displaystyle e_{\rm obs}$ $\displaystyle=\begin{bmatrix}P_{\rm obs}-P_{\rm
obj}\\\ \hat{\mathbf{q}}_{\rm obs}\ominus\hat{\mathbf{q}}_{\rm
obj}^{-1}\end{bmatrix}\in\mathbb{R}^{6}.$ (10)

The observation error velocity and acceleration are given as

$\displaystyle\alpha_{e_{\rm obs}}=\alpha_{X_{\rm obs}},\ \dot{\alpha}_{e_{\rm
obs}}=\dot{\alpha}_{X_{\rm obs}}.$ (11)

The observation task state is defined as

$\eta_{\rm obs}=\begin{bmatrix}e_{\rm obs}\\\ \alpha_{e_{\rm
obs}}\end{bmatrix}\in\mathbb{R}^{12}.$ (12)

Thus, the observation task is formulated as follows

$\displaystyle\dot{\eta}_{\rm obs}$ $\displaystyle=\begin{bmatrix}0&I\\\
0&0\end{bmatrix}\eta_{\rm obs}+\begin{bmatrix}0\\\ I\end{bmatrix}\mu_{\rm
obs},\ \mu_{\rm obs}=\dot{\alpha}_{X_{\rm obs}}$ (13)

Choosing $\mu_{\rm obs}$ in (13) a linear state feedback

$\mu_{\rm obs}=-\left[K_{s_{\rm obs}}\ K_{d_{\rm obs}}\right]\eta_{\rm
obs}=-K_{\rm obs}\eta_{\rm obs},$ (14)

with $K_{s_{\rm obs}},K_{d_{\rm obs}}\in\mathbb{R}^{6\times 6}$ are diagonal
positive-definite matrices denoting the stiffness and damping gains; it yields
to the observation-task closed-loop dynamics

$\displaystyle\dot{\eta}_{\rm obs}$ $\displaystyle=A_{\rm obs}\eta_{\rm obs},\
A_{\rm obs}=\begin{bmatrix}0&I\\\ -K_{s_{\rm obs}}&-K_{d_{\rm
obs}}\end{bmatrix},$ (15)

with $A_{\rm obs}$ Hurwitz [34]. Note that $\eta_{\rm obs}$ only converges
asymptotically to zero if the object is static ($X_{\rm obs}$ constant).
However, choosing high gain values typically enables a fast convergence and
keeps $\begin{Vmatrix}e_{\rm obs}\end{Vmatrix}$ sufficiently small.

The benefits of the observation task are three folds: (i) allows a bounded
estimation of $s_{\rm obs}$ in (2) given (13)–(15); (ii) $X_{\rm obj}$ is low-
pass filtered by the closed-loop observation task dynamics (15); (iii) enables
an online generation of a smooth twice-differentiable trajectory111This is
also known as a _reference model_ -based approach for trajectory references
generation [34, Chapter 13]. In addition, the trajectory feedforward terms are
generated in real-time since the observation task is updated at the same
frequency of QP no matter the sampling-frequency of the sensor. for the
Cartesian and orientation coordinates of ${\cal R}_{\rm obs}$ without the
needs of an offline planning methods [32]. The latter advantage is the _core
idea_ presented in this paper: the observation task outputs trajectory
references required by the trajectory-tracking task in order to achieve a
proactive handover process. More importantly, theses two tasks are integrated
in the multi-objective QP controller as shown in Section II-F. Once having
$s_{\rm obs}$, we can compute the full-state for any other frame ${\cal
R}_{\star}$ attached to the object (for which the local pose $X_{\star}^{\rm
obs}$ is known) by simply applying the classical kinematic relations for
position, velocity and acceleration:

$\displaystyle P_{\star}$ $\displaystyle\\!=\\!P_{\rm obs}+R_{\rm
obs}P_{\star}^{\rm obs},$ (16) $\displaystyle\dot{P}_{\star}$
$\displaystyle\\!=\\!\dot{P}_{\rm obs}+\begin{bmatrix}\omega_{\rm
obs}\times\end{bmatrix}R_{\rm obs}P_{\star}^{\rm obs},$ (17)
$\displaystyle\begin{split}\ddot{P}_{\star}&\\!=\\!\ddot{P}_{\rm
obs}\\!+\\!\begin{bmatrix}\dot{\omega}_{\rm obs}\times\end{bmatrix}R_{\rm
obs}P_{\star}^{\rm obs}\\!+\\!\begin{bmatrix}\omega_{\rm
obs}\times\end{bmatrix}^{2}R_{\rm obs}P_{\star}^{\rm obs}\end{split},$ (18)
$\displaystyle R_{\star}$ $\displaystyle\\!=\\!R_{\rm obs}R_{\star}^{\rm
obs}.$ (19)

Note that the rigid body assumption implies that $\dot{P}_{\star}^{\rm
obs}=0$, $\dot{R}_{\star}^{\rm obs}=0$, $\omega_{\star}=\omega_{\rm obs}$ and
$\dot{\omega}_{\star}=\dot{\omega}_{\rm obs}$.

Figure 2: Illustrative scheme showing the frames ${\cal R}_{\rm w}$, ${\cal
R}_{\rm ee}$, ${\cal R}_{\rm obj}$, ${\cal R}_{\rm obs}$ and ${\cal R}_{\rm
grasp}$. The observed object is tracking the actual object giving the data
from sensor thanks to the observation task. The end-effector tracks the
observation task state yielding an anticipatory motion toward the HOL where it
ultimately meets the object. The colored unit vectors in ${\cal R}_{\rm ee}$
track their corresponding in ${\cal R}_{\rm grasp}$. For instance, if the
object is cylindrical, only the yellow unit vectors must be aligned.

###  II-D Trajectory-Tracking Task

Let ${\cal R}_{\rm grasp}$ be the grasping frame rigidly attached to the
object and which pose is defined as

$X_{\rm grasp}=\begin{bmatrix}P_{\rm grasp}\\\ \mathbf{q}_{\rm
obs}\end{bmatrix}\in\mathbb{R}^{7},$ (20)

such that $P_{\rm grasp}\in\mathbb{R}^{3}$ denotes the coordinates of the
point where the end-effector grasps the object and is computed from (16), and
$\mathbf{q}_{\rm obs}$ is obtained from $R_{\rm grasp}$ in (19). ${\cal
R}_{\rm grasp}$ velocity and acceleration are computed as

$\displaystyle\alpha_{X_{\rm grasp}}$
$\displaystyle=\begin{bmatrix}\dot{P}_{\rm grasp}\\\ \omega_{\rm
obs}\end{bmatrix}\in\mathbb{R}^{6},\ \dot{\alpha}_{X_{\rm
grasp}}=\begin{bmatrix}\ddot{P}_{\rm grasp}\\\ \dot{\omega}_{\rm
obs}\end{bmatrix}\in\mathbb{R}^{6}.$ (21)

Let us define the trajectory-tracking task error as

$\displaystyle e_{\rm tt}$ $\displaystyle=\begin{bmatrix}P_{\rm ee}-P_{\rm
grasp}\\\ \hat{\mathbf{q}}_{\rm ee}\ominus\hat{\mathbf{q}}_{\rm
grasp}^{-1}\end{bmatrix}\in\mathbb{R}^{6},$ (22)

and which derivative is given as

$\displaystyle\alpha_{e_{\rm tt}}$ $\displaystyle=\begin{bmatrix}\dot{P}_{\rm
ee}-\dot{P}_{\rm grasp}\\\ \omega_{\rm ee}-\omega_{\rm
obs}\end{bmatrix}=\alpha_{X_{\rm ee}}-\alpha_{X_{\rm
grasp}}\in\mathbb{R}^{6}.$ (23)

Let us denote the trajectory-tracking task state

$\displaystyle\eta_{\rm tt}=\begin{bmatrix}e_{\rm tt}\\\ \alpha_{e_{\rm
tt}}\end{bmatrix}\in\mathbb{R}^{12}.$ (24)

Hence, the trajectory-tracking task dynamics is formulated

$\displaystyle\dot{\eta}_{\rm tt}$ $\displaystyle=\begin{bmatrix}0&I\\\
0&0\end{bmatrix}\eta_{\rm tt}+\begin{bmatrix}0\\\ I\end{bmatrix}\mu_{\rm tt},$
(25) $\displaystyle\mu_{\rm tt}$ $\displaystyle=\dot{\alpha}_{X_{\rm
ee}}-\dot{\alpha}_{X_{\rm grasp}}=J_{\rm ee}\ddot{q}+\dot{J}_{\rm
ee}\dot{q}-\dot{\alpha}_{X_{\rm grasp}}.$ (26)

Then, choosing $\mu_{\rm tt}$ in (26) as

$\mu_{\rm tt}=-\left[K_{s_{\rm tt}}\ K_{d_{\rm tt}}\right]\eta_{\rm
tt}=-K_{\rm tt}\eta_{\rm tt},$ (27)

with $K_{s_{\rm tt}},K_{d_{\rm tt}}\in\mathbb{R}^{6\times 6}$ are diagonal
positive-definite matrices denoting the stiffness and damping gains; it yields
to the following trajectory-tracking task closed-loop dynamics

$\dot{\eta}_{\rm tt}=A_{\rm tt}\eta_{\rm tt},\ A_{\rm
tt}=\begin{bmatrix}0&I\\\ -K_{s_{\rm tt}}&-K_{d_{\rm tt}}\end{bmatrix}.$ (28)

where $A_{\rm tt}$ is Hurwitz. This enables a global asymptotic convergence of
$\eta_{\rm tt}$ to the origin [34]. Commonly, the trajectory is planned such
that the initial reference trajectory pose is as close as possible to the
current end-effector pose $X_{\rm ee}(t_{0})$ which indeed ensures that the
tracking error $e_{\rm tt}(t_{0})$ is small and thereby enforces the end-
effector pose to stick on trajectory forward in time. However, when the
trajectory starts far from the initial end-effector pose, the latter converges
to the trajectory with an asymptotic decay of the tracking error $\eta_{\rm
tt}$. This property enables the anticipatory motion of the end-effector which
moves proactively toward the object grasping position (see Fig. 2).

###  II-E Posture Task

The posture task is mainly intended to solve the remaining redundant states
$(q,\dot{q})$ and keep them bounded. Let $q_{\rm ref}\in\mathbb{R}^{n}$ be a
given reference posture designed generally to represent a suitable robot
posture (e.g., elbow up). Let us define the posture task state as

$\eta_{\rm pos}=\begin{bmatrix}e_{\rm pos}\\\ \dot{e}_{\rm
pos}\end{bmatrix}=\begin{bmatrix}q-q_{\rm ref}\\\ \dot{q}\end{bmatrix}.$ (29)

The posture task dynamics is thereby

$\displaystyle\dot{\eta}_{\rm pos}$ $\displaystyle=\begin{bmatrix}0&I\\\
0&0\end{bmatrix}\eta_{\rm pos}+\begin{bmatrix}0\\\ I\end{bmatrix}\mu_{\rm
pos},\ \mu_{\rm pos}=\ddot{q}.$ (30)

Choosing $\mu_{\rm pos}$ in (30) as

$\mu_{\rm pos}=-\begin{bmatrix}K_{s_{\rm pos}}&K_{d_{\rm
pos}}\end{bmatrix}\eta_{\rm pos}=-K_{\rm pos},$ (31)

leads to the posture task closed-loop dynamics

$\dot{\eta}_{\rm pos}=A_{\rm pos}{\eta}_{\rm pos},\ A_{\rm
pos}=\begin{bmatrix}0&I\\\ -K_{s_{\rm pos}}&-K_{d_{\rm pos}}\end{bmatrix},$
(32)

where $A_{\rm pos}$ is Hurwitz given that the stiffness and damping gains
$K_{s_{\rm pos}}$, $K_{d_{\rm pos}}$ are diagonal positive-definite. Eq. 32
yields asymptotic convergence of $\eta_{\rm pos}$ to the origin. However, this
is only the case when the posture task is not in conflict with the other tasks
and constraints among which the posture task has the lowest priority.
Alternatively, $e_{\rm pos}$ is only ensured to be uniformly ultimately
bounded [24].

###  II-F Multi-Tasks QP Formulation

Let us denote $\chi^{T}=\begin{bmatrix}\ddot{q}^{T}&\dot{\alpha}_{\rm
obs}^{T}&f^{T}\end{bmatrix}\in\mathbb{R}^{n+6+3}$ the vector encompassing the
linear and angular acceleration of the observed rigid-body object, the joint-
acceleration of the robotic arm, and the contact force. Given the affinity of
(13) w.r.t $\dot{\alpha}_{X_{\rm obs}}$, that of (26)(30) w.r.t $\ddot{q}$ and
that of (3) w.r.t $\ddot{q}$ and $f$, then we can combine all the tasks and
constraint in a single weighted-prioritized QP formulation

$\displaystyle\underset{\chi}{\min}\sum_{i}$ $\displaystyle
w_{i}\begin{Vmatrix}E_{\chi}^{i}\chi+F_{\chi}^{i}\end{Vmatrix}^{2}$ (33a) S.t:
$\displaystyle C_{\chi}\chi\leq d_{\chi}$ (33b)

where $w_{i}>0$ is the associated weight to each task $i$. QP formulation (33)
enables to: (i) combine the different competing tasks in (33a) by settling a
soft prioritization scheme; (ii) account for unilateral constraints (33b) that
embed limitations like joint-position, velocity, acceleration and torque
limits, as well as safety aspects like (self)-collision avoidance to ensure
safe grasping [35]; (iii) find optimal $\ddot{q}$ that generates the robotic
arm motion that achieve _at best_ all the tasks while fulfilling all the
constraints; and (iv) handle multi-robot control where the robotic arm and the
object are considered as two distinct robots entities which opens the
possibility to add other robots to achieve the handover (e.g., bi-arm
handover) while still using one compact formulation [25].

Figure 3: Perception Neuron motion capture sensor-suit used for the handover
experiments and mounted on the left arm. Three IMUs (highlighted in colored
squares) mounted on the left hand are used to provide an estimation of the
hand pose (the orange square).

##  III Experimental Result

For demonstration, we implemented our approach on the open-source code
implementation of the QP controller interface mc_rtc222https://jrl-
umi3218.github.io/mc_rtc/index.html. Based on the embedded sensors data,
mc_rtc builds and solves the QP problem at each control cycle ($1$ ms).
Experiments are conducted using 7-DoF robotic arm Panda. Perception
Neuron333https://neuronmocap.com/ sensor-suit has been used for the pose
measurement (Fig. 3). An API provides the hand pose in an arbitrarily fixed-
frame at a frequency of $60$ Hz. A calibration step is necessary to correctly
coincides the sensor frame with ${\cal R}_{\rm w}$. The calibration process is
repeated at the beginning of each handover experiment to avoid drifting. When
the human is a giver, the object is assumed to be rigidly attached to the
hand, and thereby its pose can be computed based on the hand’s one. Several
handover scenarios have been performed where the robot starts from different
configurations. In addition, multiple HOLs have been tried (Fig. 4). The
observation task gains have been set to high values $K_{s_{\rm obs}}=1500I$,
$K_{d_{\rm obs}}=2\sqrt{K_{s_{\rm obs}}}$ to ensure an accurate object pose
estimation $s_{\rm obj}$ (1). Fig. 5 shows how $X_{\rm obs}$ tracks $X_{\rm
obj}$. The high observation task gains allow an accurate pose tracking.
Concurrently, it allows obtaining a good estimation of $\alpha_{X_{\rm obj}}$
and $\dot{\alpha}_{X_{\rm obj}}$ by computing $\alpha_{X_{\rm obs}}$ and
$\dot{\alpha}_{X_{\rm obs}}$ from (15). $s_{\rm obj}$ is then forwarded to the
trajectory-tracking task where the gains of the orientation part have been
chosen twice bigger than those of the translation to ensure that end-effector
reaches the object while its orientation has already converged to the target.
The stiffness of the translation part is fixed at $I$ and the corresponding
damping $2\sqrt{I}$. From Fig. 6, the object and the end-effector meet
seamlessly and smoothly at the HOL without being specified _a priori_ , see
Fig. 7. Furthermore, the end-effector adapts reactively to the variation of
the grasping point as shown in the attached video.

Figure 4: Different HOLs (red) desired by the human operator. The object is a
black cylindrical container. The transparent robots represent the initial
configuration.

Figure 5: Position (left) and orientation (right with RPY angles) of ${\cal
R}_{\rm obj}$ (dashed) and its estimation ${\cal R}_{\rm obs}$ (solid)
obtained by the observation task.

Figure 6: Position (left) and orientation (right with RPY angles) of the
frames ${\cal R}_{\rm grasp}$ (dashed) and ${\cal R}_{\rm ee}$ (solid)
obtained by the trajectory-tracking task. The sudden variation in the yaw
angle is sound because the angle is bounded between $-\pi$ and $\pi$. This
does not induce any singularity issue since the orientation is implemented
using unit quaternions.  Figure 7: 3D trajectories of ${\cal R}_{\rm grasp}$
(blue) and ${\cal R}_{\rm ee}$ (orange). The initial and terminal points are
denoted with a square and a circle, respectively.

##  IV Conclusion

We propose an original new approach for human-robot handovers formulated using
task-space optimization controller. The core idea is based on a novel
implementation of tasks interdependency, which consists in providing the state
of a task (of estimation nature) as an input to another task so that both meet
at the HOL without explicit time or object configuration specification. Our
experimental results confirmed very promising performances of simple handovers
focusing mainly on the reaching phase. This new approach raised very promising
novel features of the task-space optimization control schemes. Extension in
terms of functionalities and theoretical investigation on how observer-tasks
can be embedded in a unified task-space observer/control framework through
task interdependencies and constraints between task errors opens perspectives
in embedding scheduling in the task-space control formalism. Our on-going and
future research is focusing on this issue together with complete-phases and
more complex handovers considering force cues.

## References

  * [1] V. Ortenzi, A. Cosgun, T. Pardi, W. P. Chan, E. Croft, and D. Kulić, “Object handovers: A review for robotics,” IEEE Transactions on Robotics, vol. 37, no. 6, pp. 1855–1873, 2021.
  * [2] A. Ajoudani, A. M. Zanchettin, S. Ivaldi, A. Albu-Schäffer, K. Kosuge, and O. Khatib, “Progress and prospects of the human–robot collaboration,” Autonomous Robots, vol. 42, pp. 957–975, Jun 2018.
  * [3] A. Billard and D. Kragic, “Trends and challenges in robot manipulation,” Science, vol. 364, no. 6446, p. eaat8414, 2019.
  * [4] K. Strabala, M. K. Lee, A. Dragan, J. Forlizzi, S. S. Srinivasa, M. Cakmak, and V. Micelli, “Toward seamless human-robot handovers,” J. Hum.-Robot Interact., vol. 2, p. 112–132, feb 2013.
  * [5] S. Shibata, K. Tanaka, and A. Shimizu, “Experimental analysis of handing over,” in Proceedings 4th IEEE International Workshop on Robot and Human Communication, pp. 53–58, 1995.
  * [6] J. Waldhart, M. Gharbi, and R. Alami, “Planning handovers involving humans and robots in constrained environment,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 6473–6478, 2015.
  * [7] J. Mainprice, M. Gharbi, T. Siméon, and R. Alami, “Sharing effort in planning human-robot handover tasks,” in IEEE International Symposium on Robot and Human Interactive Communication, pp. 764–770, 2012.
  * [8] M. Huber, M. Rickert, A. Knoll, T. Brandt, and S. Glasauer, “Human-robot interaction in handing-over tasks,” in IEEE International Symposium on Robot and Human Interactive Communication, pp. 107–112, 2008.
  * [9] C. Shi, M. Shiomi, C. Smith, T. Kanda, and H. Ishiguro, “A model of distributional handing interaction for a mobile robot,” in Robotics: Science and Systems, (Berlin, Germany), June 2013.
  * [10] A. Koene, S. Endo, A. Remazeilles, M. Prada, and A. M. Wing, “Experimental testing of the coglaboration prototype system for fluent human-robot object handover interactions,” in IEEE International Symposium on Robot and Human Interactive Communication, pp. 249–254, 2014.
  * [11] A. Remazeilles, M. Prada, A. Garzo, I. Rasines, M. Controzzi, C. Cipriani, I. Strazzulla, C. Peccia, J. Canseco, D. Cabaneros, et al., “Towards human-robot object exchange lessons learned,” in Workshop on Human-Robot handover in RSS, 2015.
  * [12] M. Bdiwi, J. Suchý, and A. Winkler, “Handing-over model-free objects to human hand with the help of vision/force robot control,” in International Multi-Conferences on Systems, Signals & Devices, pp. 1–6, 2013.
  * [13] K. Nagata, Y. Oosaki, M. Kakikura, and H. Tsukune, “Delivery by hand between human and robot based on fingertip force-torque information,” in IEEE/RSJ International Conference on Intelligent Robots and Systems., vol. 2, pp. 750–757 vol.2, 1998.
  * [14] W. P. Chan, C. A. Parker, H. M. V. der Loos, and E. A. Croft, “A human-inspired object handover controller,” International Journal of Robotics Research, vol. 32, no. 8, pp. 971–983, 2013.
  * [15] J. R. Medina, F. Duvallet, M. Karnam, and A. Billard, “A human-inspired controller for fluid human-robot handovers,” in IEEE-RAS International Conference on Humanoid Robots, pp. 324–331, 2016.
  * [16] G. Solak and L. Jamone, “Learning by demonstration and robust control of dexterous in-hand robotic manipulation skills,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 8246–8251, 2019\.
  * [17] M. Costanzo, G. De Maria, and C. Natale, “Handover control for human-robot and robot-robot collaboration,” Frontiers in Robotics and AI, vol. 8, 2021\.
  * [18] M. Prada, A. Remazeilles, A. Koene, and S. Endo, “Implementation and experimental validation of dynamic movement primitives for object handover,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2146–2153, 2014.
  * [19] N. Vahrenkamp, H. Arnst, M. Wächter, D. Schiebener, P. Sotiropoulos, M. Kowalik, and T. Asfour, “Workspace analysis for planning human-robot interaction tasks,” in IEEE-RAS International Conference on Humanoid Robots, pp. 1298–1303, 2016.
  * [20] G. J. Maeda, G. Neumann, M. Ewerton, R. Lioutikov, O. Kroemer, and J. Peters, “Probabilistic movement primitives for coordination of multiple human–robot collaborative tasks,” Autonomous Robots, vol. 41, pp. 593–612, Mar 2017\.
  * [21] D. Widmann and Y. Karayiannidis, “Human motion prediction in human-robot handovers based on dynamic movement primitives,” in European Control Conference, pp. 2781–2787, 2018.
  * [22] H. Nemlekar, D. Dutia, and Z. Li, “Object transfer point estimation for fluent human-robot handovers,” in International Conference on Robotics and Automation, pp. 2627–2633, 2019.
  * [23] A. Paolillo, K. Chappellet, A. Bolotnikova, and A. Kheddar, “Interlinked visual tracking and robotic manipulation of articulated objects,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 2746–2753, 2018.
  * [24] K. Bouyarmane and A. Kheddar, “On weight-prioritized multitask control of humanoid robots,” IEEE Transactions on Automatic Control, vol. 63, pp. 1632–1647, June 2018.
  * [25] K. Bouyarmane, K. Chappellet, J. Vaillant, and A. Kheddar, “Quadratic programming for multirobot and task-space force control,” IEEE Transactions on Robotics, vol. 35, pp. 64–77, Feb 2019.
  * [26] A. Kheddar, S. Caron, P. Gergondet, A. Comport, A. Tanguy, C. Ott, B. Henze, G. Mesesan, J. Englsberger, M. A. Roa, P.-B. Wieber, F. Chaumette, F. Spindler, G. Oriolo, L. Lanari, A. Escande, K. Chappellet, F. Kanehiro, and P. Rabaté, “Humanoid robots in aircraft manufacturing: The airbus use cases,” IEEE Robotics Automation Magazine, vol. 26, no. 4, pp. 30–45, 2019\.
  * [27] A. Bolotnikova, P. Gergondet, A. Tanguy, S. Courtois, and A. Kheddar, “Task-space control interface for softbank humanoid robots and its human-robot interaction applications,” in IEEE/SICE International Symposium on System Integration, (Iwaki, Japan), pp. 560–565, 11-14 January 2021\.
  * [28] M. Rana and V. Mittal, “Wearable sensors for real-time kinematics analysis in sports: A review,” IEEE Sensors Journal, vol. 21, no. 2, pp. 1187–1207, 2021.
  * [29] M. Jongeneel, A. Bernardino, N. Van De Wouw, and A. Saccon, “Model-Based Visual Object Tracking with Impact Collision Models,” in International Conference on Intelligent Robots and Systems, (Prague, Czech Republic), Sept. 2021.
  * [30] M. K. X. J. Pan, E. A. Croft, and G. Niemeyer, “Exploration of geometry and forces occurring within human-to-robot handovers,” in IEEE Haptics Symposium, pp. 327–333, 2018.
  * [31] T.-H. Pham, N. Kyriazis, A. A. Argyros, and A. Kheddar, “Hand-object contact force estimation from markerless visual tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 12, pp. 2883–2896, 2018\.
  * [32] L. Sciavicco, B. Siciliano, and B. Sciavicco, Modelling and Control of Robot Manipulators.  Berlin, Heidelberg: Springer-Verlag, 2nd ed., 2000.
  * [33] A. Bolotnikova, S. Courtois, and A. Kheddar, “Autonomous initiation of human physical assistance by a humanoid,” in IEEE International Conference on Robot and Human Interactive Communication, pp. 857–862, 2020.
  * [34] H. K. Khalil, Nonlinear systems; 3rd ed. Upper Saddle River, NJ: Prentice-Hall, 2002.
  * [35] M. Djeha, A. Tanguy, and A. Kheddar, “Adaptive-gains enforcing constraints in closed-loop qp control,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 6504–6511, 2020.