11institutetext:  State Key Laboratory of Networking and Switching Technology
22institutetext: Beijing Key Laboratory of Intelligent Telecommunications
Software and Multimedia 33institutetext: Beijing University of Posts and
Telecommunications, China

# Decomposed Vector-Quantized Variational Autoencoder for Human Grasp
Generation

Zhe Zhao 112233 0009-0001-2722-5563 Mengshi Qi  Corresponding author:
<EMAIL_ADDRESS>0000-0002-6955-6635 Huadong Ma 112233
0000-0002-7199-5047

###### Abstract

Generating realistic human grasps is a crucial yet challenging task for
applications involving object manipulation in computer graphics and robotics.
Existing methods often struggle with generating fine-grained realistic human
grasps that ensure all fingers effectively interact with objects, as they
focus on encoding hand with the whole representation and then estimating both
hand posture and position in a single step. In this paper, we propose a novel
Decomposed Vector-Quantized Variational Autoencoder (DVQ-VAE) to address this
limitation by decomposing hand into several distinct parts and encoding them
separately. This part-aware decomposed architecture facilitates more precise
management of the interaction between each component of hand and object,
enhancing the overall reality of generated human grasps. Furthermore, we
design a newly dual-stage decoding strategy, by first determining the type of
grasping under skeletal physical constraints, and then identifying the
location of the grasp, which can greatly improve the verisimilitude as well as
adaptability of the model to unseen hand-object interaction. In experiments,
our model achieved about $14.1\%$ relative improvement in the quality index
compared to the state-of-the-art methods in four widely-adopted benchmarks.
Our source code is available at https://github.com/florasion/D-VQVAE.

###### Keywords:

Grasp Generation Decomposed Architecture Variational Autoencoder

##  1 Introduction

Generating how a human would grasp different given objects [12, 14, 13, 40]
has already become crucial in many domains, including robotics, human-computer
interaction, and augmented reality applications. Although 3D hand pose
estimation [7, 41, 1, 4] and hand-object 3D reconstruction [11, 5, 34] have
made significant strides in recent years, grasp generation extends beyond them
and becomes imperative to have a finer-grained comprehension of the individual
components of the highly articulated human hand.

Figure 1: Illustration of the proposed grasp generation model. Initially, we
utilize Decomposed VQ-VAE (DVQ-VAE) to learn the prior distributions of the
object and each hand component (_i.e_., five fingers and the palm) during
training. Specifically, we divide the decoding process into two stages hand
posture and position generation. During inference, we perform autoregression
using the object as a guide to obtain the realistically generated human grasp.

To generate high-quality grasps, the main challenge lies in accurately
modeling the interaction between the hand and objects. In recent years,
numerous efforts [12, 14, 13, 40] have been adopted into this field, which is
primarily focused on predicting the contact map to ensure the hand comes into
full contact with the object. However, such methods often lead to generated
unreasonable grasps that resemble touches rather than grasps [16]. In
addition, existing methods predominantly employ Conditional Variational
Autoencoders (CVAE) [12, 32, 13, 14, 16, 40], while some have incorporated
Generative Adversarial Networks (GANs) [3, 8]. But they utilize a continuous
latent space to model all kinds of different grasps, which does not reflect
the discrete and categorical nature of real-life human hands, resulting in
poor diversity and fallacious grasps. In contrast, Vector Quantized
Variational Autoencoder (VQ-VAE) [35] can lead to a more controllable
generation of grasps by directly utilizing discrete latent codes instead of
randomly sampling from a Gaussian distribution. Therefore, we opt for VQ-VAE
[35] as our core network and extend it in a new decomposed manner to take full
advantage of each hand component and derive more diversity from the
autoregressive inference.

Another challenge is that existing methods cannot fit well into unseen or out-
of-domain objects. Several research proposes test-time adaptation [12, 13],
which is an optimization-based refinement during inference by directly
optimizing MANO parameters [30]. However, such approaches will lead to a
significant time cost. To mitigate the time overhead during testing, we can
break down the complex process of grasping into two distinct stages, _i.e_.,
predicting the hand pose by considering the possible grasp type and then
finding the suitable position for the manipulated object.

In this paper, we present a novel Decomposed VQ-VAE (DVQ-VAE) to generate
human grasps that exhibit both diversity and authenticity when given an
object. Our innovation lies in our model can partition the hand into multiple
components for encoding them into several discrete latent spaces, and then
executing grasp generation by the newly designed dual-stage decoding strategy.
Specifically, we propose a part-aware decomposed architecture to decouple the
hand into six distinct components including five fingers and the palm, each of
which is encoded into its respective codebook. Furthermore, grasp generation
occurs in two sequential steps by initially generating the posture of the hand
followed by the determination of the grasp location.

Our main contributions can be summarized as follows:

(1) We propose a novel DVQ-VAE for human grasp generation, which is based on a
part-aware decomposed architecture to encode multi-part of the hand and allows
for progressively generating the complete hands. To the best of our knowledge,
we are the first to propose such VQ-VAE [35] based framework for this specific
task.

(2) We introduce a new dual-stage decoding strategy to improve the quality of
generated grasps for unseen objects, by gradually determining the grasp type
under skeletal physical constraints and its position.

(3) We benchmark our proposed method on the four popular datasets, with $18\%$
relative improvements over the state-of-the-art approaches in the penetration
volume. More importantly, we achieve nearly $14.1\%$ relative improvement in
the quality index metric.

##  2 Related Work

Grasp Generation. With the development of virtual reality and robotics
technology, significant efforts [3, 12, 16, 13, 17, 18, 25, 26] have been
devoted to generating the physically plausible and diverse grasps, most of
which are based on the generative models, including GAN [8, 3, 37, 28] and VAE
[15, 12, 16, 13, 40]. A representative work is Jiang  _et al._ [12] introduced
a CVAE [31] to generate grasps, taking objects into account as conditions.
This approach incorporated test-time adaptation by using ContactNet [12].
Furthermore, VQ-VAE [35] has demonstrated its superiority in multiple areas,
such as image generation and 3D synthesis [20, 23]. However, it is noteworthy
that, to the best of our knowledge, there exists no prior research that has
applied VQ-VAE [35] to the field of grasp generation. Drawing inspiration from
the work of Pi  _et al._ [23], who segmented the human body into five parts
and encoded them into multiple codebooks for generating human motions, we
extended this concept to encode the hand into multiple components. Through the
utilization of VQ-VAE [35], we encoded the data as discrete variables, thereby
enabling us to acquire a prior distribution of hand-object interactions and
then exploit it in the grasp prediction.

Hand-Object Interaction. Modeling hand-object interaction [30, 11, 2, 9, 33,
38, 39, 27] is a crucial task, but the considerable degrees of freedom in hand
pose present significant challenges. A pioneer work is Romero  _et al._ [30]
proposed the parameterized hand model MANO to address the difficulties in
reconstructing hand pose and shape. Different from modeling hand-object
interaction assumes that both the hand and the object are visible in the
input, we aim to predict a realistic grasp for the observed objects without
seeing the hand.

##  3 Proposed Approach

Figure 2: Overall architecture of the proposed DVQ-VAE model which is based on
the encoder-decoder paradigm. During training, the model takes both hand
vertices and object point clouds as inputs and maps them into discrete latent
spaces consisting of seven codebooks (_i.e_., one for object and six for hand)
based on different hand components to generate hands. While, at the inference
phase, we only use object point clouds as input to generate hands capable of
grasping the given object.

In our work, our goal is to generate a stable and physically plausible hand
mesh for grasping by giving an object point cloud as input. To achieve this,
we design a novel Decomposed VQ-VAE and divide the decoding process into two
stages to obtain the final grasps, as illustrated in Fig. 2.

###  3.1 Overview

Training. As shown in the top part of Fig. 2, given the sampled object points
$P^{o}\in\mathbb{R}^{N_{o}*3}$ and hand vertices $P^{h}\in\mathbb{R}^{778*3}$
as input, we utilize two encoders to extract object features, denoted as
$z_{t}$ and $z_{p}$. Simultaneously, we partition the hand vertices into
$N(N=6)$ segments, comprising five distinct finger components (_i.e_., thumb,
index finger, middle finger, ring finger, and little finger) and the palm,
resulting in six distinct segments that are individually encoded to obtain
hand features $z_{f}=\\{z_{1},...,z_{N}\\}$. These hand features, along with
$z_{t}$, are projected into an discrete latent embedding space where $N+1$
separate codebooks $Z=\\{Z_{o},Z_{1},...,Z_{N}\\}$ are learned. During
decoding, We use MANO [30] to generate hand mesh, and MANO [30] has several
sets of parameters, _i.e_., $M_{\alpha}\in\mathbb{R}^{10}$ handles person-
specific hand shape, $M_{\beta}\in\mathbb{R}^{45}$ manages joint rotations,
$M_{\gamma}\in\mathbb{R}^{3}$ controls hand translation, and
$M_{\delta}\in\mathbb{R}^{3}$ deals with hand rotation. Among these,
$M_{\alpha}$ and $M_{\beta}$ are related to hand posture, and we combine them
denoted as posture $\hat{M}_{posture}$. While $M_{\gamma}$ and $M_{\delta}$
are related to hand position, and we combine them referred to position
$\hat{M}_{position}$. The returned closest matching vectors
$\hat{z}_{f}=\\{\hat{z}_{1},...,\hat{z}_{N}\\}$ of the hand features from $Z$
are then concatenated with $z_{t}$ and fed into the Posture Decoder to decode
$\hat{M}_{posture}$. Then, we concatenate the encoded result of
$\hat{M}_{posture}$, _i.e_., $z_{h}$, with $z_{p}$ and directed into the
Position Decoder to obtain $\hat{M}_{position}$. These two sets of MANO [30]
parameters are then processed through the MANO [30] layer to derive the final
hand mesh.

Inference. As shown in the bottom part of Fig. 2, our model utilizes the
sampled object points $P^{o}\in\mathbb{R}^{N_{o}*3}$ as input, and achieves
object features, $z_{t}$ and $z_{p}$ through the two object encoders. Then,
$z_{t}$ is matched with the nearest vector in the object codebook $Z_{o}$ to
obtain an index $l_{o}$. Subsequently, we adopt an autoregressive model
conditioned on $l_{o}$ to generate a sequence of hand codebook indices
$l_{h}=\\{l_{1},...,l_{N}\\}$. These hand codebook indices are then matched in
the learned hand codebooks to obtain the corresponding hand features
$\hat{z}_{f}=\\{\hat{z}_{1},...,\hat{z}_{N}\\}$. Finally, these hand features,
along with $z_{t}$ and $z_{p}$, are fed into the grasp generator to produce
the desired hand mesh.

In our model, PointNet [24] serves as the point cloud encoder for both objects
and hands, and all decoders are MLPs. For the inference phase, we employ
PixelCNN [22] as the self-regressive model.

###  3.2 Object Encoder

Different from existing methods [12, 14, 13, 16, 40, 32], we propose two
object encoders: a type encoder and a pose encoder. The idea behind the type
encoder is that all grasp types are uncountable and each of them is relevant
to the shape of the given object, we can extract the $z_{t}$ to help learn an
object codebook during the latent space embedding. And the codebook records
these types and even clusters them into several common grasp categories so
that our model can know how to grasp any input object. While, the pose encoder
focuses on extracting features that aid in decoding grasp positions, which
will determine where the hand should contact with the object.

###  3.3 Part-Aware Decomposed Architecture

Here we describe the proposed part-aware decomposed architecture of DVQ-VAE.
Traditional VQ-VAEs [35] are typically used for image generation and employ a
single codebook. This stems from the same codebook indices can be located in
various positions of the image, while during autoregression the same codebook
also can be used to infer the pixel at any position. However, in the case of
human hands, the positions of different fingers are fixed. Therefore, we
propose to extend VQ-VAE [35] to a part-aware decomposed architecture, _i.e_.,
encoding the object and the N parts of the hand as $z_{t}$ and $z_{f}$,
respectively, and search for the vectors with the minimum Euclidean distance
in their respective discrete codebooks during the latent embedding. To prevent
the loss of object features, we use the discovered $\hat{z}_{f}$ in
combination with the original $z_{t}$ as input for the grasp generator. In
this scenario, the object’s latent variable serves solely as a condition
during autoregression. Thus, the training of the object codebook is conducted
through unsupervised learning. In this context, to encourage $z_{t}$ and
$z_{f}$ to closely approximate the matched vectors in the codebooks, the loss
function $\mathcal{L}_{E}$ for codebook training can be defined as follows:

$\mathcal{L}_{h}=\sum_{i=1}^{N}\left\|sg(\hat{z}_{i})-z_{i}\right\|^{2}_{2}+\beta\sum_{i=1}^{N}\left\|sg(z_{i})-\hat{z}_{i}\right\|^{2}_{2},$
(1)
$\mathcal{L}_{o}=\left\|sg(\hat{z}_{t})-z_{t}\right\|^{2}_{2}+\beta\left\|sg(z_{t})-\hat{z}_{t}\right\|^{2}_{2},$
(2) $\mathcal{L}_{E}=\lambda_{e}\cdot(\mathcal{L}_{h}+\mathcal{L}_{o}),$ (3)

where

$\hat{z}_{i}=e_{k},~{}\text{where}~{}k=argmin_{j}\left\|z_{i}-e_{j}\right\|_{2},$
(4)
$\hat{z}_{t}=e_{m},~{}\text{where}~{}m=argmin_{j}\left\|z_{t}-e_{j}\right\|_{2},$
(5)

where the operator $sg(\cdot)$ signifies cessation of the gradient flow,
impeding the propagation of gradients into its associated parameter, $\beta$
is the hyperparameter set to 0.25, and $\lambda_{e}$ is the hyper-parameter,
$e$ represents the embedding in the codebook.

During inference, we follow VQ-VAE [35] to employ PixelCNN [22] as our
autoregressive model, and we use the object’s codebook indices as both the
condition and the initial sequence for PixelCNN [22] to predict a sequence of
codebook indices for the hand components.

###  3.4 Dual-Stage Decoding Strategy

Previous methods [12, 16] improved grasp quality by generating 61 MANO [30]
parameters in one step and optimizing these parameters over multiple steps,
which resulted in significant time consumption. In order to overcome the
weaknesses of single-step decoding, we design a dual-stage decoding strategy
by generating the grasp posture and grasp position in sequence. In the spirit
of decoupling, we divide the MANO parameters into two parts: position and
posture, and we first generate the more numerous grasp posture parameters
$\hat{M}_{posture}$ and then the fewer position parameters
$\hat{M}_{position}$ by combining hand features with the object’s
characteristics.

1) Firstly, to generate a reasonable grasp during decoding, we utilize the L2
distance of the ground truth $M_{posture}$ and the predictions:

$\mathcal{L}_{posture}=\left\|M_{posture}-\hat{M}_{posture}\right\|_{2}.$ (6)

Furthermore, we introduce physical constraints based on hand-skeletal
dynamics. We extract skeletal key points $J$ from the reconstructed hand and
calculate angles between adjacent key points. From these angles, we generate
gating information to correct grasps that do not conform to hand-skeletal
dynamics:

$\theta_{i}=\cos^{-1}\left(\frac{\overrightarrow{J_{i}J_{i-1}}\cdot\overrightarrow{J_{i}J_{i+1}}}{\|\overrightarrow{J_{i}J_{i-1}}\|\|\overrightarrow{J_{i}J_{i+1}}\|}\right),$
(7) $\hat{M}_{posture}=\hat{M}_{posture}+G(\theta)\odot T(\hat{M}_{posture}),$
(8)

where $\theta$ represents the joint angles, $\overrightarrow{J_{i}J_{i+1}}$
represents the vector from key point $J_{i}$ to $J_{i+1}$, $G(\cdot)$ denotes
the network used to generate gating information, and $T(\cdot)$ stands for the
transformer layer used to produce correction values.

2) Secondly, for training the position decoder, we stop the gradient
propagation at $z_{h}$ and compute the L2 distance of $M_{position}$ and hand
vertices formulated as:

$\hat{M}_{position}=Dec[sg(z_{h}),z_{p}],$ (9)
$\mathcal{L}_{position}=\left\|M_{position}-\hat{M}_{position}\right\|_{2},$
(10)
$\mathcal{L}_{v}=\left\|P^{h}-M(\hat{M}_{position},\hat{M}_{posture})\right\|_{2},$
(11)

where the operator $Dec[\cdot]$ represents decoding through the position
decoder, and the operator $M(\cdot)$ represents passing through the MANO layer
[30]. Then the reconstruction loss $L_{R}$ is obtained by combining
$\mathcal{L}_{posture}$, $\mathcal{L}_{position}$, and $\mathcal{L}_{v}$:

$\mathcal{L}_{R}=\lambda_{h}\cdot(\mathcal{L}_{posture}+\mathcal{L}_{position})+\lambda_{v}\cdot\mathcal{L}_{v},$
(12)

where $\lambda$ are hyper-parameters. To prevent the model from inadvertently
learning hand position information before the position decoder, we center the
hand vertices by subtracting their mean coordinates during training.

###  3.5 Optimization

Finally, our total objective loss consists of three parts, _i.e_.,
$\mathcal{L}_{E}$ for the discrete latent embeddings in Sec. 3.3,
$\mathcal{L}_{R}$ for constraining the morphology of generated hand in Sec.
3.4, and $\mathcal{L}_{contact}$ for constraining contact between the hand and
the manipulated object that we will describe below.

Following [12], we use object-centric contact loss $\mathcal{L}_{c}$ and
contact map consistency loss $\mathcal{L}_{m}$ to enhance the contact between
the hand and the object formulated as:

$\mathcal{L}_{c}=\sum_{p_{m}\in P_{m}}\min_{p_{c}\in
P_{c}}\left|p_{m}-p_{c}\right|,$ (13)
$\mathcal{L}_{m}=\frac{\left|P_{m}\right|\cap\left|\hat{P}_{m}\right|}{\left|P_{m}\right|},$
(14)

where the operator $\left|\cdot\right|$ indicates the number of points in the
point set. $P_{m}$ and $\hat{P}_{m}$ represent the point set of the ground
truth and the predicted grasp contact map, respectively. $P_{c}$ denotes the
points on the hand that could potentially contact the object. $p_{c}$ and
$p_{m}$ refer to each point in the point set $P_{c}$ and $P_{m}$,
respectively.

We also employ penetration loss to enhance the physical reality of the
generated grasp, formulated as:

$\mathcal{L}_{p}={\textstyle\sum_{p\in
P_{in}}\left\|p-P^{o}_{i}\right\|^{2}_{2}},$ (15)

where

$P^{o}_{i}=\text{argmin}_{p_{i}\in{P^{o}}}\left\|p-p_{i}\right\|,$ (16)

where $P_{in}$ is the subset of hand points that enter into the object. $p$
and $p_{i}$ denote as each point in the point set $P_{in}$ and $P^{o}$,
respectively. Hence, we can obtain the contact loss as follows:

$\mathcal{L}_{contact}=\lambda_{m}\cdot\mathcal{L}_{m}+\lambda_{c}\cdot\mathcal{L}_{c}+\lambda_{p}\cdot\mathcal{L}_{p}.$
(17)

Where $\lambda$ are hyper-parameters.

Finally, we combine $\mathcal{L}_{contact}$ with $\mathcal{L}_{E}$ and
$\mathcal{L}_{R}$ to form our overall loss function:

$\mathcal{L}=\mathcal{L}_{R}+\mathcal{L}_{E}+\mathcal{L}_{contact}.$ (18)

##  4 Experiments

###  4.1 Datasets

Obman Dataset [11] constitutes a comprehensive compilation of manipulation
interaction synthesized data, produced by GraspIt [19]. It encompasses a vast
collection of 150,000 grasps across 2,772 distinct objects. Following [16,
12], 141,550 of these grasps are designated for training purposes, while the
remaining 6,285 are allocated for testing.

HO-3D [10], FPHA [6] and GRAB [32] dataset. Following [12], due to the Obman
dataset [11] containing the greatest variety of objects, we chose to train on
the Obman dataset [11] but test on these three datasets. The test data can be
regarded as out-of-domain objects used to evaluate the adaptability.
Specifically, HO3D [10] contains 10 objects, FPHA [6] contains 4 objects, and
GRAB [32] contains 51 objects from 10 subjects.

###  4.2 Metrics

For a fair comparison, following [16, 12, 14, 13, 11, 33], we evaluate the
results in terms of the below metrics, of which the _quality index_ is our
proposed new metric.

1) Contact Ratio ($\%$). It calculates the ratio of grasps that can contact
the given object in all generated grasps.

2) Hand-Object Interpenetration Volume ($cm^{3}$). It is the volume shared by
the 3D voxelized models of the hand and object. Following [16, 13], We
voxelize the mesh of both hand and object with size 0.1 $cm^{3}$.

3) Grasp Simulation Displacement (Grasp Disp) ($cm$). It quantifies the
stability of the generated grasp when the simulated gravity is added, and we
report the average displacement of the object’s center of mass.

4) Entropy and Cluster Size. Following [13, 16], we divide the generated
grasps into 20 clusters using K-means and then measured the entropy and
averaged cluster size to assess the diversity of generated grasps. Higher
entropy values and larger cluster sizes indicate more diversity.

5) Time ($s$). It evaluates the inference speed of the model in generating one
batch of grasps.

6) Quality Index. Note that we observe the visualized results with only low
penetration or low physical simulation displacement does not necessarily
indicate it is a high-quality grasp. For example, insufficient contact can
reduce penetration but also decrease the stability of the grasp, while the
severe penetrating of the hand into the object can reduce displacement in the
physical simulator. Inspired by [36], to more scientifically quantify the
quality of grasping, we adopt a utility function:

$Q=a\cdot x+(1-a)\cdot y,$ (19)

where $a$ represents the weight, used to balance the gap between grasp
displacement and penetration,$x$ represents penetration volume, $y$ represents
grasp disp. According to [36], we measure the displacement and penetration for
each grasp in the obman and grab datasets. And we calculate $a=0.301$ using
the same method as described in [36].

###  4.3 Compared Methods

We compare our proposed DVQ-VAE against the following state-of-the-art
methods: 1) GraspTTA [12]: Trained on the Obman dataset [11] and utilizes CVAE
[31] to generate grasps conditioned on the object, and further refines the
generated grasps using ContactNet with test-time adaptation (TTA). 2)
GraspCVAE [12]: A variant of GraspTTA without TTA. 3) ContactGen [16]: Trained
on the Grab dataset [32] and utilizes CVAE [31] to generate contact maps of
the object, and then optimizes the hand parameters based on them. 4) Grasping
Field [14]: Trained on the Obman dataset [11] and utilizes VAE [15] to
generate grasps based on the signed distance fields of the object and the
hand.

Dataset | Method |  | Contact
---
Ratio
$(\%)$↑
| Penetration
---
Volume
$(cm^{3})$↓
| Grasp
---
Disp
$(cm)$ ↓
| Time
---
$(s)$ ↓
| Entr-
---
opy ↑
| Cluster
---
Size ↑
| Quality
---
Index↓
HO-3D [10] | GraspCVAE [12] | 99.60 | 7.23 | 2.78 | 0.0040 | 2.96 | 0.81 | 4.12
GraspTTA [12] | 100 | 9.00 | 2.65 | 19.67 | 2.87 | 0.80 | 4.56
ContactGen [16] | 90.10 | 6.53 | 3.72 | 119.4 | 2.94 | 4.79 | 4.57
GraspingField [14] | 89.60 | 20.05 | 4.14 | 57.49 | 2.91 | 3.31 | 8.93
Our DVQ-VAE | 99.50 | 5.36 | 2.75 | 0.14 | 2.80 | 3.84 | 3.54
FPHA [6] | GraspCVAE [12] | 98.98 | 7.46 | 2.97 | 0.0038 | 2.91 | 0.81 | 4.32
GraspTTA [12] | 100 | 8.26 | 2.75 | 19.29 | 2.93 | 0.76 | 4.41
ContactGen [16] | 94.00 | 10.43 | 3.64 | 237.38 | 2.84 | 2.88 | 5.68
GraspingField [14] | 97.00 | 29.78 | 5.47 | 58.57 | 2.85 | 2.36 | 12.79
Our DVQ-VAE | 97.96 | 4.58 | 3.35 | 0.14 | 2.86 | 3.53 | 3.72
GRAB [32] | GraspCVAE [12] | 97.10 | 3.54 | 2.02 | 0.0041 | 2.93 | 0.81 | 2.48
GraspTTA [12] | 100 | 5.05 | 1.74 | 20.42 | 2.88 | 0.93 | 2.74
GraspingField [14] | 74.80 | 10.56 | 3.80 | 62.46 | 2.91 | 3.53 | 5.83
Our DVQ-VAE | 98.60 | 3.18 | 2.13 | 0.15 | 2.83 | 3.64 | 2.45
Obman [11] | GraspCVAE [12] | 99.20 | 4.32 | 1.81 | 0.0040 | 2.95 | 1.50 | 2.57
GraspTTA [12] | 100 | 5.85 | 2.06 | 19.70 | 2.96 | 1.50 | 3.20
GraspingField [14] | 74.62 | 10.53 | 3.81 | 60.06 | 2.81 | 2.33 | 5.83
Our DVQ-VAE | 99.82 | 3.93 | 2.70 | 0.14 | 2.90 | 3.98 | 3.07
Table 1: Performance comparison of our proposed DVQ-VAE and the state-of-the-
art methods in terms of grasp generation on the widely-adopted benchmarks,
_i.e_., HO-3D [10], FPHA [6], GRAB [32], and Obman [11].

###  4.4 Implementation Details

We train our model on Obman dataset [11], and sample $N_{o}=3000$ points from
the object mesh as input. We employ Adam optimizer with an initial learning
rate of 1e-4 for 200 epochs, halving the learning rate at epochs 60, 120, 160,
and 180, and set
$\lambda_{e}=10,\lambda_{m}=-50,\lambda_{c}=1500,\lambda_{p}=5,\lambda_{h}=0.1,\lambda_{v}=10$.
For training PixelCNN [22], we use Adam optimizer with a learning rate of 3e-4
for 100 epochs. We implement our model based on Pytorch on a single NVIDIA RTX
3090 GPU, the training time is 1000 minutes.

Method | Score
---|---
Ours | 3.36
ContactGen [16] | 3.25
GraspTTA [12] | 3.23

Table 2: Average scores in human evaluation.

|  | Contact
---
Ratio $(\%)$↑
| Penetration
---
Volume $(cm^{3})$↓
| Grasp
---
Disp $(cm)$ ↓
$50\%$Masked | 99.60 | 5.66 | 2.80
$90\%$Masked | 99.70 | 7.37 | 2.67

Table 3: Results after masking part of the object point cloud

###  4.5 Results

Quantitative Results. In experiments, we report the performance in Tab. 1 of
our proposed model and the other state-of-the-art methods which are all only
trained on Obman [11] and test on HO3D [10], Grab [32], and FPHA [6]. Note
that the objects in HO3D, Grab, and FPHA are not present in the Obman training
set and are never seen during the training process. As shown in the table, we
can find our proposed DVQ-VAE can achieve the lowest penetration and grasp
displacement approaches state-of-the-art benchmarks, demonstrating our
proposed method allows for fine-grained controlling over touch interactions.
Compared to the model [12] w.r.t quality index, our model shows a remarkable
$13.3\%$ relative improvement on HO-3D [10] dataset, $13.7\%$ on FPHA [6]
dataset, and $1.2\%$ on GRAB [32] dataset. And competitive results have also
been obtained in terms of contact ratio and grasp disp used to assess grasping
stability. Although GraspTTA [12] exhibits low grasp disp, its limited grasp
diversity results in generating nearly identical grasp for a given object as
mentioned in [16]. Moreover, our model achieved a relative improvement of
$22.6\%$ in the cluster size metric, suggesting the part-aware decomposed
architecture we extended into VQ-VAE [35] allows our model to approach or even
surpass the performance of leading methods in terms of grasp diversity.
Meanwhile, we measured the ratio of high-quality grasps (both penetration and
displacement are below certain thresholds) generated by each model, as shown
in Fig. 3 we report the ratio of each model on objects in the HO-3D dataset
[10] as the penetration threshold varies from $0~{}cm^{3}$ to $10~{}cm^{3}$,
and we can clearly see our model outperform the others most of times. These
findings indicate the superiority and efficiency of our proposed DVQ-VAE.

Human Evaluation. We also conducted subjective experiments by inviting 10
human participants to evaluate the generated grasps of our DVQ-VAE, ContactGen
[16] and GraspTTA [12]. During the testing, given eight objects from the HO-3D
[10] dataset, the models randomly generated 4 grasps for each object.
Participants rated the grasps with a range from 1 to 5 in terms of
_penetration depth_ , _grasp stability_ , and _naturalness_ , and the higher
scores indicate the grasp is closer to a real human grasp. We report the
results in Fig. 5 and Tab. 2, it is evident that our proposed model produces
grasps with the highest 5-point ratio and the lowest 1-point ratios, as well
as achieving the highest average human evaluation score.

Figure 3: Performance comparison of our method and other models in high-
quality ratio w.r.t the penetration threshold for different models on the
HO-3D dataset.

Robustness. We also tested the performance of our model when only partial
point clouds of objects are provided, as shown in Tab. 3. Our model is capable
of generating plausible results even with only partial point clouds, because
of the powerful prior knowledge embedded in our codebook and autoregressive
model. In contrast, ContactGen [16] and GraspTTA [12] can generate grasps only
from a fixed number of point clouds.

Figure 4: The number of indices used in each codebook for our DVQ-VAE.

Figure 5: The percentage of each score of each model in human evaluation.

Figure 6: Qualitative results by comparing our DVQ-VAE with ContactGen [16]
and GraspTTA [12] on the HO-3D dataset [10].

Inference Time. As shown in Tab. 1, we achieved a reduction of
$99.8\%$/$99.3\%$ time cost compared to ContactGen [16] and GraspTTA [12],
respectively, demonstrates our proposed DVQ-VAE obtains faster grasp
generation speed compared to methods using optimization methods during the
generation period.

Qualitative Results. We visualize the grasps generated for different objects
in various datasets, and each grasp is presented from two different
perspectives. As Fig. 6 illustrated, we can clearly observe our model
generates grasps with fewer penetrations and higher stability than others,
showing strong adaptability when applied in such out-of-domain objects.
Furthermore, we also visualize the various types of grasps generated for the
same object in Fig. 10, and it can be observed that our model can generate
grasps with more diversity and different postures. Additionally, we show
several visualized failure cases during testing in Fig. 9, we can find our
model may generate grasps with insufficient contact when applied to objects
with complex geometric shapes. Enhanced representations of objects through the
Signed Distance Field (SDF) [21] will be promising to mitigate this challenge.

Figure 7: Performance comparison of different parts used in our proposed DVQ-
VAE on the HO-3D [10] dataset. We present the variation trend of evaluation
metrics as the number of parts varied.

Figure 8: Comparison of the generated grasp by the model with or without our
proposed dual-stage decoding strategy, denoted as “after” and “before”,
respectively.

Figure 9: Visualization examples of our model’s failure cases.

Figure 10: Diverse grasps generated by our DVQ-VAE for the same object in GRAB
[32].

###  4.6 Ablation Study

Method |  | Penetration
---
Volume
$(cm^{3})$↓
| Grasp
---
Disp
$(cm)$ ↓
| Entro-
---
py ↑
| Cluster
---
Size↑
| Quality
---
Index ↓
VQ-VAE | 6.67 | 7.21 | 2.82 | 1.69 | 7.05
VQ-VAE2 | 5.18 | 9.87 | 2.96 | 4.29 | 8.46
DVQ-VAE | 10.88 | 4.98 | 2.94 | 4.24 | 6.76
VQ-VAE+Dual-Stage(Two Encoders) | 4.44 | 3.61 | 2.79 | 1.73 | 3.86
DVQ-VAE+Dual-Stage(One Encoder) | 11.20 | 4.57 | 2.95 | 4.31 | 6.57
DVQ-VAE+Dual-Stage(Reverse) | 7.56 | 2.93 | 2.79 | 2.67 | 4.32
DVQ-VAE+Dual-Stage(Two Encoders) | 5.36 | 2.75 | 2.80 | 3.84 | 3.54
Table 4: Performance comparison of DVQ-VAE and its variants in the ablation
study.

Object Encoder We compare the results of encoding the given object with only
one object encoder and with our proposed two object encoders in Tab. 4. We can
find our proposed DVQ-VAE with two encoders outperforms the model with only
one encoder across most of the metrics, which is attributed to the two
encoders that can empower the object representation by decoupling the object’s
feature into type and pose parts. In addition, Fig. 4 illustrates the
distribution of index usage in various codebooks of our DVQ-VAE, it can be
seen that objects are clustered into 21 categories based on the latent grasp
type. From the usage of indexes, it can be observed that the degrees of
freedom of the index finger and the ring finger are higher compared to other
parts of the hand, they have more hand posture prototypes.

Part-aware Decomposed Architecture We ablate the decomposed architecture and
show the performance in Tab. 4, and we can find the proposed architecture
exhibits a relative improvement of $22.4\%$ in the grasp stability compared to
vanilla VQ-VAE [35], along with a relative improvement of $151\%$ in cluster
size. This implies that our part-aware decomposed architecture not only
enhances the diversity of generated grasps but also improves the quality of
the generated grasps. Meanwhile, we compared the results in Fig. 7 by
utilizing DVQ-VAE with different numbers of parts on HO-3D [10]. It can be
observed that dividing the hand into six components yields reasonable results
by achieving a good trade-off considering all metrics. Furthermore, in the
selection of the backbone network, we also tested VQ-VAE 2.0 [29] structure
that incorporates global features. However, as shown in Tab. 4, the results
were not satisfactory. Instead, the autoregressive model PixelCNN [22] used in
our model can represent latent or implicit global hand features for
reconstruction during inference.

Dual-Stage Decoding Strategy. In Tab. 4, we evaluate the effectiveness of our
proposed dual-stage decoding strategy. Compared to models without the
strategy, our full model can increase the contact ratio, and reduce
penetration and displacement, thereby improving the overall grasping quality.
Specifically, taking full use of this strategy results in $45.2\%$ and
$47.6\%$ relative increases in the quality index of our proposed DVQ-VAE and
VQ-VAE [35], respectively. We also tested decoding the position before the
posture, but as indicated by the "Dual-Stage (Reverse)" results in the table,
the outcome is not promising. Additionally, we also found that the position
generation module in our trained DVQ-VAE can optimize the grasps generated by
other models by improving the grasping positions. As shown in Fig. 8, the
optimized grasps exhibit reduced penetration, validating the effectiveness of
our dual-stage decoding strategy.

##  5 Conclusion

In this paper, we present Decomposed VQ-VAE for human grasp generation,
including a part-aware decomposed architecture to account for different
components of the hand with latent codebooks, and the dual-stage decoding
strategy make the hand posture fit into the position in order. We have shown
that each one of these components is important and helps outperform state-of-
the-art methods.

## Acknowledgements

This work is partly supported by the Funds for the NSFC Project under Grant
62202063, Beijing Natural Science Foundation (L243027), the Innovation
Research Group Project of the NSFC under Grant 61921003.

## References

  * [1] Boukhayma, A., Bem, R.d., Torr, P.H.: 3d hand shape and pose from images in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10843–10852 (2019)
  * [2] Brahmbhatt, S., Tang, C., Twigg, C.D., Kemp, C.C., Hays, J.: Contactpose: A dataset of grasps with object contact and hand pose. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16. pp. 361–378. Springer (2020)
  * [3] Corona, E., Pumarola, A., Alenya, G., Moreno-Noguer, F., Rogez, G.: Ganhand: Predicting human grasp affordances in multi-object scenes. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 5031–5041 (2020)
  * [4] Dibra, E., Melchior, S., Balkis, A., Wolf, T., Oztireli, C., Gross, M.: Monocular rgb hand pose inference from unsupervised refinable nets. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp. 1075–1085 (2018)
  * [5] Fan, H., Su, H., Guibas, L.J.: A point set generation network for 3d object reconstruction from a single image. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 605–613 (2017)
  * [6] Garcia-Hernando, G., Yuan, S., Baek, S., Kim, T.K.: First-person hand action benchmark with rgb-d videos and 3d hand pose annotations. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 409–419 (2018)
  * [7] Ge, L., Cai, Y., Weng, J., Yuan, J.: Hand pointnet: 3d hand pose estimation using point sets. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 8417–8426 (2018)
  * [8] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural information processing systems 27 (2014)
  * [9] Grady, P., Tang, C., Twigg, C.D., Vo, M., Brahmbhatt, S., Kemp, C.C.: Contactopt: Optimizing contact to improve grasps. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1471–1481 (2021)
  * [10] Hampali, S., Rad, M., Oberweger, M., Lepetit, V.: Honnotate: A method for 3d annotation of hand and object poses. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3196–3206 (2020)
  * [11] Hasson, Y., Varol, G., Tzionas, D., Kalevatykh, I., Black, M.J., Laptev, I., Schmid, C.: Learning joint reconstruction of hands and manipulated objects. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 11807–11816 (2019)
  * [12] Jiang, H., Liu, S., Wang, J., Wang, X.: Hand-object contact consistency reasoning for human grasps generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 11107–11116 (2021)
  * [13] Karunratanakul, K., Spurr, A., Fan, Z., Hilliges, O., Tang, S.: A skeleton-driven neural occupancy representation for articulated hands. In: 2021 International Conference on 3D Vision (3DV). pp. 11–21. IEEE (2021)
  * [14] Karunratanakul, K., Yang, J., Zhang, Y., Black, M.J., Muandet, K., Tang, S.: Grasping field: Learning implicit representations for human grasps. In: 2020 International Conference on 3D Vision (3DV). pp. 333–344. IEEE (2020)
  * [15] Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
  * [16] Liu, S., Zhou, Y., Yang, J., Gupta, S., Wang, S.: Contactgen: Generative contact modeling for grasp generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 20609–20620 (2023)
  * [17] Lv, C., Qi, M., Li, X., Yang, Z., Ma, H.: Sgformer: Semantic graph transformer for point cloud-based 3d scene graph generation. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 4035–4043 (2024)
  * [18] Lv, C., Zhang, S., Tian, Y., Qi, M., Ma, H.: Disentangled counterfactual learning for physical audiovisual commonsense reasoning. Advances in Neural Information Processing Systems 36 (2024)
  * [19] Miller, A.T., Allen, P.K.: Graspit! a versatile simulator for robotic grasping. IEEE Robotics & Automation Magazine 11(4), 110–122 (2004)
  * [20] Mittal, P., Cheng, Y.C., Singh, M., Tulsiani, S.: Autosdf: Shape priors for 3d completion, reconstruction and generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 306–315 (2022)
  * [21] Oleynikova, H., Millane, A., Taylor, Z., Galceran, E., Nieto, J., Siegwart, R.: Signed distance fields: A natural representation for both mapping and planning. In: RSS 2016 workshop: geometry and beyond-representations, physics, and scene understanding for robotics. University of Michigan (2016)
  * [22] Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A., et al.: Conditional image generation with pixelcnn decoders. Advances in neural information processing systems 29 (2016)
  * [23] Pi, H., Peng, S., Yang, M., Zhou, X., Bao, H.: Hierarchical generation of human-object interactions with diffusion probabilistic models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 15061–15073 (2023)
  * [24] Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 652–660 (2017)
  * [25] Qi, M., Li, W., Yang, Z., Wang, Y., Luo, J.: Attentive relational networks for mapping images to scene graphs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3957–3966 (2019)
  * [26] Qi, M., Qin, J., Li, A., Wang, Y., Luo, J., Van Gool, L.: stagnet: An attentive semantic rnn for group activity recognition. In: Proceedings of the European conference on computer vision (ECCV). pp. 101–117 (2018)
  * [27] Qi, M., Qin, J., Yang, Y., Wang, Y., Luo, J.: Semantics-aware spatial-temporal binaries for cross-modal video retrieval. IEEE Transactions on Image Processing 30, 2989–3004 (2021)
  * [28] Qi, M., Wang, Y., Li, A., Luo, J.: Stc-gan: Spatio-temporally coupled generative adversarial networks for predictive scene parsing. IEEE Transactions on Image Processing 29, 5420–5430 (2020)
  * [29] Razavi, A., Van den Oord, A., Vinyals, O.: Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems 32 (2019)
  * [30] Romero, J., Tzionas, D., Black, M.J.: Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.02610 (2022)
  * [31] Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. Advances in neural information processing systems 28 (2015)
  * [32] Taheri, O., Ghorbani, N., Black, M.J., Tzionas, D.: Grab: A dataset of whole-body human grasping of objects. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. pp. 581–600. Springer (2020)
  * [33] Tzionas, D., Ballan, L., Srikantha, A., Aponte, P., Pollefeys, M., Gall, J.: Capturing hands in action using discriminative salient points and physics simulation. International Journal of Computer Vision 118, 172–193 (2016)
  * [34] Tzionas, D., Gall, J.: 3d object reconstruction from hand-object interactions. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 729–737 (2015)
  * [35] Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. Advances in neural information processing systems 30 (2017)
  * [36] Wang, C., Wang, L.: Adaptive weight learning for multiple outcome optimization with continuous treatment. arXiv preprint arXiv:2402.11092 (2024)
  * [37] Wang, H., Wang, M., Che, Z., Xu, Z., Qiao, X., Qi, M., Feng, F., Tang, J.: Rgb-depth fusion gan for indoor depth completion. In: Proceedings of the ieee/cvf conference on computer vision and pattern recognition. pp. 6209–6218 (2022)
  * [38] Wang, X., Wu, Y., Zhu, L., Yang, Y.: Symbiotic attention with privileged information for egocentric action recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 12249–12256 (2020)
  * [39] Wang, X., Zhu, L., Wang, H., Yang, Y.: Interactive prototype learning for egocentric action recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8168–8177 (2021)
  * [40] Zheng, Y., Shi, Y., Cui, Y., Zhao, Z., Luo, Z., Zhou, W.: Coop: Decoupling and coupling of whole-body grasping pose generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2163–2173 (2023)
  * [41] Zimmermann, C., Brox, T.: Learning to estimate 3d hand pose from single rgb images. In: Proceedings of the IEEE international conference on computer vision. pp. 4903–4911 (2017)


# Role of the isospin diffusion on cluster transfer in 12,14C + 209Bi
reactions

Zepeng Gao1 Yinu Zhang1 Long Zhu1,2, Corresponding author:
<EMAIL_ADDRESS>Zehong Liao1 Yu Yang1 Chenchen Guo1 Jun Su1,2 1Sino-
French Institute of Nuclear Engineering and Technology, Sun Yat-sen
University, Zhuhai 519082, China
2Guangxi Key Laboratory of Nuclear Physics and Nuclear Technology, Guangxi
Normal University, Guilin 541004, China

(August 30, 2024)

###### Abstract

Heavy-ion collisions at near-barrier energies provide a crucial pathway for
investigating nucleon correlations and clustering structures. Recent
experimental results showed that the valence neutrons in light projectiles
obviously enhance the $\alpha$ transfer. This finding is extremely puzzled and
fascinating, because it violates the ground-state $Q$ value systematics
unexpectedly. In this work, the time-dependent Hartree-Fock approach is
utilized to investigate the cluster transfer. By comparing the reactions
12,14C + 209Bi, we discover that above puzzling behavior is because of the
strong correlation between isospin diffusion and clustering. Our calculations
clearly show that the equilibrium of neutron-to-proton ratio strongly inhibits
the clustering. This work opens a prospect for investigating the clustering in
open quantum system.

††preprint: APS/123-QED

Introduction. The phenomenon of cluster within atomic nuclei is widely
recognized. Clustering has been regarded as a transitional phenomenon between
crystalline and quantum-liquid phases of fermionic systems Ebran  _et al._
(2012), which is more prevalent in light nuclei von Oertzen  _et al._ (2006);
Freer  _et al._ (2018). The strong correlation among nucleons is proposed to
play a significant role in maintaining the cluster configuration Carlson and
Schiavilla (1998); Wiringa  _et al._ (2014). Especially in the case of
symmetric even-even nuclei, like 8Be, 12C, and others, these nuclei can be
viewed as being composed of multiple $\alpha$-particles. The second 0+ state
of 12C called Hoyle state, interestingly, which holds great significance in
explaining the abundance of carbon in the universe and understanding the
origins of life, dominants degrees of freedom are those of $\alpha$-particle
clusters rather than nucleons Freer and Fynbo (2014); Otsuka  _et al._ (2022);
Epelbaum  _et al._ (2012). Furthermore, extreme spin or isospin can enhance
the stability of cluster configurations Zhao  _et al._ (2015); Suhara and
Kanada-En’yo (2010). High isospin carbon nuclei with cluster configurations of
linear-chain molecular band, such as 14C Han  _et al._ (2023) and 16C Liu  _et
al._ (2020); Han  _et al._ (2022), have also been observed experimentally.

Presently, there is growing emphasis on studying the influence of various
structural properties of the nucleus during nuclear reactions, including
deformed nuclei, neutron skins, halo nuclei, as well as $\alpha$ clusters.
Additionally, valuable information about the initial structure can be
extracted by analyzing the observables of the reaction’s final state. Examples
for certain cluster nuclei have been shown that the presence of potential
$\alpha$-particles on the surface of Sn isotopes Tanaka  _et al._ (2021), as
well as the existence of potential four-neutron resonance Duer  _et al._
(2022), can be investigated through proton knockout reactions. Furthermore,
the clustering phenomenon within atomic nuclei becomes evident in both nuclear
giant resonances He  _et al._ (2014, 2016) and heavy-ion reaction processes
Piantelli  _et al._ (2023); Frosin  _et al._ (2023); Broniowski and Arriola
(2014); Zhang  _et al._ (2017); Guo  _et al._ (2019); Li  _et al._ (2020);
Dasgupta  _et al._ (2023).

The cluster effects become more pronounced at low-energy and are primarily
evident in transfer reactions Hodgson and Běták (2003). In the case of weakly
bound nuclei such as 6Li, 7Li, and 8Be, the ground-state wave functions often
exhibit a substantial component comprising the maximum number of particles
coupled with the remaining nucleons. Large cross-sections for cluster transfer
reactions is a consequence of internal clustering phenomena and low separation
energies Canto  _et al._ (2009, 2015, 2006). Moreover, in the case of strongly
bound nuclei like 12C and 16O, a certain degree of cluster transfer cross-
sections still remains evident Le Beyec  _et al._ (1972); Hamada  _et al._
(2011); Becchetti  _et al._ (1980); Jin  _et al._ (1980). In Ref. Sahu  _et
al._ (2001); Biswas  _et al._ (2012), it was observed experimentally that the
cross sections of two neutrons correlated cluster transfer are strongly
enhanced in the 18O induced reaction comparing to 16O with 174Yb target as
well as 208Pb target. This behavior is extremely puzzled and fascinating,
because it violates the ground state $Q$ value systematics unexpectedly.
Furthermore, interestingly such enhancement disappear for colliding with the
63Cu target Shorto  _et al._ (2008); Crema  _et al._ (2018). Collisions of the
heavy-ions involve a complicated interplay and exchange of energy between
relative motion and intrinsic states of nuclei, and the equilibrium of the
collective degree of freedom. In multinucleon transfer reactions, the
equilibrium of neutron-to-proton ratio plays an important role on the nucleons
rearrangement, which is related to the mechanism of quantum fluctuation and
thermal dynamics. The key question that arises is: do the equilibration
processes affect the clustering in open quantum systems?

This work is mainly focused on the $\alpha$ and 8Be clustering. The reactions
12,14C+209Bi reactions are studied. The dynamic process of cluster transfer
and the role of the isospin diffusion of fermionic systems on the cluster
transfer probabilities are revealed within the framework of the time-dependent
Hartree-Fock (TDHF) method. The TDHF method is a fully microscopic density
functional approach Schuetrumpf  _et al._ (2018) that relies on effective
nucleon-nucleon interactions, which has found extensive applications in low-
energy heavy-ion reactions and fission dynamics Simenel (2012); Simenel and
Umar (2018); Stevenson and Barton (2019); Sekizawa (2019). Furthermore, the
structural characteristics of atomic nuclei and their dynamics during nuclear
reactions can be uniformly and self-consistently described by only one set of
Skyrme parameters Sun  _et al._ (2023); Simenel  _et al._ (2013). Our research
focuses on cluster transfer in heavy-ion reactions within the near-barrier
energy region, in which nuclear reactions are highly sensitive to the initial
structural information and TDHF has the capability to leverage its advantages.

Theoretical framework. In this work, the calculation of static nuclei and the
dynamic process are described coherently by static HF and TDHF methods, where
the clustering effect during the nuclear reaction can be explained self-
consistently. The Skyrme density function SLy5 Chabanat  _et al._ (1998) has
been utilized in both static and dynamic processes, which has been adopted in
many investigations. The static HF are employed using the damped gradient
iteration method, and the box size was established to be 24 × 24 × 24 fm3 with
a mesh spacing of 1.0 fm, while 50 × 24 × 50 fm3 of box size for dynamical
simulation was further fixed.

A detailed explanation, highlighting its distinction from regular
initialization, will be provided for a static process that incorporates the
cluster effect. The trial wave function is expressed in the form of Gaussian
functions, and two of the 2n-2p (pre-cluster) wave functions are translated
upwards and downwards along the x-axis, respectively. The solution with
3$\alpha$ linear-chain configuration for 12C can thus yield from static HF,
where a local minimum energy exists. The two-dimensional density distribution
are shown in Fig. 1(a), where the 3$\alpha$ configuration is shown clearly.
Moreover, an equilateral triangular configuration can be obtained using
density-constrained HF(DC-HF) Umar  _et al._ (2023), which is shown in Fig.
1(d). We would like to emphasize that in this work we aim to reveal the
effects of N/Z equilibration on the cluster transfer by investigating and
focusing on the distinction between 12C and 14C involved reactions, even
though the linear-chain and triangle are not ground state configurations. The
truth is that several studies Otsuka  _et al._ (2022); Shen  _et al._ (2023)
suggest that the cluster state of 12C, adopting a configuration resembling a
blunt-angled triangle, may exist in the ground state with a certain
probability, allowing the linear-chain and equilateral triangular
configurations to be approximated as such. Also, during the collision the
dissipated kinetic energy could excite the projectiles, then enhance the
probabilities of the molecular states of clusters.

Figure 1:  Two-dimensional density distribution for the linear-chain structure
of 12CLin.(a), 14CLin.(b) and the difference between them (c) in the static HF
calculations and the triangular structure of 12CTri.(d), 14CTri.(e) and the
difference between them (f) in the DC-HF calculations with the SLy5
parametrization of the Skyrme EDF.

The isospin effects on cluster transfer are investigated by comparing 12C and
14C projectiles. The density distribution of 14CLin. obtained from static HF
calculation is shown in Fig. 1(b) and 14CTri. obtained from DC-HF calculation
is shown in Fig. 1(e). The superscripts "Lin." and "Tri." denote the linear-
chain and triangle configurations, respectively. Higher density compared to
12CLin. and 12CTri., as shown in Fig. 1(c)(f), is attributed to the
contribution of two valence neutrons. Fascinatingly, for valence neutrons in
linear-chain structure, after iterating through static HF calculation to reach
the local minimum density, the density distribution of these two valence
neutrons becomes more concentrated on one side, specifically between the two
$\alpha$ particles, even if they were positioned on opposite sides at the
beginning. The existence of lower energy and longer linear-chain lifetime in
such configuration, as an answer, has been extensively confirmed Ren  _et al._
(2020).

For dynamic process, the TDHF equation is written as a set of nonlinear
Schrödinger-like equations for the occupied single-particle wave functions:

$i\hbar\frac{\partial}{\partial
t}\psi_{i}(\boldsymbol{r},t)=h\psi_{i}(\boldsymbol{r},t)\quad(i=1,\ldots,A),$
(1)

where $h$ is the single-particle Hamiltonian and $A$ is the number of single-
particle states. These non-linear equations are solved on a three-dimensional
Cartesian grid, allowing for a comprehensive analysis without imposing any
symmetry restrictions.

To enhance the analysis of cluster transfer processes, we utilized the
density-constrained TDHF (DC-TDHF) method Umar and Oberacker (2006) to
calculate the interaction potential between the projectile and target nuclei
throughout the reaction, which is widely employed in heavy-ion fusion
reactions Sun  _et al._ (2022); Godbey  _et al._ (2019); Umar and Oberacker
(2009); Keser  _et al._ (2012); Umar  _et al._ (2014); Godbey  _et al._
(2022); Washiyama (2015); Jiang  _et al._ (2013, 2014). At certain times, the
instantaneous density is utilized to carry out a static Hartree-Fock (HF)
minimization. In this process, the neutron and proton densities are
constrained to match the corresponding instantaneous TDHF densities,

$\displaystyle E_{\rm{DC}}(t)=$
$\displaystyle\min_{\rho}\left\\{E\left[\rho_{n},\rho_{p}\right]+\int\mathrm{d}\mathbf{r}\lambda_{n}(\mathbf{r})\left[\rho_{n}(\mathbf{r})-\rho_{n}^{\mathrm{TDHF}}(\mathbf{r},t)\right]\right.$
(2)
$\displaystyle\left.+\int\mathrm{d}\mathbf{r}\lambda_{p}(\mathbf{r})\left[\rho_{p}(\mathbf{r})-\rho_{p}^{\mathrm{TDHF}}(\mathbf{r},t)\right]\right\\}.$

One can thus define the ion–ion potential as:

$V(R)=E_{\rm{DC}}(R)-E_{1}-E_{2},$ (3)

where $E_{1}$ and $E_{2}$ are energies of projectile and target nucleus
obtained from static HF calculations, respectively. The excitation energy in
nuclear reactions can be expressed as:

$E^{*}(R)=E_{\rm TDHF}(R)-E_{\rm DC}(R)-V(R).$ (4)

Furthermore, the particle number projection (PNP) method Simenel (2010), which
is widely applied in multinucleon transfer reactions Evers  _et al._ (2011);
Sekizawa and Yabana (2013, 2016); Sekizawa (2017); Wu  _et al._ (2022, 2019);
Jiang and Wang (2020), will be employed to calculate the probability of
cluster transfer. The transfer probabilities in transfer reactions can be
obtained from the post-collision TDHF wave function, in a spatial volume $V$
containing the fragment, onto $N$ protons or neutrons that represent a certain
transfer channel. The projection operator can be expressed as an integral over
the gauge angle $\theta$, in the following form:

$\hat{P}_{V}^{q}(N)=\frac{1}{2\pi}\int_{0}^{2\pi}{\rm
e}^{i\theta\left(\hat{N}_{V}^{q}-N\right)}{\rm d}\theta.$ (5)

The probability of finding $N$ particles with an isospin of $q$ in the volume
region $V$ is:

$P_{V}^{q}(N)=\frac{1}{2\pi}\int_{0}^{2\pi}e^{-i\theta
N}\left\langle\Psi\left|{\rm
e}^{i\theta\hat{N}_{V}^{q}}\right|\Psi\right\rangle{\rm d}\theta.$ (6)

The probability of 4He and 8Be transfers can thus be expressed as:

$\displaystyle P(^{4}{\rm He})=P^{\rm n}(N_{0}-2)P^{\rm p}(Z_{0}-2),$ (7)
$\displaystyle P(^{8}{\rm Be})=P^{\rm n}(N_{0}-4)P^{\rm p}(Z_{0}-4),$

where $N_{0}$ and $Z_{0}$ denote neutron number and proton number of 12C or
14C, respectively.

Figure 2:  The DC-TDHF potential (upper panels) calculated for the
12CLin.+209Bi at $E_{\rm c.m.}=75$ MeV and the corresponding particle number
(lower panels) of nucleon transfer from of projectile to target as functions
of distance of the center-of-mass at impact parameter $b=5.5$ fm (a),(b) and
$b=6.5$ fm (c),(d). The snapshots of density distribution are also shown.

Results and discussions. The interaction potential and nucleon transfer in the
reaction of 12CLin.+209Bi at $E_{\rm c.m.}=75$ MeV are shown in Fig. 2. With
an impact parameter of $b=5.5$ fm, the ion-ion interaction potential initially
increases as the center-of-mass distance decreases. At $R=12.8$ fm, near the
range of nuclear forces, the potential energy starts to decrease. This
signifies the initiation of mass transfer between the two nuclei as shown in
Fig. 2 (b). When $R=10.5$ fm, the projectile-like fragment departs from the
target with a significant portion of mass transferring from 12CLin. to 209Bi,
while approximately one $\alpha$ particle scattered. Notably, there are two
turning points($R=11.7$ fm and $R=10.5$ fm) in the descending potential curve,
precisely corresponding to the transfer of two and four protons from 12CLin.
to the target (see Fig. 2 (b)). Thus, this turning point provides compelling
evidence for cluster transfer.

For the impact parameter of 6.5 fm, a turning point is also observed. Due to
the grazing nature of the collision, the duration of the nuclear reaction is
reduced, leading to a relatively small number of transferred nucleons.
Intriguingly, a notable disparity between numbers of the transferred protons
and neutrons is shown. In grazing collisions, the substantial neutron skin of
209Bi results in a significant isospin asymmetry in the neck. This, in turn,
enhances the effects of symmetry energy and promotes proton transfer.
Therefore, one conjecture can be made that this behavior arises from the
competition between cluster transfer and isospin diffusion.

Figure 3:  The nucleon transfer number as functions of three reaction systems
at $E_{\rm c.m.}$ = 70 MeV, $b$ = 6 fm. The nucleon transfer from the
projectile to the target (a), from the target to the projectile (b) and the
net transfer number are both shown.

The isospin diffusion effects in the reaction process can be explained by the
bidirectionality of nucleon transfer. To clarify the role of isospin diffusion
on the cluster transferring, 14C is compared to 12C in the collisions with
209Bi. The presence of two valence neutrons in 14C outside the 12C core is
expected to influence the nucleon transfer process. The number and the
direction of nucleon transfer which obtained from the integration for specific
wave functions are illustrated in Fig. 3. Note that there is a mere 0.5 MeV
difference in the Coulomb barriers of the two systems. As a result, we employ
identical incident energies to contrast the impacts related to isospin and
valence neutrons. For 12CLin.+209Bi, about two protons and two neutrons are
transferred from the projectile to the target, suggesting that microscopic
analysis of transfer reactions can involve considering them as the concurrent
transfer of both a proton pair and a neutron pair Rotter (1969); Kurath and
Towner (1974). Meanwhile, a significant neutron reverse-transfer (from 209Bi
to 12CLin.) takes place, while there is almost no proton flow from 209Bi to
12CLin.. This is because the neck region is neutron-rich due to the neutron
skin of 209Bi and the isospin diffusion plays an important role.

On the other hand, for 14CLin. with the valence neutrons, the net neutron
transfer to the target is enhanced. The valence neutrons of 0.7 transfer to
the target has been find, which can offset a significant portion of the
reverse-transfer neutron. Moreover, the neutron-proton ratio of 14CLin. more
closely resembles 209Bi than 12CLin., leading to less proton transfer to the
target. The combined effects of the aforementioned factors influence the net
nucleon transfer, ultimately resulting in one neutron-proton pair of net
transfer from 14CLin..

Figure 4:  Time evolution of the excitation energy from DC-TDHF (a), the net
nucleon transferring number from the projectile to the target (b) and the
isospin variation of projectile-like fragments (c) for 12CLin.+209Bi
(circles), 14CLin.+209Bi (lower triangles) and 14C${}_{\rm T}^{\rm
Lin.}$+209Bi (upper triangles) at $E_{\rm c.m.}$ = 70 MeV, $b$ = 6 fm.

We also show the results in the reaction 14C${}_{\rm T}^{\rm Lin.}$+209Bi.
Here, 14C${}_{\rm T}^{\rm Lin.}$+209Bi denotes the configuration that the two
valence neutrons are more concentrated near the neck region during the
collision compared to 14CLin.. The 14C${}_{\rm T}^{\rm Lin.}$ is obtained from
performing a spatial inversion along the $x$-axis on the wave function of
14CLin.. More intense nucleon transfer and exchange are shown. As illustrated
in Fig. 4(a)(b), because of high neutron-richness around the neck region,
14C${}_{\rm T}^{\rm Lin.}$ induced reaction present larger energy dissipation,
as well as about one more neutron-proton pair transfer to the target in
comparison to 14CLin.. This means that a roughly alpha particle is purely
transferred from 14C${}_{\rm T}^{\rm Lin.}$ to 209Bi. However, for the
reaction 12CLin.+209Bi, the number of pure proton transfer is almost two times
more than that of neutron transfer. As illustrated in Fig. 4(c), for the case
of 12CLin. the value of $\Delta I$ (where $\Delta I=I-I(t=0)$,
$I=(N-Z)/(N+Z)$) increases rapidly compared to those of the 14CLin. and
14C${}_{\rm T}^{\rm Lin.}$ configurations. The isospin diffusion weakens the
proton and neutron paring during the collisions, which consequently could
lower down the probabilities of the alpha cluster transfer.

Figure 5:  The transfer probabilities of 4He (upper panels) and 8Be (lower
panels) as functions of impact parameter $b$. The dots with a horizontal line,
open dots and solid dots denote 12C, 14C and 14CT projectiles, respectively.
Circles, squares and diamonds denote $E_{\rm c.m.}=70$ MeV, 75 MeV and 80 MeV,
respectively. The open triangles and solid triangles denote 12CTri. and
14CTri. at $E_{\rm c.m.}$ = 80 MeV, respectively. The lines are only to guide
the eyes.

To further clarify the effects of isospin diffusion on the cluster transfer,
we quantitatively show the probabilities of transferring clusters of 4He and
8Be in Fig. 5. Owing to the spatial dispersion characteristic of the wave
function, the probabilities of cluster transfer can be extracted by using the
PNP method. It can be seen that, at same incident energy and impact parameter,
14CLin. shows higher cluster transfer probability than 12CLin., especially for
transferring the 8Be cluster, although 209Bi(12C,4He)217Fr having a higher
$Q_{\rm gg}$ of $-21.6$ MeV compared to 209Bi(14C,6He)217Fr of $-33.8$ MeV.
The cluster transfer in 12CLin. induced reaction, as shown in Fig. 3, is
restrained by the reverse-transfer of neutron from the target nucleus, an
effect that becomes less pronounced in the case of 14CLin. due to the presence
of two valence neutrons, especially for the configuration of 14C${}_{\rm
T}^{\rm Lin.}$. We would like to emphasize that the Coulomb barriers and
elongation along the $x$ direction are very close for 12C and 14C induced
reactions. Therefore, it is reasonable to compare these reactions at same
incident energy and with same impact parameter. To verify that the above
behavior does not depend on the configurations, we also show the results in
the triangular case. The strong enhancement of 8Be transfer in 14CTri. induced
reaction is clearly shown. Furthermore, the 4He transfer probabilities are
also much larger in the reaction 14CTri. \+ 209Bi because of the valence
neutrons. The obvious behavior of the 4He and the 8Be transferring
probabilities testifies our conjecture that the isospin diffusion strongly
inhibits the cluster transferring.

Summary. We find the enhancement of the cluster transfer in the reactions
involving the light projectile with valence neutrons. The same behavior also
had been noticed experimentally. However, why does it violate the ground state
$Q$ value systematics? Do the equilibrium processes affect the clustering in
open quantum systems? In this work, the fully microscopic approach TDHF was
employed to investigate the role of the isospin diffusion on the cluster
transfer by comparing the 12,14C+209Bi systems. The evidence of cluster
transfer phenomena, by analyzing the nuclear interaction potential and nucleon
transfer, was obtained. A bidirectional neutron transfer process has been
observed, exerting a dampening effect on cluster transfer probabilities for
the 12C+209Bi system. In contrast, within the projectile of 14C, the transfer
of valence neutrons partially offsets the reverse-transfer of neutrons from
the target nucleus. Enhanced energy dissipation, prolonged contact time, and
high neutron-richness near neck region results in an elevated cluster transfer
probability. The quantitative comparisons of 12C and 14C in both the linear-
chain and triangular configurations certify the strongly correlation between
the isospin diffusion and the clustering. It is found for the first time that
the process of isospin diffusion results in a hindrance of cluster transfer.
This is one direct evidence that the statistical equilibration processes
affect the clustering in open quantum many-body systems.

Acknowledgments. The authors thank C. J. Lin, H. M. Jia, P. W. Wen, P. W. Zhao
and D. D. Zhang for useful discussions. This work was supported by the
National Natural Science Foundation of China under Grants No. 12075327; The
Open Project of Guangxi Key Laboratory of Nuclear Physics and Nuclear
Technology under Grant No. NLK2022-01; Fundamental Research Funds for the
Central Universities, Sun Yat-sen University under Grant No. 23lgbj003.

## References

  * Ebran  _et al._ (2012) J.-P. Ebran, E. Khan, T. Nikšić, and D. Vretenar, Nature 487, 341 (2012).
  * von Oertzen  _et al._ (2006) W. von Oertzen, M. Freer, and Y. Kanada-En’yo, Physics Reports 432, 43 (2006).
  * Freer  _et al._ (2018) M. Freer, H. Horiuchi, Y. Kanada-En’yo, D. Lee, and U.-G. Meißner, Reviews of Modern Physics 90, 035004 (2018).
  * Carlson and Schiavilla (1998) J. Carlson and R. Schiavilla, Reviews of Modern Physics 70, 743 (1998).
  * Wiringa  _et al._ (2014) R. B. Wiringa, R. Schiavilla, S. C. Pieper, and J. Carlson, Physical Review C 89, 024305 (2014).
  * Freer and Fynbo (2014) M. Freer and H. O. U. Fynbo, Progress in Particle and Nuclear Physics 78, 1 (2014).
  * Otsuka  _et al._ (2022) T. Otsuka, T. Abe, T. Yoshida, Y. Tsunoda, N. Shimizu, N. Itagaki, Y. Utsuno, J. Vary, P. Maris, and H. Ueno, Nature Communications 13, 2234 (2022).
  * Epelbaum  _et al._ (2012) E. Epelbaum, H. Krebs, T. A. Lähde, D. Lee, and U.-G. Meißner, Physical review letters 109, 252501 (2012).
  * Zhao  _et al._ (2015) P. Zhao, N. Itagaki, J. Meng, _et al._ , Physical Review Letters 115, 022501 (2015).
  * Suhara and Kanada-En’yo (2010) T. Suhara and Y. Kanada-En’yo, Physical Review C 82, 044301 (2010).
  * Han  _et al._ (2023) J. Han, Y. Ye, J. Lou, X. Yang, Q. Li, Z. Yang, Y. Yang, J. Wang, J. Xu, Y. Ge, _et al._ , Communications Physics 6, 220 (2023).
  * Liu  _et al._ (2020) Y. Liu, Y. Ye, J. Lou, X. Yang, T. Baba, M. Kimura, B. Yang, Z. Li, Q. Li, J. Xu, _et al._ , Physical Review Letters 124, 192501 (2020).
  * Han  _et al._ (2022) J. Han, Y. Liu, Y. Ye, J. Lou, X. Yang, T. Baba, M. Kimura, B. Yang, Z. Li, Q. Li, _et al._ , Physical Review C 105, 044302 (2022).
  * Tanaka  _et al._ (2021) J. Tanaka, Z. Yang, S. Typel, S. Adachi, S. Bai, P. van Beek, D. Beaumel, Y. Fujikawa, J. Han, S. Heil, _et al._ , Science 371, 260 (2021).
  * Duer  _et al._ (2022) M. Duer, T. Aumann, R. Gernhäuser, V. Panin, S. Paschalis, D. Rossi, N. Achouri, D. Ahn, H. Baba, C. Bertulani, _et al._ , Nature 606, 678 (2022).
  * He  _et al._ (2014) W. He, Y. Ma, X. Cao, X. Cai, G. Zhang, _et al._ , Physical Review Letters 113, 032506 (2014).
  * He  _et al._ (2016) W. He, Y. Ma, X. Cao, X. Cai, G. Zhang, _et al._ , Physical Review C 94, 014301 (2016).
  * Piantelli  _et al._ (2023) S. Piantelli, G. Casini, P. Ottanelli, L. Baldesi, S. Barlini, B. Borderie, R. Bougault, A. Camaiani, A. Chbihi, C. Ciampi, _et al._ , Physical Review C 107, 044607 (2023).
  * Frosin  _et al._ (2023) C. Frosin, S. Piantelli, G. Casini, A. Ono, A. Camaiani, L. Baldesi, S. Barlini, B. Borderie, R. Bougault, C. Ciampi, _et al._ , Physical Review C 107, 044614 (2023).
  * Broniowski and Arriola (2014) W. Broniowski and E. R. Arriola, Physical Review Letters 112, 112501 (2014).
  * Zhang  _et al._ (2017) S. Zhang, Y. Ma, J. Chen, W. He, and C. Zhong, Physical Review C 95, 064904 (2017).
  * Guo  _et al._ (2019) C.-C. Guo, Y.-G. Ma, Z.-D. An, B.-S. Huang, _et al._ , Physical Review C 99, 044607 (2019).
  * Li  _et al._ (2020) Y.-A. Li, S. Zhang, Y.-G. Ma, _et al._ , Physical Review C 102, 054907 (2020).
  * Dasgupta  _et al._ (2023) P. Dasgupta, R. Chatterjee, G.-L. Ma, _et al._ , Physical Review C 107, 044908 (2023).
  * Hodgson and Běták (2003) P. E. Hodgson and E. Běták, Physics reports 374, 1 (2003).
  * Canto  _et al._ (2009) L. Canto, P. Gomes, J. Lubian, L. Chamon, and E. Crema, Nuclear Physics A 821, 51 (2009).
  * Canto  _et al._ (2006) L. Canto, P. Gomes, R. Donangelo, and M. S. Hussein, Physics reports 424, 1 (2006).
  * Canto  _et al._ (2015) L. Canto, P. Gomes, R. Donangelo, J. Lubian, and M. S. Hussein, Physics Reports 596, 1 (2015).
  * Le Beyec  _et al._ (1972) Y. Le Beyec, M. Lefort, and M. Sarda, Nuclear Physics A 192, 405 (1972).
  * Hamada  _et al._ (2011) S. Hamada, N. Burtebayev, K. Gridnev, and N. Amangeldi, Nuclear Physics A 859, 29 (2011).
  * Becchetti  _et al._ (1980) F. Becchetti, K. Hecht, J. Jänecke, D. Overway, and G. Kekelis, Nuclear Physics A 339, 132 (1980).
  * Jin  _et al._ (1980) G.-M. Jin, Y.-X. Xie, Y.-T. Zhu, W.-G. Shen, X.-J. Sun, J.-S. Guo, G.-X. Liu, J.-S. Yu, C.-C. Sun, and J. Garrett, Nuclear Physics A 349, 285 (1980).
  * Sahu  _et al._ (2001) P. Sahu, R. Choudhury, D. Biswas, and B. Nayak, Physical Review C 64, 014609 (2001).
  * Biswas  _et al._ (2012) D. Biswas, P. Roy, Y. Gupta, B. Joshi, B. Nayak, L. Danu, B. John, R. Vind, N. Deshmukh, S. Mukherjee, _et al._ , in  _Journal of Physics: Conference Series_ , Vol. 381 (IOP Publishing, 2012) p. 012091.
  * Shorto  _et al._ (2008) J. Shorto, E. Crema, R. Simoes, D. Monteiro, J. Huiza, N. Added, and P. Gomes, Physical Review C 78, 064610 (2008).
  * Crema  _et al._ (2018) E. Crema, V. A. B. Zagatto, J. M. B. Shorto, B. Paes, J. Lubian, R. F. Simões, D. d. S. Monteiro, J. F. P. Huíza, N. Added, M. C. Morais, _et al._ , Physical Review C 98, 044614 (2018).
  * Schuetrumpf  _et al._ (2018) B. Schuetrumpf, P.-G. Reinhard, P. Stevenson, A. S. Umar, and J. A. Maruhn, Computer Physics Communications 229, 211 (2018).
  * Simenel (2012) C. Simenel, The European Physical Journal A 48, 152 (2012).
  * Simenel and Umar (2018) C. Simenel and A. Umar, Progress in Particle and Nuclear Physics 103, 19 (2018).
  * Stevenson and Barton (2019) P. Stevenson and M. Barton, Progress in Particle and Nuclear Physics 104, 142 (2019).
  * Sekizawa (2019) K. Sekizawa, Frontiers in Physics 7, 20 (2019).
  * Sun  _et al._ (2023) X.-X. Sun, L. Guo, _et al._ , Physical Review C 107, L011601 (2023).
  * Simenel  _et al._ (2013) C. Simenel, M. Dasgupta, D. Hinde, and E. Williams, Physical Review C 88, 064604 (2013).
  * Chabanat  _et al._ (1998) E. Chabanat, P. Bonche, P. Haensel, J. Meyer, and R. Schaeffer, Nuclear Physics A 635, 231 (1998).
  * Umar  _et al._ (2023) A. Umar, K. Godbey, and C. Simenel, Physical Review C 107, 064605 (2023).
  * Shen  _et al._ (2023) S. Shen, S. Elhatisari, T. A. Lähde, D. Lee, B.-N. Lu, and U.-G. Meißner, Nature Communications 14, 2777 (2023).
  * Ren  _et al._ (2020) Z. Ren, P. Zhao, and J. Meng, Physics Letters B 801, 135194 (2020).
  * Umar and Oberacker (2006) A. Umar and V. Oberacker, Physical Review C 74, 021601 (2006).
  * Sun  _et al._ (2022) X.-X. Sun, L. Guo, A. Umar, _et al._ , Physical Review C 105, 034601 (2022).
  * Godbey  _et al._ (2019) K. Godbey, L. Guo, A. Umar, _et al._ , Physical Review C 100, 054612 (2019).
  * Umar and Oberacker (2009) A. Umar and V. Oberacker, The European Physical Journal A 39, 243 (2009).
  * Keser  _et al._ (2012) R. Keser, A. Umar, and V. Oberacker, Physical Review C 85, 044606 (2012).
  * Umar  _et al._ (2014) A. Umar, C. Simenel, and V. Oberacker, Physical Review C 89, 034611 (2014).
  * Godbey  _et al._ (2022) K. Godbey, A. Umar, and C. Simenel, Physical Review C 106, L051602 (2022).
  * Washiyama (2015) K. Washiyama, Physical Review C 91, 064607 (2015).
  * Jiang  _et al._ (2013) X. Jiang, S. Yan, and J. A. Maruhn, Physical Review C 88, 044611 (2013).
  * Jiang  _et al._ (2014) X. Jiang, J. A. Maruhn, and S. Yan, Physical Review C 90, 064618 (2014).
  * Simenel (2010) C. Simenel, Physical review letters 105, 192701 (2010).
  * Evers  _et al._ (2011) M. Evers, M. Dasgupta, D. Hinde, D. Luong, R. Rafiei, R. Du Rietz, and C. Simenel, Physical Review C 84, 054614 (2011).
  * Sekizawa and Yabana (2013) K. Sekizawa and K. Yabana, Physical Review C 88, 014614 (2013).
  * Sekizawa and Yabana (2016) K. Sekizawa and K. Yabana, Physical Review C 93, 054616 (2016).
  * Sekizawa (2017) K. Sekizawa, Physical Review C 96, 014615 (2017).
  * Wu  _et al._ (2022) Z. Wu, L. Guo, Z. Liu, and G. Peng, Physics Letters B 825, 136886 (2022).
  * Wu  _et al._ (2019) Z. Wu, L. Guo, _et al._ , Physical Review C 100, 014612 (2019).
  * Jiang and Wang (2020) X. Jiang and N. Wang, Physical Review C 101, 014604 (2020).
  * Rotter (1969) I. Rotter, Nuclear Physics A 135, 378 (1969).
  * Kurath and Towner (1974) D. Kurath and I. Towner, Nuclear Physics A 222, 1 (1974).


# Lexicon and Rule-based Word Lemmatization Approach for the Somali Language

Shafie Abdi Mohamed$\diamond$, Muhidin Abdullahi Mohamed$\star$,$\diamond$
$\diamond$ Center for Graduate Studies, Jamhuriya University, Mogadishu,
Somalia
$\star$ Operations and Information Management, Aston University, Birmingham,
UK
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Lemmatization is a Natural Language Processing (NLP) technique used to
normalize text by changing morphological derivations of words to their root
forms. It is used as a core pre-processing step in many NLP tasks including
text indexing, information retrieval, and machine learning for NLP, among
others. This paper pioneers the development of text lemmatization for the
Somali language, a low-resource language with very limited or no prior
effective adoption of NLP methods and datasets. We especially develop a
lexicon and rule-based lemmatizer for Somali text, which is a starting point
for a full-fledged Somali lemmatization system for various NLP tasks. With
consideration of the language morphological rules, we have developed an
initial lexicon of 1247 root words and 7173 derivationally related terms
enriched with rules for lemmatizing words not present in the lexicon. We have
tested the algorithm on 120 documents of various lengths including news
articles, social media posts, and text messages. Our initial results
demonstrate that the algorithm achieves an accuracy of 57% for relatively long
documents (e.g. full news articles), 60.57% for news article extracts, and
high accuracy of 95.87% for short texts such as social media messages.

##  1 Introduction

An estimated population of over 22
million111https://www.worlddata.info/languages, mainly in Somalia, Djibouti,
Kenya, Ethiopia, the UK, the USA, and Europe speaks the Somali language. It is
well known that there is no way one can understand and access information
better than using one’s own language, however, there are significant digital
limitations associated with the Somali language preventing it from fully
participating in today’s digital world. For example, Somali is a low-resource
language, which means that there are very limited or no datasets available for
AI research and applications like translation, transcription, classification,
language modeling, and many others. This puts the Somali-speaking people at a
disadvantage and limits them from utilizing such essential AI and NLP
technologies. As such, the initial research in this paper will be a starting
point in meeting these limitations by pioneering the development of
lemmatization for the Somali language and the construction of a lexicon
dedicated to this task.

Lemmatization is an NLP method of transforming words of the same root to their
base shape. There are different methods that can be used to find the root
words, for example, using rules, a dictionary-like look-up method, or a
combination of the methods mentioned above. Different NLP methods and
lemmatization techniques have been proposed and datasets created for many
developed main languages, such as the English and French, but such resources
are not available for many under-resourced languages including Somali and this
is where the importance of this study comes in.

In this work, we study the problem of lemmatization in the Somali language to
develop an approach for normalizing words derived from the same roots. The
study mainly focuses on the lemmatization of the “MAXAA TIRI”, which is the
main and written dialect of the Somali Language. The main objective of the
lemmatizer is to extract meaningful root words from their inflected forms by
applying look-up and rule-based methods. In other words, we built two ways to
find the root words: a lexicon or dictionary based (i.e., searching words in
the dictionary) and rule-based (looking at the beginning of the term and
determining the root of the given word). For the lexicon-based, we constructed
an initial dictionary built on the morphological association of the word
lemmas and their inflected forms. Finally, the combined lexicon and rule-based
methods are applied to a test dataset, achieving an average lemmatization
accuracy of over 95% for short documents. This performance shows that there is
a potential for developing a full-fledged Somali Lemmatizer by scaling the
lexicon and consolidating the rules. The words ‘dictionary’ and ‘lexicon’ are
used interchangeably in the context of this paper.

The contributions of this work are three-fold:

  1. 1.

First, we have developed an initial Somali lexicon for word lemmatization with
the consideration of the language morphological rules.

  2. 2.

Second, we have designed a set of rules for normalizing words not covered in
the dictionary and developed a Somali word lemmatization algorithm built on
the lexicon and rules.

  3. 3.

Third, we have tested the algorithm on 120 documents of various lengths,
including news articles and social media posts, to estimate the performance of
the proposed algorithm.

##  2 Related work

Lemmatization is a well-studied NLP task for high-resource languages including
English, French, and Chinese but remains an open research problem for under-
resourced languages such as Somali [11, 5, 6]. Besides, lexicon and rule-based
approaches are some of the most common methods used for word lemmatization,
particularly for languages with limited resources. For example, authors in
[10] studied the development of lemmatization algorithms based on _if-then-
rules_ for Slovene. Similarly, Prathibha and Padma [12] put forward a rule-
based lemmatization algorithm for the Kannada language by combining linguistic
rules and meaningful root words. On the other hand, the work by Eger et. al.
[4] conducted a comparative study of lexicon-based approaches for Latin and
found that a lexicon-based lemmatization can achieve a high accuracy given the
existence of a considerable sized dataset.

Very few studies have been conducted about the application of NLP methods in
the Somali language or the construction of related datasets. In one of the
very few related studies, Mohammed [7] investigates the problem of part-of-
speech (POS) tagging for the Somali language using statistical and machine-
learning approaches. The work reports that the introduced Somali POS tagger
achieved a high accuracy of 87.51% on tenfold cross-validation experiments.
Furthermore, the work by Abdillahi, et. al. [1] presents a study on the
automatic transcription of the Somali language. Their work constructs a corpus
from the audio speech of about 10 hours achieving about a 21-word error rate
(WER).

In another relevant recent study built on existing pre-trained models,
Adelani, et. al. [2] investigated the possibility of developing Machine
Translation (MT) models for 16 low-resourced African languages including
Somali. The study’s results suggested that a translation corpus of around two
thousand sentence pairs is enough to develop MT models for low-resourced
languages with a reasonable performance by fine-tuning pre-trained models.
There are also several other related works that have studied approaches for
mitigating the data scarcity and adoption of pre-trained models such as BERT
for low-resourced languages including Somali [13, 3, 8, 9].

To the best of our knowledge, there have not been any previous works on the
subject of lemmatization for the Somali language, which makes this work the
first of its kind for normalizing text written in Somali. It, therefore, paves
the way for further research on the creation of NLP resources and techniques
for the Somali language.

##  3 Proposed lemmatization approach

Figure 1 provides a high-level illustration of the proposed Somali language
word lemmatizer. As shown, constructing the lexicon is the first step of the
method in which criteria for pairing root and derived words were developed
before commencing the actual compilation of the lexicon. The lexicon is
organized into key-value pairs with the keys representing root words and
values constituting the derivational forms of the roots. Because of the
limited size of the lexicon created in this initial work, we have also
developed simple rules for handling words that are not found in the lexicon to
improve the lemmatizer accuracy. In other words, the proposed lemmmatization
is a two-stage process (denoted by step 1 and step 2 in Figure 1). In the
first stage, the algorithm attempts to lemmatize a given word by searching for
it in the constructed lexicon and applies the written rules in stage 2 if the
root word is not found in the dictionary.

Figure 1: Proposed lexicon and rule-based Somali word lemmatizer

The lemmatization proposed in this work is particularly for the main Somali
dialect, “MAXAA TIRI”. To perform the lemmatization, we first read the
prepared lexicon database. The loaded data is then converted to a JSON format
to enable us to further process the lexicon and retrieve the root words.
Before extracting the lemmas, we applied some data pre-processing such as
removing punctuation marks and stop-words as the target of the lemmatizer is
the keywords. Finally, we look for root words in the JSON-formatted lexicon.
If the base form of a given word is found in the lexicon, it is returned as
the root. For words without entries in the dictionary, the rule-based
component of the algorithm is applied to lemmatize the word. If the word is
still not normalized at this stage, it is recognized as _unsolved_ , i.e, the
algorithm could not lemmatize it.

###  3.1 The Lexicon

The authors, who are native Somali speakers, compiled an initial collection of
the most frequently used words in several domains such as news, sport, and
social media. Next, we conducted consultations with language experts at the
Somali Language Academy222https://aga.so/so/, a regulating body for the
development and promotion of the Somali language. From these consultations
with the experts at the academy, a systematic method for constructing pairs of
root and derivative words was put in place, taking into account the language’s
grammar and morphology. The collected corpus of words was predominantly of two
parts: verbs and nouns. Based on the Somali language morphology, most verbs
can be changed to their root words by returning them to the command form of
the verb, for example, the terms _“cabay”_ (drunk, past), _“cabaya”_
(drinking, continuous), _“cabid”_ (drink, noun) can all be converted to the
root word _“cab”_ (drink now, command). Other similar roots such as _“cun”_
(eat now), _“laq”_ (swallow now), can be obtained in the same way. With the
noun category, the origin of the noun is the singular form, for example, the
words _“dowladda”_ (the government), _“dowladdeena”_ (our government) can be
reduced to the lemma _“dowlad”_ (government). Each word may have different
forms with distinct meanings different from the root word, for example, let’s
see the lexicon entries for the root words, _“cab”_ and _“bariis”_ with their
derivatives:

_Example 1 : entry for the root verb “cab” (drink):
cab:[“caba”,“cabay”,“cabeen”,“caba”,“cab”]_

_Example 2 : entry for the root noun “bariis” (rice):
bariis:[“bariis”,”bariiska”]_

You may notice that word derivations are reflected with changes in the
suffixes. We have built the lexicon in the form of a key and values as
illustrated in the above examples (cf. the extract in Figure 2). This
structure helps in the search for the root words, i.e., if the key is found in
the entry, it is returned as the root reducing the time complexity of the
algorithm. Constructed in those pairs of root and derived words, the lexicon
dataset is stored in a machine-readable dictionary, which is easily searchable
and expandable if new terms and their derivations are to be added. At the time
of writing this paper, the final lexicon consisted of a total of 8420 tokens
including 1247 root words and 7173 derivatives. As an illustration of the
overall lexicon organization, Figure 2 shows an extract of our lemmatization
lexicon including the words: _“laq”_ (swallow), _“fur”_ (open),
_“ceel”_(well), _”geel”_ (camel), and their morphological derivations.

Figure 2: A sample extract from the Somali lemmatizatiom lexicon

###  3.2 The Rules

To expand the lemmatized vocabularies, we have adopted a rule-based method in
which the root is determined from word beginnings. If different inflected
forms of the same base word start with a specified sequence followed by
several possible suffixes, then a rule can be built around this to return the
root word. For example, the regular expression “$jil(c|ic|eec)\backslash
w*$”333match the sequence _‘jil’_ followed by _‘c’_ , _‘ic’_ or _‘eec’_ , then
by other letters (or equivalent _“if-then-else”_ rule) is used to lemmatize
the inflected forms of the word “jilci” (which means ‘clarify’ or ‘soften’ in
Somali). Algorithm 1 summarizes the base word identification process as
proposed in this work.

(i) Input: Load each sentence;

(iii) Perform word tokenization

(iv) Remove stop & duplicate words, and punctuation marks;

(v) for  _each word_ do

for  _key and value in lexicon_ do

/* search a specific word in the lexicon */

if  _word is found in the lexicon_ then

return the key in that lexicon entry as the root

update the corresponding statistic

end if

/* find the roots for words not found in the lexicon using rules */

if  _word match is found in the list of rules_ then

return the specified root

update the corresponding statistic;

end if

else

Error: unresolved word

end if

end for

end for

Algorithm 1 Get root words from the supplied lexicon or rules

The pseudocode in Algorithm 1 is complementary to Figure 1 in summarizing how
the root word look-up and application of the lemmatization rules work in this
proposal. Input texts are first pre-processed (e.g. tokenizing, removing
punctuation marks and stop words, etc) before searching the words in the
lexicon or with the rules. Words that can neither be found in the lexicon nor
normalized with the designed rules are classified as _unresolved_ words.

##  4 Experimental Evaluation

As previously mentioned, the proposed lemmatization method is built on the
created lexicon of 1247 root words and 7173 related inflectional forms, and a
set of enriching rules. The number of derivational words associated with each
root in the lexicon ranged from 2 to more than 20 derivative words. Some of
the primary challenges faced in relation to the lexicon construction include
the lack of any existing morphological data for the Somali language and the
difficulty of objectively judging the quality of the lemmas and their
derivatives.

Empirically speaking, to find the root for a given word in a text, a simple
lexicon search is applied in the first instance. If the word is not found in
the dictionary, the next step is to search for it in the list of designed
lemmatization rules. A user of this algorithm will normally supply a text to
be passed to the algorithm after which the text will be pre-processed and
normalized with the lemmatizer. The primary metric utilized to assess the
effectiveness of the suggested lemmatizer is accuracy. The measure is
calculated as the ratio of successfully lemmatized words to all the different
input-inflected words. The following equation shows the calculation of the
lemmatization accuracy in this work.

$Accurracy(\%)=\frac{Total\;words\;resolved}{document\;length}$ (1)

Next, we present sets of tests to evaluate the effectiveness of the proposed
lemmatization method in terms of the aforementioned accuracy measure. In this
initial evaluation, we collected 120 test documents of different sizes,
largely from the news and social media domains. The test dataset is then pre-
processed by tokenizing it, and removing punctuation marks, stop words, and
duplicates. Let’s use an example sentence to illustrate how the proposed
lemmatizer works (numbering of the example continues from the previous
examples used in the paper).

_Example 3 (Somali) : Waxaan kula taliyey inuu casriyeeyo xirfadihiisa shaqo._

_Translation (English) : I have advised him to update his job skills._

Figure 3: Lemmatizing the words in the illustrative example (Example 3)

Figure 3 demonstrates the process of normalizing the words in Example 3. The
first step of applying the lemmatizer to test data, such as this example, is
to pre-process the text (see Figure 1). As part of this pre-processing, the
terms _“waxan”_ , _“kula”_ , _“inuu”_ are recognised as stop words and
removed. Similarly, the dot (“.”) is identified as a special character and
eliminated. The blue coded words (_“tali”, “casriyee”, “xirfad”_) are the root
words obtained from the lexicon for the corresponding terms while the word
_“xirfad”_ (green) was not found in the lexicon but resolved with the rule-
based component of the algorithm.

Table 1:  Summary statistics of Example 3 lemmatization Statistic |  Values
---|---
Original document size in words |  8
Stop words(non-unique) |  3
Special characters |  1
Unresolved words |  0
Resolved words |  4
Percent found |  100%
words from lexicon |  3
words from rule-based |  1

The lemmatization summary statistics of the Example 3 sentence are also
provided in Table 1. In this case, the percentage of words that were
normalized for the example reached 100%, which means that all content words
(excluding stop words and special characters) are lemmatized. This may be due
to the fact that this is a short document, a sentence of 8 words. Unlike the
lemmatization statistics of this example, a proportion of words in any typical
text document (i.e., longer than a sentence) will normally remain unresolved -
words that the algorithm fails to lemmatize in both stages.

Overall and as part of evaluating the proposed method, we have tested the
algorithm on 120 documents of various lengths including general news articles,
and social media posts. For the news articles, we have used extracts (i.e.,
title and first 1-2 paragraphs) as well as the full articles to see the effect
of document length. The results we found for these different document
categories are summarized in Table 2. The notations $\\#Docs$,
$Avg\>Doc\>Len$, and $Avg\>Acc.$ in the table respectively represent the
number of documents, average document length in words, and average
lemmatization accuracy. As shown, the results demonstrate that the algorithm
achieves a relatively good accuracy of 57% for moderately long documents (e.g.
news articles), 60.57% for news article extracts, and high accuracy of 95.87%
for short texts such as social media messages. This initial result is
promising for a language with extremely limited or no mentionable previous NLP
resources.

Table 2: Test results of the lemmatization algorithm Type |  # Docs |  Avg Doc Len |  Avg Acc.
---|---|---|---
News (extract) |  75 |  74.63 |  60.57%
News (full) |  10 |  175.9 |  57%
Social media |  30 |  21.23 |  95.87%
Text messages |  5 |  25.2 |  68.59%

From these results, it can be thought that the developed lexicon and the rule-
based lemmatization approach works best for short text documents such as the
social media messages. From the results (cf. Table 1), we can also notice that
most words are normalized at the dictionary look-up level, particularly when
it comes to short and moderate-sized texts. Finally, the implementation
details including the source code, lexicon, and the collection of used test
datasets are made publicly available for the research community on this GitHub
page 444https://github.com/ShafieAbdi/SomaliLemmatizer.

##  5 Conclusion and future work

This work is the first of its kind to study text lemmatization for the Somali
language. It proposes a lexicon and rule-based lemmatizer for Somali written
texts, which can be considered a starting point for a full-fledged Somali
lemmatization system and an indispensable component of other NLP tasks. With
the consideration of the Somali language morphology, we have manually crafted
an initial lexicon of 1247 roots and 7173 derived words to serve as a
dictionary look-up database for the word lemmatization task. Furthermore, we
have built two ways to find the root words: a lexicon-based look-up method
(achieved by searching in the dictionary), and a rule-based (identifying roots
based on matches of word beginnings). Finally, we have performed initial
evaluation of the algorithm and found some promising results including an
average lemmatization accuracy of over 95% for short documents. This means
that the study has reached a reasonable level of accuracy, paving the way for
further research in enabling the Somali language with NLP techniques.

Since this research is still in progress and represents a starting point for
an important NLP task and the development of underpinning resources, it has
its own limitations. Therefore, our immediate future plan is to further
improve the proposed lemmatization approach by increasing the size of the
lexicon database and expanding the applied rules. We will also investigate the
possibility of devising a methodology for building the lemmatization lexicon
automatically from a collected Somali corpus.

## References

  * [1] Nimaan Abdillahi, Nocera Pascal and Bonastre Jean-François  “Towards automatic transcription of somali language”  In _les actes de Language Resource and Evaluation Conference (LREC)_ , 2006, pp. 131–134
  * [2] David Adelani et al.  “A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation”  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ Seattle, United States: Association for Computational Linguistics, 2022, pp. 3053–3070
  * [3] Jesujoba O Alabi, David Ifeoluwa Adelani, Marius Mosbach and Dietrich Klakow  “Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning”  In _Proceedings of the 29th International Conference on Computational Linguistics_ , 2022, pp. 4336–4349
  * [4] Steffen Eger, Tim Brück and Alexander Mehler  “Lexicon-assisted tagging and lemmatization in Latin: A comparison of six taggers and two lemmatization methods”  In _Proceedings of the 9th SIGHUM Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities (LaTeCH)_ , 2015, pp. 105–113
  * [5] Michael A Hedderich et al.  “A survey on recent approaches for natural language processing in low-resource scenarios”  In _arXiv preprint arXiv:2010.12309_ , 2020
  * [6] Enrique Manjavacas, Ákos Kádár and Mike Kestemont  “Improving lemmatization of non-standard languages with joint learning”  In _arXiv preprint arXiv:1903.06939_ , 2019
  * [7] Siraj Mohammed  “Using machine learning to build POS tagger for under-resourced language: the case of Somali”  In _International Journal of Information Technology_ 12.3 Springer, 2020, pp. 717–729
  * [8] Kelechi Ogueji, Yuxin Zhu and Jimmy Lin  “Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages”  In _Proceedings of the 1st Workshop on Multilingual Representation Learning_ , 2021, pp. 116–126
  * [9] Odunayo Jude Ogundepo et al.  “AfriTeVA: Extending? Small Data? Pretraining Approaches to Sequence-to-Sequence Models”  In _Proceedings of the Third Workshop on Deep Learning for Low-Resource Natural Language Processing_ , 2022, pp. 126–135
  * [10] Joël Plisson, Nada Lavrac and Dunja Mladenic  “A rule based approach to word lemmatization”  In _Proceedings of IS_ 3, 2004, pp. 83–86
  * [11] Martin F Porter  “An algorithm for suffix stripping”  In _Program_ 14.3 MCB UP Ltd, 1980, pp. 130–137
  * [12] RJ Prathibha and MC Padma  “Design of rule based lemmatizer for kannada inflectional words”  In _2015 International Conference on Emerging Research in Electronics, Computer Science and Technology (ICERECT)_ , 2015, pp. 264–269  IEEE
  * [13] Zihan Wang, Stephen Mayhew and Dan Roth  “Extending multilingual BERT to low-resource languages”  In _arXiv preprint arXiv:2004.13640_ , 2020