# Learn to Grasp with Less Supervision:
A Data-Efficient Maximum Likelihood Grasp Sampling Loss

Xinghao Zhu1, Yefan Zhou1, Yongxiang Fan2, Lingfeng Sun1, Jianyu Chen1, and
Masayoshi Tomizuka1 1Mechanical Systems Control Lab, UC Berkeley, CA USA.
{zhuxh, yefan0726, lingfengsun, jianyuchen<EMAIL_ADDRESS>Advanced Research Lab, FANUC America Corporation, CA USA.
<EMAIL_ADDRESS>refer to marking the image to indicate a
successful robotic grasp.

###### Abstract

Robotic grasping for a diverse set of objects is essential in many robot
manipulation tasks. One promising approach is to learn deep grasping models
from large training datasets of object images and grasp labels. However,
empirical grasping datasets are typically sparsely labeled (i.e., a small
number of successful grasp labels∗ in each image). The data sparsity issue can
lead to insufficient supervision and false-negative labels, and thus results
in poor learning results. This paper proposes a Maximum Likelihood Grasp
Sampling Loss (MLGSL) to tackle the data sparsity issue. The proposed method
supposes that successful grasps are stochastically sampled from the predicted
grasp distribution and maximizes the observing likelihood. MLGSL is utilized
for training a fully convolutional network that generates thousands of grasps
simultaneously. Training results suggest that models based on MLGSL can learn
to grasp with datasets composing of 2 labels per image. Compared to previous
works, which require training datasets of 16 labels per image, MLGSL is
8$\times$ more data-efficient. Meanwhile, physical robot experiments
demonstrate an equivalent performance at a 90.7% grasp success rate on
household objects. Codes and videos are available at [1].

##  I Introduction

Robotic grasping in unstructured environments can benefit applications from
warehouse automation to home servicing. Supervised machine learning approaches
have demonstrated promising results in planning grasps under various
uncertainties. One kind of approach is to sample grasp candidates and evaluate
[2, 3, 4, 5, 6, 7, 8, 9, 10, 11]. Such two-step methods, however, might be
time-consuming at execution.

An alternative is to train grasp planning models in end-to-end manners [12,
13, 14, 15, 16, 17, 18, 19, 20, 21]. These approaches directly generate grasps
and have a shorter planning time. While end-to-end models typically require
densely labeled ground-truth samples in training, existing empirical datasets,
generated by physical execution or human labeling, only include scarce labels
for each image [22, 23]. Though many other good grasps exist, they are
unfortunately not labeled. The sparsity issue can lead to a lot of false
negatives in training end-to-end models. Previous works [14, 15, 16, 17, 18]
assume that unlabeled points are not valid and treat them as failed grasps.
However, robust grasps may still exist in these areas but are mistakenly
labeled as negatives. These wrong labels can harm the learning results.

Figure 1: Grasp planning and execution pipeline. When an object is presented
in the workspace, a stereo camera captures a depth image; a trained generative
model $f_{\theta}(\cdot)$ rapidly computes grasp configuration maps
$Q_{\theta}$, $W_{\theta}$, and $\Phi_{\theta}$. The best grasp is generated
based on configuration maps and executed with the robot manipulator. The grasp
model is trained offline with empirical datasets and the proposed loss
function.

This paper proposes a maximum likelihood grasp sampling loss (MLGSL) to
improve data efficiency in training grasp planners with single-view depth
images. The main difference between MLGSL and other works is the assumption
toward unlabeled regions. Previous works [14, 15, 16, 17, 18] regard unlabeled
areas as failures and estimate the success rate for each pixel. In contrast,
we leave unlabeled pixels intact and solely apply supervisions with labeled
grasps. Specifically, we propose a stochastic grasp selection process to
estimate the likelihood for each pixel to be the best grasp point and maximize
such likelihood for labeled grasps. Since no labels are generated for
unlabeled areas, MLGSL can reduce the false-negative problem in training grasp
planners. Training results demonstrate that models with MLGSL can learn to
grasp with fewer labels compared to previous works [15, 16, 17, 24], while
physical experiments show a similar grasp success rate at 90.7% (Fig. 1).
Moreover, this paper demonstrates that attention mechanisms do not contribute
to dense grasp plannings. Furthermore, a dataset is constructed with multiple-
object scenes and collision-free grasp labels to improve the performance in
clutter.

Related works are introduced in Section II. Section III presents the proposed
approach. Training and experiments are presented in Section IV and V. Section
VI concludes the paper and suggests the future work.

##  II Related Works

###  II-A Grasp Planning and Datasets

Analytic and machine learning methods have been studied to plan grasps across
various objects. Existing analytic methods [25, 26, 27] can be used to search
for optimal grasps. These techniques, however, are less robust in practice due
to perceptional limitations and unseen geometries.

Machine learning is an alternative approach to plan grasps. Current methods
show that it is preferable to learn grasp quality functions and optimize them
at the runtime [2, 4, 5, 6, 7, 8, 9, 10, 11]. However, sampling or
optimization makes the algorithm time-consuming and requires pre-defined
heuristics. Other end-to-end approaches propose to infer grasp poses from the
raw input directly. These approaches can be generative or discriminative.
Generative models [14, 12, 15, 16, 13, 28, 29, 30, 31] perform grasp pose
regression and grasp quality assessment simultaneously. Discriminative
approaches [19, 17] use fully convolutional networks to evaluate thousands of
grasps simultaneously without direct sampling. Reinforcement learning has also
been introduced in grasp planning [24, 32]. Q learning is leveraged to
estimate the state-action qualities with iterative updates.

Machine learning approaches typically require large datasets consisting of
sensor readings and ground-truth grasp labels. Synthetic datasets can be
rapidly generated using analytic quality metrics and simulated sensors [2, 3,
13]. However, synthetic data can cause robustness issues due to the
simulation-to-reality (sim-to-real) gap. Specifically, analytic metrics might
disagree with physical grasp results in complex circumstances. Alternatively,
empirical methods collect data from human labeling or grasp execution with
correlations of physical success [4, 5, 22, 23]. Nevertheless, empirical data
can be expensive to acquire and are typically sparsely labeled (i.e.,
inadequate grasp labels in each image). This paper chooses to use empirical
datasets to avoid the sim-to-real gap.

###  II-B Dense Grasp Planning

Recent works leverage dense pixel-wise evaluation and regression for grasp
planning. These approaches utilize Fully Convolutional Networks (FCNs) to
evaluate millions of grasps in parallel. FCNs in [15, 16] predict the grasp
success rate and generate grasp configurations for each pixel. In [17, 19],
FCNs are trained to predict the grasp success rate for pre-defined grasp
primitive actions.

During training, FCNs require sufficient labels to perform the image-to-image
learning [17]. However, existing datasets typically include scarce grasp
labels for each image [22, 23]. Some works have been proposed to resolve the
sparsity issue. A label generation method is proposed in [15, 16], which
assumes that grasps close to successful labels are robust and treats unlabeled
grasps as failures. However, this approach introduces both false negatives and
false positives. On the one hand, robust grasps might be away from existing
labels and are wrongly labeled as negatives. On the other hand, generated
grasps that are close to existing successful labels might be unstable. In [19,
21], networks are converted from discriminative evaluators according to the
injective mapping between convolutional and fully connected layers. A grasp-
push policy represented by FCNs is trained in [24] with Q learnings. Although
these methods eliminate the need for dense labels, they still require large
datasets or rollouts. This paper proposes a loss function capable of training
the FCN from scratch with inadequate labels.

###  II-C Attention Mechanism

Attention mechanisms have achieved promising results in computer vision [33,
34, 35]. Previous works have introduced spatial attention mechanism (SAM) to
robotics for interest-region extractions and feature reductions. In [36, 37],
hierarchical SAM is employed to constrain sampling in reinforcement learning
(RL) and demonstrated improved efficiency in action-space sampling. SAM is
used in [38, 39] to detect grasps in clutter, achieving improved grasping
success rate with RL. However, experiments in this paper demonstrate that
attention mechanisms do not contribute to dense grasp planning with FCNs.

##  III Grasp Planning with Maximum Likelihood Grasp Sampling Loss

This section first introduces notations of the problem. The proposed loss
function is then illustrated and compared with previous works. Finally, the
network architectures and datasets are presented.

###  III-A Notations

The grasp planning problem is defined as detecting a grasp that allows the
robot to pick up objects. Moreover, no explicit knowledge of the object is
given beyond camera readings.

Figure 2: Grasp Representation $g=(p,\phi,w)$. The planar pose $(p,\phi,w)$ in
(a) represents the grasp’s center position, orientation, and width in the
image. Grasp $g$ in (a) has quality $q=1$ since it is a successful grasp. $g$
is executed perpendicular to the image plane at point $p_{world}$ in the
Cartesian frame as shown in (b), where $p_{world}$ is $p$ in the world frame.
The gripper further moves $\epsilon$ cm below $p_{world}$ in the direction of
the camera’s z-axis.

####  III-A1 Grasp

Let $I\in\mathbb{R}^{\mathcal{H}\times\mathcal{W}}$ define a given depth image
with height $\mathcal{H}$ and width $\mathcal{W}$. The $i$-th grasp is defined
in the image $I$ and denoted by $g_{i}=(p_{i},\phi_{i},w_{i})$, where $p_{i}$
is a pixel in the image representing the grasp center.
$\phi_{i}\in[-\frac{\pi}{2},\frac{\pi}{2}]$ is the gripper’s rotation, and
$w_{i}\in[0,150]$ pixels is the gripper’s width in the image (Fig. 2). Each
grasp has a quality measurement $q_{i}\in[0,1]$ that indicates the grasp
success rate.

####  III-A2 Grasp Labels

Empirical datasets contain $k$ success grasp labels $\tilde{g}$ for each
image, where $\tilde{g}=\\{\tilde{g_{i}}\\}$ for $i\in[1,...,k]$ and
$k\ll\mathcal{H}\times\mathcal{W}$. Note $\tilde{q_{i}}=1$ since they are
guaranteed to succeed.

####  III-A3 Grasp Configuration Maps

Similar to [15], we refer to the set of grasps in the image as the grasp
configuration map
$G=(Q,\Phi,W)\in\mathbb{R}^{3\times\mathcal{H}\times\mathcal{W}}$, where
$Q,\Phi,W\in\mathbb{R}^{\mathcal{H}\times\mathcal{W}}$ contain values of
$q_{i},\phi_{i},w_{i}$ at each pixel of $I$. In practice, we use two
components $\Phi_{s}=\sin(2\Phi),\Phi_{c}=\cos(2\Phi)$ for $\Phi$ to resolve
the symmetricity of antipodal grasps.

Figure 3:  (a) shows the input depth image $I$ with $k=2$ success grasp labels
$\tilde{g}$ (red). (b-d) show predicted grasp configuration maps
$\hat{G}=(\hat{Q},\hat{\Phi},\hat{W})$ respectively. Different colors
represent different values as in the color bar. (e) shows 5 selected grasps
based on grasp configuration maps $\hat{G}$.

####  III-A4 Grasp Planning Models

This paper uses a grasp neural network $f_{\theta}(\cdot)$ to approximate the
dense grasp configuration maps,
$\hat{G}=(\hat{Q},\hat{\Phi},\hat{W})=f_{\theta}(I)$. Predicted grasps are
$\hat{g}=\\{\hat{g_{i}}\\}$ for $i\in[1,...,\mathcal{H}\times\mathcal{W}]$ at
each pixel.

###  III-B Maximum Likelihood Grasp Sampling Loss

Following definitions in Section III-A, this paper uses two grasp notations
for each image $I$: the predicted grasp configuration maps $\hat{G}$ and the
grasp labels $\tilde{g}$. The former maps are densely predicted by the
network, which have values at each pixel. In contrast, the latter $\tilde{g}$
only has $k$ successful grasp labels, as shown in Fig. 3. The objective of the
grasp model is to estimate grasp configuration maps by
$\hat{G}=f_{\theta}(I)$, which yield the highest grasp success rate.

At execution, grasps are selected based on the predictions $\hat{G}$. First, a
grasp pixel $p$ is drawn from the image conditioned on the quality map
$\hat{Q}$. Categorical distribution is used to describe the sampling process.
Each pixel is a category with a probability proportional to the predicted
quality $\hat{q_{i}}$. This suggests that for each grasp pixel, the higher the
grasp quality is, the more chance it can be selected to execute. Then,
corresponding rotation $\hat{\phi}$ and width $\hat{w}$ are selected from
dense prediction maps $\hat{\Phi}$ and $\hat{W}$ at pixel $p$.

This paper assumes that success grasp labels $\tilde{g}$ are chosen from the
prediction maps $\hat{G}$ based on the above selection procedure. To optimize
the learning performance, we propose to maximize the chance of observing
$\tilde{g}$ from $\hat{G}$ using a maximum likelihood estimation (MLE).

The grasp selection process is first modeled mathematically. The grasp
configuration $\boldsymbol{p},\boldsymbol{\phi},\boldsymbol{w}$ are regarded
as random variables. The pixel location $\boldsymbol{p}$ is discrete and takes
on a value in the pixel indexes, i.e.,
$p\in[1,...,\mathcal{H}\times\mathcal{W}]$. The probability mass function of
$\boldsymbol{p}$ is a Categorical distribution defined as $P(p|\hat{Q})$,
representing the chance that the grasp at pixel $p$ is selected for execution.
The rotation $\boldsymbol{\phi}$ and the width $\boldsymbol{w}$ are real
values in the configuration space, whose density functions are Gaussians as
$P(\phi|p,\hat{\Phi})$ and $P(w|p,\hat{W})$.

Based on the above model, the probability of observing a grasp label
$\tilde{g_{i}}$ from the prediction map $\hat{G}$ is
$P(\tilde{g_{i}}|\hat{G})$. The weights of the network $\theta$ are trained to
maximize such a probability:

$\displaystyle\max_{\theta}$ $\displaystyle P(\tilde{g_{i}}|\hat{G})$
$\displaystyle=$ $\displaystyle\max_{\theta}$ $\displaystyle
P(\tilde{p_{i}},\tilde{\phi_{i}},\tilde{w_{i}}|\hat{G})$ (1a) $\displaystyle=$
$\displaystyle\max_{\theta}$ $\displaystyle P(\tilde{p_{i}}|\hat{G})\cdot
P(\tilde{\phi_{i}}|\tilde{p_{i}},\hat{G})\cdot
P(\tilde{w_{i}}|\tilde{\phi_{i}},\tilde{p_{i}},\hat{G})$ (1b) $\displaystyle=$
$\displaystyle\max_{\theta}$ $\displaystyle P(\tilde{p_{i}}|\hat{Q})\cdot
P(\tilde{\phi_{i}}|\tilde{p_{i}},\hat{\Phi})\cdot
P(\tilde{w_{i}}|\tilde{p_{i}},\hat{W})$ (1c)

In (1a), $\tilde{g_{i}}$ is replaced with grasp configurations as in III-A.
Chain rules are applied to obtain (1b). Equation (1c) is modified based on the
grasp selection model. The first term maximizes the chance to select a
successful grasp at pixel $\tilde{p_{i}}$ for execution. The second and third
terms maximize the chance of observing $\tilde{\phi_{i}}$ and $\tilde{w_{i}}$
at pixel $\tilde{p_{i}}$ in $\hat{\Phi}$ and $\hat{W}$.

Equation (1c) is then jointly optimized for all grasp labels $\tilde{g}$. This
paper further assumes that each label is independent:

$\displaystyle\max_{\theta}$
$\displaystyle\prod_{i=1}^{k}P(\tilde{g_{i}}|\hat{G})$ (2a)
$\displaystyle\approx$ $\displaystyle\max_{\theta}$
$\displaystyle\prod_{i=1}^{k}P(\tilde{p_{i}}|\hat{Q})\cdot
P(\tilde{\phi_{i}}|\tilde{p_{i}},\hat{\Phi})\cdot
P(\tilde{w_{i}}|\tilde{p_{i}},\hat{W})$ (2b) $\displaystyle\propto$
$\displaystyle\max_{\theta}$ $\displaystyle\sum_{i=1}^{k}\log
P(\tilde{p_{i}}|\hat{Q})+\log P(\tilde{\phi_{i}}|\tilde{p_{i}},\hat{\Phi})$
(2c) $\displaystyle\ \ \ \ \ +\log P(\tilde{w_{i}}|\tilde{p_{i}},\hat{W})$
$\displaystyle=$ $\displaystyle\max_{\theta}$ $\displaystyle\sum_{i=1}^{k}\log
P(\tilde{p_{i}}|\hat{Q})-\text{MSE}(\hat{\phi_{i}},\tilde{\phi_{i}})$ (2d)
$\displaystyle\ \ \ \ \ -\text{MSE}(\hat{w_{i}},\tilde{w_{i}})$

Three terms are maximized in (2d). The first term maximizes the chance of
selecting robust grasps to execute. The second and third terms are modified
from (2c) to minimize the mean square error (MSE) between predictions and
labels, as [40] suggests. Such modifications stabilize the training process
without loss of accuracy.

From all above, the grasp model $f_{\theta}(\cdot)$ is trained as:

$\displaystyle\theta=\underset{\theta}{\mathrm{argmin}}$
$\displaystyle\mathcal{L}(\hat{G},\tilde{g})$ (3)

where

$\displaystyle\hat{G}$
$\displaystyle=(\hat{Q},\hat{\Phi},\hat{W})=f_{\theta}(I)$
$\displaystyle\mathcal{L}(\hat{G},\tilde{g})$
$\displaystyle=\sum_{i=1}^{k}-\log
P(\tilde{p_{i}}|\hat{Q})+\text{MSE}(\hat{\phi_{i}},\tilde{\phi_{i}})$
$\displaystyle\ \ \ \ \ \ \ \ +\text{MSE}(\hat{w_{i}},\tilde{w_{i}})$

The maximum likelihood grasp sampling loss (MLGSL) in (3) minimizes a pixel
selection loss and two pixel-wise regression losses. The main difference
between MLGSL and others is MLGSL only uses existing labels. Previous works
[15, 16, 17, 19, 24] regard unlabeled pixels as negatives and use regression
or spatial cross-entropy losses to estimate the quality for the entire image.
In contrast, MLGSL only computes losses for pixels that have successful grasp
labels (i.e., only at ${\tilde{p_{i}}}$), and it directly predicts whether a
pixel is the best grasp point. Specifically, MLGSL estimates the likelihood
that a pixel is the most robust grasp with the grasp selection model and
maximizes the likelihood for successful grasp labels. As suggested in [41,
42], estimating grasp qualities for each pixel is challenging due to only
scarce label exists and the false-negative problem. By contrast, MLGSL
directly optimizes the likelihood for each grasp and is more reliable and
easier to converge [42].

###  III-C Model Architectures

The grasp planning model is used to predict dense grasp configuration maps
$\hat{G}$, consisting of $\hat{Q},\hat{\Phi}_{c},\hat{\Phi}_{s},\hat{W}$. Note
that angle maps $\Phi$ is calculated by
$\Phi=\frac{1}{2}\tan^{-1}\frac{\Phi_{s}}{\Phi_{c}}$. The model uses a fully
convolutional topology, similar to [16]. The architecture includes four
downsampling layers, two dilated layers, and two upsampling layers.
Downsampling layers use kernel size of $[11,5,5,5]$ respectively, activated by
ReLU and max-pooling. Two dilated layers apply $[5,5]$ kernels with dilation
$[2,4]$. Upsampling layers employ transpose convolutional kernels with size 3
and striding 2.

SAM blocks are also added as described in [34]. SAM utilizes both max-pooling
and average-pooling along the channel axis and forwards them to a convolution
layer. Outputs are then integrated into input features.

###  III-D Dataset

The network is trained with a single object dataset and a cluttered object
dataset.

This paper directly adopt Jacquard [22] as the single object dataset. The
dataset contains more than 50k images of 11k objects and 1 million success
grasp labels. We apply random rotation and zoom to images and resize them to
$300\times 300$.

The cluttered object dataset is generated based on Jacquard. A few images are
randomly selected from the single object dataset and fused into a cluttered
sample. Before fusing, each single object image is segmented, rotated, zoomed,
and translated in the image plane. Success grasp labels are then merged and
pruned according to collision constraints. Since data in Jacquard includes
images from different viewpoints, such operation can reflect the geometry of
cluttered scenes.

##  IV Training Results

We trained a series of models to test the proposed approach. The goals of the
experiments are three-fold: 1) to demonstrate that the proposed loss function
can increase the grasp performance with fewer labels and samples, 2) to
determine whether attention modules help in learning dense grasp
configurations, and 3) to inspect the collision-avoidance ability in cluttered
scenes.

###  IV-A Evaluation Metrics

Three metrics are utilized to evaluate models’ performance: predictions’
success rate and predictions’ accuracy and recall. For prediction success
rate, a predicted grasp $\hat{g_{i}}$ is considered success if

$\displaystyle\exists\tilde{g_{j}}\in\tilde{g},\
\ni\text{IoU}(\tilde{g_{j}},\hat{g_{i}})\geq 25\%\text{ and
}|\tilde{\phi_{j}},\hat{\phi_{i}}|\leq 30\degree$

where $\text{IoU}(\cdot)$ represents the intersection over union ratio between
two grasps. $\tilde{g}$ is the set of success grasp labels. This paper selects
top one (Top-1) and top five (Top-5) grasps to measure the success rate as in
[15, 16, 22].

Besides the grasp success rate, we measure predictions’ accuracy and recall. A
grasp quality discriminator [2] was pre-trained to evaluate the robustness of
grasps. For each validation data, $100$ predicted grasps $\hat{g_{i}}$ are
first uniformly sampled with predicted quality $\hat{q_{i}}$. Then, the
discriminator evaluates quality for $\hat{g_{i}}$, obtaining ground-truth
quality label $q_{i}$. Prediction accuracy and recall are measured based on
$q_{i}$ and $\hat{q_{i}}$ for $i\in[1,...,100]$.

###  IV-B Baseline Methods

We compare the training performance of MLGSL to the following baseline
approaches:

####  IV-B1 Image-wise MSE (ImgMSE)

ImgMSE is introduced in [15] that use the same prediction maps $\hat{G}$ as
ours. This baseline augments successful grasp labels to $\bar{G}$ as in
Section II-B. The loss used to train the model is

$\mathcal{L}_{\text{ImgMSE}}=\text{MSE}(\hat{Q},\bar{Q})+\text{MSE}(\hat{\Phi},\bar{\Phi})+\text{MSE}(\hat{W},\bar{W})$

####  IV-B2 Maximum Likelihood Sampling with LogMSE (MLGSL+Log)

MLGSL+Log can be derived from (2c), which uses $\text{MSE}(\cdot)$ to replace
$P(\cdot)$, i.e.

$\displaystyle\mathcal{L}_{\text{MLGSL+Log}}=\sum_{i=1}^{k}$
$\displaystyle-\log
P(\tilde{p_{i}}|\hat{Q})+\log\text{MSE}(\hat{\phi_{i}},\tilde{\phi_{i}})$
$\displaystyle+\log\text{MSE}(\hat{w_{i}},\tilde{w_{i}})$

####  IV-B3 Pixel-wise MSE (PixMSE)

PixMSE applies supervisions solely on labeled pixels with MSE loss, i.e.

$\mathcal{L}_{\text{PixMSE}}=\sum_{i=1}^{k}\text{MSE}(\hat{q_{i}},\tilde{q_{i}})+\text{MSE}(\hat{\phi_{i}},\tilde{\phi_{i}})+\text{MSE}(\hat{w_{i}},\tilde{w_{i}})$

###  IV-C Results

For comparisons, we train variant models with different loss function designs
and architectures. Each model is trained with different seeds for 50 epochs to
select the best one.

####  IV-C1 Baseline Comparisons

Figure 4: Comparing the Top-1 prediction success rate of MLGSL with baseline methods. Models are trained with densely-labeled datasets (16 labels per image). TABLE I: Training performance of MLGSL and baselines (Mean %) Method | Top-1 | Top-5 | Accuracy | Recall
---|---|---|---|---
MLGSL | 82.8 | 91.0 | 80.2 | 75.7
MLGSL (2 labels) | 81.6 | 90.3 | 77.3 | 73.4
ImgMSE [16] | 82.5 | 89.7 | 85.2 | 92.3
ImgMSE [16] (2 labels) | 42.4 | 45.2 | 41.9 | 17.4
Figure 5: Predicted grasp distributions with variant models. Predicted grasp
qualities are painted as heatmaps with color listed in the right-sidebar.
Detected grasps are labeled with red lines in each image. (First row) input
depth images, (Left) results from models trained with MLGSL, (Right) results
from models trained with ImgMSE, (Second row) results from datasets consisting
of 16 labels per image and 10k training images, (Third row) results from
datasets consisting of 2 labels per image and 10k training images, (Bottom)
results from datasets consisting of 16 labels per image and 1k training
images.

Our first experiment compares MLGSL to three baseline methods with a single
object dataset, in which each training sample includes 16 success grasp
labels. Top-1 and Top-5 prediction success rates are shown in Fig. 4 and Table
I. We see that MLGSL has similar performances compared to previous ImgMSE,
while MLGSL shows a higher convergency rate at first a few epochs. MLGSL with
logarithm converges to a similar point as the previous two methods but with a
slower rate, which might occur because the $\log(\cdot)$ operation lowers the
gradient in each training step. PixMSE performs the worst among the four
approaches. This likely due to it only applies supervisions on specific
pixels, resulting in unbounded other areas.

We also compare MLGSL to ImgMSE on prediction accuracy and recall. Results are
shown in Table I. It is interesting to observe that models trained with MLGSL
have lower accuracy and recall. To seek reasons for such phenomenon, we plot
several predicted $\hat{G}$ in the second rows of Fig. 5. As can be seen,
models trained with MLGSL have a conservative estimation of the grasp quality
and prefer to grasp each object’s center. This behavior leads to that only a
small area has high quality (red area in Fig. 5). When measuring accuracy and
recall, grasps are uniformly sampled in the image (Section IV-A), making many
of them out of the high-quality zone. This fact then produces false-negative
predictions and low accuracy and recall.

####  IV-C2 Less Training Labels per Sample

Figure 6: Comparing the Top-1 prediction success rate of MLGSL to ImgMSE with
different numbers of labels (n_label). Success grasp labels are down-sampled
to $[2,4]$ for each training image. Solid lines indicate models’ performance
trained with MLGSL, and dashed lines indicate that trained with ImgMSE.

We then investigate whether our method can learn grasping with fewer labels.
For this study, we down-sample success grasp labels to $[2,4]$ in each
training data and still use all labels for validation. It is a more difficult
setting; the grasp planning model learns to effect change only through
inadequate demonstrations. We report results in Fig. 6 and Table I. In the
figure, models trained with MLGSL are evaluated with the Top-1 prediction
success rate, indicated by solid lines. Dashed lines indicate performances of
models trained with ImgMSE.

From these results, we see that MLGSL is capable of learning to grasp with 2
labels per image, achieving prediction success rates at $81.6\%$ for Top-1 and
$90.3\%$ for Top-5, which is similar to models trained with $16$ labels. The
third row in Fig. 5 shows predicted $\hat{G}$ by models trained with 2 labels
per image. We also notice that ImgMSE under-performs MLGSL in such settings.
This is attributed that the label generation method used by ImgMSE generate
false-negatives. Padded $\bar{G}$ may mistakenly label high-quality grasps to
negatives since they are not close to existing success labels. For MLGSL,
unlabeled pixels are regulated indirectly with the probabilistic objective.
This procedure minimizes assumptions toward unlabeled areas, thus does not
suffer from false-negatives.

####  IV-C3 Less Training Samples

We next train models with a smaller dataset (1k data) using MLGSL and ImgMSE.
The former makes models converge to $69.8\%$, while the latter converges at
$60.2\%$. The results suggest that less training data makes it harder to learn
grasping strategies for both methods. However, we still observe our MLGSL
outperforms ImgMSE by about $10\%$. We visualize prediction results at the
bottom in Fig. 5. Compare to ImgMSE, MLGSL predicts more reasonable grasp
distributions with less training data. The less accurate predictions from
ImgMSE might due to false training labels. Models may require more data to
compensate for wrong labels; thus, we observe better prediction results in
previous experiments with more training data.

####  IV-C4 Attention Module Integration

Besides loss designs, we compare the effectiveness of attention modules by
adding SAM to downsampling layers, upsampling layers, all layers, and no
layers. Interestingly, results suggest that the attention module is not a
contributor to the dense grasp planning. All architectures converge to
$81.6\pm 1.2\%$. This could be because the backbone FCN is simple, which do
not allow SAMs to take effect. Furthermore, as suggested in [36, 37],
attentions might have a similar effect to quality map $\hat{Q}$ in our models.

####  IV-C5 Collision-Free Datasets

Networks in [15, 16] are trained with single object datasets and directly
deployed to the clutter. However, we observe collisions during grasp execution
in practice. To improve this, we train and test models with collision-free
cluttered datasets. The results show that the collision-free ratio improves to
$84.2\%$ when training with the collision-free dataset compared to $67.3\%$
with the single object dataset. Although we observe a $5\%$ down on prediction
success rate, models trained with proposed datasets improve collision
detection ability by $25\%$.

##  V Real-World Experiments

Figure 7: (a) shows experimental setups. (b) shows objects used for
single/cluttered grasping experiments.

Trained models were run on a laptop with GTX1060 GPU and 2.5GHz CPU.
Experimental setups are shown in Fig. 7(a). An Ensenso N35 camera was used to
capture depth images. Invalid depth values were inpainted using OpenCV [43].
Robots executed the grasp with an offset $\epsilon=1$cm from the selected
grasp point along the camera’s z-axis.

17 household and 1 adversarial objects were selected to test the models (Fig.
7(b)). Household objects contain items of varying sizes and shapes. Most of
the items (staple, tape, cube, robots, sprayers, glue stick) appear in
previous works. We used several additional objects that are deformable (cable)
and perceptually challenging (thin edges on the cup, helmet, board eraser,
scissor, and reflective zinc container). We also added an adversarial object
[2] to verify models’ robustness with a complex geometry.

TABLE II: Real-World performance of MLGSL and Baselines

Method | Required Data | SR (%) | CTPG (ms)
---|---|---|---
GGCNN [15] | 10k $\times$ 16 | 83.3 (30/36) | 21
GGCNN2 [16] | 10k $\times$ 16 | 88.9 (48/54) | 21
GQCNN [2] | 7m $\times$ 1 | 91.7 (33/36) | 570
FC-GQCNN [19] | 7m $\times$ 1 | 90.7 (49/54) | 29
CGPN [13] | 30k $\times$ 20 | 85.2 (46/54) | 410
MLGSL (Ours) | 10k $\times$ 2 | 90.7 (98/108) | 21

Require data represents the size of training datasets (number of images
$\times$ number of labels per image); SR and CTPG represent grasp success rate
and computation time per grasp respectively.

We performed physical grasping experiments with two arrangements: 1) isolated
single objects and 2) cluttered objects. For single object grasping, we
trained a model with MLGSL, and a dataset consists of 2 labels per image. We
compared our approach with five baselines [15, 16, 2, 19, 13]. Results are
reported in Table II. Each object was grasped for the same time. Note that all
baselines require larger well-labeled datasets to achieve listed performances.

For cluttered scenarios, objects were randomly placed inside the workspace.
The robot attempted one grasp each time, and the grasped object was removed
from the scene. The picking order is greedily determined to maximize the grasp
success rate of overall objects and avoid collisions. This procedure continues
until all objects are removed or consecutively failed five times. We ran this
experiment 10 times to measure performances. Models trained with MLGSL and
cluttered datasets achieved an object removal rate of 90%, compared to 70% in
models trained with single datasets, mainly due to undetected collisions.
Comparing to [16], we observed similar results that models trained with
cluttered datasets outperform that with single datasets.

Figure 8: Two failure cases. (a) shows the object slippage when the robot
grasps heavy objects, and (b) shows models mistakenly generate grasps toward
deformable thin covers.

Figure 8 displays two common failures of MLGSL. One failure mode occurs when
the object is heavy. The current method assumes a fixed contact force, and
heavy objects can slip without a grasp force controller. The second type of
failure occurs when a thin deformable layer is on top of the object’s main
body. It is challenging to distinguish thin layers from solid cubes in depth
images. Such ambiguity tricks the model into generating unstable grasps.

##  VI Discussion and Conclusion

This paper tackles the data sparsity issue in grasp planning with several key
contributions. First, we propose a stochastic process to select grasps with
dense grasp planners. Second, we present the MLGSL to train FCNs with a small
empirical dataset; we show that it can match the performance of the state-of-
the-art methods with fewer training data. Third, we show that the attention
mechanisms are not contributing to the dense grasp planning. Lastly, we
provide a grasping dataset to improve the performance of networks in clutter.

The present work also has limitations. This paper only shows experiments on
low-DoF grasping tasks. However, the proposed MLGSL does not constrain grasp
dimensions and thus can be generalized to high-DoF grasps. Validating MLGSL in
high-DoF tasks will be our future works. Moreover, collision-free is not
guaranteed with the proposed dataset; a collision-pruning module is still
required in practice.

## References

  * [1] Webstite for paper Learn to Grasp with Less Supervision using MLGSL, https://rolandzhu.github.io/MLGSL/.
  * [2] J. Mahler and et al., “Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics,” _Robotics: Science and Systems (RSS)_ , 2017.
  * [3] J. Mahler and et al., “Dex-net 3.0: Computing robust vacuum suction grasp targets in point clouds using a new analytic model and deep learning,” _2018 ICRA_ , pp. 1–8, 2018.
  * [4] A. Mousavian, C. Eppner, and D. Fox, “6-dof graspnet: Variational grasp generation for object manipulation,” in _International Conference on Computer Vision (ICCV)_ , 2019.
  * [5] A. Murali, A. Mousavian, C. Eppner, C. Paxton, and D. Fox, “6-dof grasping for target-driven object manipulation in clutter,” _2020 IEEE International Conference on Robotics and Automation (ICRA)_ , pp. 6232–6238, 2020.
  * [6] A. ten Pas, M. Gualtieri, K. Saenko, and R. Platt, “Grasp pose detection in point clouds,” _The International Journal of Robotics Research_ , vol. 36, no. 13-14, pp. 1455–1473, 2017.
  * [7] Y. Fan, X. Zhu, and M. Tomizuka, “optimization model for planning precision grasps with multi-fingered hands,” in _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2019, pp. 1548–1554.
  * [8] Y. Fan, T. Tang, H.-C. Lin, and M. Tomizuka, “Real-time grasp planning for multi-fingered hands by finger splitting,” in _2018 IROS_. IEEE, 2018, pp. 4045–4052.
  * [9] Y. Fan, H.-C. Lin, T. Tang, and M. Tomizuka, “Grasp planning for customized grippers by iterative surface fitting,” in _2018 IEEE 14th International Conference on Automation Science and Engineering (CASE)_. IEEE, 2018, pp. 28–34.
  * [10] H. Liang and et al., “Pointnetgpd: Detecting grasp configurations from point sets,” _2019 ICRA_.
  * [11] Q. Lu and T. Hermans, “Modeling grasp type improves learning-based grasp planning,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 784–791, 2019.
  * [12] L. Shao, F. Ferreira, M. Jorda, V. Nambiar, J. Luo, E. Solowjow, J. A. Ojea, O. Khatib, and J. Bohg, “Unigrasp: Learning a unified model to grasp with multifingered robotic hands,” _IEEE Robotics and Automation Letters_ , vol. 5, p. 2286–2293.
  * [13] X. Zhu, L. Sun, Y. Fan, and M. Tomizuka, “6-dof contrastive grasp proposal network,” _2021 IEEE International Conference on Robotics and Automation (ICRA)_.
  * [14] P. Ni, W. Zhang, X. Zhu, and Q. Cao, “Pointnet++ grasping: Learning an end-to-end spatial grasp generation algorithm from sparse point clouds,” _2020 ICRA_ , pp. 3619–3625, 2020.
  * [15] D. Morrison, P. Corke, and J. Leitner, “Closing the Loop for Robotic Grasping: A Real-time, Generative Grasp Synthesis Approach,” _Robotics: Science and Systems (RSS)_ , 2018.
  * [16] D. Morrison, P. Corke, and J. Leitner, “Learning robust, real-time, reactive robotic grasping,” _The International Journal of Robotics Research_ , vol. 39, no. 2-3.
  * [17] A. Zeng and et al., “Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching,” in _2018 ICRA_ , 2018, pp. 3750–3757.
  * [18] M. Breyer, J. J. Chung, L. Ott, S. Roland, and N. Juan, “Volumetric grasping network: Real-time 6 dof grasp detection in clutter,” in _Conference on Robot Learning_ , 2020.
  * [19] V. Satish, J. Mahler, and K. Goldberg, “On-policy dataset synthesis for learning robot grasping policies using fully convolutional deep networks,” _IEEE Robotics and Automation Letters_ , 2019.
  * [20] J. Redmon and A. Angelova, “Real-time grasp detection using convolutional neural networks,” in _2015 ICRA_ , 2015, pp. 1316–1322.
  * [21] J. Varley, J. Weisz, J. Weiss, and P. Allen, “Generating multi-fingered robotic grasps via deep learning,” in _2015 IROS_ , 2015, pp. 4415–4420.
  * [22] A. Depierre, E. Dellandréa, and L. Chen, “Jacquard: A large scale dataset for robotic grasp detection,” in _2018 IROS_ , 2018, pp. 3511–3516.
  * [23] I. Lenz, H. Lee, and A. Saxena, “Deep learning for detecting robotic grasps,” _The International Journal of Robotics Research_ , vol. 34, no. 4-5, pp. 705–724.
  * [24] A. Zeng, S. Song, S. Welker, J. Lee, A. Rodriguez, and T. Funkhouser, “Learning synergies between pushing and grasping with self-supervised deep reinforcement learning,” in _IROS_ , 2018.
  * [25] M. Roa and R. Suarez, “Grasp quality measures: Review and performance,” _Autonomous Robots_ , vol. 38, pp. 65–88, 07 2014.
  * [26] N. Shafii, S. H. Kasaei, and L. S. Lopes, “Learning to grasp familiar objects using object view recognition and template matching,” in _2016 IROS_ , 2016, pp. 2895–2900.
  * [27] R. M. Murray, S. S. Sastry, and L. Zexiang, _A Mathematical Introduction to Robotic Manipulation_. CRC Press, Inc., 1994.
  * [28] H. Zhang, X. Lan, S. Bai, X. Zhou, Z. Tian, and N. Zheng, “Roi-based robotic grasp detection for object overlapping scenes,” in _2019 IROS_ , 2019, pp. 4768–4775.
  * [29] S. Jiang, X. Zhao, Z. Cai, K. Xiang, and Z. Ju, “Single-grasp detection based on rotational region cnn,” in _Advances in Computational Intelligence Systems_. Cham: Springer International Publishing, 2020, pp. 131–141.
  * [30] Z. Luo, B. Tang, S. Jiang, M. Pang, and K. Xiang, “Grasp detection based on faster region cnn,” in _2020 5th International Conference on Advanced Robotics and Mechatronics (ICARM)_ , 2020, pp. 323–328.
  * [31] F.-J. Chu, R. Xu, and P. A. Vela, “Real-world multiobject, multigrasp detection,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 3355–3362, 2018.
  * [32] C. Bodnar, A. Li, K. Hausman, P. Pastor, and M. Kalakrishnan, “Quantile qt-opt for risk-aware vision-based robotic grasping,” in _Proceedings of Robotics: Science and Systems_ , Corvalis, Oregon, USA, July 2020.
  * [33] L. C. et al., “Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE Computer Society, 2017, pp. 6298–6306.
  * [34] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in _European Conference on Computer Vision (ECCV)_ , September 2018.
  * [35] O. Ulutan, A. S. M. Iftekhar, and B. S. Manjunath, “Vsgnet: Spatial attention network for detecting human object interactions using graph convolutions,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * [36] M. Gualtieri and R. Platt, “Learning manipulation skills via hierarchical spatial attention,” _IEEE Transactions on Robotics_ , vol. 36, no. 4, p. 1067–1078.
  * [37] M. Gualtieri and R. Platt, “Learning 6-dof grasping and pick-place using attention focus,” in _Conference on Robot Learning_ , vol. 87, 2018, pp. 477–486.
  * [38] B. Wu, I. Akinola, and P. K. Allen, “Pixel-attentive policy gradient for multi-fingered grasping in cluttered scenes,” in _IROS_ , 2019.
  * [39] B. Wu, I. Akinola, A. Gupta, F. Xu, J. Varley, D. Watkins-Valls, and P. Allen, “Generative attention learning: a “general” framework for high-performance multi-fingered grasping in clutter,” _Autonomous Robots_ , vol. 44, 07 2020.
  * [40] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _2017 IEEE International Conference on Computer Vision (ICCV)_ , 2017, pp. 2242–2251.
  * [41] J. Tsitsiklis and B. Van Roy, “An analysis of temporal-difference learning with function approximation,” _IEEE Transactions on Automatic Control_ , vol. 42, no. 5, pp. 674–690, 1997.
  * [42] C. Szepesvari, _Algorithms for Reinforcement Learning_. Morgan and Claypool Publishers, 2010.
  * [43] G. Bradski, “The OpenCV Library,” _Dr. Dobb’s Journal of Software Tools_ , 2000.


# Modelling of impurity heating during reconnections in a Reverse Field Pinch
device as due to parallel electric field acceleration and chaos-induced
thermalization

F. Sattin<EMAIL_ADDRESS>Consorzio RFX (CNR, ENEA, INFN, Università
di Padova, Acciaierie Venete SpA), Padova, Italy  D.F. Escande  Aix-Marseille
Université, CNRS, PIIM, UMR 7345, Marseille, France  M. Gobbin, I. Predebon
CNR ISTP, Istituto per la Scienza e Tecnologia dei Plasmi, Sede di Padova,
Italy; Consorzio RFX (CNR, ENEA, INFN, Università di Padova, Acciaierie Venete
SpA), Padova, Italy.

###### Abstract

The ion temperature during magnetic reconnections measured along the direction
of the magnetic field at the MST Reverse Field Pinch has not yet received a
satisfactory theoretical explanation. In this work we argue that it is
consistent with a picture of ion energization by the parallel electric fields
generated by the plasma during reconnection, and thermalization due to the
chaotic ion dynamics. Three possible sources of randomness are pointed out:
particle motion along static stochastic magnetic field lines, breakup of the
adiabatic ion dynamics caused by the strong gradient nonlinearity of the
magnetic field, parallel acceleration along stochastic time-dependent electric
fields. The three mechanisms are likely operative simultaneously; regardless
the specific mechanism active, the correct scaling laws with ion parameters
are recovered. Furthermore, we argue that quantitative agreement can be
obtained by feeding the model with realistic values for the plasma conditions.

##  I Introduction

Reconnection of magnetic field lines is a basic process, ubiquitous wherever
magnetized plasmas are present. During reconnection, a fraction of the energy
accumulated in the magnetic field is delivered to the particle component of
the plasma, ions and electrons. The ways by which energy is transferred are
potentially several, a detailed assessment of the energy budget is an area of
active investigation, both in astrophysical and laboratory plasmas; it is
usually a difficult task, possible only in a few well-controlled environments
(see, e.g., [1]); furthermore, it is likely that different mechanisms are
active among different physical scenarios.
The Reversed Field Pinch (RFP) is a toroidal device designed for fusion plasma
research, characterized by a magnetic field structure that is mostly self-
organized [2]. The sustainment of quasi-steady-state conditions requires the
generation of toroidal flux by the plasma itself, which occurs via continuous
and quasiperiodic reconnection events. Accordingly, the physics of
reconnection is a particularly important topic in these devices, from the
theory as well as the experimental side [2, 3, 4].
This work addresses the extensive campaigns of experiments carried out in the
period 2008-2014 in the MST RFP device and summarized in a long list of works
[5, 6, 7, 8, 9, 10, 11]. These papers provide a detailed characterization of
both the particle and field components of the plasma during reconnections,
including thermal and magnetic energy budget, time- and space-resolved
measurements of the temperature along the directions both aligned and
perpendicular to the magnetic field, scan over several ion species, frequency-
and wavelength-resolved power spectra of magnetic fluctuations. The data
allowed a comparison with the predictions from several of the most popular
models of ion heating, as well as some models developed ad-hoc. None of them
was found able to satisfactorily match all of the experimental evidence.
Therefore, the mechanism behind ion heating in this kind of devices is so far
unexplained. In paper [10] it was argued, in particular, that a complete
explanation of the ion heating likely requires two separate mechanisms,
delivering energy to the particles respectively along and perpendicularly to
the magnetic field.
In this work we propose a plausible explanation for the findings of the MST
team focussing on the interpretation of ion heating along the direction
parallel to the magnetic field. We show that it is quantitatively consistent
with the predictions of a model of ion energization due to parallel electric
fields coupled to some non-collisional thermalization mechanism. We identify
three different possible origins for the electric field and the thermalization
mechanism, likely co-existing. Very good agreement with the experiment may be
obtained both in quantitative values and in terms of scaling laws as far as
the ion energization along the direction parallel to the magnetic field is
concerned. The structure of the paper is the following: next section provides
a brief overview of the fundamental facts concerning ion heating in MST during
reconnections; section III describes the model and suggests a fit of the
experimental data on the basis of its predicted scaling. Some considerations
about the reasonableness of the fitting coefficients are made. Concluding
remarks are provided in section IV, where some open questions are also
discussed, among them the issue of the energization perpendicular to the
magnetic field, and the coupling between the parallel and perpendicular
degrees of freedom.

##  II A brief overview of the MST experimental results

In this section, we summarize information collected by three papers. First of
all, Fiksel et al. [7] monitored the behavior of plasma discharges with
different main ions: $H,D,He$. Measurements were performed using Rutheford
scattering, which provides information about the energy distribution along a
direction mainly perpendicular to the magnetic field.
Then, Kumar et al. [10] monitored the temperature of several charge states of
different impurities with high time resolution in standard RFP Deuterium
discharges using line-averaged spectroscopic methods. Measurements refer to
line emissions from edge impurities
$Al^{1+,2+},C^{2+,4+},N^{2+,3+},O^{2+,3+,4+}$ since the impact parameter of
the lines of sight is about 90% of the minor radius. Because of the
experimental arrangement and of the radial position of the emitting species,
the temperature measurement corresponds to the observation of Doppler
broadening of spectral lines along a direction that is well aligned with the
local direction of the magnetic field, hence the measurements refer to the
parallel temperature.
Thirdly, we recall the spectroscopic measurements of the parallel and
perpendicular temperature of $C^{6+}$ impurity ions by Magee et al. which show
comparable heating for both directions, with a mild excess of perpendicular
heating [8].
We summarize the main findings of these three works as follows: (i) ion
heating is extremely fast, it occurs simultaneously with reconnection over a
time interval of roughly 50 $\mu$s or less. For comparison, the cooling phase
is slower, by a threefold factor at least. (ii) Expressed in terms of power,
the heating phase entails an absorption of a few MeV/s per ion. (iii) The
final temperature increment ranges between a few tens to some hundreds of
eV’s. (iv) For one and the same ion, the levels of heating in the direction
parallel and perpendicular to $\bf B$ are comparable, with a moderate
predominance of perpendicular heating. (v) The scalings with the ion
properties are quite different, though. Perpendicular heating is strongly
correlated with the ion mass $M$: the increase in temperature scales roughly
with $M^{0.5}$ whereas no appreciable dependence from the ion charge $Q$ is
found. Conversely, parallel heating depends on both $Q$ and $M$ following a
scaling like $Q^{\gamma}/M$, with $\gamma$ close to the unity.

##  III Modelling of the parallel ion heating

Any interpretation of the experimental data must account for the list of
results (i-v).
The full heating process may be modeled formally through two coupled rate
equations for the parallel and perpendicular temperatures
$T_{\parallel,\perp}$ (see Eqns. 1,2 of [10]) which include separate heat
sources along the two directions, isotropization terms which mix the parallel
and perpendicular direction, and loss terms towards the background plasma.
Kumar et al. made a qualitative study of the equations and showed that
including just one heat source is sufficient to raise the temperature along
both directions, thanks to the isotropization terms. The correct scaling along
one direction may be achieved, provided that the source has the required
functional dependence upon ion parameters, but not along the two directions
simultaneously.
Conceptually, the present work is an advancement of the study done by Kumar et
al. Consistently with their claims, we drop any attempt of describing the
heating process in its wholeness, and limit to consider heating along the
parallel direction alone. Specifically, we will address, out of the previous
list, only the points (i,ii,iii) and partially (v). The qualifying aspect of
the work is the choice of the terms to be fed into the rate equation: the
energy source and the isotropization mechanism. Concerning the first, it is
well known that electric fields parallel to the magnetic one develop during
reconnections [12, 13, 14, 15], thus one natural mechanism for particle
energization is provided by acceleration along these fields. Some
thermalization mechanism must be invoked as well, since otherwise all
identical particles would be accelerated by the same amount and this does not
produce a broad energy distribution. Mechanisms based upon binary collisions
seem too slow to account for the measurements. For instance, according to ref.
[16] the collision frequency of a $C^{4+}$ ion (which is a best-case example,
given its high charge–to–mass ratio) in a $O(100)\,eV,O(10^{19})\,m^{-3}$
Deuterium background is few tens of kHz whereas, to be compliant with the
experiment, frequencies close to 100 kHz are needed. Indeed, the very Figure 5
of ref. [10], produced using collisional estimates, shows plasma dynamics
evolving over time scales slower than the experimental ones. The discrepancy
is not of orders of magnitude, yet is large enough to be noticeable, hence a
non-collisional thermalization mechanism must be invoked. In this work we will
review three potential candidate mechanisms. In the first one, thermalization
is provided by some weakly chaotic stationary magnetic field.We argue that
this mechanism could be the most likely operative one, since a lot of magnetic
chaos is actually identified during reconnections, see Fig. 10 in [4]. The
second mechanism is popular mostly among the astrophysics community. It has
been recognized for a long time that in the neighborhood of the magnetic
neutral point or, more generally, of a local minimum of a spatially
inhomogeneous magnetic field, even though the field itself is smooth, the
orbital motion of charged particles may become chaotic due to the strong
gradient nonlinearity [17, 18, 19, 20, 21, 22] as soon as the ratio between
the particle Larmor radius and the typical length scale for the variation of
$B$ takes locally values of the order of unity. The two mechanisms are not
necessarily alternative, and may coexist: recently, Lemoine [23] argued that
fully developed MHD turbulence produces a spectrum of magnetic structures
whose sizes span all the way down to the ion Larmor radius. It produces
simultaneously, therefore, both magnetic chaos and breakup of the particle’s
adiabatic dynamics. Thirdly, we acknowledge that reconnections are actually
dynamic process, involving a broadband spectrum of magnetic activity.
Fluctuating electric fields–with a component parallel to the equilibrium
magnetic field–are thus inductively generated.

###  III.1 Particle heating in chaotic static magnetic fields

Let us write the magnetic and electric fields as ${\bf B}=(b_{x},0,B),{\bf
E}=(0,0,E)$. Both the guide field $B$ and the electric field $E$ are taken
constant in space and time, whereas $|b_{x}|\ll B$ is chaotically varying in
space.
The projection of the particle equation of motion along $z$ is

${dv_{z}\over dt}={Q\over M}\left(E-b_{x}v_{y}\right)$ (1)

and the drift along $y$ is $v_{y}=Eb_{x}/B$, thus

${dv_{z}\over dt}={Q\over M}E\left(1-{b_{x}^{2}\over B^{2}}\right)\to
v_{z}=v_{z}(0)+{Q\over M}E\int dt\left(1-{b_{x}^{2}\over B^{2}}\right)$ (2)

In the previous equation the quantity $f=\int dt\left(1-(b_{x}/B)^{2}\right)$
appears. It is a stochastic variable, since the term $b_{x}=b_{x}(y(t),z(t))$
that appears inside the integral depends on the trajectory.
The temperature of an ensemble of particles is proportional to the variance of
the velocity distribution:

$T={1\over 2}M\langle(v_{z}-\langle v_{z}\rangle)^{2}\rangle={1\over
2}M\left(\langle v_{z}^{2}\rangle-\langle v_{z}\rangle^{2}\right)$ (3)

Eventually, the temperature increment is (assuming $\langle
v_{z}(0)\rangle=0$):

$\Delta T=T(t)-T(0)={Q^{2}\over 2M}E^{2}\left(\langle f^{2}\rangle-\langle
f\rangle^{2}\right)$ (4)

If $b_{x}$ were approximately constant along the particle motion, $\langle
f^{2}\rangle\approx\langle f\rangle^{2}$, and no heating would take place.
In order to assess whether Eq. 4 may be quantitatively consistent with the
data, we write $Q=Z\,e,M=\mu\,m_{p},\left(\langle f^{2}\rangle-\langle
f\rangle^{2}\right)=k^{2}t^{2}$, with $e,m_{p}$ the proton charge and mass and
$k<1$. If we replace for $t$ the typical duration of reconnection, say
$t=5\times 10^{-5}$s, we obtain

$\Delta T\,(eV)={Z^{2}\over\mu}0.25\times(Ek)^{2}$ (5)

requiring $Ek$ to be at least $O(10^{2})$ V/m in order to get $\Delta T\approx
O(10)eV$ (the ratio $Z^{2}/\mu$ varies in the range $\equiv 0.03\div 1$ for
the ions studied in MST). The estimates we have for for $E$ vary from
$O(10^{2})$ V/m (see Fig. 2b in [15]) to $O(10^{3})$ V/m [7].

###  III.2 Heating in static electric fields and non-uniform smooth magnetic
fields

In the neighborhood of a local minimum of a spatially inhomogeneous magnetic
field, the orbital motion of charged particles may become chaotic when the
ratio between the particle Larmor radius and the typical length scale for the
variation of $B$ takes locally values of the order of unity. This result was
employed to justify particle heating by very-low-frequency waves [24], poor
confinement of fast ions in spherical tokamaks [25, 26, 27] or of charged
particles in the magnetosphere [28, 29, 30, 31]. Mason and Rusbridge [32] and,
later on, Yoshida et al. [33, 34, 35], pointed out that the chaotic motion of
charged particles may be equivalent to the randomization process of the
directional motion of the current-carrying particles induced by an effective
collisionality. Numerical simulations [36] show that this anomalous
collisionality is much larger than the Spitzer one by several orders of
magnitude in low-density astrophysical plasmas, and easily by a ten- to
hundredfold factor in laboratory ones, potentially providing the amplitude
required.
Formally, all results are an exact replica of the previous subsection: Chaos
mixes parallel and perpendicular motion; along the direction aligned with the
electric field (say, $z$) particles still experience an accelerated motion but
with an effective average smaller acceleration, just like in Eq.2:

$v_{z}\approx{Q\over M}\alpha E\,t$ (6)

Here, we have employed $\alpha<1$ rather than $k$ just to emphasize its
different physical origin. Furthermore, the motion along the perpendicular
direction becomes diffusive [37].

###  III.3 Heating in fluctuating electric fields

Magnetic activity has a broadband distribution both in frequency (see fig. 1
in [9] and fig. 4 in [10]) and wavenumbers (fig. 2 in [9]). Let us consider
the two model vector potentials

${\bf A}_{0}=B_{0}x\hat{y},\quad{\bf A}_{1}=b_{w}/k\cos(kx-\omega t)\hat{z}$
(7)

The magnetic fields ${\bf B}_{0}=\nabla\times{\bf A}_{0},{\bf
B}_{1}=\nabla\times{\bf A}_{1}$ stand respectively for the homogeneous
equilibrium field, and a generic Fourier component picked up from the whole
magnetic spectrum. The term ${\bf A}_{1}$ stands for a transversal magnetic
wave propagating along a direction perpendicular to the equilibrium field. It
does not cause loss of generality; furthermore, it is consistent with the
experimental fact that the spectral density is centered close to
$k_{\parallel}\approx 0$. An electric field ${\bf E}_{1}$ is produced

${\bf E}_{1}=-{\partial{\bf A}_{1}\over\partial t}=-{\omega b_{w}\over
k}\sin(kx-\omega t)\hat{z}$ (8)

Hence, the particle experiences an acceleration along the parallel direction,
effectively aligned with the z-axis since we postulate $b_{w}\ll B_{0}$:

$M{dv_{z}\over dt}\approx QE_{1}\to v_{z}(t)=v_{z}(0)-{Q\over M}{\omega
b_{w}\over k}\int dt\sin(kx-\omega t)$ (9)

The argument then proceeds like in subsection III.1, if one supposes that the
term $\sin(kx-\omega t)$ behaves like a stochastic variable, since $x$ depends
on the specific trajectory [38]. This is likely for a full turbulent spectrum
of magnetic fluctuations–which is found experimentally. We remark, however,
that in principle chaos may take place even with a single wave.

###  III.4 Fitting of the experimental data

In conclusion, regardless of the precise mixture of mechanisms active we
expect a scaling like

$\Delta T=c\,(Z^{2}/\mu)\;eV.$ (10)

This scaling is generic, and arises whenever there is acceleration by given
fields.
In this subsection we look at the original data presented in [10] and argue
that: (i) even if the discussion therein suggested $Z/\mu$ as scaling, the
present $Z^{2}/\mu$ proposal performs at least as well, when employed to fit
the data. (ii) The absolute values matter, too. In order to match experiments,
we expect $c\approx O(10^{2})eV$, and we have argued in subsection III.1 that
this estimate does not seem unrealistic on the basis of the known plasma
conditions, at least for the first mechanism proposed.
We digitally acquired the data from Fig. 3 of [10] and made a best fit using
Eq.10 . The result is shown in Fig. 1 in terms of the original scaling
$Z/\mu$. The agreement appears excellent, the fitting parameter is $c=140$ eV
which perfectly matches our order-of-magnitude estimates.

Figure 1: Parallel temperature increase versus $Z/\mu$. Blue circles with
error bars: data acquired from Fig. 3 of [10]; orange squares, best fit using
the present model (Eq. 10), with $c=140$ eV.

##  IV Summary and concluding considerations

We have shown that a model of particle energization by parallel electric
fields coupled with chaos-induced thermalization is compatible with MST data
of impurity parallel heating during reconnections. We obtained satisfactory
results both in terms of absolute values and in terms of scaling relations.
The model involves just a few free parameters, whose best-fit numerical values
turn out to be fairly reasonable.
The difference between our modeling and Kumar’s is in the choice of the
ingredients: the thermalization mechanism and the energy source. We replace
their collisional thermalization with the chaos-induced one since argue that
the former cannot quantitatively account for the data. Furthermore, the
existence of parallel electric fields is quite well established, hence it
appears natural to take them into account. In the present work, we have
considered just the parallel heating source, nonetheless, it is unavoidable
that chaos does not just thermalize the parallel velocity distribution, it
also couples parallel motion with the perpendicular one. This entails that a
consistent interpretation of ion heating must involve a comprehensive picture,
where both parallel and perpendicular heating are included and treated
simultaneously, with feedback from one term to the other. This is not a simple
exercise, which we do not attempt to carry out here, and is further
complicated by the fact, as we have argued earlier, that presently no
candidate mechanism for perpendicular heating is fully convincing. Stochastic
heating by low-frequency electromagnetic turbulence might be a promising
candidate, except that the level of turbulence in RFP devices is definitely
too small, even though some advancements have recently been made [39].

## AUTHOR DECLARATIONS

### Conflict of Interest

The authors have no conflicts to disclose.

### Author Contributions

F. Sattin: Conceptualization (lead); Formal analysis (lead); Investigation
(lead); Validation (lead); Writing – original draft (lead); Writing – review &
editing (equal). D.F. Escande, I. Predebon, M. Gobbin: Conceptualization
(supporting); Formal analysis (supporting); Investigation (supporting);
Validation (supporting); Writing – review & editing (equal).

### DATA AVAILABILITY

The data that support the findings of this study are available within the
article and the articles listed in the references section below

## References

  * [1] S.C. Hsu, G. Fiksel, T.A. Carter, H. Ji, R.M. Kulsrud and M. Yamada, Phys. Rev. Lett. 84, 3859 (2000)
  * [2] L. Marrelli, P. Martin, M.E. Puiatti, J.S. Sarff, B.E. Chapman, J.R. Drake, D.F. Escande, and S. Masamune, Nucl. Fusion 61, 023001 (2021)
  * [3] B. Momo, et al, Nucl. Fus. 60, 056023 (2020)
  * [4] M. Gobbin et al, Nucl. Fusion 62, 026030 (2022)
  * [5] S. Gangadhara, D. Craig, D.A. Ennis, D.J. Den Hartog, G. Fiksel, and S.C. Prager, Phys. Plasmas 15, 056121 (2008)
  * [6] V. Tangri, P.W. Terry, and G. Fiksel, Phys. Plasmas 15, 112501 (2008)
  * [7] G. Fiksel, A.F. Almagri, B.E. Chapman, V.V. Mirnov, Y. Ren, J.S. Sarff, and P.W. Terry, Phys. Rev. Lett. 103, 145002 (2009)
  * [8] R.M. Magee, et al, Phys. Rev. Lett. 107, 065005 (2011)
  * [9] Y. Ren, A.F. Almagri, G. Fiksel, S.C. Prager, J.S. Sarff, and P.W. Terry, Phys. Rev. Lett. 107, 195002 (2011)
  * [10] S.T.A. Kumar, et al, Phys Plasmas 20, 056501 (2013)
  * [11] M.S. Cartolano, D. Craig, D.J. Den Hartog, S.T.A. Kumar, and M.D. Nornberg, Phys. Plasmas 21, 012510 (2014)
  * [12] J. Egedal,W. Daughton, A. Le, Nature Physics 8, 321 (2012)
  * [13] R.E. Ergun et al, Phys. Rev. Lett. 116, 235102 (2016)
  * [14] J.K. Anderson et al, AIP Conf. Proc. 1771, 040001 (2016)
  * [15] J.K. Anderson et al, Phys. Plasmas 23, 055702 (2016)
  * [16] A.S. Richardson, NRL Plasma formulary (2019)
  * [17] A. Garren et al, Proceedings of the Second UN International Conference on the Peaceful uses of Atomic Energy, paper P/383 (1958)
  * [18] R.J. Hastie et al, Proc. 3rd Int. Conf. on Plasma Physics and Controlled Nuclear Fusion Research, (Novosibirsk, 1968) vol 1, (Vienna, IAEA), p 389
  * [19] J. E. Howard, Phys. Fluids 14, 2378 (1971)
  * [20] J.H. Foote, Plasma Physics 14, 545 (1972)
  * [21] A.V. Timofeev, Sov. Phys. JETP 48, 656 (1978)
  * [22] R.F. Martin Jr. and H. Matsuoka, Phys Rev E 47, 721 (1993)
  * [23] M. Lemoine, J. Plasma Phys. 89, 175890501 (2023)
  * [24] D.F. Escande, V. Gondret, F. Sattin, Sci. Rep. 9,14274 (2019)
  * [25] J. Carlsson, Phys. Plasmas 8, 4725 (2001)
  * [26] V.A. Yavorskij et al, Nucl. Fusion 42, 1210 (2002)
  * [27] D.F. Escande, F. Sattin, Nucl. Fusion 61, 106025 (2021)
  * [28] R.F. Martin Jr, Jour. Geophys. Res. 91, 11985 (1986)
  * [29] J. Buchner and L.M. Zelenyi, Jour. Geophys. Res. 94, 11821 (1989)
  * [30] S.L. Young, R.E. Denton, B.J. Anderson and M.K. Hudson, Jour. Geophys. Res. 107, 1069 (2002)
  * [31] R. Liu, S. Liu, F. Zhu, Q. Chen, U. He and C. Cai, Chaos 32, 043104 (2022)
  * [32] V.A. Mason and M.G. Rusbridge, Plasma Physics 21, 907 (1979)
  * [33] Z. Yoshida, H. Asakura, H. Kakuno, J. Morikawa, K. Takemura, S. Takizawa and T. Uchida, Phys. Rev. Lett. 81, 2458 (1998)
  * [34] R. Numata and Z. Yoshida, Phys. Rev. Lett. 88, 045003 (2002)
  * [35] R. Numata and Z. Yoshida, Phys. Rev. E 68, 016407 (2003)
  * [36] Z. Wang, De-Jin Wu, L. Chen and Yu-Fei Hao, Res. Astron. Astrophys. 21, 071 (2021)
  * [37] M. Shang, De-Jin Wu, L. Chen, Peng-Fei Chen, Res. Astron. Astrophys. 17, 3 (2017)
  * [38] B. Dasgupta, A.K. Ram, G. Li, X. Li, AIP Conf. Proc., 1582, 213 (2014)
  * [39] F. Sattin, D.F. Escande, Phys. Rev.  E 107, 065201 (2023)