# Offline-Online Learning of Deformation Model for Cable Manipulation with
Graph Neural Networks

Changhao Wang1, Yuyou Zhang2, Xiang Zhang1, Zheng Wu1, Xinghao Zhu1
Shiyu Jin1, Te Tang3, and Masayoshi Tomizuka1 1University of California,
Berkeley, CA, USA. changhaowang, xiang_zhang_98, zheng_wu, zhuxh, jsy,
<EMAIL_ADDRESS>Jiao Tong University, Shanghai, China.
<EMAIL_ADDRESS>Advance Research Laboratory, Union City, CA, USA.
<EMAIL_ADDRESS>
###### Abstract

Manipulating deformable linear objects by robots has a wide range of
applications, e.g., manufacturing and medical surgery. To complete such tasks,
an accurate dynamics model for predicting the deformation is critical for
robust control. In this work, we deal with this challenge by proposing a
hybrid offline-online method to learn the dynamics of cables in a robust and
data-efficient manner. In the offline phase, we adopt Graph Neural Network
(GNN) to learn the deformation dynamics purely from the simulation data. Then
a linear residual model is learned in real-time to bridge the sim-to-real gap.
The learned model is then utilized as the dynamics constraint of a trust
region based Model Predictive Controller (MPC) to calculate the optimal robot
movements. The online learning and MPC run in a closed-loop manner to robustly
accomplish the task. Finally, comparative results with existing methods are
provided to quantitatively show the effectiveness and robustness. Experiment
videos are available at https://msc.berkeley.edu/research/deformable-GNN.html.

###### Index Terms:

Manipulation Planning, Dual Arm Manipulation, Deformable Object Manipulation

##  I Introduction

Manipulating deformable linear objects, especially cables, has a wide range of
applications. For example, in factory manufacturing, both stator winding and
cable harnessing require cables to be assembled in a precise manner. In
medical surgery, steering needles or catheters are essential for vascular
pathology treatment and minimally invasive surgery. In order to complete the
above tasks automatically, robustly manipulating cables to the desired shape
remains a fundamental problem for robotics. However, due to the high
dimensionality and nonlinearity, it is challenging to obtain an accurate
dynamics model for precise planning and control.

The finite element method (FEM) serves as a powerful tool to model cables.
Through discretizing the objects into small elements, the deformation can be
derived by solving a set of partial differential equations. FEM can accurately
represent the dynamics of deformable objects with fine tessellation [1].
However, it is usually computationally expensive to solve, thus posing a
challenge to use it for real-time robotic manipulation tasks.

Aside from the FEM models, data-driven approaches are often utilized to learn
the non-linear deformation dynamics [2, 3, 4]. Recently, many works have
attempted to use deep neural networks (DNN) to learn such dynamics. However,
the characteristic of deformable objects is complex, and it is found
challenging for those general-purpose network structures to capture the cable
behaviors.

Recent studies on graph neural networks (GNN) bring a new structure for model
learning. By viewing particles of the object as the graph vertices, the
dynamics is modeled as the interaction between the vertices pairs [5]. The
future object state can be predicted through a sequence of ‘message passing’
blocks, which mimic the transmission of the interaction from one graph vertex
to others. Compared with those general-purpose network structures, the GNN
encodes the prior knowledge on how the interaction may transmit inside the
deformable objects. However, as most of the machine learning based approach
does, the GNN model relies on simulation for data generation. Thus, the sim-
to-real gap still poses a significant challenge for the model to be adopted in
robotic manipulation tasks.

Figure 1: The proposed cable manipulation framework combines offline model
learning with online residual learning. In the offline phase, we trained a GNN
with simulated data as the global model of the cable. In the online phase, a
local residual model that estimates the GNN prediction error is learned in
real-time to reduce the sim-to-real gap. The learned models are then sent to
an MPC controller to obtain the optimal robot motion.

To tackle the above issues, we propose a dynamics learning framework that
consists of an offline GNN model and an online residual model, as shown in
Fig. 1. In the offline phase, we learn rough global graph dynamics from the
simulation data. In the online phase, we further refine the local predictions
through a time-varying linear residual model that estimates the error of the
GNN output. In contrast with existing sim-to-real approaches, our method does
not require pre-collecting real-world data. The linear residual model can be
learned simultaneously as the robot executes. With this framework, we can
efficiently capture the global deformation behaviors without sacrificing local
accuracy. For manipulation, a trust region based MPC formulation with the
learned model is proposed to optimize the robot movement. In summary, the main
contributions are listed as follows:

  * •

Combine the GNN with an online linear residual model for robust model
learning.

  * •

Formulate a trust region based model predictive controller to optimize the
robot movements.

  * •

Demonstrate the effectiveness of the proposed method through comparative
simulations and experiments.

##  II Related Works

###  II-A Global Deformation Model Learning

Learning-based approaches approximate the cable dynamics from the collected
data set. Nair et al. [6] proposed a predictive model learned from step-by-
step images of a rope collected during human demonstration. Yan et al. [7]
proposed to utilize bi-directional LSTM to model chain-like objects. By
recurrently applying the same LSTM block along the cable, we can enforce how
the interaction will propagate inside the model. Similarly, the graph neural
network arises as a novel structure to learn the deformation dynamics. By
explicitly modeling the vertices and edges, authors of [8] demonstrated the
effectiveness of the proposed dynamic particle interaction networks (DPI-Nets)
on fluids and deformable foam manipulation tasks. The graph network structure
proposed in [5] further improved the generalizability in terms of prediction
steps and particle scalability. Moreover, Pfaff et al. [9] generalized the
graph structure to mesh-based dynamics, which proved to be effective for more
complex simulation tasks.

###  II-B Sim-to-real Transfer for Model Learning

While the neural networks can capture complex behaviors of deformable objects,
those methods typically require sim-to-real methods as complements in case of
parameter changes or uncertainties encountered in the real-world experiment.
Matching the simulator parameters with real-world parameters and directly
applying the learned model to the real world is the most common way to utilize
the model. However, the learned model may suffer from uncertainties in the
real world. Another approach is to randomize several aspects of the domain to
provide enough variability in training [10, 11, 12]. As a result, the learned
model can cover a large fraction of scenarios. Fine-tuning the learned model
with real-world data is also demonstrated to be effective in closing the sim-
to-real gap [7]. By collecting new data in the real world and re-training the
network, we can adapt the model to the new scenario robustly.

Combining the offline model with a residual model is also beneficial to
resolve the sim-to-real gap [13, 14]. Since the offline model does not need to
match reality accurately, training in simulation is more efficient and more
manageable.

###  II-C Online Local Model Estimation

Instead of learning the global model and transferring it to the real world,
online approximating a local model is another powerful way for cable
manipulation. Visual servoing is a commonly used method for online cable
manipulation. Visual servo approaches approximate the local linear deformation
model by iteratively updating the deformation Jacobian [15, 16, 17], which
represents the mapping between the robot end-effector’s velocity and the
object state. Compared with offline global model learning approaches, there
are no sim-to-real gaps, and it can generalize to different objects and
scenarios. Jin et al. [18] further improved the robustness of the online
linear model under sensor noises and occlusions. However, as the linear
model’s expressiveness is limited, it is challenging for the online methods to
manage large deformations.

###  II-D Offline-Online Learning for Robust Manipulation

Though the online model estimation methods are useful in certain tasks, it is
still desirable to take advantage of the more powerful offline global model.
As demonstrated in [2], the offline learning provides the initial guess of the
linear Jacobian matrix, and in the online phase, the Jacobian matrix is
further updated with an adaptive control law. Distinct from the sim-to-real
methods introduced in the previous subsection, this combination does not
require additional online data collection. The sim-to-real gap can be
instantaneously resolved as the robot executes. While the motivation of our
proposed framework is similar to the motivation of [2], our algorithm is
unique in several aspects: 1) we utilize GNN to capture the global model
instead of providing the initial guess of the local Jacobian, 2) we online
learn a residual model for refinement, 3) we construct an optimization-based
MPC controller to obtain optimal robot motions within a long horizon.

##  III Proposed Framework

In this section, we introduce the proposed framework for cable manipulation as
shown in Fig. 1. We mainly focus on the task of precisely shaping a cable to
the desired curvature in a 2D and uncluttered environment. A robust state
estimation algorithm for cables is introduced first. Then the cable dynamics
are approximated by a combination of an offline graph neural network (GNN) and
an online residual model. In the end, a model predictive controller (MPC) is
utilized to control the robot to manipulate the cable to desired states.

###  III-A Cable State Estimation

Estimating the state is essential for dynamics learning and manipulation. As
shown in Fig. 1, a cable is represented by a series of key points
$X(t)=[x_{1}(t),x_{2}(t),\cdots,x_{N}(t)]^{T}$, where
$x_{i}(t)\in\mathbb{R}^{1\times 2}$ denotes the 2D position of the i-th key
point at time step $t$. The objective of state estimation is to estimate the
position of each key point at each time step from the dense, noisy, and
occluded point clouds $Y(t)=[y_{1}(t),y_{2}(t),\cdots,y_{S}(t)]^{T}$ perceived
by stereo cameras.

Following our previous work on structure preserved registration (SPR) [19], we
regard the perceived point clouds $Y(t)$ as samples from a Gaussian Mixture
Model (GMM), whose centroids are the key point positions $X(t)$. According to
the Bayes’ theorem, the probability of point $y_{s}^{t}$ to be sampled from
the mixture model can be expressed by:

$\displaystyle p(y_{s}^{t})=\sum_{n=1}^{N+1}p(n)p(y_{s}^{t}|n)$ (1)

where $p(n)$ denotes the weight of the $n$-th mixture component, and
$p(y_{s}^{t}|n)$ is the corresponding probability that $y_{s}^{t}$ is sampled
from the mixture component. As shown below, we assume all Gaussians share the
same weight, and we add a uniform distribution to account for noise and
outliers.

$p(n)=\left\\{\begin{array}[]{cc}(1-\mu)\frac{1}{N},&n=1,\dots,N\\\
\mu,&n=N+1\\\ \end{array}\right.$ (2)
$p(y_{s}^{t}|n)=\left\\{\begin{array}[]{cc}\mathcal{N}(y_{s}^{t};x_{n}^{t},\sigma^{2}\mathbf{I}),&n=1,\dots,N\\\
\frac{1}{S},&n=N+1\\\ \end{array}\right.$ (3)

The problem is then converted to a Maximum Likelihood Estimation (MLE)
problem, where we want to optimize the mixture centroids in order to maximize
the log likelihood $L$ that the point cloud $Y(t)$ is sampled as shown in (5).

$\displaystyle L(x_{n}^{t},\sigma^{2}|Y^{t})$
$\displaystyle=log\prod_{s=1}^{S}p(y_{s}^{t})$ (4)
$\displaystyle=\sum_{s=1}^{S}log(\sum_{n=1}^{N+1}p(n)p(y_{s}^{t}|n))$ (5)

The minimization can be achieved via performing E-M algorithm on the log
likelihood function. Since state estimation is not the main contribution of
this paper, we omit the derivation and refer readers to [19] and [20] for
details.

###  III-B Offline Model Learning with Graph Neural Networks

Figure 2: The structure of the Graph Neural Network (GNN). The offline GNN
takes in a graph that represents the state of the cable and outputs the
prediction of the cable movements.

Graph neural networks (GNN) is demonstrated to be effective to represent
complex dynamics [5]. In this paper, we adopt a similar idea from [5] to learn
the graph dynamics of the cable.

We assume that the connection between the robot end-effector and the grasping
point is fixed throughout task executions. The dynamics $f(\cdot)$ is then
modeled by the interactions among the cable key points and the robot end-
effectors as shown in (6). The dynamics takes in $m+1$ previous cable states
from $X(t-m)$ to $X(t)$and the current robot end-effector velocities
(translations in $x$, $y$ and rotation around $z$)
$R(t)=[r_{1}(t),r_{2}(t),\cdots,r_{Q}(t)]^{T}$, where
$r_{i}(t)\in\mathbb{R}^{1\times 3}$ is the end-effector velocities of the i-th
robot, and $Q$ is the number of robots.

$\dot{X}(t)=f(X(t-m:t),R(t))$ (6)

As studied in [5], the dynamics can be captured via a graph $G=(V,E)$, where
the graph vertices $V=[v_{1},v_{2},\cdots,v_{N}]$ correspond to the key
points, and the graph edges $E$ correspond to the interactions between the key
points pair. As shown in Fig. 2, the features of the $i$-th graph vertex
$v_{i}$ consist of a sequence of previous key point positions
$[x_{i}(t),x_{i}(t-1),\cdots,x_{i}(t-m)]$, and the robot control input at that
point. For the graph edge, it models the relative movement between vertices
$e_{i,j}=\|x_{i}(t)-x_{j}(t)\|$. An edge is constructed if two vertices are
within a ‘connective radius’.

To learn such a graph, we follow the GNS network structure [5], which contains
three steps – encoding, processing, and decoding:

Figure 3: Illustration of the graph encoder and decoder. By minimizing the
reconstruction loss in (7), the encoder can project the high dimensional graph
representation to a latent space.

Encoding: Instead of operating in the high dimensional space, we project the
graph $G$ to a low dimensional representation
$G^{\prime}=(V^{\prime},E^{\prime})$ for efficiency. Graph vertices auto-
encoder and graph edges auto-encoder are trained, respectively as shown in
Fig. 3. The reconstruction error in (7) is minimized to make the network learn
the optimal latent space that can capture all the original information.
$\phi^{v}$, $\phi^{e}$ denote the encoder for the graph vertices and edges,
and $\psi^{v}$, $\psi^{e}$ are the corresponding decoders.

$\displaystyle{\phi^{v}}^{*},{\psi^{v}}^{*}$
$\displaystyle=arg\underset{\phi^{v},\psi^{v}}{\text{min}}\|v_{i}-(\phi^{v}\circ\psi^{v})(v_{i})\|^{2}$
(7) $\displaystyle{\phi^{e}}^{*},{\psi^{e}}^{*}$
$\displaystyle=arg\underset{\phi^{e},\psi^{e}}{\text{min}}\|e_{i,j}-(\phi^{e}\circ\psi^{e})(e_{i,j})\|^{2}$

Processing: The processor mimics the dynamics of the cable by computing the
interaction between vertices in the latent graph.The message passing block in
Fig. 4 is to propagate the graph vertex, and edge features based on the
current graph state and the robot control input as shown in (8).

$\displaystyle{}^{k+1}v_{i}^{\prime}$
$\displaystyle=f^{v}(^{k}v_{i}^{\prime},\sum_{j}{{}^{k}e_{i,j}^{\prime}})$ (8)
$\displaystyle{}^{k+1}e_{i,j}^{\prime}$
$\displaystyle=f^{e}(^{k}e_{i,j}^{\prime},{{}^{k}v_{i}^{\prime}},{{}^{k}v_{j}^{\prime}})$

where $f^{v}(\cdot)$ and $f^{e}(\cdot)$ are the graph vertex network and graph
edge network, and ${}^{k+1}v_{i}^{\prime}$, ${}^{k+1}e_{i,j}^{\prime}$ denote
the latent vertex/edge features after the $k$-th message passing block.

Figure 4: Illustration of the message passing block. Each message passing
block transmit the interaction from one graph vertex to others through the
graph edge.

Decoding:  As the inversion of the encoder, the decoder converts the latent
graph representation back to get the prediction of the cable movement
$\dot{X}(t)$. The weight is directly taken from the decoder part of the auto-
encoder $\psi^{v}$ and $\psi^{e}$.

###  III-C Online Residual Model Learning

The offline GNN model is able to learn the global nonlinear dynamics and
provide a rough prediction. However, due to the sim-to-real gap, the training
data from simulation may not accurately capture the dynamics of the physical
object. To overcome the sim-to-real gap and the generalization issues, we
propose to online learn a residual model to correct the local predictions. As
shown in (9), $\delta\dot{X}(t)$ is the error between the actual cable state
increment $\dot{X}(t)$ and the predicted cable state increment
$\widehat{\dot{X}}(t)$ given by the offline learned dynamics denoted as
$\widehat{f}(\cdot)$. For simplicity, we use $X$ to represent the history key
point trajectory $X(t-m:t)$. Given the history cable trajectory $X$, we assume
that the ground truth residual model is linear with respect to $R(t)$ inside
$\|R(t)\|\leq\epsilon$, where $J(t)$ denotes the local residual dynamics. For
simplicity, we omit the bias term. It can be considered without much
modification.

$\displaystyle\delta\dot{X}(t)$
$\displaystyle=\dot{X}(t)-\widehat{\dot{X}}(t)$ (9)
$\displaystyle=f(X,R(t))-\widehat{f}(X,R(t))$
$\displaystyle=R(t)J(t),\quad\|R(t)\|<\epsilon$

To estimate the residual, we online collect $\delta\dot{\textbf{X}}$ and the
corresponding robot end-effector velocities R during the task execution as
shown in (10).

$\displaystyle\delta\dot{\textbf{X}}$
$\displaystyle=\begin{bmatrix}\delta\dot{X}(t-m)&\cdots&\delta\dot{X}(t-1)&\delta\dot{X}(t)\end{bmatrix}^{T}\in\mathbb{R}^{(m+1)\times
2N}$ (10) R
$\displaystyle=\begin{bmatrix}R(t-m)&\cdots&R(t-1)&R(t)\end{bmatrix}^{T}\in\mathbb{R}^{(m+1)\times
3Q}$

Based on the collected data $\delta\dot{\textbf{X}}$ and R, we solve a
regularized least squares (ridge regression). The problem can be decomposed to
$3Q$ independent ordinary ridge regression problem and solved in parallel.
$\delta\dot{\textbf{X}}_{n}$ denotes the $n$-th column of
$\delta\dot{\textbf{X}}$, and similarly, $J_{n}(t)$ represents the $n$-th
column of $J(t)$.

$\displaystyle\widehat{J}(t)$ $\displaystyle=arg\underset{J(t)}{\text{min}}$
$\displaystyle\|\delta\dot{\textbf{X}}-\textbf{R}J(t)\|_{F}^{2}+\lambda\|J(t)\|_{F}^{2}$
(11) $\displaystyle=\sum_{n=1}^{6N}arg\underset{J_{n}(t)}{\text{min}}$
$\displaystyle\|\delta\dot{\textbf{X}_{n}}-\textbf{R}J_{n}(t)\|_{2}^{2}+\lambda\|J_{n}(t)\|_{2}^{2}$

The solution is given below, where $\lambda$ is the regularization weight, and
$I$ denotes the identity matrix.

$\widehat{J}_{n}^{*}(t)=(\delta\dot{\textbf{X}_{n}}^{T}\delta\dot{{\textbf{X}_{n}}}+\lambda
I)^{-1}\delta\dot{\textbf{X}_{n}}^{T}\textbf{R}$ (12)

###  III-D Model Predictive Control for Manipulation

To utilize the learned model for manipulation, we adopt the idea of model
predictive control (MPC).

$\displaystyle\min_{R(0:h)}$
$\displaystyle\sum_{t=1}^{h+1}\|X(t)-X_{d}\|_{2}^{2}$ (13) s.t. $\displaystyle
X(t+1)=X(t)+\dot{X}(t)\Delta t$
$\displaystyle\dot{X}(t)=\widehat{f}(X,R(t))+R(t)\widehat{J}(t)$
$\displaystyle\|R(t)\|\leq\epsilon$ $\displaystyle t=0,1,\cdots,h$

As shown in (13), the optimization variables are the robot’s end-effector
velocities within a horizon $h$, and the objective function is to minimize the
shape position error. The constraints include the learned dynamics, where
$\Delta t$ denotes the sample time of the dynamics. More importantly, we
consider movement limit constraints to ensure that the learned model (offline
and online) is in effect in the optimized region. Borrowing the idea from the
trust-region optimization, we update the trust-region size according to the
ratio $\rho$ between the actual shape error reduction $\Delta e_{actual}$ and
the predicted shape error reduction $\Delta e_{pred}$ as shown in (14). The
pseudo-code is summarized in Alg. 1, where the trust-region size will
expand/shrink if $\rho$ is greater/smaller than a threshold.

$\rho=\frac{\Delta e_{actual}}{\Delta e_{pred}}$ (14)

0: Initialize $R$, $\epsilon$

1: while $\|X(t)-X_{d}\|\geq e$ do

2: $R\leftarrow$ Solve the optimization in (13)

3: $X(t+1)\leftarrow$ Execute the robot and obtain the new cable state

4: $\rho\leftarrow$ Calculate the ratio according to (14)

5: if $\rho\geq\eta^{+}$ then

6: $\epsilon\leftarrow\tau^{+}\epsilon$

7: else if $\rho\leq\eta^{-}$ then

8: $\epsilon\leftarrow\tau^{-}\epsilon$

9: end if

110: end while

Algorithm 1 Trust Region Based Model Predictive Control

During the task execution, the MPC controller iteratively solves for the
optimal robot movements and sends the results to the robot for execution.
Similar as our previous work [21], the optimization is solved by IPOPT [22],
which implements a primal-dual interior-point linear search algorithm. The
local convergence is guaranteed as proved in [23]. We empirically demonstrated
the effectiveness through simulation and experiments.

##  IV Simulation and Experiments

TABLE I: Parameter Values of the Proposed Approach Simulator Parameters | Value
---|---
Cable Length | $1m$
Cable Diameter | $0.01m$
Spring Elastic Stiffness | $4\times 10^{3}$
Spring Damping Stiffness | $2\times 10^{3}$
Spring Bending Stiffness | $3\times 10^{3}$
Global Scale | $2$
GNN Model Parameters | Value
Encoder |
Number of Hidden Layers | $2$
Size of Hidden Layers | $128$
Activation Function | ReLU
Processor |
Time Window Size ($m$) | $5$
Number of Message Passing ($k$) | $10$
Connective Radius | $0.2m$
Number of Hidden Layers | $2$
Size of Hidden Layers | $128$
Activation Function | ReLU
Decoder |
Number of Hidden Layers | $2$
Size of Hidden Layers | $128$
Activation Function | ReLU
GNN Training Parameters | Value
Batch Size | $1$
Learning Rate |  $10^{-4}$ to $10^{-6}$
MPC Parameters | Value
MPC Horizon ($h$) | $5$
Dynamics Sample Time ($\Delta t$) | $1s$
Regularization Weight ($\lambda$) | $10$
Initial Trust Region Size | $0.05m$
$\eta^{+}$, $\eta^{-}$ |  $0.8$, $0.4$
$\tau^{+}$, $\tau^{-}$ |  $1.05$, $0.95$
optimizer | IPOPT

Figure 5: The relation between prediction error and rollout steps of various
offline model. The models are forwarded 1 step to 20 steps with the same cable
that utilized for training. The y axis is log scaled.

###  IV-A Simulation and Experiment Setup

1) Simulation Setup: The simulation setup is shown in Fig. 7, two KUKA robots
move the ends of a cable so that the cable matches the desired shape. The
simulation environment is built upon the PyBullet Physics Engine. The cable
model is generated by Gmsh and simulated by the built-in finite element method
(FEM). Detailed cable simulation parameters are summarized in Table I.

Figure 6: Benchmark comparison results. The robots will manipulate the cable
from a straight line to three desired shapes. The curve shows the result in
the environment that we double the cable stiffness.

Figure 7: Snapshots of the proposed method. The proposed method is tested to
manipulate the cable from random initial shapes to randomly generated targets.
With the GNN model, the robot is able to find a rough global trajectory, while
the online residual model is able to refine the local behaviors.  TABLE II:
Manipulation Performance Comparison of in Simulation

| Scenario 1: U shape | Scenario 2: S shape | Scenario 3: Z shape
---|---|---|---
Terminal Error(cm) | Settling Time(s) | Terminal Error(cm) | Settling Time(s) | Terminal Error(cm) | Settling Time(s)
Online Visual Servo | $1.96\pm 0.29$ | $27.47\pm 5.44$ | $3.28\pm 0.82$ | $24.76\pm 3.27$ | $8.72\pm 1.15$ | $19.43\pm 6.11$
Direct Transfer GNN | $1.54\pm 0.53$ | $11.32\pm 1.46$ | $1.61\pm 0.42$ | $\bm{13.98\pm 1.31}$ | $1.87\pm 0.56$ | $11.91\pm 1.75$
Domain Randomization | $1.11\pm 0.21$ | $12.60\pm 1.32$ | $1.30\pm 0.33$ | $14.33\pm 1.47$ | $1.76\pm 0.41$ | $12.07\pm 1.66$
GNN + Fine-tune | $\bm{0.42\pm 0.10}$ | $\bm{8.31\pm 0.72}$ | $\bm{0.57\pm 0.08}$ | $13.58\pm 1.06$ | $1.19\pm 0.23$ | $\bm{9.82\pm 1.32}$
GNN + Online Residual | $0.58\pm 0.13$ | $12.35\pm 1.09$ | $0.61\pm 0.14$ | $13.79\pm 1.50$ | $\bm{0.78\pm 0.17}$ | $16.32\pm 1.15$

* Fine-tuning requires additional data collection before the robot execution, and its performance depends on the quality of the online dataset.

TABLE III: Performance Comparison in Experiments with the Ethernet Cable | Scenario 1: U shape | Scenario 2: S shape
---|---|---
Terminal Error(cm) | Settling Time(s) | Terminal Error(cm) | Settling Time(s)
Online Visual Servo | $5.58\pm 1.11$ | $52.04\pm 12.11$ | $1.64\pm 0.28$ | $44.18\pm 8.00$
Direct Transfer GNN | $2.54\pm 0.61$ | $25.23\pm 5.19$ | $1.57\pm 0.35$ | $24.63\pm 5.16$
GNN + Domain Randomization | $1.91\pm 0.59$ | $36.29\pm 5.45$ | $1.36\pm 0.34$ | $\bm{16.73\pm 4.37}$
GNN + Fine-tune | $1.56\pm 0.43$ | $\bm{22.29\pm 5.63}$ | $1.12\pm 0.33$ | $23.03\pm 4.80$
GNN + Online Residual | $\bm{1.03\pm 0.24}$ | $24.29\pm 7.92$ | $\bm{0.99\pm 0.16}$ | $19.29\pm 7.71$
TABLE IV: Performance Comparison in Experiments with the USB Cable | Scenario 1: U shape | Scenario 2: S shape
---|---|---
Terminal Error(cm) | Settling Time(s) | Terminal Error(cm) | Settling Time(s)
Online Visual Servo | $3.41\pm 0.91$ | $113.66\pm 20.93$ | $2.04\pm 0.16$ | $106.12\pm 17.09$
Direct Transfer GNN | $1.79\pm 0.45$ | $21.16\pm 4.38$ | $1.91\pm 0.67$ | $23.71\pm 5.13$
GNN + Domain Randomization | $1.39\pm 0.40$ | $\bm{13.65\pm 4.39}$ | $1.33\pm 0.43$ | $31.19\pm 6.90$
GNN + Fine-tune | $0.86\pm 0.38$ | $14.83\pm 3.87$ | $1.08\pm 0.24$ | $\bm{21.44\pm 4.06}$
GNN + Online Residual | $\bm{0.93\pm 0.23}$ | $18.81\pm 7.64$ | $\bm{0.82\pm 0.16}$ | $34.81\pm 8.61$

2) Experiment Setup: The experiment setup is shown in Fig. 1. Two FANUC LR-
Mate 200iD robots collaboratively manipulate the cable to different desired
shapes. An Intel Realsense RGB-D camera is utilized to obtain the cable point
clouds. The online model learning and MPC calculation are achieved on a Ubuntu
18.04 desktop. The optimized robot velocities are sent to the robot for
execution with a communication frequency of $100Hz$.

###  IV-B Data Collection and Learning

Offline Data Collection and Training: The state of the cable is approximated
by a series of key points. In simulation, we uniformly select $13$ points on
the cable. The cable is initialized with a straight line, and we randomly move
the robot end-effector to obtain a trajectory
$\\{X(t),R(t)\\}_{t=0,1,\cdots,200}$, where the trajectory contains $200$
steps of transitions. With this mechanism, we form a data-set with $10K$
trajectories. The network structure and parameters are summarized in Table I.

Online Data Collection and Learning: In the online phase, the proposed
approach does not require to pre-collect data before execution. The online
linear residual model is obtained in real-time by solving (12) with $5$
previous cable states and robot movements.

###  IV-C Simulation Results

We first evaluate the performance of the offline model. The cable is randomly
initialized, and two robots will randomly move two cable tips for $1$ to $20$
steps. We tested the GNN model with two baselines: 1) a two-layer MLP with the
hidden layer size of $128$, 2) a bi-directional LSTM model proposed in [7].
All methods are trained with the same amount of data. The results are provided
in Fig. 5. The GNN and bi-directional LSTM models outperform the MLP by a
large margin (over $10$ times). The result confirms our hypothesis that the
structural designs of the GNN and bi-directional LSTM encode the prior
knowledge on how the interaction will transmit inside the cable. Those prior
knowledge can make the training result better. While the LSTM model shares
many similarities with the GNN on how the interaction is propagated through
the vertex, the GNN model performs better in our scenarios. The average
prediction error of GNN is less than $2\times 10^{-2}$cm after $20$ steps of
predictions, which confirms our assumption that an offline GNN model is a good
approximation of the ground truth cable model.

For manipulation, we change the stiffness of the cable ($0.1\times$,
$0.5\times$, $1\times$, $2\times$, and $3\times$ of the original values) to
test the performance of the proposed online residual model. We compare the
performance of the proposed method with three sim-to-real baselines: 1) direct
transfer, 2) domain randomization, and 3) fine-tuning the network. To be more
specific, direct transfer refers to applying the offline learned model
directly to the testing scenarios without modification. Domain randomization
denotes randomly perturbs some parameters of the cable in the training
dataset. We change the spring elastic stiffness, damping stiffness, and
bending stiffness in the range of $0.1$ to $3$ times the original values. For
fine-tuning, we collect $2k$ transitions in the testing scenario and re-train
the network before executing the robot. In addition, we also compared with an
existing online visual servo control method proposed in [16, 18], where a
linear deformation Jacobian is online estimated via least squares. The
benchmark scenarios are illustrated in Fig. 6. The robots need to
collaboratively manipulate the cable from straight lines to three desired
shapes. The results are averaged over $5$ trails with the cable stiffness
changing from $0.1$ to $3$ times the original value.

The results are summarized in Table II. For settling time, the methods with
offline model outperform the online visual servo method. Since the online
visual servo method purely relies on the local information, it is easy to get
stuck in a local optimal region and spend lots of time to get out. This
phenomenon is also revealed in Fig. 6. In all three scenarios, the online
visual servo controller is able to reduce the shape error quickly in the
beginning but fails to continue this trend in the end. However, since the
visual servo control does not require offline training, the online visual
servo method still has the inherent advantage on data efficiency.

For the terminal shape error, we notice that both domain randomization and
fine-tuning can improve the test time performance than direct transfer. It is
reasonable since the training data for both approaches covers more similar
scenarios as in the test time, especially for fine-tuning, which utilizes the
data collected in the test scenario to refine the network. However, it is
worth noticing that the proposed online residual learning method does not
require additional data, and it is robust across all the scenarios. Though
fine-tuning slightly outperforms the proposed approach in the first two
scenarios, it has a larger error in the third. This may indicate the
performance of fine-tuning may greatly depend on the fine-tune dataset.

We also applied the proposed framework in more challenging scenarios as shown
in Fig. 7, where the initial state and the target state are randomly generated
in simulation. Both tasks require large deformation and accurate local
adjustment. Utilizing the proposed method, we are able to complete the tasks
robustly.

###  IV-D Experiment Results

Figure 8: Snapshots of the proposed methods. We tested the performance of the
proposed method on two cables. For each cable, we set two desired shapes: U
shape and S shape. The proposed method is able to achieve high accuracy
efficiently.

Figure 9: We show the final shapes that each method achieved for the
benchmark. The benchmark is to manipulate the cable to a U shape and an S
shape, as shown with the purple curve. The proposed method can achieve high
accuracy across all the four tasks.

We further evaluate the performance of the proposed framework on two different
cables, a blue Ethernet cable, and a black USB cable. The offline GNN dynamics
are the same as the offline dynamics in the previous experiment. The online
models are initialized by randomly moving around the initial configuration,
and the fine-tune dataset is similarly collected with $2k$ transitions. The
desired shapes are selected as U shapes and S shapes as the purple curves in
Fig. 9.

Table III and Table IV summarize the performance on both cables. The online
visual servo control method achieves good performance by gradually learning
the local model and moving the cable. However, the online method may converge
to a local optimal region and get stuck. As shown in Fig. 9(b), the online
method fails to achieve the U shape and gets stuck in an S shape that is
relative far away from the desire. The offline GNN method has fair
performances on both cables. The global model can efficiently guide the robot
to avoid potential local optimal regions for better performance. However, due
to the sim-to-real gap, the predictions from the offline model may become
inaccurate, and the final shape errors are inevitable. Domain randomization
can slightly perform better because the training data already includes the
deformation of some higher stiffness cables. However, there is still a gap
between simulation and reality. Fine-tuning performs better than domain
randomization but is not comparable with the proposed approach. As we analyzed
in the previous simulation benchmark, its performance may greatly depend on
the fine-tune dataset. Since it is expensive and time-consuming to obtain the
real-world data, it is challenging to obtain a dataset covering most of the
scenarios. The proposed method achieves the best performance in terms of the
terminal error. The result corroborates that the proposed framework is
effective and robust in manipulating cables.

##  V Discussion and Future Works

We proposed a dynamics learning framework for precise cable manipulation, and
we demonstrated the effectiveness of the proposed approach mainly in 2D and
uncluttered environments. In this section, we discuss the applicability of our
approach to more challenging scenarios and point out future research
directions.

1) Multi-step cable manipulation: Real-world manipulation tasks may require
multi-steps of cable shaping, where the robots need to change the grasping
point during the manipulation. Finding the optimal grasping point for
deformable objects is a challenging research problem, and it is widely studied
in the literature [24, 4]. We plan to utilize the Transporter Network [24] to
learn the grasping point from images and combine it with our approach for
manipulation.

2) 3D scenarios: The main challenge to transfer the proposed framework from 2D
to 3D is the prediction accuracy of the offline model. It is challenging for
the offline GNN model to precisely predict the twisting behaviors, as shown in
a recent paper [25]. Therefore, we plan to incorporate their modification on
the offline model structure and utilize the proposed residual learning
approach to manipulate the cable in 3D.

3) Cluttered environment: To the author’s knowledge, few model-based
approaches have been proposed in the literature due to the complexity of this
problem. Mitrano et al. [26] proposed to train a classifier to determine where
the offline learned model is in effect and utilize a learned policy to recover
the robot when the model is not reliable. However, this approach cannot deal
with the scenario that the cable requires to contact with the environment. We
hypothesize that our proposed online residual learning method can be a
potential way to tackle this scenario. The residual model can serve as a
contact model, and we leave it as a future research direction.

## References

  * [1] S. C. Brenner, L. R. Scott, and L. R. Scott, _The mathematical theory of finite element methods_. Springer, 2008, vol. 3.
  * [2] M. Yu, H. Zhong, F. Zhong, and X. Li, “Adaptive control for robotic manipulation of deformable linear objects with offline and online learning of unknown models,” _arXiv preprint arXiv:2107.00194_ , 2021.
  * [3] Z. Hu, T. Han, P. Sun, J. Pan, and D. Manocha, “3-d deformable object manipulation using deep neural networks,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 4, pp. 4255–4261, 2019.
  * [4] W. Yan, A. Vangipuram, P. Abbeel, and L. Pinto, “Learning predictive representations for deformable objects using contrastive estimation,” _arXiv preprint arXiv:2003.05436_ , 2020.
  * [5] A. Sanchez-Gonzalez, J. Godwin, T. Pfaff, R. Ying, J. Leskovec, and P. Battaglia, “Learning to simulate complex physics with graph networks,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 8459–8468.
  * [6] A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, and S. Levine, “Combining self-supervised learning and imitation for vision-based rope manipulation,” in _2017 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2017, pp. 2146–2153.
  * [7] M. Yan, Y. Zhu, N. Jin, and J. Bohg, “Self-supervised learning of state estimation for manipulating deformable linear objects,” _IEEE robotics and automation letters_ , vol. 5, no. 2, pp. 2372–2379, 2020.
  * [8] Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba, “Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids,” _arXiv preprint arXiv:1810.01566_ , 2018.
  * [9] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. W. Battaglia, “Learning mesh-based simulation with graph networks,” _arXiv preprint arXiv:2010.03409_ , 2020.
  * [10] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in _2017 IEEE/RSJ international conference on intelligent robots and systems (IROS)_. IEEE, 2017, pp. 23–30.
  * [11] Y. Chebotar, A. Handa, V. Makoviychuk, M. Macklin, J. Issac, N. Ratliff, and D. Fox, “Closing the sim-to-real loop: Adapting simulation randomization with real world experience,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 8973–8979.
  * [12] F. Ramos, R. C. Possas, and D. Fox, “Bayessim: adaptive domain randomization via probabilistic inference for robotics simulators,” _arXiv preprint arXiv:1906.01728_ , 2019.
  * [13] A. Ajay, J. Wu, N. Fazeli, M. Bauza, L. P. Kaelbling, J. B. Tenenbaum, and A. Rodriguez, “Augmenting physical simulators with stochastic neural networks: Case study of planar pushing and bouncing,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 3066–3073.
  * [14] A. Kloss, S. Schaal, and J. Bohg, “Combining learned and analytical models for predicting action effects,” _arXiv preprint arXiv:1710.04102_ , vol. 11, 2017\.
  * [15] D. Navarro-Alarcon, Y. Liu, J. G. Romero, and P. Li, “Visually servoed deformation control by robot manipulators,” in _2013 IEEE International Conference on Robotics and Automation_. IEEE, 2013, pp. 5259–5264.
  * [16] J. Zhu, B. Navarro, P. Fraisse, A. Crosnier, and A. Cherubini, “Dual-arm robotic manipulation of flexible cables,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 479–484.
  * [17] J. Zhu, D. Navarro-Alarcon, R. Passama, and A. Cherubini, “Vision-based manipulation of deformable and rigid objects using subspace projections of 2d contours,” _Robotics and Autonomous Systems_ , vol. 142, p. 103798, 2021\.
  * [18] S. Jin, C. Wang, and M. Tomizuka, “Robust deformation model approximation for robotic cable manipulation,” in _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2019, pp. 6586–6593.
  * [19] T. Tang and M. Tomizuka, “Track deformable objects from point clouds with structure preserved registration,” _The International Journal of Robotics Research_ , p. 0278364919841431, 2018.
  * [20] T. Tang, C. Wang, and M. Tomizuka, “A framework for manipulating deformable linear objects by coherent point drift,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 3426–3433, 2018.
  * [21] C. Wang, J. Bingham, and M. Tomizuka, “Trajectory splitting: A distributed formulation for collision avoiding trajectory optimization,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 8113–8120.
  * [22] A. Wächter and L. T. Biegler, “On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming,” _Mathematical programming_ , vol. 106, no. 1, pp. 25–57, 2006.
  * [23] A. Wachter and L. T. Biegler, “Line search filter methods for nonlinear programming: Motivation and global convergence,” _SIAM Journal on Optimization_ , vol. 16, no. 1, pp. 1–31, 2005.
  * [24] D. Seita, P. Florence, J. Tompson, E. Coumans, V. Sindhwani, K. Goldberg, and A. Zeng, “Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 4568–4575.
  * [25] Y. Yang, J. A. Stork, and T. Stoyanov, “Learning to propagate interaction effects for modeling deformable linear objects dynamics,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 1950–1957.
  * [26] P. Mitrano, D. McConachie, and D. Berenson, “Learning where to trust unreliable models in an unstructured world for deformable object manipulation,” _Science Robotics_ , vol. 6, no. 54, p. eabd8170, 2021.


the denominators of the conditional density estimators cancel out. Denote by
$Z_{\pi_{s}(1:n)}$ the resulting permutation of $Z_{1:n}$ after all swaps in
iteration $s$ have been accepted or rejected. The CP test runs an initial $S$
iterations, after which it then runs $B$ independent sequences initialized at
$Z_{\pi_{S}(1:n)}$, each for another $S$ iterations (Berrett et al., 2019,
Algorithm 2). For $b\in\\{1,\dotsc,B\\}$, denote the final permutation of each
procedure as $Z^{(b)}_{\pi_{2S}(1:n)}$. The $p$-value of the CP test is then
computed as

$\displaystyle
p_{\text{CP}}=\frac{1}{1+B}\left[1+\sum_{b=1}^{B}\mathbbm{1}\left\\{T_{\text{CP}}(X_{1:n},Z_{1:n},M(X)_{1:n})\leq
T_{\text{CP}}(X_{1:n},Z^{(b)}_{\pi_{2S}(1:n)},M(X)_{1:n})\right\\}\right]\;.$

The test rejects $H_{0}$ at level $\alpha$ if $p_{\text{CP}}\leq\alpha$. See
Berrett et al. (2019) for more details about the CP test.

###  C.2 Distribution of $p$-values in $\mathrm{SO}(3)$-invariance experiment

Figure 6 shows the $p$-value distributions for the tests for
$\mathrm{SO}(3)$-invariance conducted in Section 7.1.1.

Figure 6: Histograms showing the $p$-value distributions obtained over
$N=1000$ simulations in the $\mathrm{SO}(3)$-invariance experiment. The
$p$-value of a Kolmogorov–Smirnov test for uniformity of the distribution is
shown in the bottom-right corner of each plot.

###  C.3 Number of random projections for Nmmd and Cw in
$\mathbb{S}_{10}$-invariance experiment

Figure 7 shows the rejection rate and average computation time for Nmmd and Cw
as the number of random projections $J$ increases in the
$\mathbb{S}_{10}$-invariance experiment (Section 7.1.2).

Figure 7: Test for $\mathbb{S}_{10}$-invariance rejection rates and standard
deviations (first row) and average computation time in seconds for a single
execution (second row) over $N=1000$ simulations as the number of random
projections increases.

###  C.4 Swarm experiment

The grids used to train the kernels in Kci were manually tuned through trial
and error. The final set of grids used to obtain the results in Section 7.2
were

  * •

$k_{X}$: 31 linearly-spaced numbers between $1\text{e}^{-3}$ and $3$;

  * •

$k_{Y}$: 57 linearly-spaced numbers between $1$ and $15$; and

  * •

$k_{M(X)}$: {$5\text{e}^{-3}$, $5.5\text{e}^{-3}$, $6\text{e}^{-3}$}.

The test results for the marginal invariance and joint invariance experiments
are given in Table 3.

| Marginal invariance | Joint invariance
---|---|---
| Discrete | Continuous | Discrete | Continuous
2sMmd | $0.007$ | $0.004$ | $0.006$ | $0.011$
Mmd | $0.047$ | $0.047$ | $0.998$ | $0.999$
Nmmd | $0.060$ | $0.055$ | $0.098$ | $0.094$
Cw | $0.064$ | $0.087$ | $0.903$ | $0.897$
Table 3: Test rejection rates over $N=1000$ simulations for the Swarm data.

Histograms of the $p$-values for these tests are shown in Figure 8.

Figure 8: Histograms showing the $p$-value distributions obtained over
$N=1000$ simulations for tests for $X$-marginal invariance and $(X,Y)$-joint
invariance with respect to discrete and continuous $X$-rotations about the
geographic north pole. The $p$-value of a Kolmogorov–Smirnov test for
uniformity of the distribution is shown in each plot.

###  C.5 LHC experiment

The grid $\\{10^{-2},10^{-1},0,10\\}$ was used to train the kernels $k_{X}$,
$k_{Y}$, and $k_{M(X)}$ in Kci for the equivariance experiment in Section 7.3.
The grid $\\{10^{-3},10^{-2},10^{-1}\\}$ was used to train the kernels $k_{Y}$
and $k_{M(X)}$ in Cp.

Figure 9 shows histograms of the $p$-values obtained from the tests for joint
invariance and equivariance in Section 7.3.

Figure 9: Histograms showing the $p$-value distributions obtained over
$N=1000$ simulations for tests for joint invariance and equivariance in the
LHC experiments. The $p$-value of a Kolmogorov–Smirnov test for uniformity of
the distribution is shown in the bottom-right corner of each plot.

Figure 10 shows the rejection rate and average computation time for Nmmd and
Cw as the number of random projections $J$ increases in the LHC joint
invariance experiment.

Figure 10: LHC test for joint invariance rejection rates and standard
deviations (first row) and average computation time in seconds for a single
execution (second row) over $N=1000$ simulations as the number of random
projections increases.

###  C.6 Top quark experiment

For Kci in the top quark experiment in Section 7.4, the grid
$\\{5,7.5,10,\dotsc,50\\}$ was used to train the kernel $k_{X}$, and the grid
$\\{5,7.5,10,\dotsc,100\\}$ was used to train the kernel $k_{M(X)}$. The grids
were manually selected based on trial and error.

Figure 11 shows the $p$-value distributions obtained from Kci in the top quark
experiment.

Figure 11: Histograms showing the Kci $p$-value distributions obtained over
$N=1000$ simulations in the top quark experiment.

## References

  * Abdi (2007) H. Abdi.  Multiple correlation coefficient.  _Encyclopedia of Measurement and Statistics_ , 648:651, 2007\.
  * Aldous (1985) D. J. Aldous.  Exchangeability and related topics.  In P. L. Hennequin, editor, _École d’Été de Probabilités de Saint-Flour XIII - 1983_ , number 1117 in Lecture Notes in Mathematics, pages 1–198. Springer, 1985.
  * ATLAS Collaboration (2017) ATLAS Collaboration.  Searching for new symmetries of nature, April 2017.  https://atlas.cern/updates/briefing/searching-new-symmetries-nature.  Last visited 2023-07-26.
  * Berrett et al. (2019) T. B. Berrett, Y. Wang, R. F. Barber, and R. J. Samworth.  The conditional permutation test for independence while controlling for confounders.  _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , 82(1):175–197, 10 2019.
  * Bierlich et al. (2022) C. Bierlich, S. Chakraborty, N. Desai, L. Gellersen, I. Helenius, P. Ilten, L. Lönnblad, S. Mrenna, S. Prestel, C. T. Preuss, T. Sjöstrand, P. Skands, M. Utheim, and R. Verheyen.  A comprehensive guide to the physics and usage of PYTHIA 8.3.  _SciPost Physics Codebases_ , page 8, 2022.
  * Birman et al. (2022) M. Birman, B. Nachman, R. Sebbah, G. Sela, O. Turetz, and S. Bressler.  Data-directed search for new physics based on symmetries of the SM.  _The European Physical Journal C_ , 82(6):508, 2022.
  * Bloem-Reddy and Teh (2020) B. Bloem-Reddy and Y. W. Teh.  Probabilistic symmetries and invariant neural networks.  _Journal of Machine Learning Research_ , 21:90–1, 2020\.
  * Bogatskiy et al. (2020) A. Bogatskiy, B. Anderson, J. Offermann, M. Roussi, D. Miller, and R. Kondor.  Lorentz group equivariant neural network for particle physics.  In _Proceedings of the 37th International Conference on Machine Learning_ , pages 992–1002. PMLR, 2020.
  * Bronstein et al. (2021) M. M. Bronstein, J. Bruna, T. Cohen, and P. Veličković.  Geometric deep learning: Grids, groups, graphs, geodesics, and gauges.  _arXiv preprint arXiv:2104.13478_ , 2021.
  * Çinlar (2011) E. Çinlar.  _Probability and Stochastics_.  Springer New York, 2011.
  * Chatalic et al. (2022) A. Chatalic, N. Schreuder, L. Rosasco, and A. Rudi.  Nyström kernel mean embeddings.  In _International Conference on Machine Learning_ , pages 3006–3024. PMLR, 2022.
  * Chen et al. (2020) S. Chen, E. Dobriban, and J. H. Lee.  A group-theoretic framework for data augmentation.  _Journal of Machine Learning Research_ , 21(1):9885–9955, 2020.
  * Christie and Aston (2022) L. G. Christie and J. A. Aston.  Testing for geometric invariance and equivariance.  _arXiv preprint arXiv:2205.15280_ , 2022.
  * Christie and Aston (2023) L. G. Christie and J. A. Aston.  Estimating maximal symmetries of regression functions via subgroup lattices.  _arXiv preprint arXiv:2303.13616_ , 2023.
  * Christmann and Steinwart (2008) A. Christmann and I. Steinwart.  _Support Vector Machines_.  Springer New York, 2008.
  * Cohen et al. (2019) T. S. Cohen, M. Geiger, and M. Weiler.  A general theory of equivariant CNNs on homogeneous spaces.  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Collins et al. (2018) J. Collins, K. Howe, and B. Nachman.  Anomaly detection for resonant new physics with machine learning.  _Physical Review Letters_ , 121:241803, 2018.
  * Dawid (1985) A. P. Dawid.  Invariance and independence in multivariate distribution theory.  _Journal of Multivariate Analysis_ , 17(3):304–315, Dec. 1985.
  * De Gooijer and Zerom (2003) J. G. De Gooijer and D. Zerom.  On conditional density estimation.  _Statistica Neerlandica_ , 57(2):159–176, 2003\.
  * Dehmamy et al. (2021) N. Dehmamy, R. Walters, Y. Liu, D. Wang, and R. Yu.  Automatic symmetry discovery with lie algebra convolutional network.  _Advances in Neural Information Processing Systems_ , 34:2503–2515, 2021.
  * Desai et al. (2021) K. Desai, B. Nachman, and J. Thaler.  SymmetryGAN: Symmetry discovery with deep learning.  _arXiv preprint arXiv:2112.05722_ , 2021.
  * Dobriban (2022) E. Dobriban.  Consistency of invariance-based randomization tests.  _The Annals of Statistics_ , 50(4), 2022.
  * Dufour (2006) J.-M. Dufour.  Monte Carlo tests with nuisance parameters: A general approach to finite-sample inference and nonstandard asymptotics.  _Journal of Econometrics_ , 133(2):443–477, Aug. 2006.
  * Dufour and Neves (2019) J.-M. Dufour and J. Neves.  _Finite-sample inference and nonstandard asymptotics with Monte Carlo tests and R_ , volume 41 of _Handbook of Statistics_ , chapter 1, pages 3–31.  Elsevier, 2019.
  * Eaton (1989) M. L. Eaton.  _Group Invariance in Applications in Statistics_.  Regional Conference Series in Probability and Statistics. Institute of Mathematical Statistics and American Statistical Association, 1989.
  * Eaton (2007) M. L. Eaton.  Topological groups and invariant measures.  In _Multivariate Statistics_ , pages 184–232. Institute of Mathematical Statistics, 2007.
  * Eaton and Sudderth (1999) M. L. Eaton and W. D. Sudderth.  Consistency and strong inconsistency of group-invariant predictive inferences.  _Bernoulli_ , 5(5):833–854, 1999.
  * Elesedy (2021) B. Elesedy.  Provably strict generalisation benefit for invariance in kernel methods.  _Advances in Neural Information Processing Systems_ , 34:17273–17283, 2021.
  * Elesedy and Zaidi (2021) B. Elesedy and S. Zaidi.  Provably strict generalisation benefit for equivariant models.  In _International Conference on Machine Learning_ , pages 2959–2969. PMLR, 2021.
  * Farrell (1962) R. H. Farrell.  Representation of invariant measures.  _Illinois Journal of Mathematics_ , 6(3):447–467, 1962.
  * Folland (2016) G. B. Folland.  _A Course in Abstract Harmonic Analysis_ , volume 29.  CRC press, 2016.
  * Fraiman et al. (2021) R. Fraiman, L. Moreno, and T. Ransford.  Application of the Cramér-Wold theorem to testing for invariance under group actions.  _arXiv preprint arXiv:2109.01041_ , 2021.
  * Fraser (1961) D. A. S. Fraser.  The fiducial method and invariance.  _Biometrika_ , 48(3/4):261–280, 1961.
  * Fraser (1966) D. A. S. Fraser.  Structural probability and a generalization.  _Biometrika_ , 53(1/2):1–9, 1966.
  * Fraser (1968) D. A. S. Fraser.  _The Structure of Inference_.  Wiley, 1968.
  * Fukumizu et al. (2008) K. Fukumizu, A. Gretton, B. Schölkopf, and B. K. Sriperumbudur.  Characteristic kernels on groups and semigroups.  _Advances in Neural Information Processing Systems_ , 21, 2008.
  * Garreau et al. (2017) D. Garreau, W. Jitkrittum, and M. Kanagawa.  Large sample analysis of the median heuristic.  _arXiv preprint arXiv:1707.07269_ , 2017.
  * Gretton et al. (2012) A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola.  A kernel two-sample test.  _The Journal of Machine Learning Research_ , 13(1):723–773, 2012.
  * Gross (1996) D. J. Gross.  The role of symmetry in fundamental physics.  _Proceedings of the National Academy of Sciences_ , 93(25):14256–14259, 1996.
  * Hemerik and Goeman (2018) J. Hemerik and J. Goeman.  Exact testing with random permutations.  _TEST_ , 27(4):811–825, 2018.
  * Hora and Buehler (1966) R. B. Hora and R. J. Buehler.  Fiducial theory and invariant estimation.  _The Annals of Mathematical Statistics_ , 37(3):643–656, 1966.
  * Hora and Buehler (1967) R. B. Hora and R. J. Buehler.  Fiducial theory and invariant prediction.  _The Annals of Mathematical Statistics_ , 38(3):795–801, 1967.
  * Huang et al. (2022) K. H. Huang, P. Orbanz, and M. Austern.  Quantifying the effects of data augmentation.  _arXiv preprint arXiv:2202.09134_ , 2022.
  * Kallenberg (2007) O. Kallenberg.  Invariant measures and disintegrations with applications to palm and related kernels.  _Probability Theory and Related Fields_ , 139(1):285–310, 2007.
  * Kallenberg (2011) O. Kallenberg.  Invariant palm and related disintegrations via skew factorization.  _Probability Theory and Related Fields_ , 149(1):279–301, 2011.
  * Kallenberg (2017) O. Kallenberg.  _Random Measures, Theory and Applications_.  Springer International, 2017.
  * Karagiorgi et al. (2022) G. Karagiorgi, G. Kasieczka, S. Kravitz, B. Nachman, and D. Shih.  Machine learning in the search for new fundamental physics.  _Nature Reviews Physics_ , 4(6):399–412, 2022\.
  * Kasieczka et al. (2019) G. Kasieczka, T. Plehn, J. Thompson, and M. Russel.  Top quark tagging reference dataset, Mar. 2019.  https://doi.org/10.5281/zenodo.2603256.
  * Kasieczka et al. (2021) G. Kasieczka, B. Nachman, D. Shih, O. Amram, A. Andreassen, K. Benkendorfer, B. Bortolato, G. Brooijmans, F. Canelli, J. H. Collins, et al.  The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics.  _Reports on Progress in Physics_ , 84(12):124201, 2021.
  * Krippendorf and Syvaeri (2020) S. Krippendorf and M. Syvaeri.  Detecting symmetries with neural networks.  _Machine Learning: Science and Technology_ , 2(1):015010, 2020.
  * Larkoski et al. (2020) A. J. Larkoski, I. Moult, and B. Nachman.  Jet substructure at the Large Hadron Collider: A review of recent advances in theory and machine learning.  _Physics Reports_ , 841:1–63, 2020.
  * Lehmann and Casella (1998) E. L. Lehmann and G. Casella.  _Theory of Point Estimation_.  Springer, 2 edition, 1998.
  * Lehmann and Romano (2005) E. L. Lehmann and J. P. Romano.  _Testing Statistical Hypotheses_.  Sprinter Texts in Statistics. Springer-Verlag New York, 2005.
  * Li and Fan (2020) C. Li and X. Fan.  On nonparametric conditional independence tests for continuous variables.  _WIREs Computational Statistics_ , 12(3):e1489, 2020.
  * Lyle et al. (2020) C. Lyle, M. van der Wilk, M. Kwiatkowska, Y. Gal, and B. Bloem-Reddy.  On the benefits of invariance in neural networks.  _arXiv preprint arXiv:2005.00178_ , 2020.
  * McCormack and Hoff (2023) A. McCormack and P. D. Hoff.  Equivariant estimation of Fréchet means.  _Biometrika_ , page asad014, Feb. 2023.
  * Muandet et al. (2017) K. Muandet, K. Fukumizu, B. Sriperumbudur, B. Schölkopf, et al.  Kernel mean embedding of distributions: A review and beyond.  _Foundations and Trends in Machine Learning_ , 10(1-2):1–141, 2017.
  * Olsen et al. (2013) N. Olsen, E. Friis-Christensen, R. Floberghagen, P. Alken, C. D. Beggan, A. Chulliat, E. Doornbos, J. T. Da Encarnação, B. Hamilton, G. Hulot, et al.  The Swarm satellite constellation application and research facility (SCARF) and Swarm data products.  _Earth, Planets and Space_ , 65:1189–1200, 2013.
  * Rahimi and Recht (2007) A. Rahimi and B. Recht.  Random features for large-scale kernel machines.  _Advances in Neural Information Processing Systems_ , 20, 2007.
  * Raj et al. (2017) A. Raj, A. Kumar, Y. Mroueh, T. Fletcher, and B. Schölkopf.  Local group invariant representations via orbit embeddings.  In _Artificial Intelligence and Statistics_ , pages 1225–1235. PMLR, 2017.
  * Salam (2010) G. P. Salam.  Towards jetography.  _The European Physical Journal C_ , 67(3):637–686, 2010.
  * Shah and Peters (2020) R. D. Shah and J. Peters.  The hardness of conditional independence testing and the generalised covariance measure.  _The Annals of Statistics_ , 48(3), 2020.
  * Sriperumbudur et al. (2010) B. K. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. R. Lanckriet.  Hilbert space embeddings and metrics on probability measures.  _Journal of Machine Learning Research_ , 11:1517–1561, Aug. 2010.
  * Sriperumbudur et al. (2012) B. K. Sriperumbudur, K. Fukumizu, A. Gretton, B. Schölkopf, and G. R. G. Lanckriet.  On the empirical estimation of integral probability metrics.  _Electronic Journal of Statistics_ , 6, 2012.
  * Wijsman (1990) R. A. Wijsman.  _Invariant measures on groups and their use in statistics_ , volume 14 of _Lecture Notes–Monograph Series_.  Institute of Mathematical Statistics, 1990.
  * Winter et al. (2022) R. Winter, M. Bertolini, T. Le, F. Noé, and D.-A. Clevert.  Unsupervised learning of group invariant and equivariant representations.  _Advances in Neural Information Processing Systems_ , 35:31942–31956, 2022.
  * (67) World Data Center for Geomagnetism, Kyoto University.  Magnetic north, geomagnetic and magnetic poles.  https://wdc.kugi.kyoto-u.ac.jp/poles/polesexp.html.  Last visited 2023-07-26.
  * Yang et al. (2023) J. Yang, R. Walters, N. Dehmamy, and R. Yu.  Generative adversarial symmetry discovery.  In _International Conference on Machine Learning_ , 2023.
  * Zhang et al. (2011) K. Zhang, J. Peters, D. Janzing, and B. Schölkopf.  Kernel-based conditional independence test and application in causal discovery.  In _Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence_ , UAI’11, pages 804–813, 2011. AUAI Press.
  * Zhou et al. (2021) A. Zhou, T. Knowles, and C. Finn.  Meta-learning symmetries by reparameterization.  In _International Conference on Learning Representations_ , 2021.