# GET-Zero: Graph Embodiment Transformer
for Zero-shot Embodiment Generalization

Austin Patel Shuran Song
Stanford University
https://get-zero-paper.github.io

###### Abstract

This paper introduces GET-Zero, a model architecture and training procedure
for learning an embodiment-aware control policy that can immediately adapt to
new hardware changes without retraining. To do so, we present Graph Embodiment
Transformer (GET), a transformer model that leverages the embodiment graph
connectivity as a learned structural bias in the attention mechanism. We use
behavior cloning to distill demonstration data from embodiment-specific expert
policies into an embodiment-aware GET model that conditions on the hardware
configuration of the robot to make control decisions. We conduct a case study
on a dexterous in-hand object rotation task using different configurations of
a four-fingered robot hand with joints removed and with link length
extensions. Using the GET model along with a self-modeling loss enables GET-
Zero to zero-shot generalize to unseen variation in graph structure and link
length, yielding a 20% improvement over baseline methods. All code and
qualitative video results are on our project website.

> Keywords: Cross-embodiment learning, Dexterous manipulation

Figure 1: GET-Zero is an embodiment-aware policy that is able to zero-shot
generalize to unseen embodiment designs with varied geometry, number of
joints, and graph structure.

##  1 Introduction

Learning algorithms have significantly improved robots’ ability to sense and
adapt to external environments, yet most robots today cannot tolerate minor
changes in their internal hardware. From missing links caused by damage to
added joints for design improvements, hardware modifications often require
retraining existing policies with embodiment-specific data.

Recent methods develop embodiment-aware policies that condition on the
hardware configuration of a robot design to control a class of embodiments [1,
2, 3, 4, 5, 6], which enables efficient data reuse across embodiments.
However, these method often have incomplete embodiment representations which
ignore the graph connectivity of the robot, and thus cannot generalize well to
an embodiment with a varied graph structure. Improving the performance under
graph variations would greatly expand the applicability of embodiment-aware
methods in domains such as manipulation, where the number of fingers may vary
or in locomotion, where the leg may have a varying number of joints.

Motivated by this limitation, the goal of this work is to introduce an
embodiment representation that explicitly leverages the graph connectivity of
the robot to improve zero-shot control of robots with variations in graph
structure. We propose GET-Zero, which introduces the Graph Embodiment
Transformer (GET) model architecture that enables zero-shot adaptation to new
embodiments (Fig. 1). The key idea of GET is to leverage the robot graph
connectivity as a structural bias in the transformer attention mechanism. By
training this model using behavior data from a variety of robot embodiments,
we can then zero-shot control previously unseen embodiments with varied
geometry, number of joints, and graph structure. Specifically, GET-Zero
consist of three novel components:

  1. 1.

Graph Embodiment Transformer (GET). GET is a modified transformer architecture
[7] that encodes joints as separate tokens and uses the embodiment
connectivity as a learned bias in the transformer attention mechanism.
Attention enables local joint information to communicate and the graph
attention bias modulates this communication based on how the joints are
physically connected. This extends prior embodiment-aware transformer models
which either do not encode the embodiment connectivity [3] or have incomplete
graph representations [4].

  2. 2.

Embodiment-Aware Distillation. To train GET-Zero we first collect
demonstration data from embodiment-specific experts. By conditioning on
embodiment information, we then distill knowledge from embodiment-specific
experts into an embodiment-aware GET model using behavior cloning (BC) that
controls both training and unseen embodiments with a single set of network
weights. Our approach contrasts with prior methods [2, 3, 4] that use RL to
simultaneously learn an embodiment-aware model while learning to complete the
task across all embodiments, which is significantly more difficult and
prevents the use of expert demonstration data.

  3. 3.

Self Modeling Loss. During the distillation BC phase, we introduce an
additional self modeling loss alongside the standard policy action loss. The
self-modeling output predicts the position of each joint in 3D space (i.e.,
forward kinematics), which encourages the network to understanding the
embodiment graph and infer actions accordingly. We found that this simple and
general supervision improves generalization performance.

To validate GET-Zero, we conduct a case study to learn an embodiment-aware,
dexterous in-hand object rotation policy across different hardware
configurations of a multi-fingered robot hand. In particular, we use the LEAP
Hand [8], which is a low-cost, four-fingered hand with four joints per finger
consisting of 3D printed components and off-the-shelf motors. We create 44
hardware configurations of this hand by removing different combinations of
finger joints and associated links. Next, we train embodiment-specific expert
policies for each hand using RL in simulation and then follow the GET-Zero
training procedure to distill the expert behavior into an embodiment-aware GET
model. Our experiments in simulation and real world demonstrate how the novel
graph encoding and self-modeling in GET-Zero improves the capability of this
model to zero-shot control hand designs with unseen variations in link
geometry and graph connectivity.

##  2 Related Work

Cross-Embodiment pretraining with embodiment-specific finetuning. Recent
approaches leverage large-scale pretraining, either on visual data, robot
trajectories, language, or tasks as an initialization for policy learning or
generalist agents [9, 10, 11, 12, 13, 14, 15]. Other approaches leverage robot
data across embodiments to initialize a generalist agent, and then fine-tune
the base model or an action head to adapt to a new robot embodiments [16, 17].
These method often assume a unified action space, which is reasonable for
navigation [18] or for robot arm policies where the end-effector pose is a
shared representation across arms [17, 19, 16, 20]. However, unified actions
spaces are often not sufficient for tasks where precise, low-level joint
actions matter, such as dexterous manipulation [21, 22, 23, 24]. Moreover,
many approaches still require non-trivial amounts of additional training data
specific to the target hardware and environment. Instead, our method uses the
graph structure of the robot as a general embodiment representation to improve
zero-shot transfer for tasks where a unified action space is not possible.

Embodiment Representation. Prior works in multi-embodiment policies
demonstrate using a single policy to control different robot designs [1, 25,
2, 3, 4, 6, 5]. However, most methods show limited generalization ability to
new embodiment designs beyond the training set due to incomplete embodiment
representations. For example, Octo [1] has 3rd-person images of the robot arm
which doesn’t generalize across robot appearance changes, though this could be
addressed with robot inpainting [26]. Other methods learn cross embodiments
representations that align actions across different embodiments [27, 28, 29].
In contrast, GET-Zero uses the graph embodiment structure connecting local
sensory information as a flexible and general representation across robots.

Embodiment-aware policy architectures. Most related to our approach is the
line of work developing embodiment-aware policies. Prior methods [30, 31, 2]
structure a graph neural network (GNN) to match the embodiment graph with
joints as nodes and links as edges to control stick-like walking characters in
simulation. Kurin et al. [3] shows that a GNN matching the embodiment graph is
outperformed with the fully connected attention mechanism in transformers [7]
despite having no graph encoding. Gupta et al. [4] extends this model with
dynamics and kinematics encodings, demonstrating zero-shot generalization to
these properties. However, these methods do not generalize well to unseen
embodiment graphs due to no [3] or limited [4] graph representations. Our
method GET-Zero extends the transformer architecture in [3, 4] to include an
explicit graph representation that improves zero-shot generalization to unseen
embodiment graphs. Furthermore, unlike prior embodiment-aware RL methods that
do locomotion only in simulation [30, 31, 2, 3, 4], GET-Zero uses behavior
cloning from experts for an in-hand rotation task transferable to a real
robot.

##  3 Method: Graph Embodiment Transformer

We present Graph Embodiment Transformer (GET), an embodiment-aware transformer
encoder [7] that uses the embodiment graph as a structural bias in network
architecture (Fig. 2). An embodiment graph consists of nodes representing
local joint information and directed edges representing links connecting
parent and child joints. GET represents each joint as separate transformer
tokens containing both local sensory observations and local hardware
properties (§3.1). The graph edges (i.e., links) are encoded through a learned
attention bias in the self-attention layers (§3.2).

###  3.1 Embodiment Tokenization

For a robot with $J$ joints, there are $J$ input tokens to the transformer
encoder containing local observations and $J$ corresponding output tokens for
per-joint actions. The encoder supports a variable number of tokens which
enables compatibility as the number of joints varies across embodiments.

Observations can either be local $l$ to specific joints or global $g$ across
the entire embodiment. Additionally, observations are either fixed $f$ or
variable $v$ if they change during execution. This yields four observation
types: variable local $o_{vl}$ (e.g., joint angle/velocity), variable global
$o_{vg}$ (e.g., a global time encoding, or environment state), fixed local
$o_{fl}$ (e.g., joint position in rest pose, or joint limit ranges) and fixed
global $o_{fg}$ (e.g., a task identifier). $H$ past history steps are included
only for variable observations ($o_{vl}$ and $o_{vg}$). The local observations
for joint $j$ are defined as $o_{*l,j}$. The transformer token $T_{j}$ for
joint $j$ at timestep $t$ is constructed as $T_{j}^{t}=[o_{vg}^{t\rightarrow
t-H},o_{fg},o_{vl,j}^{t\rightarrow t-H},o_{fl,j}]$. The tokens pass through a
learned linear embedding before entering the transformer encoder.

Figure 2: Graph Embodiment Transformer (GET). GET is an embodiment-aware model
based on a transformer encoder. Each joint forms separate tokens containing
local sensory and embodiment information. The self-attention layers use an
undirected (Spatial Bias) and directed (Parent-Child Bias) graph distance to
bias the attention scores between joints according to the embodiment graph
(grid color intensity indicates distance between nodes). A policy head
predicts actions and a self modeling head predicts meta-properties about the
embodiment, such as forward kinematics.

###  3.2 Graph Encoding

One challenge is that the transformer encoder has no direct graph encoding
mechanism in the original implementation [7]. Input tokens are permutation
invariant without a positional encoding mechanism, but are often linearly
encoded with sinusoidal or learned positional encodings. Prior embodiment-
aware methods address this in multiple ways. Kurin et al. [3] use no
positional encoding meaning no graph representation is present. Gupta et al.
[4] linearize the graph using a depth-first search ordering then apply a
learned linear positional encoding. However, this approach is sensitive to
graph variations as the DFS ordering is not unique, which empirically caused a
$\sim 75\%$ drop in performance with the opposite node order in Gupta et al.
[4]. Sferrazza et al. [32] use the adjacency matrix as a binary attention mask
to limit communication early in the transformer to adjacent nodes.

Our GET model uses a learned attention bias in the encoder self-attention
mechanism to explicitly encode the graph similar to the Graphormer work by
Ying et al. [33]. Using the notation from Ying et al. [33], the inputs to the
self-attention layer given $J$ joints are the hidden states
$H=[h_{1}^{\top},...,h_{J}^{\top}]\in\mathbb{R}^{J\times d}$. $H$ is projected
by three matrices $W_{Q}\in\mathbb{R}^{d\times
d_{K}}$,$W_{K}\in\mathbb{R}^{d\times d_{K}}$, $W_{V}\in\mathbb{R}^{d\times
d_{V}}$ to produce $Q=HW_{Q}$, $K=HW_{K}$, $V=HW_{V}$. Then the attention
layers will compute an attention matrix $A\in\mathbb{R}^{J\times J}$ where
$A_{ij}$ represents the attention score computed between joints $i$ and $j$ as
follows: $A=QK^{\top}/\sqrt{d},\text{Attn}(H)=\text{softmax}(A)V$.

To encode the embodiment graph, we learn two separate biases to the attention
scores ($A$): a Spatial Encoding [33], and a novel Parent-Child Encoding,
detailed below.

Spatial Encoding. The spatial encoding computes the shortest path distance
(SPD) $\phi^{\text{SPD}}$ between node $i$ and node $j$ in the graph as
$\phi^{\text{SPD}}(i,j)$. An embedding table $s$ is indexed with the SPD
distance as $s_{\phi^{\text{SPD}}(i,j)}$ to produce a learned scalar that is
added on top of the attention score $A_{ij}$.

The intuition is that the attention mechanism can learn to pay more or less
attention to joints in the graph based on their distance. For example, nearby
joints may learn a large attention bias, while far away joints that may not
have much influence on each other hence a low spatial bias.

Parent-Child Encoding. Unlike Graphormer [33] that only encodes undirected
graph in the attention mechanism, we introduce a new parent-child attention
bias to encode directed features. For joints $i$ and $j$, we compute the
parent distance $\phi^{P}(i,j)=\phi^{\text{SPD}}(i,j)\mathbbm{1}\\{i\text{ is
parent of }j\\}$ which is the distance between $i$ and $j$ if $j$ is the child
of $i$ at some distance along the forward kinematic chain, otherwise $0$. We
compute an analogous formula for the child distance as
$\phi^{C}(i,j)=\phi^{\text{SPD}}(i,j)\mathbbm{1}\\{i\text{ is child of }j\\}$.
There are associated scalar embedding tables $p$ and $c$.

The intuition is that the model may attempt to compute forward kinematics
through the embodiment graph and the parent-child encoding enables the model
to modulate attention based on directed nature of a forward chain (child joint
angles do not impact the parent joint’s 3D pose).

Attention Bias Computation. For each attention head and encoder layer, we have
separate embeddings $s$, $p$, and $c$ and compute attention as:
$A_{ij}=\frac{(h_{i}W_{Q})(h_{j}W_{K})^{T}}{\sqrt{d}}+s_{\phi^{\text{SPD}}(i,j)}+p_{\phi^{\text{P}}(i,j)}+c_{\phi^{\text{C}}(i,j)}$.

One key aspect of this approach is that this attention bias is invariant to
the ordering of the input tokens unlike the depth-first linearization approach
[4].

Output Heads. After passing $J$ tokens through each transformer encoder, GET
produces $J$ corresponding latent features. We include a policy head to
predict actions and a self-modeling head used to predict meta-features about
the embodiment. Further details on these heads are discussed in §4.3.

##  4 Case Study: In-hand Object Rotation

We conduct a case study on applying GET-Zero to an in-hand object rotation
task with different variations of LEAP Hand [8]. Unlike pick-and-place tasks
where a unified action space across different robot arms is easy to define
(e.g., end-effector space), the task of in-hand object rotation requires the
policy to differentiate low-level joint space actions that are aware of the
embodiment structure. Applying GET-Zero to the in-hand rotation task requires
three stages (Fig. 3): 1) generate embodiment variations (§4.1), 2) train
embodiment-specific experts using RL (§4.2), and 3) distill knowledge from
experts into an embodiment-aware GET architecture (§4.3).

###  4.1 Procedural Embodiment Generation

The LEAP Hand [8] consists of 3D printed components and off-the-shelf motors,
making it highly configurable. The hand features three main fingers which are
identical in structure and a thumb which has a separate structure. For the
main fingers, we construct five variations of the fingers (4 combinations + no
finger config) and two different variations of the thumb by removing various
combinations of joints (Fig. 4). This yields $5^{3}\times 2=250$ embodiment
configurations from which we require 1) at least two fingers have at least one
joint and 2) that there is at least one main finger with two joints, yielding
236 graph variation embodiments. We additionally introduce 1.5cm link length
extensions shown in orange in Fig. 4 to the 236 designs to generate additional
hand designs.

Figure 3: Training procedure. To train GET-Zero, we follow a teacher-student
paradigam, where the teachers are separate embodiment-specific experts trained
using RL. Then we distill knowledge from the experts into a single embodiment-
aware transformer (i.e., the student policy) using behavior cloning with a
self-modeling loss. GET-Zero takes as input embodiment definition and
proprioception, and infers proper actions to perform an in-hand rotation task
for an unseen embodiment.

###  4.2 Train Embodiment-Specific Experts

We train embodiment-specific RL experts using PPO [34] in Isaac Gym [35] to
complete the in-hand rotation task, adapted from the setup by Shaw et al. [8].
This setup consists of rewards for rotation along the yaw axis, penalties for
high motor torques and dropping the cube, and domain randomization across cube
sizes. These learned policies operate only on proprioceptive joint states and
predict delta joint targets that guide a PD controller. We pick the best of
five training seeds for each embodiment and filter out embodiments that are
not able to complete a full $2\pi$ rotation within $30$s. Many hand designs
are unable to complete the rotation task under this threshold due to many
missing joints, resulting in 44 training embodiments with associated expert
policies.

###  4.3 Distill Experts into Embodiment-Aware Transformer

Figure 4: Finger Variations. We procedurally generate variations of the
fingers in the LEAP Hand [8] by removing joints and links as well as adding in
1.5cm link length extensions (orange).

By rolling out embodiment-specific experts we can collect demonstration data
for each hand. Specifically, we collect 7 hours worth of demonstrations for
each embodiment. This demonstration data is combined with embodiment
information to form the training dataset of (state, action, embodiment) tuples
used to train the embodiment-aware GET model with behavior cloning (BC).

Separating the collection of demonstration data from the training of
embodiment-aware policies gives more freedom to use any existing policy
architecture or training setup to generate expert demonstrations (e.g.
teleoperation, RL, optimization based methods, privilaged information, etc.).
Prior embodiment-aware methods [3, 4] do not separate these stages and instead
jointly learn the embodiment-aware policy along with learning how to control
training embodiments.

GET Training Setup. We train the GET architecture (§3) using BC. Specifically,
our token input consists normalized joint angles and PD target joint states
($o_{vl}$), 3D joint position and joint rotation with respect to the parent
joint from the URDF file ($o_{fl}$), and a sinusoidal phase encoding with a
period of two seconds to encode cyclic progression through the rotation cycle
($o_{vg}$). The directed embodiment graph is an additional input used in the
graph attention encoding (§3.2).

Policy Head. The network contains a policy head for predicting single-step
action. Each joint has a PD target state that is initialized to the starting
joint state. The action head predicts delta actions that are added to this
target state, which sets the target state for the low-level PD controller. We
supervise these actions using $L_{2}$ loss with the demonstration data.

Self-Model Head. The network also has a self-modeling head to predict the
current 3D pose of each joint in the robot local frame. This forward
kinematics (FK) task is simple and generally applicable across embodiments,
yet requires the network to use the spatial relationships from the graph and
current joint angles along the FK chain. We supervise with the ground truth FK
pose with $L_{2}$ loss. As the policy and self-modeling heads operate on the
same latent representations output by GET, self-modeling encourages the
actions to be predictable from an embodiment-rich representation.

##  5 Experiments

We procedurally generate 236 graph variations of the LEAP hand [8] by removing
various combinations of joints and associated links from the embodiment
(§4.1). From these, we obtain 44 embodiments with graph variations with
associated expert RL policies that achieve a baseline level of performance
(§4.2). These 44 embodiments serve as training embodiments and only have graph
variations (link length extensions are not seen during training). Our
experiments consist of using demonstration data from these embodiments to
train various baselines and ablations of the GET architecture to evaluate
zero-shot performance to unseen embodiment graphs and link extensions.

Comparisons. MetaMorph [4] and Amorpheus [3] models are two prior embodiment-
aware transformer methods. These methods also use a transformer but do not
contain a full graph encoding or self modeling loss. We re-implement the
positional encoding aspects of these method in our setup to focus on comparing
the impact of our graph representation and self-modeling rather than
differences in task (locomotion v.s. manipulation), training procedure (RL
v.s. BC) or observation format.

  * •

ET. The Embodiment Transformer (ET) matches the architecture and token format
of GET except without the graph encoding (§3.2) or the self modeling loss.
This ET baseline most similarly matches the encoding method in Amorpheus [3],
which uses no graph encoding. This means that the model has no explicit
connectivity information about the joints.

  * •

ET+DFS. This baseline uses the same positional encoding scheme as MetaMorph
[4] that linearizes the embodiment graph in depth-first search (DFS) ordering
and then applies a learned positional encoding onto the linear sequence.

We also ablate different components of our method such as self-modeling loss
(SL), spatial encoding (SE), parent-child encoding (PE). Results are
summarized in Tab. 1.

Evaluation Metric. The task performance is measured by average rotational
velocity along the yaw axis in degrees/second including standard deviation
across five seeds. We compute results by rolling out the embodiment-aware
policies in a physics simulator for a total of 42 minutes of execution time
per embodiment per seed. Each category averages over 10 unseen embodiments
(Appendix Fig. 6).

###  5.1 Key Results and Findings

| Training Graph | New Graph | New Geo | New Graph & Geo
---|---|---|---|---
RL experts | 16.50 | — | — | —
ET [3] | 14.82$\pm$0.15 | 8.68$\pm$0.34 | 12.92$\pm$0.36 | 8.12$\pm$0.46
ET+DFS [4] | 15.03$\pm$0.37 | 6.45$\pm$0.28 | 14.37$\pm$0.34 | 6.27$\pm$0.22
ET+SL | 14.68$\pm$0.33 | 8.19$\pm$0.74 | 12.71$\pm$0.72 | 7.60$\pm$1.12
ET+PE+SE | 16.03$\pm$0.19 | 9.65$\pm$0.23 | 15.30$\pm$0.29 | 9.05$\pm$0.42
ET+PE+SL | 16.10$\pm$0.22 | 9.56$\pm$0.47 | 15.59$\pm$0.32 | 8.81$\pm$0.42
ET+SE+SL | 16.24$\pm$0.21 | 10.04$\pm$0.15 | 15.74$\pm$0.25 | 9.75$\pm$0.20
ET+PE+SE+SL (GET-Zero) | 16.32$\bm{\pm}$0.24 | 10.07$\bm{\pm}$0.58 | 15.80$\bm{\pm}$0.29 | 9.75$\bm{\pm}$0.54
Table 1: Simulation results (average rotational velocity deg/sec). RL experts
are embodiment-specific policies. ET: Embodiment transformer, DFS: depth-first
graph encoding, SL: self-modeling loss, SE: spatial graph encoding, PE:
parent-child graph encoding. We observe GET-Zero consistently outperforms
prior works [3, 4] through the use of graph encoding and self-modeling meta-
task.

Table 1 summarizes GET-Zero’s performance on training embodiments (Training
Graph) and on multiple categories of zero-shot embodiments: unseen graph
variations (New Graph), unseen link length variations (New Geo) and both (New
Graph & Geo). Note that link length variations (Fig. 4) were never seen during
training.

GET-Zero matches the performance of embodiment-specific experts on training
embodiments. Despite being trained to control 44 embodiments with graph
changes, GET-Zero reaches 99% of the performance of the embodiment-specific RL
experts with a single set of network weights (Tab. 1).

GET-Zero improves zero-shot capabilities to graph and geometry variations.
Compared to the best performing baseline for each category, GET-Zero achieves
a 16% improvement with zero-shot to new graph, an 10% improvement with zero-
shot to link length variations, and a 20% improvement with zero-shot to both
graph and link length variations (Tab. 1).

Self-modeling improves performance only with graph encoding present. We
observe that adding self-modeling (SL) to the baseline (ET) consistently
decreases zero-shot performance by an average of 4.6% across the three
categories, but yields an average increase of 5.1% when added to the model
with graph encoding (ET+PE+SE) (Table 1). This result intuitively makes sense
as the ET baseline has no graph encoding which is needed to complete the
forward kinematics task and thus self-modeling provides a poor training
signal. In the presence of graph encoding, forward kinematics serves as a
useful meta-task to improve cross-embodiment transfer.

Depth-first graph linearization lowers performance under graph variations. The
DFS linearization (ET+DFS) serves as a simple, yet incomplete graph
representation as the DFS ordering is not unique. Additionally, if a joint
earlier in the linearization is removed, then all subsequent joints will shift
down and receive a new positional encoding. When only link length variations
are present, linearization improves zero-shot performance over no positional
encoding (ET) by 11% (Tab. 1). However, when unseen graph changes are present,
performance drops by 26%. This result intuitively makes sense as the
linearized positional encoding can overfit to training embodiment graphs, but
fails to generalize with graph changes due to the aforementioned issue with
DFS. This motivates the need for the more complete graph representation used
in GET-Zero.

Spatial and parent-child encoding help policy learning. Relative to ET with
self-modeling (ET+SL), adding the parent-child graph encoding (ET+PE+SL)
improves performance by 19% and adding the spatial graph encoding (ET+SE+SL)
improves performance by 25% on average across zero-shot tasks. We note that
adding parent-child encoding provides only a minor improvement when the
spatial encoding is present, but that best performance is when both encodings
are present.

Figure 5: Impact of training embodiments on zero-shot graph generalization.

Training diversity improves zero-shot. As shown in Fig. 5, increasing
diversity of training embodiments improves zero shot generalization
performance to unseen graph changes. GET-Zero also achieve a reasonable
rotation performance even with only 10 training embodiments, and we observe
that GET-Zero achieves more data efficient generalization performance than ET.

|  |  |  |  |  |
---|---|---|---|---|---|---
Variation | Training | New Graph | New Graph | New Geo | New Graph+Geo | New Graph+Geo
ET | 11.5, 55%, 0 | 5.9, 50%, 0 | 9.2, 93%, 1 | 8.6, 57%, 2 | 9.6, 67%, 18 | -0.6, -6%, 0
GET-Zero | 20.5, 70%, 0 | 8.3, 65%, 0 | 21.8, 133%, 0 | 19.0, 83%, 1 | 9.0, 53%, 16 | 1.2, 9%, 0

Table 2: Real world evaluation. We evaluate sim-to-real capabilities of GET-
Zero and the ET baseline on LEAP Hand [8] design variations. [Col 1] An AR tag
tracks cube rotation. [Col 2] The unmodified LEAP Hand (seen during training).
[Col 3-7] Zero-shot embodiments with unseen graph variations and/or link
length extensions (orange). We report (across 3 minute trial): 1) average cube
angular velocity in degrees/second 2) real average velocity as a % of sim
average velocity for the same embodiment and policy (sim-to-real gap) and 3)
number of times the hand dropped the cube.

###  5.2 Real-world Evaluation

We present results from a real-world evaluation of GET-Zero on unseen
embodiments in Tab. 2. We observe GET-Zero outperforms the ET baseline for
both the original LEAP Hand [8] as well as unseen graph and link length
variations. Empirically, the baseline ET policy struggles to continuously
rotate the cube, and we observe high finger precision is required for stable
control. We observe a sim-to-real gap, but GET-Zero achieves zero-shot
performance above simulation for the third embodiment. For another embodiment
(second from right), a missing index finger causes the hand to drop the cube
many times. For the right-most embodiment, a shortened index finger and thumb
make it challenging for the hand to start the rotation cycle. Training with
link length extensions could help improve zero-shot performance to New
Graph+Geo. Please refer to the appendix for real-world evaluation methodology
and the website for videos.

###  5.3 Limitations

While GET-Zero improves zero-shot performance, it’s unlikely that our model
trained on LEAP Hand configurations would zero-shot transfer to a different
robot hand. For example, we train with at most 16 joints, and expect poor
performance for hands with more joints or fingers. Through compatible with the
GET tokenization, our model does not explicitly encode joint limit ranges,
motor strength, friction properties or finger shape, all of which vary with a
new hand and are important in manipulation tasks. We also assume that the hand
design space has enough variations to train on to expect reasonable
generalization performance. Additionally, GET-Zero assumes a fixed set of
embodiments, though future work could extend the model to iteratively improve
a robot’s design.

##  6 Conclusion

We present GET-Zero, an embodiment-aware model architecture and training
procedure that enables zero-shot control of new robot designs. Through a case
study on an in-hand object rotation task, we demonstrate the ability of our
model to control a wide range of hardware configurations of a multi-fingered
robot hand under variations in embodiment graph and geometry. Our results in
simulation and real demonstrate that the graph encoding and self-modeling
features in GET-Zero improve cross-embodiment transfer. Our goal is for GET-
Zero to serve as a useful, embodiment-aware method for the robotics community
to share knowledge across similar robot designs.

#### Acknowledgments

We would like to thank Kenneth Shaw, Prof. Pathak’s lab at CMU, and the Prof.
Liu’s Movement lab at Stanford for sharing the LEAP Hand hardware. We would
also like to thank Huy Ha, Xiaomeng Xu, Mengda Xu and Haochen Shi for their
helpful feedback and fruitful discussions. This work was supported in part by
Sloan Fellowship and NSF #2132519. The views and conclusions contained herein
are those of the authors and should not be interpreted as necessarily
representing the official policies, either expressed or implied, of the
sponsors.

## References

  * Octo Model Team et al. [2023] Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, Y. Tan, D. Sadigh, C. Finn, and S. Levine.  Octo: An open-source generalist robot policy.  https://octo-models.github.io, 2023.
  * Huang et al. [2020] W. Huang, I. Mordatch, and D. Pathak.  One policy to control them all: Shared modular policies for agent-agnostic control.  In _ICML_ , 2020.
  * Kurin et al. [2021] V. Kurin, M. Igl, T. Rocktäschel, W. Boehmer, and S. Whiteson.  My body is a cage: the role of morphology in graph-based incompatible control.  In _International Conference on Learning Representations_ , 2021.  URL https://openreview.net/forum?id=N3zUDGN5lO.
  * Gupta et al. [2022] A. Gupta, L. Fan, S. Ganguli, and L. Fei-Fei.  Metamorph: Learning universal controllers with transformers.  In _International Conference on Learning Representations_ , 2022.  URL https://openreview.net/forum?id=Opmqtk_GvYL.
  * Shafiee et al. [2023] M. Shafiee, G. Bellegarda, and A. Ijspeert.  Manyquadrupeds: Learning a single locomotion policy for diverse quadruped robots.  _arXiv preprint arXiv:2310.10486_ , 2023.
  * Bohlinger et al. [2024] N. Bohlinger, G. Czechmanowski, M. P. Krupka, P. Kicki, K. Walas, J. Peters, and D. Tateo.  One policy to run them all: Towards an end-to-end learning approach to multi-embodiment locomotion.  In _Workshop on Embodiment-Aware Robot Learning_ , 2024.  URL https://openreview.net/forum?id=HVWusz2zv5.
  * Vaswani et al. [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin.  Attention is all you need.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc., 2017.  URL https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
  * Shaw et al. [2023] K. Shaw, A. Agarwal, and D. Pathak.  Leap hand: Low-cost, efficient, and anthropomorphic hand for robot learning.  _Robotics: Science and Systems (RSS)_ , 2023.
  * Brohan et al. [2023] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, P. Florence, C. Fu, M. G. Arenas, K. Gopalakrishnan, K. Han, K. Hausman, A. Herzog, J. Hsu, B. Ichter, A. Irpan, N. Joshi, R. Julian, D. Kalashnikov, Y. Kuang, I. Leal, L. Lee, T.-W. E. Lee, S. Levine, Y. Lu, H. Michalewski, I. Mordatch, K. Pertsch, K. Rao, K. Reymann, M. Ryoo, G. Salazar, P. Sanketi, P. Sermanet, J. Singh, A. Singh, R. Soricut, H. Tran, V. Vanhoucke, Q. Vuong, A. Wahid, S. Welker, P. Wohlhart, J. Wu, F. Xia, T. Xiao, P. Xu, S. Xu, T. Yu, and B. Zitkovich.  Rt-2: Vision-language-action models transfer web knowledge to robotic control.  In _arXiv preprint arXiv:2307.15818_ , 2023.
  * Radosavovic et al. [2023] I. Radosavovic, B. Shi, L. Fu, K. Goldberg, T. Darrell, and J. Malik.  Robot learning with sensorimotor pre-training.  In _Conference on Robot Learning_ , pages 683–693. PMLR, 2023.
  * Reed et al. [2022] S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-maron, M. Giménez, Y. Sulsky, J. Kay, J. T. Springenberg, T. Eccles, J. Bruce, A. Razavi, A. Edwards, N. Heess, Y. Chen, R. Hadsell, O. Vinyals, M. Bordbar, and N. de Freitas.  A generalist agent.  _Transactions on Machine Learning Research_ , 2022.  ISSN 2835-8856.  URL https://openreview.net/forum?id=1ikK0kHjvj.  Featured Certification, Outstanding Certification.
  * Schubert et al. [2023] I. Schubert, J. Zhang, J. Bruce, S. Bechtle, E. Parisotto, M. Riedmiller, J. T. Springenberg, A. Byravan, L. Hasenclever, and N. Heess.  A generalist dynamics model for control.  _arXiv preprint arXiv:2305.10912_ , 2023.
  * Stone et al. [2023] A. Stone, T. Xiao, Y. Lu, K. Gopalakrishnan, K.-H. Lee, Q. Vuong, P. Wohlhart, S. Kirmani, B. Zitkovich, F. Xia, C. Finn, and K. Hausman.  Open-world object manipulation using pre-trained vision-language models.  In _arXiv preprint_ , 2023.
  * Driess et al. [2023] D. Driess, F. Xia, M. S. M. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, Y. Chebotar, P. Sermanet, D. Duckworth, S. Levine, V. Vanhoucke, K. Hausman, M. Toussaint, K. Greff, A. Zeng, I. Mordatch, and P. Florence.  Palm-e: An embodied multimodal language model.  In _arXiv preprint arXiv:2303.03378_ , 2023.
  * Kalashnikov et al. [2018] D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, et al.  Scalable deep reinforcement learning for vision-based robotic manipulation.  In _Conference on robot learning_ , pages 651–673. PMLR, 2018.
  * Collaboration et al. [2023] O. X.-E. Collaboration, A. Padalkar, A. Pooley, A. Jain, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, A. Rai, A. Singh, A. Brohan, A. Raffin, A. Wahid, B. Burgess-Limerick, B. Kim, B. Schölkopf, B. Ichter, C. Lu, C. Xu, C. Finn, C. Xu, C. Chi, C. Huang, C. Chan, C. Pan, C. Fu, C. Devin, D. Driess, D. Pathak, D. Shah, D. Büchler, D. Kalashnikov, D. Sadigh, E. Johns, F. Ceola, F. Xia, F. Stulp, G. Zhou, G. S. Sukhatme, G. Salhotra, G. Yan, G. Schiavi, H. Su, H.-S. Fang, H. Shi, H. B. Amor, H. I. Christensen, H. Furuta, H. Walke, H. Fang, I. Mordatch, I. Radosavovic, I. Leal, J. Liang, J. Kim, J. Schneider, J. Hsu, J. Bohg, J. Bingham, J. Wu, J. Wu, J. Luo, J. Gu, J. Tan, J. Oh, J. Malik, J. Tompson, J. Yang, J. J. Lim, J. Silvério, J. Han, K. Rao, K. Pertsch, K. Hausman, K. Go, K. Gopalakrishnan, K. Goldberg, K. Byrne, K. Oslund, K. Kawaharazuka, K. Zhang, K. Majd, K. Rana, K. Srinivasan, L. Y. Chen, L. Pinto, L. Tan, L. Ott, L. Lee, M. Tomizuka, M. Du, M. Ahn, M. Zhang, M. Ding, M. K. Srirama, M. Sharma, M. J. Kim, N. Kanazawa, N. Hansen, N. Heess, N. J. Joshi, N. Suenderhauf, N. D. Palo, N. M. M. Shafiullah, O. Mees, O. Kroemer, P. R. Sanketi, P. Wohlhart, P. Xu, P. Sermanet, P. Sundaresan, Q. Vuong, R. Rafailov, R. Tian, R. Doshi, R. Martín-Martín, R. Mendonca, R. Shah, R. Hoque, R. Julian, S. Bustamante, S. Kirmani, S. Levine, S. Moore, S. Bahl, S. Dass, S. Song, S. Xu, S. Haldar, S. Adebola, S. Guist, S. Nasiriany, S. Schaal, S. Welker, S. Tian, S. Dasari, S. Belkhale, T. Osa, T. Harada, T. Matsushima, T. Xiao, T. Yu, T. Ding, T. Davchev, T. Z. Zhao, T. Armstrong, T. Darrell, V. Jain, V. Vanhoucke, W. Zhan, W. Zhou, W. Burgard, X. Chen, X. Wang, X. Zhu, X. Li, Y. Lu, Y. Chebotar, Y. Zhou, Y. Zhu, Y. Xu, Y. Wang, Y. Bisk, Y. Cho, Y. Lee, Y. Cui, Y. hua Wu, Y. Tang, Y. Zhu, Y. Li, Y. Iwasawa, Y. Matsuo, Z. Xu, and Z. J. Cui.  Open X-Embodiment: Robotic learning datasets and RT-X models.  https://arxiv.org/abs/2310.08864, 2023.
  * Bousmalis et al. [2023] K. Bousmalis, G. Vezzani, D. Rao, C. Devin, A. X. Lee, M. Bauza, T. Davchev, Y. Zhou, A. Gupta, A. Raju, et al.  Robocat: A self-improving foundation agent for robotic manipulation.  _arXiv preprint arXiv:2306.11706_ , 2023.
  * Shah et al. [2023] D. Shah, A. Sridhar, N. Dashora, K. Stachowicz, K. Black, N. Hirose, and S. Levine.  Vint: A foundation model for visual navigation.  _arXiv preprint arXiv:2306.14846_ , 2023.
  * Yang et al. [2024] J. Yang, C. Glossop, A. Bhorkar, D. Shah, Q. Vuong, C. Finn, D. Sadigh, and S. Levine.  Pushing the limits of cross-embodiment learning for manipulation and navigation.  _arXiv preprint arXiv:2402.19432_ , 2024.
  * Martín-Martín et al. [2019] R. Martín-Martín, M. Lee, R. Gardner, S. Savarese, J. Bohg, and A. Garg.  Variable impedance control in end-effector space. an action space for reinforcement learning in contact rich tasks.  In _Proceedings of the International Conference of Intelligent Robots and Systems (IROS)_ , 2019.
  * Han and Trinkle [1998] L. Han and J. Trinkle.  Dextrous manipulation by rolling and finger gaiting.  In _Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146)_ , volume 1, pages 730–735 vol.1, 1998.  doi:10.1109/ROBOT.1998.677060.
  * Saut et al. [2007] J.-P. Saut, A. Sahbani, S. El-Khoury, and V. Perdereau.  Dexterous manipulation planning using probabilistic roadmaps in continuous grasp subspaces.  In _2007 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pages 2907–2912, 2007.  doi:10.1109/IROS.2007.4399090.
  * Rus [1999] D. Rus.  In-hand dexterous manipulation of piecewise-smooth 3-d objects.  _The International Journal of Robotics Research_ , 18(4):355–381, 1999.  doi:10.1177/02783649922066268.  URL https://doi.org/10.1177/02783649922066268.
  * Bai and Liu [2014] Y. Bai and C. K. Liu.  Dexterous manipulation using both palm and fingers.  In _2014 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1560–1565, 2014.  doi:10.1109/ICRA.2014.6907059.
  * Shao et al. [2020] L. Shao, F. Ferreira, M. Jorda, V. Nambiar, J. Luo, E. Solowjow, J. A. Ojea, O. Khatib, and J. Bohg.  Unigrasp: Learning a unified model to grasp with multifingered robotic hands.  _IEEE Robotics and Automation Letters_ , 5(2):2286–2293, 2020.  doi:10.1109/LRA.2020.2969946.
  * Chen et al. [2024] L. Y. Chen, K. Hari, K. Dharmarajan, C. Xu, Q. Vuong, and K. Goldberg.  Mirage: Cross-embodiment zero-shot policy transfer with cross-painting.  In _Proceedings of Robotics: Science and Systems_ , Delft, Netherlands, 2024.
  * Xu et al. [2023] M. Xu, Z. Xu, C. Chi, M. Veloso, and S. Song.  XSkill: Cross embodiment skill discovery.  In _7th Annual Conference on Robot Learning_ , 2023.  URL https://openreview.net/forum?id=8L6pHd9aS6w.
  * Chen et al. [2018] T. Chen, A. Murali, and A. Gupta.  Hardware conditioned policies for multi-robot transfer learning.  In _Advances in Neural Information Processing Systems_ , pages 9355–9366, 2018.
  * Zakka et al. [2021] K. Zakka, A. Zeng, P. Florence, J. Tompson, J. Bohg, and D. Dwibedi.  Xirl: Cross-embodiment inverse reinforcement learning.  _Conference on Robot Learning (CoRL)_ , 2021.
  * Wang et al. [2018] T. Wang, R. Liao, J. Ba, and S. Fidler.  Nervenet: Learning structured policy with graph neural networks.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=S1sqHMZCb.
  * Pathak et al. [2019] D. Pathak, C. Lu, T. Darrell, P. Isola, and A. A. Efros.  Learning to control self- assembling morphologies: A study of generalization via modularity.  In _NeurIPS_ , 2019.
  * Sferrazza et al. [2024] C. Sferrazza, D.-M. Huang, F. Liu, J. Lee, and P. Abbeel.  Body transformer: Leveraging robot embodiment for policy learning.  In _Workshop on Embodiment-Aware Robot Learning_ , 2024.  URL https://openreview.net/forum?id=IbXqRpANPD.
  * Ying et al. [2021] C. Ying, T. Cai, S. Luo, S. Zheng, G. Ke, D. He, Y. Shen, and T.-Y. Liu.  Do transformers really perform badly for graph representation?  In _Thirty-Fifth Conference on Neural Information Processing Systems_ , 2021.  URL https://openreview.net/forum?id=OeWooOxFwDa.
  * Schulman et al. [2017] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ , 2017.
  * Makoviychuk et al. [2021] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State.  Isaac gym: High performance GPU based physics simulation for robot learning.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_ , 2021.  URL https://openreview.net/forum?id=fgFBtYgJQX_.

##  A Appendix

We provide additional details on the procedural embodiment generation (§A.1),
real-world evaluations (§A.2), a forward-kinematics task (§A.3), and the GET
model and training parameters (§A.4).

###  A.1 Procedural Generation

In this section, we supplement the discussion in §4.1 with more details on the
procedural generation of LEAP Hand embodiments. In Fig. 6 we show the
diversity of zero-shot embodiments we use to evaluate GET-Zero spanning both
graph and link length variations. We were able to train embodiment-specific RL
experts for all 10 embodiments with graph variations (Fig. 6, row a). Doing so
lets us verify that these hardware configurations are capable of completing
the task with sufficient performance ($2\pi$ rotation completed in under 30
seconds), making them reasonable targets for zero-shot generalization.

We generate the zero-shot link length extension embodiments (Fig. 6 rows b and
c) by introducing link extensions to the graph variation embodiments. In
particular, for row b we randomly select 10 embodiments from our set of 44
training embodiments and introduce link extensions. For row c, we use the same
10 embodiments used to evaluate zero-shot graph generalization (row a). We
open source all of our code for this work on our project website, including
the scripts to procedurally generate embodiments as well as the generated URDF
files and assets.

Figure 6: Zero-shot embodiments. We evaluate zero-shot transfer to (a) new
graph variations, (b) link length extensions (orange), and (c) both link and
graph variations. For each category we evaluate ten embodiments in simulation
(pictured) and a subset to evaluate in real. Hands in the third row (link and
graph variations) match those in the first row (connectivity variations), but
with extensions applied to some links. We train on 44 hands with connectivity
variations (not pictured).

###  A.2 Real-world Evaluation Methodology

We conduct real world evaluation across 3 minutes of execution per embodiment
and sample our GET model at 20Hz. Every 20 seconds we pause execution,
automatically reset the hand to a starting pose, and manually place the cube
at the center of the palm. This matches the reset procedure used in the
simulation evaluations and addresses the issue where the cube could get stuck
in one challenging position for the entire evaluation.

Additionally, we attach an AR tag to the top of the cube and use the RGB
camera in an Intel Realsense D415 camera to track the AR tag pose to provide
the rotation metric. If the cube falls off the hand, the AR tag is no longer
visible and we pause execution, increment the drop counter, reset the hand
pose, and reset the cube.

###  A.3 Forward Kinematics Task

To further understand the self-modeling component of GET, we train a GET model
with only the forward kinematics prediction head and without the action head.
The main objectives are to validate 1) that the model can leverage the
embodiment graph to complete a downstream task and 2) that the learned graph
encoding appropriately transfers to previously unseen embodiments. We observe
that GET-Zero is able to achieve much higher precision predicting the per-
joint positions because of the graph encoding mechanism, which provides joint
graph connectivity information that is needed to accurately compute forward
kinematics (Tab. 3).

| New Graph
---|---
ET [3] | 7.97$\pm 1.36$
ET+DFS [4] | 9.03$\pm$3.22
GET-Zero (Ours) | 0.82$\bm{\pm}$0.81
Table 3: Zero-shot forward kinematics performance. Measurements are a 5 seed
average per-joint forward kinematics error in millimeters across 10 unseen
embodiments with graph variations.

###  A.4 GET Model and Training Parameters

We include the parameter choices for the GET architecture, the behavior
cloning training, and the dataset in Tab. 4. We refer to Shaw et al. [8] for
the embodiment-specific model architecture and training parameters used for RL
experts.

| Parameter | Value
---|---|---
| Token dimension | 512
| Token dropout | 0.1
| Encoder layers | 8
| Attention heads | 16
GET Model | Attention dropout | 0.1
| Graph embedding initialization | $\mathcal{N}(0,0.02)$
| Feedforward dimension | 1024
| Head hidden dimension | 16
| Head activation | elu
| Action loss weight | 1
| Self-modeling loss weight | 2
| Optimizer | Adam
| Adam $\beta_{1}$, $\beta_{2}$ | 0.9, 0.98
Behavior Cloning | Epochs | 7
| LR schedule | 15% Linear warmup with sqrt decay
| LR max | 1.5e-4
| Grad clip | 1
| Training + sim eval time | 8 hours on NVIDIA RTX A6000
| Training embodiments | 44
Dataset | Train samples per embodiment | 5e5
| Zero-shot embodiments (Fig. 6) | 30
Table 4: GET model, training and dataset parameters.