# Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for
Visual Robotic Manipulation

Hyunwoo Ryu1, Jiwoo Kim1, Junwoo Chang1, Hyun Seok Ahn1
Joohwan Seo2, Taehan Kim3, Yubin Kim4, Jongeun Choi1,2, Roberto Horowitz2,
1Yonsei University, 2University of California, Berkeley,
3Samsung Research, 4Massachusetts Institute of Technology
<EMAIL_ADDRESS>
{joohwan_seo<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Corresponding author: Jongeun Choi<EMAIL_ADDRESS>
###### Abstract

Recent studies have verified that equivariant methods can significantly
improve the data efficiency, generalizability, and robustness in robot
learning. Meanwhile, denoising diffusion-based generative modeling has
recently gained significant attention as a promising approach for robotic
manipulation learning from demonstrations with stochastic behaviors. In this
paper, we present _Diffusion-EDFs_ , a novel approach that incorporates
spatial roto-translation equivariance, i.e., $SE(3)$-equivariance to diffusion
generative modeling. By integrating $SE(3)$-equivariance into our model
architectures, we demonstrate that our proposed method exhibits remarkable
data efficiency, requiring only 5 to 10 task demonstrations for effective end-
to-end training. Furthermore, our approach showcases superior generalizability
compared to previous diffusion-based manipulation methods. Codes are available
at: https://github.com/tomato1mule/diffusion_edf

##  1 Introduction

Denoising diffusion-based generative models are increasingly being recognized
as superior methods for modeling stochastic and multimodal policies [1, 2, 3,
4, 5, 6, 7, 8, 9, 10, 11]. In particular, _$SE(3)$ -Diffusion Fields_ [7]
apply diffusion-based learning on the $SE(3)$ manifold to generate grasp poses
of the end-effector. However, while being proficient at learning highly
stochastic behaviors, current diffusion-based approaches require significant
amount of demonstration data for training.

On the other hand, extensive work has shown that incorporating equivariance
can significantly enhance the data efficiency, generalizability, and
robustness in robot learning [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 9, 10].
In particular, several recent works explore the use of $SE(3)$-equivariant
models for learning 6-DoF manipulation tasks from point cloud observations
[15, 13, 14, 19, 17]. _Equivariant Descriptor Fields_ (EDFs) [19] achieve few-
shot level data efficiency and generalizability in 6-DoF manipulation learning
by employing an $SE(3)$ _bi-equivariant_ [19, 20] energy-based model (EBM).
EDFs are distinguished from other $SE(3)$-equivariant methods in that they can
be trained and deployed in a fully end-to-end manner without requiring any
pre-training or object segmentation. However, training energy-based models is
typically slow and unstable due to the Markov Chain Monte Carlo sampling. For
instance, more than 10 hours are generally required for EDFs to stably learn
only a few task demonstrations. On the other hand, training is significantly
faster and more stable for diffusion models. Consequently, $SE(3)$-equivariant
diffusion-based generative modeling is highly desirable.

In this paper, we present _Diffusion-EDFs_ for bi-equivariant diffusion
generative modeling on the $SE(3)$-manifold. Bi-equivariance [19, 20] requires
the model to be equivariant to both the objects in the scene and grasp (see
Appendix A), enabling previously learned policies to be generalized to novel
object configurations. We first begin by formally discussing the required
conditions for achieving bi-equivariance in diffusion models. Next, we provide
novel practical methods to address the challenges that are posed to meet these
conditions. Finally, we validate the data efficiency, robustness, and
generalizability of Diffusion-EDFs by comparing them to other state-of-the-art
methods.

##  2 Preliminaries

###  2.1 SO(3) Group Representation Theory

A _representation_ $\mathbf{D}(g)$ is a map from a group $\mathcal{G}$ to a
linear map on a vector space $\mathcal{W}$ that is homomorphic to the group
action such that

$\mathbf{D}(g)\mathbf{D}(h)=\mathbf{D}(gh)\quad\forall g,h\in\mathcal{G}$ (1)

The vector space $\mathcal{W}$ where $\mathbf{D}(g)$ acts on is called the
_representation space_ of $\mathbf{D}(g)$. Two representations $\mathbf{D}(g)$
and $\mathbf{D}^{\prime}(g)$ are said to be _equivalent_ (not to be confused
with _equi variant_) if there exists a change of basis $W$ such that
$\mathbf{D}(g)=W\mathbf{D}^{\prime}(g)W^{-1}$. A representation is said to be
_reducible_ if it is equivalent to a block-diagonal of smaller
representations. _Irreducible representations_ , or _irreps_ are
representations that cannot be reduced anymore. Hence, irreps are the building
blocks of any larger representation of $\mathcal{G}$. We follow the E3NN [22]
formalism and use the term _irreducible vectors_ or _irreducible features_ to
refer to the vectors in the representation space of irreps.

For the special orthogonal group $SO(3)$, it is known that every irreps are
classified according to their angular frequency $l\in\\{0,1,2,\ldots\\}$, a
non-negative integer number also refered to as the _type_ , or _spin_. It is
known that any type-$l$, or spin-$l$ representations are equivalent
representations of the _real Wigner D-matrix_ of degree $l$, denoted as
$\mathbf{D}_{l}(R):SO(3)\rightarrow\mathbb{R}^{(2l+1)\times(2l+1)}$. We refer
to the irreducible vectors of $\mathbf{D}_{l}(R)$ as _type- $l$_, or _spin-
$l$ vectors_. Type-$0$ vectors are one-dimensional scalars that are invariant
to rotations such that $\mathbf{D}_{0}(R)=1$. On the other hand, type-$1$
vectors are typical $3$-dimensional Euclidean vectors that are rotated by the
rotation matrix $R$. In this paper, we follow the convention of E3NN [22] and
use the $x$-$y$-$z$ basis in which $\mathbf{D}_{1}(R)=R$. Note that
$\mathbf{D}_{0}(R)=1$ and $\mathbf{D}_{1}(R)=R$ have the angular frequency of
$0$ and $1$. Therefore, $\mathbf{D}_{0}$ is invariant to any rotation and
$\mathbf{D}_{1}$ is identical under full $360^{\circ}$ rotations. In general,
$\mathbf{D}_{l}(R)$ is identical when rotated by $\theta=2\pi/l$.

###  2.2 Equivariant Descriptor Fields and Bi-equivariance

An Equivariant Descriptor Field (EDF) [19] $\bm{\varphi}({\bm{x}}|O)$ is an
$SE(3)$-equivariant111More precisely, it is $SO(3)$-equivariant and
translation-invariant, forming a subset of $SE(3)$-equivariance. vector field
on $\mathbb{R}^{3}$ generated by a point cloud $O\in\mathcal{O}$. EDFs can be
reduced into the direct sum of irreducible subspaces

$\bm{\varphi}({\bm{x}}|O)=\bigoplus_{n=1}^{N}\bm{\varphi}^{(n)}({\bm{x}}|O)$

where
$\bm{\varphi}^{(n)}({\bm{x}}|O):\mathbb{R}^{3}\times\mathcal{O}\rightarrow\mathbb{R}^{2l_{n}+1}$
is an $SE(3)$-equivariant1 type-$l_{n}$ vector field generated by $O$. An EDF
$\bm{\varphi}({\bm{x}}|O)$ is transformed by $\Delta g=(\Delta{\bm{p}},\Delta
R)\in SE(3),\ \Delta{\bm{p}}\in\mathbb{R}^{3},\ \Delta R\in SO(3)$ as

$\bm{\varphi}(\Delta g\,{\bm{x}}|\Delta g\cdot O)=\mathbf{D}(\Delta
R)\bm{\varphi}({\bm{x}}|O)\quad\forall\ \Delta g=(\Delta{\bm{p}},\Delta R)\in
SE(3)$ (2)

where $\mathbf{D}(R)$ is the block-diagonal matrix whose sub-matrices are
Wigner D-matrices $\left\\{\mathbf{D}_{l_{n}}(R)\right\\}_{n=1}^{n=N}$. Ryu et
al. [19] proposed to use EDFs to construct a bi-equivariant energy-based model
for robotic object rearrangement tasks. We provide more explanations on bi-
equivariance in Appendix A.

###  2.3 Brownian Diffusion on the SE(3) Manifold

Let $g_{t}\in SE(3)$ be generated by diffusing $g_{0}\in SE(3)$ for time $t$.
The Brownian diffusion process is defined by the following Lie group
stochastic differential equation (SDE)

$g_{t+dt}=g_{t}\exp{\left[dW\right]}$ (3)

where $dW$ is the standard Wiener process on the Lie algebra
$\mathfrak{se}(3)$, which is the 6-dimensional vector space that serves as the
tangent space at the identity of $SE(3)$. The Brownian diffusion kernel
$P_{t|0}(g_{t}|g_{0})=\mathcal{B}_{t}(g_{0}^{-1}g_{t})$ corresponding to the
SDE in Equation (3) can be decomposed into rotational and translational parts
[23, 24], i.e.,

$\displaystyle\mathcal{B}_{t}(g)=$ $\displaystyle\
\mathcal{N}({\bm{p}};{\bm{\mu}}=\bm{0},\Sigma=tI)\
\mathcal{IG}_{SO(3)}(R;{\epsilon}=t/2)$ (4)
$\displaystyle\mathcal{IG}_{SO(3)}(R;\epsilon)=$
$\displaystyle\left[\sum_{l=0}^{\infty}(2l+1)e^{-\epsilon
l(l+1)}\frac{\sin{((2l+1)\theta/2)}}{\sin{\theta/2}}\right]$ (5)

where $\mathcal{N}$ is the normal distribution on $\mathbb{R}^{3}$,
$\mathcal{IG}_{SO(3)}$ is the isotropic Gaussian on $SO(3)$ [25, 26, 27, 28],
$g=({\bm{p}},R)\in SE(3),\ {\bm{p}}\in\mathbb{R}^{3},\ R\in SO(3)$, and
$\theta$ is the rotation angle of $SO(3)$ in the axis-angle parameterization.
$R$ can be sampled from $\mathcal{IG}_{SO(3)}$ with the following procedure
[27, 19, 28]. First, the rotation axis is sampled uniformly from the unit
sphere $S^{2}$. Next, $\theta$ is sampled using the inverse-CDF method,
ranging from $\theta=0$ to $\pi$. When calculating the cumulative distribution
function (CDF), Equation (5) must be multiplied by $(1-\cos{\theta})/\pi$.
This is because the invariant volume element of $SO(3)$ is
$dR=(1-\cos{\theta})/\pi\,d\theta dS^{2}$ in the axis-angle parameterization.

###  2.4 Annealed Langevin Markov Chain Monte Carlo on the SE(3) Manifold

Let $dP_{0}(g)=P_{0}(g)dg$ be a distribution on $SE(3)$ where $P_{0}(g)$ is
the invariant probability distribution function (PDF). The _Langevin dynamics_
for $dP_{0}(g)$ on $SE(3)$ is defined as follows [29, 30]:

$\displaystyle g_{\tau+d\tau}$
$\displaystyle=g_{\tau}\exp{\left[\frac{1}{2}\nabla\log{P_{0}(g)}d\tau+dW\right]}$

where the $i$-th component of $\nabla$ is the _Lie derivative_ [30] of $SE(3)$
along the $i$-th basis of the $\mathfrak{se}(3)$ Lie algebra. A Lie derivative
$\mathcal{L}_{\mathcal{V}}$ along $\mathcal{V}\in\mathfrak{se}(3)$ acts on a
differentiable function $f(g)$ on $SE(3)$ as

$\mathcal{L}_{\mathcal{V}}f(g)=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}f(g\exp\left[\epsilon\mathcal{V}\right])$

For brevity, we denote the Lie derivative along the $i$-th basis
$\hat{{\bm{e}}}_{i}\in\mathfrak{se}(3)$ as $\mathcal{L}_{i}$ instead of
$\mathcal{L}_{\hat{{\bm{e}}}_{i}}$ such that

$\nabla f(g)=\sum_{i=1}^{6}\mathcal{L}_{i}f(g)\,\hat{{\bm{e}}}_{i}.$

We denote the time for the Langevin dynamics as $\tau$, as we will reserve the
notation $t$ for the diffusion time. It is known that under mild assumptions,
this process converges to $P_{0}(g)$ as $\tau\rightarrow\infty$ regardless of
the initial distribution. Therefore, one may sample from $P_{0}(g)$ with
Langevin dynamics if the _score function_ ${\bm{s}}(g)=\nabla\log P_{0}(g)$ is
known.

However, it is known to be difficult and unstable to train and sample with the
score function for a sparse distribution [31, 32]. To address this issue,
_Annealed Langevin Markov Chain Monte Carlo_ [31, 7] method leverages the
score of the diffused marginal $P_{t}$ instead of $P_{0}$. A diffused marginal
$P_{t}(g)$ for a diffusion kernel $P_{t|0}(g|g_{0})$ is defined on the $SE(3)$
manifold as

$P_{t}(g)=\int_{g_{0}\in SE(3)}dg_{0}P_{t|0}(g|g_{0})P_{0}(g_{0})$ (6)

Annealed Langevin MCMC on the $SE(3)$ manifold is defined as follows [7, 23,
24, 33, 34, 28]:

$g_{\tau+d\tau}=g_{\tau}\exp{\left[\frac{1}{2}\nabla\log{P_{t(\tau)}(g)}d\tau+dW\right]}$
(7)

where $t(\tau)$ is the diffusion time scheduling, which is gradually annealed
to zero as $\tau\rightarrow\infty$, such that $t(\tau=\infty)=0$. Assuming the
ergodicity of Langevin dynamics, this process will converge to $P_{0}(g)$
regardless of the initial distribution if it is annealed sufficiently slowly
and $\underset{t\rightarrow 0}{\lim}P_{t}=P_{0}$.

Figure 1:  Overview of Diffusion-EDFs. (a) The target end-effector pose
$g_{0}$ is diffused in a bi-equivariant manner. (b) The end-effector pose is
sampled from the policy by denoising with learned bi-equivariant score
function. Due to the bi-equivariance, the trained policy can be effectively
generalized to unseen configurations in the observation of the scene and the
grasp.

##  3 Bi-equivariant Score Matching on the SE(3) Manifold

###  3.1 Problem Formulation

Let the target policy distribution222For notational simplicity, we do not
distinguish the probability distribution $dP=Pdg$ from the probability
distribution function (PDF) $P$ where $dg$ denotes the bi-invariant volume
form [30, 35, 36] on $SE(3)$. be $P_{0}(g_{0}|O^{(s)},O^{(b)})$, where
$g_{0}\in SE(3)$ is the target end-effector pose, and $O^{(s)}$ and $O^{(b)}$
are the observed point clouds of the scene and the grasped object,
respectively. Following Ryu et al. [19], we model this target policy to be bi-
equivariant (see Appendix A):

$P_{0}(\Delta g\,g|\Delta g\cdot
O^{(s)},O^{(b)})=P_{0}(g|O^{(s)},O^{(b)})=P_{0}(g\Delta g^{-1}|O^{(s)},\Delta
g\cdot O^{(b)})$ (8)

Now let $g_{t}\in SE(3)$ be the samples that are noised from $g_{0}$ by some
diffusion process, where $t$ denotes the diffusion time. A detailed
explanation of this diffusion process will be deferred to a subsequent
section. Our goal is to train a model that denoises $g_{t}$, which is sampled
from the diffused marginal distribution $P_{t}(g_{t}|O^{(s)},O^{(b)})$, into a
denoised sample $g$, which follows the target distribution
$P_{0}(g|O^{(s)},O^{(b)})$. This can be achieved with Annealed Langevin MCMC
if the _score function_ (see Section 2.4) of the diffused marginal
distribution is known. The score function
${\bm{s}}(g|O^{(s)},O^{(b)}):SE(3)\rightarrow\mathfrak{se}(3)$ for a
probability distribution $P(g|O^{(s)},O^{(b)})$ is defined as

${\bm{s}}(g|O^{(s)},O^{(b)})=\nabla\log{P(g|O^{(s)},O^{(b)})}=\sum_{i=1}^{6}\mathcal{L}_{i}\log{P(g|O^{(s)},O^{(b)})}\,\hat{{\bm{e}}}_{i}.$
(9)

###  3.2 Bi-equivariant Score Function

For a bi-equivariant distribution, the score function satisfies special
equivariance conditions that can be exploited for our model design.

######  Proposition 1.

Let a probability distribution $P(g|O^{(s)},O^{(b)})$ be bi-equivariant such
that it satisifies Equation (8). The score function then satisfies the
following two equivariance conditions:

$\displaystyle{\bm{s}}(\Delta g\,g|\Delta g\cdot O^{(s)},O^{(b)})$
$\displaystyle={\bm{s}}(g|O^{(s)},O^{(b)})$ $\displaystyle\forall\ \Delta g\in
SE(3)$ (10) $\displaystyle{\bm{s}}(g\,\Delta g^{-1}|O^{(s)},\Delta g\cdot
O^{(b)})$ $\displaystyle=\left[\mathrm{Ad}_{\Delta
g}\right]^{-T}{\bm{s}}(g|O^{(s)},O^{(b)})$ $\displaystyle\forall\ \Delta g\in
SE(3)$ (11)

$\mathrm{Ad}_{g}$ is the adjoint representation [30, 35, 37] of $SE(3)$ with
$g=({\bm{p}},R)$, ${\bm{p}}\in\mathbb{R}^{3}$, and $R\in SO(3)$

$Ad_{g}=\begin{bmatrix}R&[{\bm{p}}]^{\wedge}R\\\ \emptyset&R\end{bmatrix}$
(12)

where $[{\bm{p}}]^{\wedge}$ denotes the skew-symmetric $3\times 3$ matrix of
${\bm{p}}$. We follow the notation from Murray et al. [35] and place the
translational part in the upper rows. See Appendix C.1 for the proof of
Proposition 1. We refer to Equation (10) as the _left invariance_ of the
score, since $\Delta g$ comes to the left side of $g$, and the score remains
invariant. Likewise, we refer to Equation (11) as the _right equivariance_ of
the score, as the inverse of $\Delta g$ comes to the right side of $g$, and
the score is adjoint-transformed accordingly.

###  3.3 Bi-equivariant Diffusion Process

Let the point cloud conditioned diffusion kernel be
$P_{t|0}(g|g_{0},O^{(s)},O^{(b)})$ such that the diffused marginal
$P_{t}(g|O^{(s)},O^{(b)})$ for the initial target distribution
$P_{0}(g|O^{(s)},O^{(b)})$ is defined as follows:

$P_{t}(g|O^{(s)},O^{(b)})=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g|g_{0},O^{(s)},O^{(b)})P_{0}(g_{0}|O^{(s)},O^{(b)})$
(13)

If the diffused marginal $P_{t}(g|O^{(s)},O^{(b)})$ is bi-equivariant, one may
leverage the equivariance property of the target score function $\nabla\log
P_{t}(g|O^{(s)},O^{(b)})$ to design the score model.

######  Definition 1.

A bi-equivariant diffusion kernel $P_{t|0}(g|g_{0},\ O^{(s)},O^{(b)})$ is a
square-integrable kernel that satisfies the following equations for all
$\Delta g\in SE(3)$, except on a set of measure zero:

$\displaystyle P_{t|0}(g|g_{0},\ O^{(s)},O^{(b)})$
$\displaystyle=P_{t|0}(\Delta g\,g|\Delta g\,g_{0},\ \Delta g\cdot
O^{(s)},O^{(b)})$ (14) $\displaystyle=P_{t|0}(g\,\Delta g^{-1}|g_{0}\,\Delta
g^{-1},\ O^{(s)},\Delta g\cdot O^{(b)})$ (15)

######  Proposition 2.

The diffused margianl $P_{t}$ is guaranteed to be bi-equivariant for all bi-
equivariant initial distribution $P_{0}$ if and only if the diffusion kernel
$P_{t|0}$ is bi-equivariant.

See Appendix C.2 for the proof of Proposition 2. However, the Brownian
diffusion kernel $P_{t|0}(g|g_{0})=\mathcal{B}_{t}(g_{0}^{-1}g)$ in Equation
(4) is left invariant3 but not right invariant333 We use the term _invariance_
instead of equivariance since the kernel is neither conditioned by $O^{(s)}$
nor $O^{(b)}$. , that is

$P_{t|0}(\Delta g\,g|\Delta g\,g_{0})=P_{t|0}(g|g_{0}),\quad P_{t|0}(g\,\Delta
g^{-1}|g_{0}\,\Delta g^{-1})\neq P_{t|0}(g|g_{0})\qquad\forall\,\Delta g\in
SE(3)$

In fact, there exist no square-integrable kernel on $SE(3)$ that is bi-
invariant3 (see Appendix C.3). Therefore, we propose to let the diffusion
kernel be at least dependent on either $O^{(s)}$ or $O^{(b)}$ to absorb the
left or right group action of $\Delta g$. For generality, we keep both
$O^{(s)}$ and $O^{(b)}$ in the following.

While there may be numerous ways to implement such bi-equivariant diffusion
kernels, we use a diffusion kernel with an equivariant _diffusion frame
selection mechanism_ $P_{ref}(g_{ref}|g_{0}^{-1}\cdot O^{(s)},O^{(b)})$

$P_{t|0}(g|g_{0},O^{(s)},O^{(b)})=\int_{g_{ref}\in
SE(3)}dg_{ref}P_{ref}(g_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)})K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$ (16)

where $K_{t}(g_{0}^{-1}g)$ is any left invariant kernel444Note that any left
invariant kernel $K_{t}(g,g_{0})$ can be written as $K_{t}(g_{0}^{-1}g)$ (see
Appendix C.3). and $g_{ref}\in SE(3)$ is the reference frame of the diffusion
in the end-effector frame. The diffusion procedure is as follows:

  * D1.

A target pose $g_{0}\sim P_{0}(g_{0}|O^{(s)},O^{(b)})$ is sampled from the
demonstration policy.

  * D2.

A diffusion reference frame $g_{ref}\sim P_{ref}(g_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)})$ is sampled from the diffusion frame selection mechanism.

  * D3.

A diffusion displacement $\Delta g_{t|0}\sim K_{t}(\Delta g_{t|0})$ is sampled
from $K_{t}$ in Equation (16).

  * D4.

The sampled $\Delta g_{t|0}$ is applied to $g_{0}$ within the reference frame
of diffusion $g_{ref}$ such that $g_{t}=g_{0}\,g_{ref}\,\Delta
g_{t|0}\,g_{ref}^{-1}$ where $g_{t}\sim P_{t}$ is the diffused end-effector
pose.

######  Proposition 3.

The diffusion kernel in Equation (16) is bi-equivariant if the diffusion frame
selection mechanism $P_{ref}(g_{ref}|g_{0}^{-1}\cdot O^{(s)},O^{(b)})$
satisfies the following property:

$P_{ref}(g_{ref}|g_{0}^{-1}\cdot O^{(s)},O^{(b)})=P_{ref}(\Delta
g\,g_{ref}|(\Delta g\,g_{0}^{-1})\cdot O^{(s)},\Delta g\cdot O^{(b)})$ (17)

See Appendix C.4 for the proof. In practice, however, the orientational part
of the frame selection mechanism may be difficult to implement. Remarkably,
for the specific case in which $K_{t}$ is the Brownian diffusion kernel
$\mathcal{B}_{t}$, only the translation part of the frame selection is
required for Equation (16) to be bi-equivariant. For example, consider the
following frame selection mechanism

$P_{ref}(g_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)})=P_{origin}({\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)})\delta_{SO(3)}(R_{ref})$ (18)

where $\delta_{SO(3)}$ is the Dirac delta on $SO(3)$ such that the orientation
part of $g_{ref}$ is always same as the identity, and
$P_{origin}({\bm{p}}_{ref}|g_{0}^{-1}\cdot O^{(s)},O^{(b)})$ is the _diffusion
origin selection mechanism_.

######  Proposition 4.

The diffusion kernel $P_{t|0}$ in Equation (16) with the frame selection
mechanism $P_{ref}$ in Equation (18) is bi-equivariant if $K_{t}$ in Equation
(16) is the Brownian diffusion kernel555 In fact, any left-invariant kernel
that is also right-invariant to rotation as in Equation (40) can be used.
$\mathcal{B}_{t}$ and the origin selection mechanism $P_{origin}$ in Equation
(18) is equivariant such that

$P_{origin}\left({\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)=P_{origin}\left(\Delta g\,{\bm{p}}_{ref}|(\Delta
g\,g_{0}^{-1})\cdot O^{(s)},\Delta g\cdot O^{(b)}\right)$ (19)

We provide the proof in Appendix C.5. A concrete realization of such
equivariant diffusion origin selection mechanism
$P_{origin}({\bm{p}}_{ref}|g_{0}^{-1}\cdot O^{(s)},O^{(b)})$ is discussed in
Section 4.1.

###  3.4 Score Matching Objectives

In contrast to Song and Ermon [31], Urain et al. [7], our diffusion kernel
$P_{t|0}(g|g_{0},O^{(s)},O^{(b)})$ in Equation (16) is not the Brownian
kernel. Still, the following mean squared error (MSE) loss can be used to
train our score model ${\bm{s}}_{t}(g|O^{(s)},O^{(b)})$ without requiring the
integration of Equation (16):

$\mathcal{J}_{t}=\mathbb{E}_{g,g_{{}_{0}},g_{{}_{ref}},O^{(s)},O^{(b)}}\left[\frac{1}{2}\left\|{\bm{s}}_{t}(g|O^{(s)},O^{(b)})-\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right\|^{2}\right]$ (20)

where $g_{0}\sim P_{0}(g_{0}|O^{(s)},O^{(b)})$, $\ g_{ref}\sim
P_{ref}(g_{ref}|g_{0}^{-1}\cdot O^{(s)},O^{(b)})$, and $\ g\sim
P_{t|0}(g|g_{0},O^{(s)},O^{(b)})$. We optimize $\mathcal{J}_{t}$ for sampled
reference frame $g_{ref}$ and diffusion time $t$. The minimizer of
$\mathcal{J}_{t}$ is neither $\nabla\log K_{t}$ nor $\nabla\log P_{t|0}$ but
the score function of the diffused marginal $\nabla\log P_{t}$, that is

$\operatorname*{arg\,min}_{{\bm{s}}_{t}(g|O^{(s)},O^{(b)})}\mathcal{J}_{t}\ =\
{\bm{s}}_{t}^{*}(g|O^{(s)},O^{(b)})\ =\
\nabla\log{P_{t}({g|O^{(s)},O^{(b)}})}$ (21)

Although Equation (21) is a straightforward adaptation of the MSE minimizer
formula [31], we still provide the derivation in Appendix C.6 for
completeness. In practice, we use the Brownian diffusion kernel
$\mathcal{B}_{t}$ for $K_{t}$ to exploit Proposition 4. Therefore, training
with Equation (20) requires the computation of
$\nabla\log\mathcal{B}_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$. While autograd
packages are commonly used for the calculation of Lie derivaties [27, 19, 28,
7, 24, 23], we provide the explicit form in Appendix B.

###  3.5 Bi-equivariant Score Model

To model the left invariance and the right equivariance of the score function
as in Equation (10) and Equation (11), we propose the following score model on
$SE(3)$:

$\displaystyle{\bm{s}}_{t}(g|O^{(s)},O^{(b)})$
$\displaystyle={\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})\oplus{\bm{s}}_{\omega;t}(g|O^{(s)},O^{(b)})$
(22) $\displaystyle{\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\
\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$ (23)
$\displaystyle\begin{split}{\bm{s}}_{\omega;t}(g|O^{(s)},O^{(b)})&=\uwave{\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\
{\bm{x}}\times\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})}_{\raisebox{8.5359pt}{$\
{}_{\text{Orbital term}}$}}\\\ &\phantom{=}\,\,+\
\uwave{\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\ \rho_{\omega;t}({\bm{x}}|O^{(b)})\
\widetilde{{\bm{s}}}_{\omega;t}(g,{\bm{x}}|O^{(s)},O^{(b)})}_{\raisebox{8.5359pt}{$\
{}_{\text{Spin term}}$}}\end{split}$ (24)

where $\oplus$ and $\times$ denote direct sum (concatenation) and cross
product, respectively. We denote the translational part with subscript $\nu$
and rotational part with subscript $\omega$. In Equation (22), we decompose
the score model
${\bm{s}}_{t}(\cdot|O^{(s)},O^{(b)}):SE(3)\rightarrow\mathfrak{se}(3)\cong\mathbb{R}^{6}$
into the translational score model
${\bm{s}}_{\nu;t}(\cdot|O^{(s)},O^{(b)}):SE(3)\rightarrow\mathbb{R}^{3}$ and
the rotational score model
${\bm{s}}_{\omega;t}(\cdot|O^{(s)},O^{(b)}):SE(3)\rightarrow\mathfrak{so(3)}\cong\mathbb{R}^{3}$.
Next, we model these translational and rotational score model using two
different types of equivariant fields: 1) the equivariant density field
$\rho_{{}_{\square};t}(\cdot|O^{(b)}):\mathbb{R}^{3}\rightarrow\mathbb{R}_{\geq
0}$, and 2) the time-conditioned score field
$\widetilde{{\bm{s}}}_{{}_{\square};t}(\cdot|O^{(s)},O^{(b)}):SE(3)\times\mathbb{R}^{3}\rightarrow\mathbb{R}^{3}$,
where $\square$ is either $\omega$ or $\nu$.

######  Proposition 5.

The score model in Equation (22) is left invariant and right equivariant if
for $\square=\omega,\nu$ the density and score fields satisfy the following
equivariance conditions

$\displaystyle\rho_{{}_{\square};t}(\Delta g\,{\bm{x}}|\Delta g\cdot O^{(b)})$
$\displaystyle=\rho_{{}_{\square};t}({\bm{x}}|O^{(b)})$ $\displaystyle\forall\
\Delta g\in SE(3)$ (25)
$\displaystyle\widetilde{{\bm{s}}}_{{}_{\square};t}(\Delta
g\,g,{\bm{x}}|\Delta g\cdot O^{(s)},O^{(b)})$
$\displaystyle=\widetilde{{\bm{s}}}_{{}_{\square};t}(g,{\bm{x}}|O^{(s)},O^{(b)})$
$\displaystyle\forall\ \Delta g\in SE(3)$ (26)
$\displaystyle\widetilde{{\bm{s}}}_{{}_{\square};t}(g\,{\Delta g}^{-1},\Delta
g\,{\bm{x}}|O^{(s)},\Delta g\cdot O^{(b)})$ $\displaystyle=\Delta
R\,\widetilde{{\bm{s}}}_{{}_{\square};t}(g,{\bm{x}}|O^{(s)},O^{(b)})$
$\displaystyle\forall\ \Delta g\in SE(3)$ (27)

See Appendix C.7 for the proof. To achieve the left invariance (Equation (26))
and right equivariance (Equation (27)) of the score field, we propose using
the following model with two EDFs:

$\widetilde{{\bm{s}}}_{\square;t}(g,{\bm{x}}|O^{(s)},O^{(b)})=\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\
\otimes_{\square;t}^{(\rightarrow 1)}\
\mathbf{D}(R^{-1})\,\bm{\varphi}_{\square;t}(g\,{\bm{x}}|O^{(s)})$ (28)

where $\bm{\varphi}_{\square;t}$ and $\bm{\psi}_{\square;t}$ are two different
EDFs that respectively encode the point clouds $O^{(s)}$ and $O^{(b)}$, and
$\otimes_{\square;t}^{(\rightarrow 1)}$ is the time-conditioned equivariant
low-rank tensor product [38, 39] that maps the highly over-parametrized
equivariant descriptors into a type-$1$ vector.

######  Proposition 6.

The score field model in Equation (28) satisfies Equation (26) and Equation
(27).

We provide the proof of Proposition 6 in Appendix C.8. Note that the resulting
score field model
$\widetilde{{\bm{s}}}_{\square;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$ can be
considered as the type-$1$ generalization to the energy function of Ryu et al.
[19], which is a type-$0$ (rotation invariant scalar) function.

##  4 Implementation

In this section, we first provide the specific implementation of the bi-
equivariant diffusion frame selection mechanism, which was postponed in
Section 3.3. We thereafter propose the novel architectures of EDFs and density
fields proposed in Section 3.5.

###  4.1 Bi-equivariant Diffusion Frame Selection with Contact Heuristics

Real-world objects may be decomposed into local sub-geometries. For the most
of the cases, some sub-geometries are more significant than others in terms of
deciding the target end-effector pose. Therefore, a desirable model should
exhibit equivariance to important local sub-geometries instead of the global
geometry. Such importance of incorporating locality in equivariant methods has
recently been reported in various works [15, 19, 20, 40, 41].

To capture such important sub-geometries, we take into account the local
contact points. It is highly likely that sub-geometries with rich contact are
more important than those without any contact. A straightforawd way to inject
this inductive bias is to _select the reference frame of diffusion near the
contact points_. We find that this strategy enables our models to identify and
pay more attention to such contact-rich and relevant sub-geometries without
explicit supervision. Note that it only suffices to sample the origin of
diffusion, due to Proposition 4.

Let $n_{r}({\bm{x}},O)$ be the number of points in a point cloud $O$ that is
within a contact radius $r$ from a point ${\bm{x}}\in\mathbb{R}^{3}$. We use
the following diffusion origin selection mechanism with $r$ as a
hyperparameter.

$P_{origin}\left({\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)\propto\begin{cases}n_{r}\left({\bm{p}}_{ref},\
g_{0}^{-1}\cdot O^{(s)}\right)&\text{if }\ {\bm{p}}_{ref}\in O^{(b)}\\\
0&\text{else}\end{cases}$ (29)

For each point in $O^{(b)}$, we first calculate the number of neighboring
points in $g_{0}^{-1}\cdot O^{(s)}$. We then randomly sample a point from
$O^{(b)}$ as the origin of diffusion with the probability proportional to the
number of neighbors. Although this specific frame origin selection mechanism
allows tractable integration of Equation (16), we efficiently approximate it
by using the loss function in Equation (20) with a minibatch of a few
${\bm{p}}_{ref}$. Note that this approach is more general and can be applied
for an arbitrary frame selection mechanism whose integration of Equation (16)
is intractable.

###  4.2 Architecture of Equivariant Descriptor Fields

In practice, we use shared EDFs and density fields for $\square=\omega$ and
$\nu$ to save on computation. Therefore, the equivariant low-rank tensor
product $\otimes_{\square;t}^{(\rightarrow 1)}$ is the only operation that
distinguishes the linear score ($\nu$) from the angular score ($\omega$). We
also drop the time dependence for the density field ($\rho$) and the grasp EDF
($\bm{\psi}$) because evaluating them for every diffusion timestep $t$ is
inefficient. Hence, we denote these models as
$\bm{\varphi}_{t}({\bm{x}}|O^{(s)}),\ \bm{\psi}({\bm{x}}|O^{(b)}),\
\rho({\bm{x}}|O^{(b)})$. Consequently, time dependence is only left for the
tensor product $\otimes_{\square;t}^{(\rightarrow 1)}$ and the scene EDF
$\bm{\varphi}_{t}({\bm{x}}|O^{(s)})$.

Similar to Ryu et al. [19], Chatzipantazis et al. [40], we separate the EDFs
into a feature extractor and a field model (see Figure 2). The feature
extractor is a deep $SE(3)$-equivariant GNN based encoder that is run only
once in the beginning of the denoising process. Note that the EDFs used in our
score models must have significantly wide receptive fields for denoising
diffusion, such that they can cover the whole scene while keeping the local
high-frequency details. Therefore, a hierarchical modeling is required to
effectively capture this multiscale representations. For this reason, we
propose an UNet-like architecture based on Equiformer [42] for the feature
extractor, which is analogous to diffusion models for image generation.
However, the original Equiformer assumes source and destination points to be
identical, which cannot be met in our application. To address this
discrepancy, we slightly modify the architecture to accommodate skip
connections between source and destination points. We use the _Farthest Point
Sampling_ (FPS) algorithm [43] for the point pooling.

The encoded multiscale feature points from the feature extractor are then
passed into the much shallower field model for each denoising timestep. The
field model is also an Equiformer-based $SE(3)$-equivariant GNN but is much
shallower and hence much faster than the feature extractor. We find that only
a single layer is enough for the field model. While it is possible to stack
multiple layers for the field model as Chatzipantazis et al. [40], we find
this does not improve and even damages the accuracy of the estimated score in
our case, presumably due to the limited number of training data. The overview
of the modules in our architecture is illustrated in Figure 3.

Figure 2:  Architecture of multiscale EDF. Our multiscale EDF model is
composed of a feature extracting part and a field model part. See Figure 3 for
details on each module in the architecture. (a) The feature extractor encodes
the input point cloud into multiscale featured point clouds. We use an U-Net-
like GNN architecture for the feature extractor part. (b) The encoded
multiscale point clouds are passed into the field model part along with the
query point and optional time embedding. The field model part outputs the
time-conditioned EDF field value at the query point. We simply add the field
value for each scale to obtain the multiscale EDF field value, which is used
to infer the score.  Figure 3:  Overview of modules used in multiscale EDF.
(a) We employ Equiformer [42] to achieve $SE(3)$-equivariance in our model.
(b) We use an equivariant feed forward network with gate activation from
Equiformer. (c) We use radius graph to construct graph from points. Graph edge
length and orientation are respectively encoded by a radial encoder and
spherical harmonics [38, 39, 42]. (d) Multiple equiformer layers are stacked
and form the equivariant U-Net Block. FPS pooling is used in downward blocks
to obtain destination points of larger scale.

###  4.3 Score Model

#### Density Field.

We use the weighted query points model of Ryu et al. [19] for
$\rho({\bm{x}}|O)$

$\displaystyle{\bm{x}}^{(q)}_{n}$ $\displaystyle={\bm{q}}_{n}(O^{(b)})$ (30)
$\displaystyle\rho({\bm{x}}|O^{(b)})$
$\displaystyle=\sum_{n=1}^{N_{q}}w({\bm{x}}|O^{(b)})\delta^{(3)}({\bm{x}}-{\bm{x}}^{(q)}_{n})$
(31)

where ${\bm{q}}_{n}(O^{(b)})\in\mathbb{R}^{3}$ is the $n$-th _query point
function_ and $w(\cdot|O^{(b)}):\mathbb{R}^{3}\rightarrow\mathbb{R}_{\geq 0}$
is the _query weight field_ that assigns non-negative weights to each query
points. The query point function and query weight field are
$SE(3)$-equivariant such that

$\displaystyle{\bm{q}}_{n}(\Delta g\cdot O^{(b)})$ $\displaystyle=\Delta
g\,{\bm{q}}_{n}(O^{(b)})$ $\displaystyle\forall\Delta g\in SE(3)$
$\displaystyle w(\Delta g\,{\bm{x}}|\Delta g\cdot O^{(b)})$
$\displaystyle=w({\bm{x}}|O^{(b)})$ $\displaystyle\forall\Delta g\in SE(3)$

We use FPS algorithm for the query point function ${\bm{q}}_{n}(O)$. Although
it is not strictly deterministic, we observe negligible impact from this
stochasticity. For the implementation of the query weight field
$w({\bm{x}}|O)$, we use an EDF with a single scalar output.

#### Non-dimensionalization.

We assume that the output of the score field model in Equation (28) is a
dimensionless quantity. Therefore, we obtain the dimensionful score by taking

$\displaystyle\widetilde{{\bm{s}}}_{\nu;t}\rightarrow\frac{1}{L\sqrt{t}}\widetilde{{\bm{s}}}_{\nu;t},\qquad\
\widetilde{{\bm{s}}}_{\omega;t}\rightarrow\frac{1}{\sqrt{t}}\widetilde{{\bm{s}}}_{\omega;t}$

where $L$ is the characteristic length scale unit. The reason for dividing
$1/\sqrt{t}$ is because the norm of the target score tend to scale with
$O(1/\sqrt{t})$. Likewise, we divide the linear score field by $L$ because
score field is a gradient and thus scales reciprocally to the characteristic
length scale.

#### Score Function.

In conclusion, Equation (23) and Equation (24) become tractable summation
forms

$\displaystyle{\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})$
$\displaystyle=\frac{1}{L\sqrt{t}}\sum_{n=1}^{N_{q}}w({\bm{x}}^{(q)}_{n}|O^{(b)})\left[\bm{\psi}({\bm{x}}^{(q)}_{n}|O^{(b)})\,\otimes_{\nu;t}^{(\rightarrow
1)}\,\mathbf{D}(R^{-1})\,\bm{\varphi}_{t}(g\,{\bm{x}}^{(q)}_{n}|O^{(s)})\right]$
$\displaystyle{\bm{s}}_{\omega;t}(g|O^{(s)},O^{(b)})$
$\displaystyle=\frac{1}{\sqrt{t}}\sum_{n=1}^{N_{q}}w({\bm{x}}^{(q)}_{n}|O^{(b)})\,\frac{{\bm{x}}^{(q)}_{n}}{L}\times\left[\bm{\psi}({\bm{x}}^{(q)}_{n}|O^{(b)})\,\otimes_{\nu;t}^{(\rightarrow
1)}\,\mathbf{D}(R^{-1})\,\bm{\varphi}_{t}(g\,{\bm{x}}^{(q)}_{n}|O^{(s)})\right]$
$\displaystyle+\frac{1}{\sqrt{t}}\sum_{n=1}^{N_{q}}w({\bm{x}}^{(q)}_{n}|O^{(b)})\left[\bm{\psi}({\bm{x}}^{(q)}_{n}|O^{(b)})\,\otimes_{\omega;t}^{(\rightarrow
1)}\,\mathbf{D}(R^{-1})\,\bm{\varphi}_{t}(g\,{\bm{x}}^{(q)}_{n}|O^{(s)})\right]$

###  4.4 Sampling with Annealed Langevin Dynamics

For denoising, we use the trained scores for the annealed Langevin MCMC (see
section 2.4). The annealed Langevin MCMC for our trained score model on
$SE(3)$ is written as follows:

$\begin{split}g_{\tau+d\tau}=g_{\tau}\exp{\left[\frac{1}{2}{\bm{s}}_{t(\tau)}(g_{\tau}|O^{(s)},O^{(b)})d\tau+dW\right]}\end{split}$
(32)

This SDE can be discretized using the forward Euler-Maruyama method such that

$\begin{split}g_{n+1}=g_{n}\exp{\left[\frac{1}{2}{\bm{s}}_{t[n]}(g_{n}|O^{(s)},O^{(b)})\alpha[n]+\sqrt{\alpha[n]}{\bm{z}}_{n}\right]},\
\ {\bm{z}}_{n}\sim\mathcal{N}(\bm{0},I)\end{split}$ (33)

where $t[n]$ and $\alpha[n]$ are respectively the diffusion time and Langevin
step size, both of which are scheduled according to the step count $n$. A
commonly used scheduling scheme is taking $\alpha[n]\propto t[n]$ with either
a linear or log-linear $t[n]$ schedule [31, 44, 7]. However, the convergence
is very slow with this scheduling. Therefore, we use $\alpha[n]\propto
t[n]^{k_{1}}$ schedule with a hyperparameter ${k_{1}}<1$. To suppress the
instability caused by large step sizes when $t$ is small, we also gradually
lower the _temperature_ 666This temperature annealing should not be confused
with that of the _‘annealed’_ Langevin MCMC in which the diffusion time $t$ is
decreased. of the process. This can be done by using
$\sqrt{\alpha[n]T[n]}{\bm{z}}_{n}$ instead of $\sqrt{\alpha[n]}{\bm{z}}_{n}$
for the noise term with the temperature schedule $T[n]=t[n]^{k_{2}}$, where
${k_{2}}\geq 0$ is another hyperparameter. Intuitively, this makes the
sampling process to smoothly transition into a simple gradient descent
optimization as $t[n]\rightarrow 0$, and hence $T[n]\rightarrow 0$. We
empirically found that this strategy significantly improves the convergence
time without compromising the accuracy and diversity of the sampled poses. The
resulting proposed sampling algorithm with a small number $\epsilon$ is

$\begin{split}g_{n+1}=g_{n}\exp{\left[\frac{\epsilon}{2}{\bm{s}}_{t[n]}(g_{n}|O^{(s)},O^{(b)})t[n]^{k_{1}}+t[n]^{\frac{{k_{1}}+{k_{2}}}{2}}{\bm{z}}_{n}\right]},\
\ {\bm{z}}_{n}\sim\mathcal{N}(\bm{0},I)\end{split}$ (34)

We use ${k_{1}}=0.5$ and ${k_{2}}=1.0$ for the step size and temperature
scheduling. For the diffusion time $t[n]$, we use piecewise linear scheduling.
For example, we linearly schedule the diffusion time for $t=1$ to $t=0.1$ and
then with $t=0.1$ to $t=0.01$. Similar to diffusion-based image generation
models, we separate a low-resolution model and high-resolution model instead
of using a single model. We use the low-resolution model for higher $t$ and
the high-resolution model for lower $t$.

##  5 Experimental Results

We evaluate the data-efficiency, generalizability, and robustness of our
methods following a similar evaluation protocol as in Simeonov et al. [13,
14], Ryu et al. [19], Biza et al. [45]. In particular, we evaluate the pick-
and-place success rate for two different object categories: mugs and bottles,
in which a mug should be placed on a rack and a bottle on a tray. All the
models are trained with 10 task demonstrations (two demonstrations for five
different object instances), performed by humans, for each pick and place
task. Each demonstration is a tuple of $\left(O^{(s)},O^{(b)},g_{0}\right)$
where $O^{(s)}$ and $O^{(b)}$ are the point cloud observation of the scene and
grasp, and $g_{0}$ is the target end-effector pose.

Table 1: Pick-and-place success rates in various out-of-distribution settings
in simulated environment.

---
| Mug | Bottle
| Pick | Place | Total | Pick | Place | Total
Default (Trained Setup) |  |  |  |  |  |
R-NDFs† [14] w/ seg. |  0.83 |  0.97 |  0.81 |  0.91 |  0.73 |  0.67
R-NDFs† [14] w/o seg. |  0.00 |  0.00 |  0.00 |  0.00 |  0.00 |  0.00
SE(3)-DiffusionFields‡ [7] w/ seg. |  0.75 |  (n/a) |  (n/a) |  0.47 |  (n/a) |  (n/a)
SE(3)-DiffusionFields‡ [7] w/o seg. |  0.11 |  (n/a) |  (n/a) |  0.01 |  (n/a) |  (n/a)
Diffusion-EDFs∗ (Ours) |  0.99 |  0.96 |  0.95 |  0.97 |  0.85 |  0.83
Previously Unseen Instances |  |  |  |  |  |
R-NDFs† [14] w/ seg. |  0.73 |  0.70 |  0.51 |  0.90 |  0.87 |  0.79
R-NDFs† [14] w/o seg. |  0.00 |  0.00 |  0.00 |  0.00 |  0.00 |  0.00
SE(3)-DiffusionFields‡ [7] w/ seg. |  0.55 |  (n/a) |  (n/a) |  0.57 |  (n/a) |  (n/a)
SE(3)-DiffusionFields‡ [7] w/o seg. |  0.14 |  (n/a) |  (n/a) |  0.00 |  (n/a) |  (n/a)
Diffusion-EDFs∗ (Ours) |  0.96 |  0.96 |  0.92 |  0.99 |  0.91 |  0.90
Previously Unseen Poses |  |  |  |  |  |
R-NDFs† [14] w/ seg. |  0.84 |  0.93 |  0.78 |  0.65 |  0.72 |  0.47
R-NDFs† [14] w/o seg. |  0.00 |  0.00 |  0.00 |  0.00 |  0.00 |  0.00
SE(3)-DiffusionFields‡ [7] w/ seg. |  0.75 |  (n/a) |  (n/a) |  0.47 |  (n/a) |  (n/a)
SE(3)-DiffusionFields‡ [7] w/o seg. |  0.00 |  (n/a) |  (n/a) |  0.04 |  (n/a) |  (n/a)
Diffusion-EDFs∗ (Ours) |  0.98 |  0.98 |  0.96 |  0.98 |  0.81 |  0.79
Previously Unseen Clutters§ |  |  |  |  |  |
R-NDFs† [14] w/o seg. |  0.01 |  0.00 |  0.00 |  0.00 |  0.00 |  0.00
SE(3)-DiffusionFields‡ [7] w/o seg. |  0.06 |  (n/a) |  (n/a) |  0.03 |  (n/a) |  (n/a)
Diffusion-EDFs∗ (Ours) |  0.91 |  1.00 |  0.91 |  0.96 |  0.91 |  0.87
Previously Unseen |  |  |  |  |  |
Instances, Poses, & Clutters§ |  |  |  |  |  |
R-NDFs† [14] w/ seg. |  0.71 |  0.75 |  0.53 |  0.85 |  0.84 |  0.72
R-NDFs† [14] w/o seg. |  0.00 |  0.00 |  0.00 |  0.00 |  0.00 |  0.00
SE(3)-DiffusionFields‡ [7] w/ seg. |  0.58§ |  (n/a) |  (n/a) |  0.59§ |  (n/a) |  (n/a)
SE(3)-DiffusionFields‡ [7] w/o seg. |  0.03§ |  (n/a) |  (n/a) |  0.00§ |  (n/a) |  (n/a)
Diffusion-EDFs∗ (Ours) |  0.89 |  0.89 |  0.79 |  0.98 |  0.89 |  0.87
|  |  |  |  |  |

∗ No pre-training or segmentation pipeline is used for Diffusion-EDFs (Ours).
† Category specific self-supervised pre-training is used for R-NDFs.
‡ Rotational data augmentation is applied for training SE(3)-Diffusion Fields.
§ Models with segmented inputs are only tested for scenes without cluttered
objects to guarantee the perfectness of object segmentation.

We assess the generalizability and robustness of the methods under four
previously unseen scenarios: 1) previously unseen object instances (within the
trained object category), 2) previously unseen object poses, 3) previously
unseen clutters of distracting objects, 4) and all of the three combined.
Evaluations are performed in a simulated enviroment with SAPIEN [46] with nine
ceiling-mounted depth cameras and a Franka Panda robot. To minimize the
influence of the quality of the motion planning, we turn off the collision
between the environment and the robot during moving. Nonetheless, we fully
simulate the grasp, because in reality, the actual grasp pose may differ from
the intended pose, especially when a low-quality pose is inferred.

We compare our method (Diffusion-EDFs) against a state-of-the-art $SE(3)$ bi-
equivariant method (R-NDFs [14]) and a state-of-the-art denoising diffusion-
based method ($SE(3)$-Diffusion Fields [7]). We train our models only with ten
task demonstrations without resorting to any type of pre-training or object
segmentation or detection pipelines. In contrast, R-NDFs and $SE(3)$-Diffusion
Fields greatly suffer when the target object is not segmented out from the
scene point cloud. Therefore, we evaluate R-NDFs and $SE(3)$-Diffusion Fields
both with and without object segmentation pipelines. Furthermore, to achieve
reasonable success rate, R-NDFs require additional self-supervised training
which require massive amount of category specific object geometry data.
Therefore, we use the pre-trained weights from the original implementation of
R-NDFs [14] trained with ShapeNet [47] dataset and procedurally generated
object meshes. Still, we observe that R-NDFs fail to place the mug on a rack
due to the discrepancy of the shape of the rack in our experiment and the pre-
trained ones. Therefore, we do R-NDFs an additional favor of using the pre-
trained rack instead of our rack for the evaluation. Lastly, we help
$SE(3)$-Diffusion Fields by using rotational data augmentation, as they are
not $SE(3)$-equivariant. The experimental results are summarized in Table 1.

#### Analysis.

As shown in Table 1, Diffusion-EDFs consitently outperform both the
$SE(3)$-equivariant baseline (R-NDFs [14]) and diffusion model baseline
($SE(3)$-DiffusionFields [7]) in almost all scenarios. Notably, the baseline
models completely fail with unsegmented observations, whereas Diffusion-EDFs
maintain a total success rate exceeding 80% in most scenarios.

When dealing with segmented observations, the baseline models can perform
object pick/place tasks, yet their success rates remain inferior to those of
Diffusion-EDFs. Remarkably, Diffusion-EDFs achieve comparable or superior
performance without necessitating any form of pretraining, in contrast to
R-NDFs that rely on self-supervised pretraining that requires copious amount
of category-specific object geometry data. $SE(3)$-Diffusion Fields, despite
being trained with rotationally augmented data, lag significantly behind both
Diffusion-EDFs and R-NDFs. This sub-par performance is likely attributed to
the absence of SE(3)-equivariance in the model’s architecture.

Interestingly, all three methods consistently exhibit better performance for
the bottle pick-and-place task with previously unseen object instances. We
find this is due to an outlier bottle instance which has much taller shape
than others. The lower success rate of R-NDFs for bottles in previously unseen
poses is due to the occlusion of the point cloud observations. We also find
that $SE(3)$-DiffusionFields cannot distinguish the bottom of the bottle with
the top of the bottle, resulting in low success rate. We presume this is due
to the artifact of the global $SO(3)$ augmentation. On the other hand,
Diffusion-EDFs exhibit robustness to such partial occlusion or spurious
correlation with global geometry by only attending to relevant local sub-
geometry. We achieve such property with local contact frame referencing in
training as well as with careful architectural design to preserve locality
[19, 20, 15].

##  6 Related Works

#### Equivariant Robot Learning.

Several recent works utilize the planar roto-translation equivariance, or the
$SE(2)$-equivariance to improve data-efficiency, generalizability, and
robustness in robot learning with behavior cloning [12, 16, 48, 49, 50, 51,
52] and reinforcement learning [18, 53, 54, 55]. These approaches have also
been shown to be effective for problems that are not strictly
$SE(2)$-symmetric [56, 57]. However, these methods suffer with highly spatial
out-of-plane robotic manipulation tasks [19, 21]. To address this issue,
incorporating the full spatial roto-translation equivariance, or the
$SE(3)$-equivariance has been explored [13, 14, 19, 15, 20, 17, 9, 10].
Equivariant modeling has also been shown to be effective for learning robot
control [20, 58, 59, 60].

#### SE(3)-Equivariant Graph Neural Networks.

$SO(3)$\- and $SE(3)$-equivariant graph neural networks (GNNs) [38, 39, 61,
62, 42, 63, 64] are widely used to model the 3-dimensional roto-translation
symmetry in various domains including bioinformatics [65, 66, 67, 24, 23],
chemistry [38, 39, 42, 68], computer vision [40, 41, 69, 70, 71], and robotics
[13, 19, 17, 72]. In this paper, we use Equiformer [42] for the
$SE(3)$-equivariant GNN layers.

#### Diffusion Models.

Diffusion models are rapidly replacing previous generative models in various
fields including computer vision [44, 73, 74, 75, 76, 77, 78], bioinformatics
[23, 24, 79, 80], and robotics [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 81, 82]. In
particular, recent works studied diffusion-based generative modeling for non-
Euclidean Riemannian manifolds [33, 34] such as Lie groups [27, 28, 7, 24, 23,
5]. In robotics, Urain et al. [7], Simeonov et al. [5] apply diffusion-based
generative modeling directly on the end-effector pose space, which is the
$SE(3)$ manifold. The application of diffusion guidance is also explored in
robotics to modulate the denoising process so that actions with higher reward
or lower cost are more liekly to be sampled [1, 7, 8, 11]. $SE(3)$-equivariant
diffusion models on the product space of $SE(3)$ has been explored in
bioinformatics [23, 24] but not yet in robotics.

##  7 Conclusion

In this paper, we demonstrate the required conditions as well as specific
models and algorithms to incorporate the bi-equivariance in diffusion-based
generative modeling on $SE(3)$ with point cloud observations of the scene and
grasp. Our proposed method, Diffusion-EDFs, significantly enhance the slow
training time and instability of EDFs without losing the end-to-end
trainability. By experiment, we show that Diffusion-EDFs trained with only a
few task demonstrations can be generalized to previously unseen object
instances and poses, as well as the existence of previously unseen cluttered
distracting objects. In practice, however, the observed point clouds are often
noisy and with occlusions. Future research may explore methods that work
directly from images to address this issue. Another limitation of Diffusion-
EDFs is the inability of control-level or trajectory-level inference. The
application of geometric control framework [83, 59] or guided diffusion with
motion planning cost [1, 7] can be considered in subsequent works.

## Acknowledgments and Disclosure of Funding

This work was supported by the National Research Foundation of Korea (NRF)
grants funded by the Korea government (MSIT) (No.RS-2023-00221762 and No.
2021R1A2B5B01002620). This work was also supported by the Korea Institute of
Science and Technology (KIST) intramural grants (2E31570), and a Berkeley
Fellowship.

## References

  * Janner et al. [2022] Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine.  Planning with diffusion for flexible behavior synthesis.  _arXiv preprint arXiv:2205.09991_ , 2022.
  * Black et al. [2023] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine.  Training diffusion models with reinforcement learning.  _arXiv preprint arXiv:2305.13301_ , 2023.
  * Pearce et al. [2023] Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, et al.  Imitating human behaviour with diffusion models.  _arXiv preprint arXiv:2301.10677_ , 2023.
  * Chi et al. [2023] Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song.  Diffusion policy: Visuomotor policy learning via action diffusion.  _arXiv preprint arXiv:2303.04137_ , 2023.
  * Simeonov et al. [2023a] Anthony Simeonov, Ankit Goyal, Lucas Manuelli, Lin Yen-Chen, Alina Sarmiento, Alberto Rodriguez, Pulkit Agrawal, and Dieter Fox.  Shelving, stacking, hanging: Relational pose diffusion for multi-modal rearrangement.  _arXiv preprint arXiv:2307.04751_ , 2023a.
  * Liu et al. [2022] Weiyu Liu, Tucker Hermans, Sonia Chernova, and Chris Paxton.  Structdiffusion: Object-centric diffusion for semantic rearrangement of novel objects.  _arXiv preprint arXiv:2211.04604_ , 2022.
  * Urain et al. [2023] Julen Urain, Niklas Funk, Jan Peters, and Georgia Chalvatzaki.  SE(3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion.  _IEEE International Conference on Robotics and Automation (ICRA)_ , 2023.
  * Ajay et al. [2022] Anurag Ajay, Yilun Du, Abhi Gupta, Joshua Tenenbaum, Tommi Jaakkola, and Pulkit Agrawal.  Is conditional generative modeling all you need for decision-making?  _arXiv preprint arXiv:2211.15657_ , 2022.
  * Brehmer et al. [2023a] Johann Brehmer, Joey Bose, Pim De Haan, and Taco Cohen.  Edgi: Equivariant diffusion for planning with embodied agents.  _arXiv preprint arXiv:2303.12410_ , 2023a.
  * Brehmer et al. [2023b] Johann Brehmer, Pim De Haan, Sönke Behrends, and Taco Cohen.  Geometric algebra transformers.  _arXiv preprint arXiv:2305.18415_ , 2023b.
  * Mishra and Chen [2023] Utkarsh A Mishra and Yongxin Chen.  Reorientdiff: Diffusion model based reorientation for object manipulation.  _arXiv preprint arXiv:2303.12700_ , 2023.
  * Zeng et al. [2020] Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, and Johnny Lee.  Transporter networks: Rearranging the visual world for robotic manipulation.  _Conference on Robot Learning (CoRL)_ , 2020.
  * Simeonov et al. [2022] Anthony Simeonov, Yilun Du, Andrea Tagliasacchi, Joshua B Tenenbaum, Alberto Rodriguez, Pulkit Agrawal, and Vincent Sitzmann.  Neural descriptor fields: SE(3)-equivariant object representations for manipulation.  In _2022 International Conference on Robotics and Automation (ICRA)_ , pages 6394–6400. IEEE, 2022.
  * Simeonov et al. [2023b] Anthony Simeonov, Yilun Du, Yen-Chen Lin, Alberto Rodriguez Garcia, Leslie Pack Kaelbling, Tomás Lozano-Pérez, and Pulkit Agrawal.  SE(3)-equivariant relational rearrangement with neural descriptor fields.  In _Conference on Robot Learning_ , pages 835–846. PMLR, 2023b.
  * Chun et al. [2023] Ethan Chun, Yilun Du, Anthony Simeonov, Tomas Lozano-Perez, and Leslie Kaelbling.  Local neural descriptor fields: Locally conditioned object representations for manipulation.  _arXiv preprint arXiv:2302.03573_ , 2023.
  * Huang et al. [2022a] Haojie Huang, Dian Wang, Robin Walters, and Robert Platt.  Equivariant transporter network.  _arXiv preprint arXiv:2202.09400_ , 2022a.
  * Huang et al. [2023] Haojie Huang, Dian Wang, Xupeng Zhu, Robin Walters, and Robert Platt.  Edge grasp network: A graph-based SE(3)-invariant approach to grasp detection.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 3882–3888. IEEE, 2023.
  * Wang et al. [2022a] Dian Wang, Robin Walters, Xupeng Zhu, and Robert Platt.  Equivariant $q$ learning in spatial action spaces.  In _Conference on Robot Learning_ , pages 1713–1723. PMLR, 2022a.
  * Ryu et al. [2023] Hyunwoo Ryu, Hong in Lee, Jeong-Hoon Lee, and Jongeun Choi.  Equivariant descriptor fields: SE(3)-equivariant energy-based models for end-to-end visual robotic manipulation learning.  In _The Eleventh International Conference on Learning Representations_ , 2023.  URL https://openreview.net/forum?id=dnjZSPGmY5O.
  * Kim et al. [2023] Jiwoo Kim, Hyunwoo Ryu, Jongeun Choi, Joohwan Seo, Nikhil Potu Surya Prakash, Ruolin Li, and Roberto Horowitz.  Robotic manipulation learning with equivariant descriptor fields: Generative modeling, bi-equivariance, steerability, and locality.  In _RSS 2023 Workshop on Symmetries in Robot Learning_ , 2023.
  * Lin et al. [2023a] Yen-Chen Lin, Pete Florence, Andy Zeng, Jonathan T Barron, Yilun Du, Wei-Chiu Ma, Anthony Simeonov, Alberto Rodriguez Garcia, and Phillip Isola.  Mira: Mental imagery for robotic affordances.  In _Conference on Robot Learning_ , pages 1916–1927. PMLR, 2023a.
  * Geiger and Smidt [2022] Mario Geiger and Tess Smidt.  e3nn: Euclidean neural networks, 2022.  URL https://arxiv.org/abs/2207.09453.
  * Yim et al. [2023] Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola.  SE(3) diffusion model with application to protein backbone generation.  _arXiv preprint arXiv:2302.02277_ , 2023.
  * Corso et al. [2023] Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola.  Diffdock: Diffusion steps, twists, and turns for molecular docking.  _International Conference on Learning Representations (ICLR)_ , 2023\.
  * Nikolayev and Savyolov [1970] Dmitry I Nikolayev and Tatjana I Savyolov.  Normal distribution on the rotation group SO(3).  _Textures and Microstructures_ , 29, 1970.
  * Savyolova [1994] TM Ivanova TI Savyolova.  Normal distributions on SO(3).  In _Programming And Mathematical Techniques In Physics-Proceedings Of The Conference On Programming And Mathematical Methods For Solving Physical Problems_ , page 220. World Scientific, 1994.
  * Leach et al. [2022] Adam Leach, Sebastian M Schmon, Matteo T Degiacomi, and Chris G Willcocks.  Denoising diffusion probabilistic models on SO(3) for rotational alignment.  In _ICLR 2022 Workshop on Geometrical and Topological Representation Learning_ , 2022.
  * Jagvaral et al. [2022] Yesukhei Jagvaral, Francois Lanusse, and Rachel Mandelbaum.  Diffusion generative models on SO(3).  2022\.
  * Brockett [1997] Roger Brockett.  Notes on stochastic processes on manifolds.  In _Systems and Control in the Twenty-first Century_ , pages 75–100. Springer, 1997.
  * Chirikjian [2011] Gregory S Chirikjian.  _Stochastic models, information theory, and Lie groups, volume 2: Analytic methods and modern applications_ , volume 2.  Springer Science & Business Media, 2011.
  * Song and Ermon [2019] Yang Song and Stefano Ermon.  Generative modeling by estimating gradients of the data distribution.  _Advances in neural information processing systems_ , 32, 2019.
  * Koehler et al. [2022] Frederic Koehler, Alexander Heckett, and Andrej Risteski.  Statistical efficiency of score matching: The view from isoperimetry.  _arXiv preprint arXiv:2210.00726_ , 2022.
  * Bortoli et al. [2022] Valentin De Bortoli, Emile Mathieu, Michael John Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet.  Riemannian score-based generative modelling.  In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, _Advances in Neural Information Processing Systems_ , 2022.  URL https://openreview.net/forum?id=oDRQGo8I7P.
  * Huang et al. [2022b] Chin-Wei Huang, Milad Aghajohari, Joey Bose, Prakash Panangaden, and Aaron C Courville.  Riemannian diffusion models.  _Advances in Neural Information Processing Systems_ , 35:2750–2761, 2022b.
  * Murray et al. [2017] Richard M Murray, Zexiang Li, and S Shankar Sastry.  _A mathematical introduction to robotic manipulation_.  CRC press, 2017.
  * Zee [2016] Anthony Zee.  _Group theory in a nutshell for physicists_ , volume 17.  Princeton University Press, 2016.
  * Lynch and Park [2017] Kevin M Lynch and Frank C Park.  _Modern robotics_.  Cambridge University Press, 2017.
  * Thomas et al. [2018] Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley.  Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds.  _arXiv preprint arXiv:1802.08219_ , 2018.
  * Fuchs et al. [2020] Fabian Fuchs, Daniel Worrall, Volker Fischer, and Max Welling.  SE(3)-transformers: 3d roto-translation equivariant attention networks.  _Advances in neural information processing systems_ , 33:1970–1981, 2020.
  * Chatzipantazis et al. [2022] Evangelos Chatzipantazis, Stefanos Pertigkiozoglou, Edgar Dobriban, and Kostas Daniilidis.  SE(3)-equivariant attention networks for shape reconstruction in function space.  _arXiv preprint arXiv:2204.02394_ , 2022.
  * Deng et al. [2023] Congyue Deng, Jiahui Lei, Bokui Shen, Kostas Daniilidis, and Leonidas Guibas.  Banana: Banach fixed-point network for pointcloud segmentation with inter-part equivariance.  _arXiv preprint arXiv:2305.16314_ , 2023.
  * Liao and Smidt [2022] Yi-Lun Liao and Tess Smidt.  Equiformer: Equivariant graph attention transformer for 3d atomistic graphs.  _arXiv preprint arXiv:2206.11990_ , 2022.
  * Qi et al. [2017] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas.  Pointnet++: Deep hierarchical feature learning on point sets in a metric space.  _Advances in neural information processing systems_ , 30, 2017.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ , 33:6840–6851, 2020.
  * Biza et al. [2023] Ondrej Biza, Skye Thompson, Kishore Reddy Pagidi, Abhinav Kumar, Elise van der Pol, Robin Walters, Thomas Kipf, Jan-Willem van de Meent, Lawson LS Wong, and Robert Platt.  One-shot imitation learning via interaction warping.  _arXiv preprint arXiv:2306.12392_ , 2023.
  * Xiang et al. [2020] Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, Li Yi, Angel X. Chang, Leonidas J. Guibas, and Hao Su.  SAPIEN: A simulated part-based interactive environment.  In _The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2020.
  * Chang et al. [2015] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al.  Shapenet: An information-rich 3d model repository.  _arXiv preprint arXiv:1512.03012_ , 2015.
  * Seita et al. [2021] Daniel Seita, Pete Florence, Jonathan Tompson, Erwin Coumans, Vikas Sindhwani, Ken Goldberg, and Andy Zeng.  Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks.  In _2021 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 4568–4575. IEEE, 2021.
  * Wu et al. [2022] Hongtao Wu, Jikai Ye, Xin Meng, Chris Paxton, and Gregory S Chirikjian.  Transporters with visual foresight for solving unseen rearrangement tasks.  In _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 10756–10763. IEEE, 2022.
  * Teng et al. [2022] Yadong Teng, Huimin Lu, Yujie Li, Tohru Kamiya, Yoshihisa Nakatoh, Seiichi Serikawa, and Pengxiang Gao.  Multidimensional deformable object manipulation based on dn-transporter networks.  _IEEE Transactions on Intelligent Transportation Systems_ , 24(4):4532–4540, 2022.
  * Lim et al. [2022] Michael H Lim, Andy Zeng, Brian Ichter, Maryam Bandari, Erwin Coumans, Claire Tomlin, Stefan Schaal, and Aleksandra Faust.  Multi-task learning with sequence-conditioned transporter networks.  In _2022 International Conference on Robotics and Automation (ICRA)_ , pages 2489–2496. IEEE, 2022.
  * Jia et al. [2023] Mingxi Jia, Dian Wang, Guanang Su, David Klee, Xupeng Zhu, Robin Walters, and Robert Platt.  Seil: Simulation-augmented equivariant imitation learning.  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , pages 1845–1851. IEEE, 2023.
  * Wang et al. [2022b] Dian Wang, Robin Walters, and Robert Platt.  SO(2)-equivariant reinforcement learning.  _arXiv preprint arXiv:2203.04439_ , 2022b.
  * Zhu et al. [2022] Xupeng Zhu, Dian Wang, Ondrej Biza, Guanang Su, Robin Walters, and Robert Platt.  Sample efficient grasp learning using equivariant models.  _arXiv preprint arXiv:2202.09468_ , 2022.
  * Wang et al. [2022c] Dian Wang, Mingxi Jia, Xupeng Zhu, Robin Walters, and Robert Platt.  On-robot learning with equivariant models.  _arXiv preprint arXiv:2203.04923_ , 2022c.
  * Wang et al. [2022d] Dian Wang, Jung Yeon Park, Neel Sortur, Lawson LS Wong, Robin Walters, and Robert Platt.  The surprising effectiveness of equivariant models in domains with latent symmetry.  _arXiv preprint arXiv:2211.09231_ , 2022d.
  * Wang et al. [2023] Dian Wang, Xupeng Zhu, Jung Yeon Park, Robert Platt, and Robin Walters.  A general theory of correct, incorrect, and extrinsic equivariance.  _arXiv preprint arXiv:2303.04745_ , 2023.
  * Zhao et al. [2023] Linfeng Zhao, Jung Yeon Park, Xupeng Zhu, Robin Walters, and Lawson LS Wong.  SE(3) frame equivariance in dynamics modeling and reinforcement learning.  In _ICLR 2023 Workshop on Physics for Machine Learning_ , 2023.
  * Seo et al. [2023] Joohwan Seo, Nikhil Potu Surya Prakash, Xiang Zhang, Changhao Wang, Jongeun Choi, Masayoshi Tomizuka, and Roberto Horowitz.  Robot manipulation task learning by leveraging SE(3) group invariance and equivariance.  _arXiv preprint arXiv:2308.14984_ , 2023.
  * Kohler et al. [2023] Colin Kohler, Anuj Shrivatsav Srikanth, Eshan Arora, and Robert Platt.  Symmetric models for visual force policy learning.  _arXiv preprint arXiv:2308.14670_ , 2023.
  * Satorras et al. [2021] Vıctor Garcia Satorras, Emiel Hoogeboom, and Max Welling.  E(n) equivariant graph neural networks.  In _International conference on machine learning_ , pages 9323–9332. PMLR, 2021.
  * Deng et al. [2021] Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas J Guibas.  Vector neurons: A general framework for SO(3)-equivariant networks.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 12200–12209, 2021.
  * Du et al. [2022] Weitao Du, He Zhang, Yuanqi Du, Qi Meng, Wei Chen, Nanning Zheng, Bin Shao, and Tie-Yan Liu.  SE(3) equivariant graph neural networks with complete local frames.  In _International Conference on Machine Learning_ , pages 5583–5608. PMLR, 2022.
  * Liao et al. [2023] Yi-Lun Liao, Brandon Wood, Abhishek Das, and Tess Smidt.  Equiformerv2: Improved equivariant transformer for scaling to higher-degree representations.  _arXiv preprint arXiv:2306.12059_ , 2023.
  * Cramer [2021] Patrick Cramer.  Alphafold2 and the future of structural biology.  _Nature structural & molecular biology_, 28(9):704–705, 2021.
  * Lee et al. [2022] Jae Hyeon Lee, Payman Yadollahpour, Andrew Watkins, Nathan C Frey, Andrew Leaver-Fay, Stephen Ra, Kyunghyun Cho, Vladimir Gligorijević, Aviv Regev, and Richard Bonneau.  Equifold: Protein structure prediction with a novel coarse-grained structure representation.  _bioRxiv_ , pages 2022–10, 2022.
  * Ganea et al. [2021] Octavian-Eugen Ganea, Xinyuan Huang, Charlotte Bunne, Yatao Bian, Regina Barzilay, Tommi Jaakkola, and Andreas Krause.  Independent SE(3)-equivariant models for end-to-end rigid protein docking.  _arXiv preprint arXiv:2111.07786_ , 2021.
  * Batzner et al. [2022] Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky.  E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials.  _Nature communications_ , 13(1):2453, 2022.
  * Lin et al. [2023b] Chien Erh Lin, Jingwei Song, Ray Zhang, Minghan Zhu, and Maani Ghaffari.  SE(3)-equivariant point cloud-based place recognition.  In _Conference on Robot Learning_ , pages 1520–1530. PMLR, 2023b.
  * Zhu et al. [2023] Minghan Zhu, Maani Ghaffari, William A Clark, and Huei Peng.  E2pn: Efficient SE(3)-equivariant point network.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1223–1232, 2023.
  * Lei et al. [2023] Jiahui Lei, Congyue Deng, Karl Schmeckpeper, Leonidas Guibas, and Kostas Daniilidis.  Efem: Equivariant neural field expectation maximization for 3d object segmentation without scene supervision.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 4902–4912, 2023.
  * Fu et al. [2023] Jiahui Fu, Yilun Du, Kurran Singh, Joshua B Tenenbaum, and John J Leonard.  Neuse: Neural SE(3)-equivariant embedding for consistent spatial understanding with objects.  _arXiv preprint arXiv:2303.07308_ , 2023.
  * Song et al. [2020a] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  _arXiv preprint arXiv:2010.02502_ , 2020a.
  * Song et al. [2020b] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations.  _arXiv preprint arXiv:2011.13456_ , 2020b.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat gans on image synthesis.  _Advances in neural information processing systems_ , 34:8780–8794, 2021.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 1(2):3, 2022\.
  * Nichol et al. [2021] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen.  Glide: Towards photorealistic image generation and editing with text-guided diffusion models.  _arXiv preprint arXiv:2112.10741_ , 2021.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10684–10695, 2022.
  * Watson et al. [2022] Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E Eisenach, Woody Ahern, Andrew J Borst, Robert J Ragotte, Lukas F Milles, et al.  Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models.  _BioRxiv_ , pages 2022–12, 2022.
  * Du et al. [2023] Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl.  Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc.  In _International Conference on Machine Learning_ , pages 8489–8510. PMLR, 2023.
  * Chen et al. [2023] Hongyi Chen, Yilun Du, Yiye Chen, Joshua Tenenbaum, and Patricio A Vela.  Planning with sequence models through iterative energy minimization.  _arXiv preprint arXiv:2303.16189_ , 2023.
  * Dai et al. [2023] Yilun Dai, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel.  Learning universal policies via text-guided video generation.  _arXiv preprint arXiv:2302.00111_ , 2023.
  * Seo et al. [2022] Joohwan Seo, Nikhil Potu Surya Prakash, Alexander Rose, and Roberto Horowitz.  Geometric impedance control on SE(3) for robotic manipulators.  _arXiv preprint arXiv:2211.07945_ , 2022.
  * Chirikjian [2015] Gregory S Chirikjian.  Partial bi-invariance of SE(3) metrics.  _Journal of Computing and Information Science in Engineering_ , 15(1), 2015.
  * Chirikjian [2000] Gregory S Chirikjian.  _Engineering applications of noncommutative harmonic analysis: with emphasis on rotation and motion groups_.  CRC press, 2000.
  * Kyatkin and Chirikjian [1998] Alexander B Kyatkin and Gregory S Chirikjian.  Regularized solutions of a nonlinear convolution equation on the euclidean group.  _Acta Applicandae Mathematica_ , 53:89–123, 1998.

##  Appendix A Bi-equivariance

Figure 4:  Scene Equivariance and Grasp Equivariance. (a) The end-effector
pose must follow the transformation of the placement target within the scene.
This scene equivariance can be achieved by multiplying the transformation
$\Delta g$ on the left side of the end-effector pose. Therefore, we also refer
to this property as the left equivariance. (b) The end-effector pose must move
contravariantly to the transformation of the grasped object to compensate for
the changes. This grasp equivariance involves the inverse transformation
$\Delta g^{-1}$ coming to the right side of the end-effector pose. Therefore,
we also refer to this property as the right equivariance.

For robust pick-and-place manipulation, the trained policy needs to be
generalizable to previously unseen configurations of the target objects to
pick/place. This can be achieved by inferring end-effector poses that keep the
relative pose between the grasped object and the placement target. Note that
in our formulation, picking is essentially a special case of placing tasks, in
which the gripper is _placed_ at appropriate grasp points of the target object
to pick with an appropriate orientation.

Consider the scenario in which the policy is trained with a demonstration
$\left(g_{{}_{WE}},O^{(s)},O^{(b)}\right)$ in which $g_{{}_{WE}}$ is the end-
effector pose, and $O^{(s)}$ and $O^{(b)}$ are respectively the point cloud
observations of the scene and grasp. We denote the world frame using subscript
$W$ and the end-effector frame using subscript $E$. Note that $O^{(s)}$ is
observed in frame $W$ and $O^{(b)}$ in frame $E$. Now, let the placement
target be moved by $\Delta g=g_{{}_{W^{\prime}W}}$, inducing the
transformation of the observation $O^{(s)}\rightarrow\Delta g\,O^{(s)}$. This
is equivalent to changing the world reference frame from $W$ to $W^{\prime}$
with respect to the observation. Therefore, the end-effector pose should also
be transformed equivariantly as $g_{{}_{WE}}\rightarrow
g_{{}_{W^{\prime}E}}=\Delta g\,g_{{}_{WE}}$ (see Figure 4-(a)). This _scene
equivariance_ is also referred to as _left equivariance_ [19, 20], as the
transformation $\Delta g$ comes to the left side of $g_{{}_{WE}}$.

On the other hand, consider the transformation of the grasped object $\Delta
g=g_{{}_{E^{\prime}E}}$, which induces the transformation of the observation
$O^{(b)}\rightarrow\Delta g\,O^{(b)}$. This is equivalent to changing the end-
effector reference frame from $E$ to $E^{\prime}$ with respect to the
observation. In the world frame, this corresponds to the transformation of the
end-effector pose by $g_{{}_{WE}}\rightarrow
g_{{}_{WE^{\prime}}}=g_{{}_{WE}}\Delta g^{-1}$ (see Figure 4-(b)). This _grasp
equivariance_ is also referred to as _right equivariance_ [19, 20], as the
transformation $\Delta g^{-1}$ comes to the right side of $g_{{}_{WE}}$.
Combining these left and right equivariance conditions, we obtain the bi-
equivariance condition, which can be formally expressed in a probabilistic
form as Equation (8).

##  Appendix B Analytic Form of the Target Score in Equation (20)

In this section, we provide the analytic form of the target score function in
Equation (20)

$\nabla\log\mathcal{B}_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$ (35)

By definition, the $i$-th component of the target score function is calculated
as follows:

$\displaystyle\mathcal{L}_{i}\log\mathcal{B}_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{\mathcal{B}_{t}\left(g_{ref}^{-1}g_{0}^{-1}g\exp[\epsilon\hat{{\bm{e}}}_{i}]g_{ref}\right)}$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{\mathcal{B}_{t}\left(g_{ref}^{-1}g_{0}^{-1}g\,g_{ref}\exp[\epsilon
Ad_{g_{ref}^{-1}}\hat{{\bm{e}}}_{i}]\right)}$
$\displaystyle=\left[\mathcal{L}_{Ad_{g_{ref}^{-1}}\hat{{\bm{e}}}_{i}}\log\mathcal{B}_{t}\right]\left(g_{ref}^{-1}g_{0}^{-1}gg_{ref}\right)$
$\displaystyle=\left[\sum_{j=1}^{6}\left[Ad_{g_{ref}^{-1}}\right]_{ji}\mathcal{L}_{j}\log\mathcal{B}_{t}\right]\left(g_{ref}^{-1}g_{0}^{-1}gg_{ref}\right)$
$\displaystyle=\sum_{j=1}^{6}\left[Ad_{g_{ref}^{-1}}\right]_{ji}\left[\mathcal{L}_{j}\log\mathcal{B}_{t}\vphantom{1^{1^{1^{1^{1}}}}}\right]\left(g_{ref}^{-1}g_{0}^{-1}gg_{ref}\right)$
$\Rightarrow\nabla\log\mathcal{B}_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})=\left[Ad_{g_{ref}}\right]^{-T}\left[\nabla\log\mathcal{B}_{t}\right](g_{ref}^{-1}g_{0}^{-1}gg_{ref})$
(36)

Therefore, all we need is the score of the Brownian diffusion kernel
$\nabla\log\mathcal{B}_{t}(g)$ which can be decomposed into its translation
and rotation parts using Equation (4)

$\displaystyle\nabla\log\mathcal{B}_{t}(g)$
$\displaystyle=\nabla\log\mathcal{N}({\bm{p}};{\bm{\mu}}=\bm{0},\Sigma=tI)+\nabla\log\mathcal{IG}_{SO(3)}(R;{\epsilon}=t/2)$

where
$\nabla\log\mathcal{N}({\bm{p}};{\bm{\mu}}=\bm{0},\Sigma=tI)=-{\bm{p}}/t$ can
be easily computed. A common practice for the calculation of the rotational
part $\nabla\log\mathcal{IG}_{SO(3)}(R;{\epsilon}=t/2)$ is to use automatic
differentiation packages [27, 19, 28, 7, 24, 23]. However, the explicit form
can be easily calculated without automatic differentiation packages.

$\mathcal{L}_{i}\
\log\mathcal{IG}_{SO(3)}(R;{\epsilon})=\frac{\mathcal{L}_{i}\
\mathcal{IG}_{SO(3)}(R;{\epsilon})}{\mathcal{IG}_{SO(3)}(R;{\epsilon})}$ (37)
$\begin{split}&\mathcal{L}_{i}\ \mathcal{IG}_{SO(3)}(R;{\epsilon})\\\
&=\sum_{l=0}^{\infty}(2l+1)\exp\left[-l(l+1)\epsilon\right]\left[\frac{(l+1)\sin(l\theta)-l\sin((l+1)\theta)}{\cos(\theta)-1}\right]\left[\frac{-\mathrm{tr}\left[R[\hat{{\bm{e}}}_{i}]^{\wedge}\right]}{2\sin{\theta}}\right]\end{split}$
(38)

We denote the skew-symmetric matrix of the $i$-th $\mathfrak{so}(3)$ basis
$\hat{{\bm{e}}}_{i}$ as $[\hat{{\bm{e}}}_{i}]^{\wedge}$, whose matrix element
is $[\hat{{\bm{e}}}_{i}]^{\wedge}_{jk}=-\epsilon_{ijk}$ where $\epsilon_{ijk}$
is the Levi-Civita symbol.

The derivation is as follows. First, we rewrite Equation (5) with the
_character_ $\mathcal{X}(R)$ of $SO(3)$ [36].

$\displaystyle\mathcal{IG}_{SO(3)}(R;{\epsilon})$
$\displaystyle=\sum_{l=0}^{\infty}(2l+1)\exp\left[-l(l+1)\epsilon\right]\mathcal{X}_{l}(R)$
$\displaystyle\mathcal{X}_{l}(R)=\mathrm{tr}\left[D_{l}(R)\right]$
$\displaystyle=\sin\left((2l+1)\frac{\theta}{2}\right)/\sin(\frac{\theta}{2})$

$\theta\in(0,\pi)$ is the rotation angle of $R$. Now we calculate the Lie
derivative of $\mathcal{IG}_{SO(3)}$ as follows:

$\displaystyle\mathcal{L}_{i}\,\mathcal{IG}_{SO(3)}(R;{\epsilon})$
$\displaystyle=\sum_{l=0}^{\infty}(2l+1)\exp\left[-l(l+1)\epsilon\right]\mathcal{L}_{i}\,\mathcal{X}_{l}(R)$
$\displaystyle\mathcal{L}_{i}\,\mathcal{X}_{l}(R)$
$\displaystyle=\left[\frac{(l+1)\sin(l\theta)-l\sin((l+1)\theta)}{\cos(\theta)-1}\right]\mathcal{L}_{i}\,\theta$
$\displaystyle\mathcal{L}_{i}\,\theta$
$\displaystyle=\left[\frac{-1}{\sin{\theta}}\right]\mathcal{L}_{i}\left[\cos{\theta}\right]$

The last line can be easily calculated using
$\cos{\theta}=\frac{1}{2}\left(\mathrm{tr}\left[R\right]-1\right)$ and
$\mathcal{L}_{\mathcal{V}}\,\mathrm{tr}\left[R\right]=\mathrm{tr}\left[R[\mathcal{V}]^{\wedge}\right]$.

$\mathcal{L}_{i}\left[\cos{\theta}\right]=\frac{1}{2}\left(\mathrm{tr}\left[R[\hat{{\bm{e}}}_{i}]^{\wedge}\right]\right)$

Combining these results, one can derive Equation (38). In practice, the
infinite sum in Equation (38) is approximated with $\sum_{l=0}^{l_{max}}$
where $l_{max}=1000\sim 10000$, which can be computed within a millisecond
when appropriately parallelized. Although we have derived Equation (38) for
$\theta=(0,\pi)$, the result can be asymptotically extended to $\theta=0$ and
$\pi$ as $\mathcal{IG}_{SO(3)}$ is infinitely differentiable [25].

##  Appendix C Proofs and Derivations

###  C.1 Proof of Proposition 1

###### Proof of the left invariance of the score function.

$\displaystyle\mathcal{L}_{i}\log{P(\Delta g\,g|\Delta g\cdot
O^{(s)},O^{(b)})}$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{P}(\Delta
g\,g\exp{\left[\epsilon\hat{{\bm{e}}}_{i}\right]}|\Delta g\cdot
O^{(s)},O^{(b)})$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{P}(g\exp{\left[\epsilon\hat{{\bm{e}}}_{i}\right]}|O^{(s)},O^{(b)})$
$\displaystyle=\mathcal{L}_{i}\log{P}(g|O^{(s)},O^{(b)})$

where we used $P(\Delta g\,g|\Delta g\cdot
O^{(s)},O^{(b)})=P(g|O^{(s)},O^{(b)})$ in the second line. ∎

###### Proof of the right equivariance of the score function.

$\displaystyle\mathcal{L}_{i}\log{P}(g\,\Delta g^{-1}|O^{(s)},\Delta g\cdot
O^{(b)})$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{P}(g\,\Delta
g^{-1}\exp{\left[\epsilon\hat{{\bm{e}}}_{i}\right]}|O^{(s)},\Delta g\cdot
O^{(b)})$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{P}(g\,\Delta
g^{-1}\exp{\left[\epsilon\hat{{\bm{e}}}_{i}\right]}\Delta g|O^{(s)},O^{(b)})$
$\displaystyle=\left.\frac{d}{d\epsilon}\right|_{\epsilon=0}\log{P}(g\exp{\left[\epsilon
Ad_{\Delta g^{-1}}\hat{{\bm{e}}}_{i}\right]}|O^{(s)},O^{(b)})$
$\displaystyle=\mathcal{L}_{Ad_{\Delta
g^{-1}}\hat{{\bm{e}}}_{i}}\log{P}(g|O^{(s)},O^{(b)})$
$\displaystyle=\mathcal{L}_{\sum_{j}{\left[Ad_{\Delta
g^{-1}}\right]_{ji}\hat{{\bm{e}}}_{j}}}\log{P}(g|O^{(s)},O^{(b)})$
$\displaystyle=\sum_{j=1}^{6}{\left[Ad_{\Delta
g^{-1}}\right]_{ji}\mathcal{L}_{j}\log{P}(g|O^{(s)},O^{(b)})}$ ($\because$
Linearity of Lie-derivatives [30]
$\mathcal{L}_{\sum_{i}v_{i}\hat{{\bm{e}}}_{i}}=\sum_{i}v_{i}\,\mathcal{L}_{i}$)
$\displaystyle\Rightarrow\nabla\log{P}(g\,\Delta g^{-1}|O^{(s)},\Delta g\cdot
O^{(b)})$ $\displaystyle=\left[Ad_{\Delta
g^{-1}}\right]^{T}\nabla\log{P}(g|O^{(s)},O^{(b)})$
$\displaystyle=\left[Ad_{\Delta
g}\right]^{-T}\nabla\log{P}(g|O^{(s)},O^{(b)})$

where we denote the $(j,i)$-th matrix element of $Ad_{\Delta g^{-1}}$ with
$\left[Ad_{\Delta g^{-1}}\right]_{ji}\,$. We used $P(g\,\Delta
g^{-1}|O^{(s)},\Delta g\cdot O^{(b)})=P(g|O^{(s)},O^{(b)})$ in the second
line. ∎

###  C.2 Proof of Proposition 2

It is straightforward to prove the bi-equivariance of the diffused marginal
using the bi-invariance of the integral measure (Haar measure) $dg$

$\int_{g\in SE(3)}d(\Delta g\,g)=\int_{g\in SE(3)}dg=\int_{g\in
SE(3)}d(g\,\Delta g)\qquad\forall\Delta g\in SE(3)$ (39)

where
$dg=dR\,d{\bm{p}}=\frac{1}{8\pi^{2}}\left(\sin{\beta}\right)\,d\alpha\,d\beta\,d\gamma\,dx\,dy\,dz$
in the rotation-translation coordinate with the Euler angles
$\alpha,\beta,\gamma$ and the frame origin $x,y,z$. See Chirikjian [84, 30,
85], Murray et al. [35] and Appendix A of Ryu et al. [19] for more details on
the bi-invariant integral measure of $SE(3)$.

We first prove that the marginal is bi-equivariant if the kernel is bi-
equivariant.

###### Proof of left equivariance.

$\displaystyle P_{t}(g|O^{(s)},O^{(b)})$ $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g|g_{0},O^{(s)},O^{(b)})P_{0}(g_{0}|O^{(s)},O^{(b)})$
$\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g|g_{0},O^{(s)},O^{(b)})P_{0}(\Delta g\,g_{0}|\Delta
g\cdot O^{(s)},O^{(b)})$ ($\because$ Equation (8))
$\displaystyle=\int_{g_{0}\in SE(3)}dg_{0}\;P_{t|0}(\Delta g\,g|\Delta
g\,g_{0},\Delta g\cdot O^{(s)},O^{(b)})P_{0}(\Delta g\,g_{0}|\Delta g\cdot
O^{(s)},O^{(b)})$ ($\because$ Equation (14)) $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(\Delta g\,g|g_{0},\Delta g\cdot
O^{(s)},O^{(b)})P_{0}(g_{0}|\Delta g\cdot O^{(s)},O^{(b)})$ ($\because$
Equation (39), $\Delta g\,g_{0}\rightarrow g_{0}$) $\displaystyle=P_{t}(\Delta
g\,g|\Delta g\cdot O^{(s)},O^{(b)})$

∎

###### Proof of right equivariance.

$\displaystyle P_{t}(g|O^{(s)},O^{(b)})$ $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g|g_{0},O^{(s)},O^{(b)})P_{0}(g_{0}|O^{(s)},O^{(b)})$
$\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g|g_{0},O^{(s)},O^{(b)})P_{0}(g_{0}\,\Delta
g^{-1}|O^{(s)},\Delta g\cdot O^{(b)})$ ($\because$ Equation (8))
$\displaystyle=\int_{g_{0}\in SE(3)}dg_{0}\;P_{t|0}(g\,\Delta
g^{-1}|g_{0}\,\Delta g^{-1},O^{(s)},\Delta g\cdot O^{(b)})P_{0}(g_{0}\,\Delta
g^{-1}|O^{(s)},\Delta g\cdot O^{(b)})$ ($\because$ Equation (15))
$\displaystyle=\int_{g_{0}\in SE(3)}dg_{0}\;P_{t|0}(g\,\Delta
g^{-1}|g_{0},O^{(s)},\Delta g\cdot O^{(b)})P_{0}(g_{0}|O^{(s)},\Delta g\cdot
O^{(b)})$ ($\because$ Equation (39), $g_{0}\,\Delta g^{-1}\rightarrow g_{0}$)
$\displaystyle=P_{t}(g\,\Delta g^{-1}|O^{(s)},\Delta g\cdot O^{(b)})$

∎

Similarly, it can be proven that the kernel must be bi-equivariant (up to
measure zero) to guarantee the bi-equivariance of the diffused marginal for
any arbitrary initial distribution $dP_{0}=P_{0}\,dg_{0}$.

###### Proof.

$\displaystyle P_{t}(g|O^{(s)},O^{(b)})$ $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g|g_{0},O^{(s)},O^{(b)})P_{0}(g_{0}|O^{(s)},O^{(b)})$
$\displaystyle P_{t}(\Delta g\,g|\Delta g\cdot O^{(s)},O^{(b)})$
$\displaystyle=\int_{g_{0}\in SE(3)}dg_{0}\;P_{t|0}(\Delta g\,g|g_{0},\Delta
g\cdot O^{(s)},O^{(b)})P_{0}(g_{0}|\Delta g\cdot O^{(s)},O^{(b)})$
$\displaystyle=\int_{g_{0}\in SE(3)}dg_{0}\;P_{t|0}(\Delta g\,g|g_{0},\Delta
g\cdot O^{(s)},O^{(b)})P_{0}(\Delta g^{-1}\,g_{0}|O^{(s)},O^{(b)})$
($\because$ Equation (8)) $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(\Delta g\,g|\Delta g\,g_{0},\Delta g\cdot
O^{(s)},O^{(b)})P_{0}(g_{0}|O^{(s)},O^{(b)})$ ($\because$ Equation (39),
$g_{0}\rightarrow\Delta g\,g_{0}$) $\displaystyle P_{t}(g\,\Delta
g^{-1}|O^{(s)},\Delta g\cdot O^{(b)})$ $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g\,\Delta g^{-1}|g_{0},O^{(s)},\Delta g\cdot
O^{(b)})P_{0}(g_{0}|O^{(s)},\Delta g\cdot O^{(b)})$
$\displaystyle=\int_{g_{0}\in SE(3)}dg_{0}\;P_{t|0}(g\,\Delta
g^{-1}|g_{0},O^{(s)},\Delta g\cdot O^{(b)})P_{0}(g_{0}\,\Delta
g|O^{(s)},O^{(b)})$ ($\because$ Equation (8)) $\displaystyle=\int_{g_{0}\in
SE(3)}dg_{0}\;P_{t|0}(g\,\Delta g^{-1}|g_{0}\,\Delta g^{-1},O^{(s)},\Delta
g\cdot O^{(b)})P_{0}(g_{0}|O^{(s)},O^{(b)})$ ($\because$ Equation (39),
$g_{0}\rightarrow g_{0}\,\Delta g^{-1}$) $\displaystyle\Rightarrow\ $
$\displaystyle\int
dg_{0}\;P_{0}(g_{0}|O^{(s)},O^{(b)})\times\left[P_{t|0}(g|g_{0},O^{(s)},O^{(b)})-P_{t|0}(\Delta
g\,g|\Delta g\,g_{0},\Delta g\cdot O^{(s)},O^{(b)})\right]=0$
$\displaystyle\int
dg_{0}\;P_{0}(g_{0}|O^{(s)},O^{(b)})\times\left[P_{t|0}(g|g_{0},O^{(s)},O^{(b)})-P_{t|0}(g\,\Delta
g^{-1}|g_{0}\,\Delta g^{-1},O^{(s)},\Delta g\cdot O^{(b)})\right]=0$

Therefore, for this equation to hold for any arbitrary bi-equivariant initial
distribution $dP_{0}=P_{0}\,dg_{0}$, the diffusion kernel must be bi-
equivariant $\forall g,\Delta g\in SE(3)$

$\displaystyle P_{t|0}(g|g_{0},O^{(s)},O^{(b)})-$ $\displaystyle
P_{t|0}(\Delta g\,g|\Delta g\,g_{0},\Delta g\cdot O^{(s)},O^{(b)})=0$
$\displaystyle P_{t|0}(g|g_{0},O^{(s)},O^{(b)})-$ $\displaystyle
P_{t|0}(g\,\Delta g^{-1}|g_{0}\,\Delta g^{-1},O^{(s)},\Delta g\cdot
O^{(b)})=0$ $\displaystyle\Rightarrow\ P_{t|0}(g|g_{0},O^{(s)},O^{(b)})$
$\displaystyle=P_{t|0}(\Delta g\,g|\Delta g\,g_{0},\Delta g\cdot
O^{(s)},O^{(b)})$ $\displaystyle=P_{t|0}(g\,\Delta g^{-1}|g_{0}\,\Delta
g^{-1},O^{(s)},\Delta g\cdot O^{(b)})$

∎

###  C.3 Non-existence of Bi-Invariant Diffusion Kernels on SE(3)

Due to the left invariance of the kernel, it must depend solely on the group
difference $g_{0}^{-1}g$

$P_{t|0}(\Delta g\,g|\Delta g\,g_{0})=P_{t|0}(g|g_{0})\quad\forall\,\Delta
g,g\quad\Rightarrow\quad
P_{t|0}(g|g_{0})=P_{t|0}(g_{0}^{-1}g|I)\quad\forall\,g$

Therefore, we can simply denote any left-invariant kernel as
$K_{t}(g)=P_{t|0}(g|I)$. The right invariance requires this kernel to satisfy
$K_{t}(\Delta g\,g\,\Delta g^{-1})=K_{t}(g)$, meaning that it is a _class
function_ , which does not exist for $L^{2}(SE(3))$ [86, 85].

###  C.4 Proof of Proposition 3

###### Proof.

The right equivariance can be proved as follows.

$\displaystyle P_{t|0}(g|g_{0},O^{(s)},O^{(b)})$ $\displaystyle=\int
dg_{ref}P_{ref}(g_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)})K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$ $\displaystyle=\int
dg_{ref}P_{ref}(\Delta g\,g_{ref}|(\Delta g\,g_{0}^{-1})\cdot O^{(s)},\Delta
g\cdot O^{(b)})K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$ ($\because$ Equation
(17)) $\displaystyle=\int dg_{ref}P_{ref}(g_{ref}|(\Delta g\,g_{0}^{-1})\cdot
O^{(s)},\Delta g\cdot O^{(b)})K_{t}(g_{ref}^{-1}\left(g_{0}\,\Delta
g^{-1}\right)^{-1}\left(g\,\Delta g^{-1}\right)g_{ref})$ ($\because$
invariance of integral $\int dg_{ref}$ under $g_{ref}\rightarrow\Delta
g^{-1}\,g_{ref}$) $\displaystyle=P_{t|0}(g\,\Delta g^{-1}|g_{0}\,\Delta
g^{-1},O^{(s)},\Delta g\cdot O^{(b)})$

The left equivariance proof is straightforward using the following equations:

$\displaystyle g_{0}^{-1}\,g$ $\displaystyle=\left(\Delta
g\,g_{0}\right)^{-1}\left(\Delta g\,g\right)$ $\displaystyle g_{0}^{-1}\cdot
O^{(s)}$ $\displaystyle=\left(\Delta g\,g_{0}\right)^{-1}\cdot\left(\Delta
g\cdot O^{(s)}\right)$

∎

###  C.5 Proof of Proposition 4

Note that the Brownian diffusion kernel $\mathcal{B}_{t}(g)$ is right-
invariant to rotation, that is,

$\begin{split}\mathcal{B}_{t}\left((g_{0}\,\Delta R)^{-1}\,(g\,\Delta
R)\right)&=\mathcal{B}_{t}(g_{0}^{-1}\,g)\\\ \Rightarrow\mathcal{B}_{t}(\Delta
R^{-1}\,g\,\Delta R)&=\mathcal{B}_{t}(g)\qquad\forall\Delta R\in
SO(3)\end{split}$ (40)

where we abuse the notation to denote the action of a pure rotation $\Delta R$
on $g=({\bm{p}},R)$ as $\Delta R\,g=(\Delta R\,{\bm{p}},\Delta R\,R)$ and
$g\,\Delta R=({\bm{p}},R\,\Delta R)$. Equation (40) holds because the Gaussian
distribution in Equation (4) is rotation-invariant and $\mathcal{IG}_{SO(3)}$
in Equation (5) is a linear combination of _characters_ of $SO(3)$, which are
_class functions_ due to the permutation invariance of trace operations (see
Appendix B and C.3). Consider the following diffusion kernel with the
equivariant origin selection mechanism in Equation (19):

$P_{t|0}(g|g_{0},O^{(s)},O^{(b)})=\int_{\mathbb{R}^{3}}d{\bm{p}}_{ref}P_{origin}\left({\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)\mathcal{B}_{t}\left(\left(g_{0}\triangleleft{\bm{p}}_{ref}\right)^{-1}\left(g\triangleleft{\bm{p}}_{ref}\right)\right)$

where $\triangleleft\,{\bm{p}}_{ref}:SE(3)\rightarrow SE(3)$ denotes the right
action of a pure translation ${\bm{p}}_{ref}\in\mathbb{R}^{3}$ onto
$g=({\bm{p}},R)\in SE(3)$ such that

$g\,\triangleleft\,{\bm{p}}_{ref}=\begin{bmatrix}R&{\bm{p}}\\\
\emptyset&1\end{bmatrix}\begin{bmatrix}I&{\bm{p}}_{ref}\\\
\emptyset&1\end{bmatrix}=\begin{bmatrix}R&R\,{\bm{p}}_{ref}+{\bm{p}}\\\
\emptyset&1\end{bmatrix}$

Note that the following equation holds for all $g_{1},g_{2}\in SE(3)$ and
${\bm{p}}_{ref}\in\mathbb{R}^{3}$:

$\begin{split}(g_{1}\,g_{2})\,\triangleleft\,{\bm{p}}_{ref}&=\begin{bmatrix}R_{1}&{\bm{p}}_{1}\\\
\emptyset&1\end{bmatrix}\begin{bmatrix}R_{2}&{\bm{p}}_{2}\\\
\emptyset&1\end{bmatrix}\begin{bmatrix}I&{\bm{p}}_{ref}\\\
\emptyset&1\end{bmatrix}\\\
&=\begin{bmatrix}R_{1}\,R_{2}&R_{1}\left(R_{2}{\bm{p}}_{ref}+{\bm{p}}_{2}\right)+{\bm{p}}_{1}\\\
\emptyset&1\end{bmatrix}\\\ &=\begin{bmatrix}R_{1}&{\bm{p}}_{1}\\\
\emptyset&1\end{bmatrix}\begin{bmatrix}I&g_{2}\,{\bm{p}}_{ref}\\\
\emptyset&1\end{bmatrix}\begin{bmatrix}R_{2}&\mathbf{0}\\\
\emptyset&1\end{bmatrix}\\\
&=\left(g_{1}\triangleleft\left(g_{2}\,{\bm{p}}_{ref}\right)\right)\,\Delta
R_{2}\end{split}$ (41)

The bi-equivariance of $P_{t|0}$ can be proved using Equation (40) and
Equation (41).

###### Proof.

The proof of left equivariance is straightforward as $g_{0}^{-1}\cdot
O^{(s)}=\left(\Delta g\,g_{0}\right)^{-1}\cdot\left(\Delta g\cdot
O^{(s)}\right)$. The proof of right equivariance is as follows:

$\displaystyle P_{t|0}(g\,\Delta g^{-1}|g_{0}\,\Delta g^{-1},O^{(s)},\Delta
g\cdot O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d{\bm{p}}_{ref}P_{origin}\left({\bm{p}}_{ref}|(\Delta
g\,g_{0}^{-1})\cdot O^{(s)},\Delta g\cdot
O^{(b)}\right)\mathcal{B}_{t}\left(\left((g_{0}\,\Delta
g^{-1})\triangleleft{\bm{p}}_{ref}\right)^{-1}\left((g\,\Delta
g^{-1})\triangleleft{\bm{p}}_{ref}\right)\right)$
$\displaystyle=\int_{\mathbb{R}^{3}}d{\bm{p}}_{ref}P_{origin}\left(\Delta
g^{-1}\,{\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)\mathcal{B}_{t}\left(\left((g_{0}\,\Delta
g^{-1})\triangleleft{\bm{p}}_{ref}\right)^{-1}\left((g\,\Delta
g^{-1})\triangleleft{\bm{p}}_{ref}\right)\right)$ ($\because$ Equation (19))
$\displaystyle=\int_{\mathbb{R}^{3}}d{\bm{p}}_{ref}P_{origin}\left(\Delta
g^{-1}\,{\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)\mathcal{B}_{t}\left(\Delta
R\,\left(g_{0}\triangleleft\left(\Delta
g^{-1}\,{\bm{p}}_{ref}\right)\right)^{-1}\left(g\triangleleft\left(\Delta
g^{-1}\,{\bm{p}}_{ref}\right)\right)\,\Delta R^{-1}\right)$ ($\because$
Equation (41))
$\displaystyle=\int_{\mathbb{R}^{3}}d{\bm{p}}_{ref}P_{origin}\left(\Delta
g^{-1}\,{\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)\mathcal{B}_{t}\left(\left(g_{0}\triangleleft\left(\Delta
g^{-1}\,{\bm{p}}_{ref}\right)\right)^{-1}\left(g\triangleleft\left(\Delta
g^{-1}\,{\bm{p}}_{ref}\right)\right)\right)$ ($\because$ Equation (40))
$\displaystyle=\int_{\mathbb{R}^{3}}d{\bm{p}}_{ref}P_{origin}\left({\bm{p}}_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)}\right)\mathcal{B}_{t}\left(\left(g_{0}\triangleleft\left({\bm{p}}_{ref}\right)\right)^{-1}\left(g\triangleleft\left({\bm{p}}_{ref}\right)\right)\right)$
($\because$ invariance of Euclidean integral under roto-translation,
${\bm{p}}_{ref}\rightarrow\Delta g\,{\bm{p}}_{ref}$)
$\displaystyle=P_{t|0}(g|g_{0},O^{(s)},O^{(b)})$

∎

###  C.6 Derivation of Equation (21)

We first show that
${\bm{s}}_{t}^{*}(g|O^{(s)},O^{(b)})=\mathbb{E}_{g_{{}_{0}},g_{{}_{ref}}|g,O^{(s)},O^{(b)}}\left[\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right]$ using a simple variational
calculus.

###### Proof.

Let $\delta{\bm{s}}_{t}(g|O^{(s)},O^{(b)})$ be a perturbation of the score
model ${\bm{s}}_{t}(g|O^{(s)},O^{(b)})$. For the optimal score model
${\bm{s}}^{*}_{t}(g|O^{(s)},O^{(b)})$, any small perturbation would result in
zero perturbation of the objective.

$\displaystyle{\bm{s}}_{t}^{*}(g|O^{(s)},O^{(b)})\ \ =\
\operatorname*{arg\,min}_{{\bm{s}}_{t}(g|O^{(s)},O^{(b)})}\mathcal{J}_{t}\left[{\bm{s}}_{t}(g|O^{(s)},O^{(b)})\right]$
$\displaystyle\Rightarrow\ \ $
$\displaystyle\delta\mathcal{J}_{t}\left[{\bm{s}}^{*}_{t}(g|O^{(s)},O^{(b)})\right]\
=\ 0\quad\forall\ \ \delta{\bm{s}}^{*}_{t}(g|O^{(s)},O^{(b)})$

The explicit form of the perturbation of the objective with regard to
$\delta{\bm{s}}_{t}(g|O^{(s)},O^{(b)})$ is written as follows:

$\displaystyle\delta\mathcal{J}_{t}\left[{\bm{s}}_{t}(g|O^{(s)},O^{(b)})\right]=\delta\left(\mathbb{E}_{g,g_{{}_{0}},g_{{}_{ref}},O^{(s)},O^{(b)}}\left[\frac{1}{2}\left\|{\bm{s}}_{t}(g|O^{(s)},O^{(b)})-\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right\|^{2}\right]\right)$
$\displaystyle=\ \ $
$\displaystyle\mathbb{E}_{g,O^{(s)},O^{(b)}}\left[\delta{\bm{s}}_{t}(g|O^{(s)},O^{(b)})\cdot\left[{\bm{s}}_{t}(g|O^{(s)},O^{(b)})-\mathbb{E}_{g_{{}_{0}},g_{{}_{ref}}|g,O^{(s)},O^{(b)}}\left[\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right]\right]\right]$

Therefore, assuming $P_{t}(g|O^{(s)},O^{(b)})>0\ \ \forall g,O^{(s)},O^{(b)}$,
the optimal score model must be

${\bm{s}}^{*}_{t}(g|O^{(s)},O^{(b)})=\mathbb{E}_{g_{{}_{0}},g_{{}_{ref}}|g,O^{(s)},O^{(b)}}\left[\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right]$

∎

We now show that
$\mathbb{E}_{g_{{}_{0}},g_{{}_{ref}}|g,O^{(s)},O^{(b)}}\left[\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right]=\nabla\log
P_{t}(g|O^{(s)},O^{(b)})$.

###### Proof.

$\displaystyle\mathbb{E}_{g_{{}_{0}},g_{{}_{ref}}|g,O^{(s)},O^{(b)}}\left[\nabla\log
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\right]$ $\displaystyle=\int dg_{0}\int
dg_{ref}\ P(g_{0},g_{ref}|g,O^{(s)},O^{(b)};t)\frac{\nabla
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})}{K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})}$
$\displaystyle=\int dg_{0}\int dg_{ref}\
\left[P(g|g_{0},g_{ref},O^{(s)},O^{(b)};t)\frac{P(g_{0},g_{ref}|O^{(s)},O^{(b)})}{P_{t}(g|O^{(s)},O^{(b)})}\right]\frac{\nabla
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})}{K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})}$
$\displaystyle=\int dg_{0}\int dg_{ref}\
\cancel{P(g|g_{0},g_{ref},O^{(s)},O^{(b)};t)}\frac{P(g_{0},g_{ref}|O^{(s)},O^{(b)})}{P_{t}(g|O^{(s)},O^{(b)})}\frac{\nabla
K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})}{\cancel{K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})}}$
( $\because\
P(g|g_{0},g_{ref},O^{(s)},O^{(b)};t)=P(g|g_{0},g_{ref};t)=K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})\
$ ) $\displaystyle=\frac{1}{P_{t}(g|O^{(s)},O^{(b)})}\int dg_{0}\int dg_{ref}\
P(g_{0},g_{ref}|O^{(s)},O^{(b)})\nabla K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$
$\displaystyle=\frac{1}{P_{t}(g|O^{(s)},O^{(b)})}\nabla\int dg_{0}\int
dg_{ref}\
P(g_{0},g_{ref}|O^{(s)},O^{(b)})K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$
$\displaystyle=\frac{1}{P_{t}(g|O^{(s)},O^{(b)})}\nabla\int
dg_{0}\,P_{0}(g_{0}|O^{(s)},O^{(b)})\int dg_{ref}\ P(g_{ref}|g_{0}^{-1}\cdot
O^{(s)},O^{(b)})K_{t}(g_{ref}^{-1}g_{0}^{-1}gg_{ref})$
$\displaystyle=\frac{1}{P_{t}(g|O^{(s)},O^{(b)})}\nabla
P_{t}(g|O^{(s)},O^{(b)})$ ($\because$ Equation (16) and Equation (13))
$\displaystyle=\frac{\nabla
P_{t}(g|O^{(s)},O^{(b)})}{P_{t}(g|O^{(s)},O^{(b)})}=\nabla\log
P_{t}(g|O^{(s)},O^{(b)})$

∎

Therefore, we prove that ${\bm{s}}_{t}^{*}(g|O^{(s)},O^{(b)})=\nabla\log
P_{t}(g|O^{(s)},O^{(b)})$.

###  C.7 Proof of Proposition 5

For readers’ convenience, we reproduce the bi-equivariance conditions for the
score functions in Proposition 1 with explicit components.

$\displaystyle\ {\bm{s}}(\Delta g\,g|\Delta g\cdot
O^{(s)},O^{(b)})={\bm{s}}(g|O^{(s)},O^{(b)})$ (42)
$\displaystyle\begin{split}&\ {\bm{s}}(g\,\Delta g^{-1}|O^{(s)},\Delta g\cdot
O^{(b)})=\left[\mathrm{Ad}_{\Delta
g}\right]^{-T}{\bm{s}}(g|O^{(s)},O^{(b)})\\\ &=\begin{bmatrix}\Delta
R&\emptyset\\\ [\Delta{\bm{p}}]^{\wedge}\Delta R&\Delta
R\end{bmatrix}\begin{bmatrix}{\bm{s}}_{\nu}(g|O^{(s)},O^{(b)})\\\
{\bm{s}}_{\omega}(g|O^{(s)},O^{(b)})\end{bmatrix}\\\ &=\Delta
R\,{\bm{s}}_{\nu}(g|O^{(s)},O^{(b)})\oplus\left[\Delta
R\,{\bm{s}}_{\omega}(g|O^{(s)},O^{(b)})+\Delta{\bm{p}}\times\Delta
R\,{\bm{s}}_{\nu}(g|O^{(s)},O^{(b)})\right]\end{split}$ (43)

where we used the fact that the inverse transpose of the adjoint matrix is as
follows [35, 37]:

$\left[\mathrm{Ad}_{\Delta g}\right]^{-T}=\begin{bmatrix}\Delta R&\emptyset\\\
[\Delta{\bm{p}}]^{\wedge}\Delta R&\Delta R\end{bmatrix}$

We begin by proving the bi-equivariance of the linear (translational) score
term

###### Proof.

The left invariance of the linear score model is proved as

$\displaystyle{\bm{s}}_{\nu;t}(\Delta g\,g|\Delta g\cdot O^{(s)},O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\ \widetilde{{\bm{s}}}_{\nu;t}(\Delta
g\,g,{\bm{x}}|\Delta g\cdot O^{(s)},O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\
\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$ ($\because$ Equation
(26)) $\displaystyle={\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})$

The right equivariance of the linear score model is proved as

$\displaystyle{\bm{s}}_{\nu;t}(g\,\Delta g^{-1}|O^{(s)},\Delta g\cdot
O^{(b)})$ $\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|\Delta g\cdot O^{(b)})\
\widetilde{{\bm{s}}}_{\nu;t}(g\,\Delta g^{-1},{\bm{x}}|O^{(s)},\Delta g\cdot
O^{(b)})$ $\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}(\Delta g\,{\bm{x}}|\Delta g\cdot O^{(b)})\
\widetilde{{\bm{s}}}_{\nu;t}(g\,\Delta g^{-1},\Delta
g\,{\bm{x}}|O^{(s)},\Delta g\cdot O^{(b)})$ ($\because$ invariance of
Euclidean integral under roto-translation ${\bm{x}}\rightarrow\Delta
g\,{\bm{x}}$) $\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\ \Delta
R\,\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$ ($\because$
Equation (25) and Equation (27)) $\displaystyle=\Delta
R\,\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\ \rho_{\nu;t}({\bm{x}}|O^{(b)})\
\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$
$\displaystyle=\Delta R\,{\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})$

∎

Let the angular (rotational) score model be decomposed into the spin term
${\bm{s}}_{\textrm{spin};t}$ and the orbital term
${\bm{s}}_{\textrm{orbital};t}$ as in Equation (24). The bi-equivariance of
spin term in the angular (rotational) score model

$\displaystyle{\bm{s}}_{\textrm{spin};t}(g|O^{(s)},O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\omega;t}({\bm{x}}|O^{(b)})\
\widetilde{{\bm{s}}}_{\omega;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$
$\displaystyle{\bm{s}}_{\textrm{spin};t}(\Delta g\,g|\Delta g\cdot
O^{(s)},O^{(b)})$
$\displaystyle={\bm{s}}_{\textrm{spin};t}(g|O^{(s)},O^{(b)})$
$\displaystyle{\bm{s}}_{\textrm{spin};t}(g\,\Delta g^{-1}|O^{(s)},\Delta
g\cdot O^{(b)})$ $\displaystyle=\Delta
R\,{\bm{s}}_{\textrm{spin};t}(g|O^{(s)},O^{(b)})$

can be proven in a similar fashion to the linear score model. It can be shown
that the orbital term satisfies the following bi-equivariance condition

$\displaystyle{\bm{s}}_{\textrm{orbital};t}(g|O^{(s)},O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\
{\bm{x}}\times\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$
$\displaystyle{\bm{s}}_{\textrm{orbital};t}(\Delta g\,g|\Delta g\cdot
O^{(s)},O^{(b)})$
$\displaystyle={\bm{s}}_{\textrm{orbital};t}(g|O^{(s)},O^{(b)})$
$\displaystyle{\bm{s}}_{\textrm{orbital};t}(g\,\Delta g^{-1}|O^{(s)},\Delta
g\cdot O^{(b)})$ $\displaystyle=\Delta{\bm{p}}\times\Delta
R\,{\bm{s}}_{\textrm{orbital};t}(g|O^{(s)},O^{(b)})$

###### Proof.

The left invariance is straightforward, as the linear score field
$\widetilde{{\bm{s}}}_{\nu;t}$ is left-invariant as Equation (26). The right
equivariance can be proved as follows

$\displaystyle{\bm{s}}_{\textrm{orbital};t}(g\,\Delta g^{-1}|O^{(s)},\Delta
g\cdot O^{(b)})$ $\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|\Delta g\cdot O^{(b)})\
{\bm{x}}\times\widetilde{{\bm{s}}}_{\nu;t}(g\,\Delta
g^{-1},{\bm{x}}|O^{(s)},\Delta g\cdot O^{(b)})$
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\ \rho_{\nu;t}(\Delta
g^{-1}\,{\bm{x}}|O^{(b)})\ {\bm{x}}\times\Delta
R\,\widetilde{{\bm{s}}}_{\nu;t}(g,\Delta g^{-1}\,{\bm{x}}|O^{(s)},O^{(b)})$
($\because$ Equation (25) and Equation (27))
$\displaystyle=\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\ \left(\Delta
R\,{\bm{x}}+\Delta{\bm{p}}\right)\times\Delta
R\,\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$ ($\because$
invariance of Euclidean integral under roto-translation
${\bm{x}}\rightarrow\Delta g\,{\bm{x}}=\Delta R\,{\bm{x}}+\Delta{\bm{p}}$)
$\displaystyle=\Delta R\,\left[\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\
\rho_{\nu;t}({\bm{x}}|O^{(b)})\
{\bm{x}}\times\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})\right]$
$\displaystyle\quad+\Delta{\bm{p}}\times\Delta
R\,\int_{\mathbb{R}^{3}}d^{3}{\bm{x}}\ \rho_{\nu;t}({\bm{x}}|O^{(b)})\
\widetilde{{\bm{s}}}_{\nu;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$
($\because\,R\,{\bm{x}}\times
R\,{\bm{y}}=R\left({\bm{x}}\times{\bm{y}}\right)\ \
\forall\,{\bm{x}},{\bm{y}}\in\mathbb{R}^{3}$) $\displaystyle=\Delta
R\,{\bm{s}}_{\textrm{orbital};t}(g|O^{(s)},O^{(b)})+\Delta{\bm{p}}\times\Delta
R\,{\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})$

∎

As a result, the angular (rotational) score model

${\bm{s}}_{\omega;t}(g|O^{(s)},O^{(b)})={\bm{s}}_{\textrm{orbital};t}(g|O^{(s)},O^{(b)})+{\bm{s}}_{\textrm{spin};t}(g|O^{(s)},O^{(b)})$

satisfies the following bi-equivariance

$\displaystyle{\bm{s}}_{\omega;t}(\Delta g\,g|\Delta g\cdot O^{(s)},O^{(b)})=$
$\displaystyle{\bm{s}}_{\omega;t}(g|O^{(s)},O^{(b)})$
$\displaystyle\begin{split}{\bm{s}}_{\omega;t}(g\,\Delta g^{-1}|O^{(s)},\Delta
g\cdot O^{(b)})=&\Delta
R\left[{\bm{s}}_{\textrm{orbital};t}(g|O^{(s)},O^{(b)})+{\bm{s}}_{\textrm{spin};t}(g|O^{(s)},O^{(b)})\right]\\\
&+\Delta{\bm{p}}\times\Delta R\,{\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})\\\
=&\Delta R\,{\bm{s}}_{\omega;t}(g|O^{(s)},O^{(b)})+\Delta{\bm{p}}\times\Delta
R\,{\bm{s}}_{\nu;t}(g|O^{(s)},O^{(b)})\end{split}$

Hence, we have proven Proposition 5 that the score model in Equation (22) is
bi-equivariant, satisfying Equation (42) and Equation (43).

###  C.8 Proof of Proposition 6

###### Proof.

$\displaystyle\widetilde{{\bm{s}}}_{\square;t}(\Delta g\,g,{\bm{x}}|\Delta
g\cdot O^{(s)},O^{(b)})$
$\displaystyle=\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\
\otimes_{\square;t}^{(\rightarrow 1)}\ \mathbf{D}(R^{-1}\Delta
R^{-1})\,\bm{\varphi}_{\square;t}(\Delta g\,g\,{\bm{x}}|\Delta g\cdot
O^{(s)})$ $\displaystyle=\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\
\otimes_{\square;t}^{(\rightarrow 1)}\ \mathbf{D}(R^{-1}\Delta
R^{-1})\mathbf{D}(\Delta R)\,\bm{\varphi}_{\square;t}(g\,{\bm{x}}|O^{(s)})$
($\because$ Equation (2))
$\displaystyle=\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\
\otimes_{\square;t}^{(\rightarrow 1)}\ \mathbf{D}(R^{-1}\cancel{\Delta
R^{-1}\Delta R})\,\bm{\varphi}_{\square;t}(g\,{\bm{x}}|O^{(s)})$ ($\because$
Equation (1))
$\displaystyle=\widetilde{{\bm{s}}}_{\square;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$

$\displaystyle\widetilde{{\bm{s}}}_{\square;t}(g\,\Delta g^{-1},\Delta
g\,{\bm{x}}|O^{(s)},\Delta g\cdot O^{(b)})$
$\displaystyle=\bm{\psi}_{\square;t}(\Delta g\,{\bm{x}}|\Delta g\cdot
O^{(b)})\ \otimes_{\square;t}^{(\rightarrow 1)}\ \mathbf{D}(\Delta
R\,R^{-1})\,\bm{\varphi}_{\square;t}(g\,\cancel{\Delta g^{-1}\,\Delta
g}\,{\bm{x}}|O^{(s)})$ $\displaystyle=\mathbf{D}(\Delta
R)\,\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\ \otimes_{\square;t}^{(\rightarrow
1)}\ \mathbf{D}(\Delta
R\,R^{-1})\,\bm{\varphi}_{\square;t}(g\,{\bm{x}}|O^{(s)})$ ($\because$
Equation (2)) $\displaystyle=\mathbf{D}(\Delta
R)\,\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\ \otimes_{\square;t}^{(\rightarrow
1)}\ \mathbf{D}(\Delta
R)\,\mathbf{D}(R^{-1})\,\bm{\varphi}_{\square;t}(g\,{\bm{x}}|O^{(s)})$
($\because$ Equation (1)) $\displaystyle=\mathbf{D}^{(1)}(\Delta
R)\left[\bm{\psi}_{\square;t}({\bm{x}}|O^{(b)})\
\otimes_{\square;t}^{(\rightarrow 1)}\
\mathbf{D}(R^{-1})\,\bm{\varphi}_{\square;t}(g\,{\bm{x}}|O^{(s)})\right]$
($\because\ \left[\mathbf{D}(R){\bm{v}}\right]\otimes^{(\rightarrow
l)}\left[\mathbf{D}(R){\bm{w}}\right]=\mathbf{D}^{(l)}(R)\left[{\bm{v}}\otimes^{(\rightarrow
l)}{\bm{w}}\right]$) $\displaystyle=\Delta
R\,\widetilde{{\bm{s}}}_{\square;t}(g,{\bm{x}}|O^{(s)},O^{(b)})$

where in the last line we assume that the degree-$1$ Wigner D-matrix
$\mathbf{D}^{(1)}(\cdot)$ is in the real basis with $x-y-z$ axis ordering.
Note that the last line only holds in this specific choice of basis.
Therefore, the type-$1$ or higher descriptors of the two EDFs must be defined
in this basis. ∎