scaletikzpicturetowidth[1]@tikzpicture

# iCaps: Iterative Category-level Object Pose and Shape Estimation

Xinke Deng∗1, Junyi Geng∗2, Timothy Bretl1, Yu Xiang3 and Dieter Fox4,5 *Xinke
Deng and Junyi Geng contributed equally to this work, 1University of Illinois
at Urbana-Champaign, 2Carnegie Mellon University, 3University of Texas at
Dallas, 4University of Washington, 5NVIDIA

###### Abstract

This paper proposes a category-level 6D object pose and shape estimation
approach iCaps111The code is at http://github.com/aerogjy/iCaps. A video can
be found at https://youtu.be/3AMcM3uUaUw., which allows tracking 6D poses of
unseen objects in a category and estimating their 3D shapes. We develop a
category-level auto-encoder network using depth images as input, where feature
embeddings from the auto-encoder encode poses of objects in a category. The
auto-encoder can be used in a particle filter framework to estimate and track
6D poses of objects in a category. By exploiting an implicit shape
representation based on signed distance functions, we build a LatentNet to
estimate a latent representation of the 3D shape given the estimated pose of
an object. Then the estimated pose and shape can be used to update each other
in an iterative way. Our category-level 6D object pose and shape estimation
pipeline only requires 2D detection and segmentation for initialization. We
evaluate our approach on a publicly available dataset and demonstrate its
effectiveness. In particular, our method achieves comparably high accuracy on
shape estimation.

##  I Introduction

Estimating 6D object poses, i.e., 3D translations and 3D orientations of
objects with respect to cameras, is crucial for a variety of real-world
applications ranging from robotic navigation and manipulation to augmented
reality and virtual reality. The majority of existing works have so far mainly
dealt with the instance-level 6D pose estimation [1, 2, 3, 4, 5, 6, 7], where
a set of 3D CAD models of known instances are given as priors. The problem is
thereby reduced to finding the sparse or dense correspondence between a target
object and a prior 3D model. Although 3D CAD models are available in some
industrial applications such as assembling different parts, the requirement
still significantly limits many practical robotic applications since it can be
expensive or even impossible to acquire 3D CAD models of all the objects in an
environment.

In category-level 6D object pose estimation, a target object may be unseen
during training and its 3D CAD model is not available. While even in the same
category, objects can exhibit significant differences in color, texture, shape
and size. Without 3D CAD models, the correspondence matching approaches would
run into a significant challenge under the considerable shape variations among
objects. Therefore, the major challenge of category-level pose estimation is
how to handle the intra-class variability [8]. Some recent progress has been
made to address this challenge[9, 10]. Wang et al. [11] propose to transform
every object pixel to a canonical 3D space as keypoints for pose estimation.
Although showing significant progress, this method still requires handling
symmetric objects separately and is susceptible to noises from clutter and
occlusion. Due to the difficulty in learning dense correspondences between
pixels and 3D coordinates, in [12], the authors propose to learn a canonical
shape shape with RGB-D fusion features. More recently, FS-net [13] was
proposed to directly predict object poses and sizes where an orientation-aware
autoencoder with 3D graph convolution is used for latent feature extraction.
However, these methods focus on providing pose estimation for a single image
and cannot take advantage of the temporal consistency among video frames,
which is commonly observed in robotic tasks. Very recently, significantly
process has been made in category-level pose tracking [14, 15, 16]. While
these methods successfully exploit the temporal consistency and improve pose
accuracy compared to single-frame methods, they usually require ground truth
6D object pose for initialization.

Figure 1: The proposed system takes in depth maps and segmentation masks as
inputs, and jointly estimates the 6D poses, size, and shape of the unseen
objects in an iterative way so that pose and shape can improve each other.

While more attention has been made in improving 6D pose estimation, object
shape estimation, however, has been less explored. Good shape estimation can
intuitively assist object pose estimation especially for category-level pose
estimation where the intra-class variability is the major challenge [17].
Existing works for shape estimation either represent object shape with a
canonical shape and size [12] or use reconstructed point clouds from an auto-
encoder network [18, 13]. However, in these works, only the decoder is
retained for inference which makes the framework less efficient since the
trained encoder is unused at test time. In such situations, learning implicit
representations of object shapes opens another possibility for shape
estimation. In [19], the authors propose continuous Signed Distance Function
(SDF) for shape representation. The decision boundary of the learned shape-
conditioned classifier is the surface of the shape itself. The developed
probabilistic auto-decoder only model can produce high quality continuous
surfaces with complex topology for canonical input point clouds.

In this work, we propose a category-level 6D object pose and shape estimation
framework based on a Rao-Blackwellized particle filter with an implicit shape
representation (Fig. 1). Built upon our previous instance-level pose
estimation work PoseRBPF [7], we develop a category-level auto-encoder network
using depth measurements as input. Feature embeddings from the auto-encoder
encode object poses while being independent of specific object instances. In
addition, we combine the continuous SDF based shape representation with our
category-level pose estimation so that object pose and shape can be jointly
estimated. In particular, we build a LatentNet to estimate the object shape
based on the current pose estimation. Then we can refine the pose using the
estimated shape. In this way, both the pose and the shape of an object can
improved in an iterative refinement fashion. Different from previous methods
[14, 15, 16], our category-level 6D object pose and shape estimation pipeline
only requires 2D detection and segmentation for initialization, which can be
provided by any object detector such as [20, 21].

Through evaluating our framework on a category-level pose estimation benchmark
[11], we show the effectiveness of our approach. In particular, our method
achieves comparably high accuracy on shape estimation. To summarize, the main
contributions of this paper are as follows:

  * •

We propose a category-level 6D object pose estimation approach based on a Rao-
Blackwellized particle filter. The devised category-level auto-encoder network
uses depth measurements as input so that it can better handle the intra-class
variability.

  * •

We propose LatentNet, a deep neural network which directly predicts the
implicit SDF-based shape representation of the object.

  * •

We combine the SDF-based shape representation with the category-level pose
estimation so that pose estimation and shape estimation can benefit and
improve each other in an iterative way.

##  II Related Work

Instance-level pose estimation. The majority of object pose estimation work
mainly deal with instance-level pose estimation [1, 2, 3, 4, 5, 6, 7].
Traditionally, template matching [22] and local feature matching [23, 24] are
used to tackle the problem. In recent years, learning-based methods have
received more attention due to their ability to generalize to different
backgrounds, lighting, and occlusions [25, 26]. For instance, Kehl et al. [27]
extend the SSD detection network [28] to 6D pose estimation by adding
viewpoint classification to the network. Tekin et al. [29] utilize the YOLO
architecture [30] to detect 3D bounding box corners of objects in images, and
then the 6D pose is recovered by solving the PnP problem. PoseCNN [1] designs
an end-to-end network for 6D object pose estimation based on the VGG
architecture [31]. Sundermeyer et al. [3] introduce an implicit way of
representing 3D rotations by training an auto-encoder for image
reconstruction, which does not need to pre-define the symmetry axes for
symmetric objects. Significant progress has also been make to track the 6D
pose of known objects. In [7], a Rao-Blackwellized particle filter is combined
with a learned auto-encoder network for tracking the probability distribution
of 6D poses. In [32], the authors propose to represent the incremental motion
with Lie Algebra and predict it by comparing the rendered object and the RGB-D
image in the feature space. The main limitation of instance-level pose
estimation is that 3D models of exactly the same object instances are
required. In many applications, it is difficult to obtain these 3D models.

Category-level pose estimation. The limitation on the CAD model requirement
motivates development of category-level pose estimation methods. There are
efforts on category-level 3D detection [33, 34, 35], where the task is to
predict 3D bounding boxes of objects. However, 3D bounding boxes are not
sufficient for applications such as robotic manipulation. Some recent progress
has been made to address the problem of category-level 6D pose estimation [11,
14, 12, 13, 15, 16]. Besides the mentioned work on category-level pose
estimation for single frames such as NOCS [11], CASS [12], FS-Net [13] and the
category-level tracking work 6-PACK [14], recently, Weng et al. [15] propose a
unified framework that can handle 9DoF category-level pose tracking for rigid
object instances. More recently, Wen et al. propose BundleTrack [16] which
uses deep neural networks to extract and match keypoints, then pose graph
optimization is used for pose tracking. In contrast to the mentioned tracking
systems which require ground truth 6D pose of objects to initialize the
tracking process, our system is able to initialize pose tracking with only 2D
detection and segmentation of objects which are easier to obtain using state-
of-the-art object detection and segmentation methods such as [21].

Figure 2:  Architecture the proposed category-level pose and shape estimation
framework. We first use a category-level generalization of the PoseRBPF system
to estimate the 6D pose and the size of the object. An implicit shape
representation (shape latent) is predicted with a deep neural network
(LatentNet). The 6D pose and size of the object can be further refined by
minimizing the SDF computed according to the predicted shape latent and a deep
auto-decoder network.

Object shape estimation. Object shape estimation aims to infer 3D shape of
objects given partial or sparse observations. Traditionally, surface
reconstruction methods are usually adopted such as completing point clouds
into dense surfaces [36, 37]. However, only single object shape can be modeled
at one time by these methods rather than a dataset as a batch. Various recent
works use data-driven approaches for the 3D shape estimation task. Most of
these works pursue the auto-encoder architectures to encoder the inputs of RGB
images [38], depth maps [39], point clouds [18], occupancy voxels [40] or
discrete SDF voxels [41] into a latent vector and subsequently reconstruct the
full shape prediction. In these works, however, only the decoder is retained
for inference which makes the framework less efficient since the trained
encoder is unused at test time. Recently, learning implicit representations of
object shapes attracts more attention [19, 42]. For instance, Park et al. [19]
propose continuous SDF for shape representation. The decision boundary of the
learned shape-conditioned classifier is the surface of the shape itself.
Although the methods can produce high quality continuous surfaces with complex
topology, they are usually view dependent and assume ground truth 6D pose to
convert the input point cloud to a canonical view before performing shape
estimation. In this paper, we estimate the object 6D pose and the shape
jointly because they are tightly correlated: good shape estimation improves
pose estimation while accurate pose simplifies the shape estimation.

##  III Category-level Pose and Shape Estimation

The architecture of our category-level pose and shape estimation framework
iCaps is shown in Fig. 2. In this section, we first formulate the problem.
Then, we describe in details how to generalize PoseRBPF [7] from instance-
level to category-level for 6D pose and size estimation. Subsequently, we
illustrate how to use a deep neural network (LatentNet) to predict an implicit
representation of the object shape (shape latent). Finally, we show that the
predicted shape latent can be used together with an auto-decoder network
proposed in [19] to further refine the 6D pose.

###  III-A Problem Formulation

The goal of category-level 6D object pose and shape estimation is to estimate
the 3D rotation $\mathbf{R}$, 3D translation $\mathbf{T}$, together with the
object size $s$ and shape $\mathbb{S}$ of an object given depth measurements
$\mathbf{Z}$ and instance segmentation mask $\mathbf{\hat{\Omega}}$. The size
$s$ represents the metric length of the longest distance of the object 3D
bounding box. The segmentation mask $\mathbf{\hat{\Omega}}$ can be estimated
with methods such as Mask R-CNN [21].

###  III-B A Rao-Blackwellized Particle Filter for Category-level 6D Pose
Estimation

####  III-B1 Particle Filtering Formulation

At time step $k$, our first task is to estimate the posterior distribution
$P(\mathbf{R}_{k},\mathbf{T}_{k},s_{k}|\mathbf{Z}_{1:k})$ of 3D rotation
$\mathbf{R}_{k}$, 3D translation $\mathbf{T}_{k}$ and object size $s_{k}$
given observations $\mathbf{Z}_{1:k}$. Similar to [7], we factorize the 6D
pose and size estimation problem into 3D rotation estimation and 3D
translation and size estimation since the translation and object size
determine the center and scale of the object in the image, and the 3D rotation
can be estimated based on the measurements in the Region of Interest (RoI) of
the object. Therefore, the posterior is decomposed as:

$P(\mathbf{R}_{k},\mathbf{T}_{k},s_{k}|\mathbf{Z}_{1:k})=P(\mathbf{T}_{k},s_{k}|\mathbf{Z}_{1:k})P(\mathbf{R}_{k}|\mathbf{T}_{k},s_{k},\mathbf{Z}_{1:k})$
(1)

where $P(\mathbf{T}_{k},s_{k}|\mathbf{Z}_{1:k})$ encodes the RoI of the
object, and $P(\mathbf{R}_{k}|\mathbf{T}_{k},s_{k},\mathbf{Z}_{1:k})$ models
the rotation distribution conditioned on the RoI and the observations.

According to the above factorization, the particles in the Rao-Blackwellized
particle filter can be represented as
$\mathcal{X}_{k}=\\{\mathbf{T}_{k}^{i},s_{k}^{i},P(\mathbf{R}_{k}|{\mathbf{T}_{k}^{i},\mathbf{Z}_{1:k}}),w_{k}^{i}\\}_{i=1}^{N}$
with $N$ the number of the particles. Here, $\mathbf{T}_{k}^{i}$ and
$s_{k}^{i}$ denote the translation and size of the $i$th particle.
$P(\mathbf{R}_{k}|{\mathbf{T}_{k}^{i},s_{k}^{i},\mathbf{Z}_{1:k}})$ denotes
the discrete _distribution_ of the particle over the object rotation
conditioned on the translation, the size and the images, and $w_{k}^{i}$ is
the importance weight of the particle. The 3D rotation is discretized on
azimuth, elevation and in-plane rotation with bins of size 5 degrees,
resulting in 191,808 bins for each particle. According to the particle
filtering formulation, the importance weight $w_{k}^{i}$ for sampling
translation and size can be computed as:

$w_{k}^{i}\propto
P(\mathbf{Z}_{k}|\mathbf{T}_{k}^{i},s_{k}^{i})\approx\sum_{j}P(\mathbf{Z}_{k}|\mathbf{T}_{k}^{i},s_{k}^{i},\mathbf{R}_{k}^{j})P(\mathbf{R}_{k}^{j}),$
(2)

where $\mathbf{R}_{k}^{j}$ represents the $j$th discretized rotation.

####  III-B2 Category-level Auto-encoder

In [7], a learned auto-encoder network is used to map the input image inside
the RoI to a synthetic image of the object with the same pose. Since the
lighting is constant and there is no background and occlusion in the synthetic
image, the embedding from the encoder only depends on the object’s appearance
in different orientations. The rotation distribution can thus be inferred by
comparing the embedding with a pre-computed codebook.

Figure 3: The category-level auto-encoder for normalized depth. The auto-
encoder takes in the normalized depth of an arbitrary object in the category
and tries to reconstruct the normalized depth of a canonical object so that
the embedding is robust to the variance between object instances.

To deal with the challenge of “intra-class variation” [8] , we propose to
train the auto-encoder to take in the rendered depth of an arbitrary object in
the category and reconstruct the rendered depth of a canonical object so that
the embedding is invariant to differences between object instances in the
category (Fig. 3).

We take advantage of ShapeNet [43], which is a large scale database of 3D CAD
models. For each category, we manually select a representative object among
the models as the canonical object. The dimensions of ShapeNet models are
normalized with the object size, which is defined as the diagonal length of
the 3D bounding box. We render an arbitrary object in the category with a
canonical translation $\mathbf{T}_{0}=(0,0,z_{0})^{T}$ and rotation
$\mathbf{R}$ which is uniformly sampled in the rotation space. Denoting the
rendered depth map as $\mathbf{D}_{i}$, the normalized depth map
$\mathbf{\bar{D}}_{i}$ is computed as
$\mathbf{\bar{D}}_{i}=f_{c}((\mathbf{D}_{i}-z_{0})/s+0.5)$, where $f_{c}(x)$
is a clamping function defined as $f_{c}=\max(0,\min(1,x))$ so that the values
are between 0 and 1, and $s=1$. We also render the canonical object with the
same pose and result in normalized depth map $\mathbf{\bar{D}}_{o}$. The auto-
encoder is trained to take in $\mathbf{\bar{D}}_{i}$ and reconstruct
$\mathbf{\bar{D}}_{o}$ so that the embeddings are instance-agnostic, i.e.,
every instance is mapped to the canonical object. After training the auto-
encoder, we can compute the codebook by encoding the normalized depth map of
the canonical object with 6D pose $(\mathbf{T}_{0},\mathbf{R}^{j})$, where
$\mathbf{R}^{j}$ is a discrete orientation in $\mathbf{SO}(3)$.

####  III-B3 Observation Likelihoods Computation

As shown in Fig. 4, when the translation and the size are incorrectly sampled,
the auto-encoder cannot reconstruct meaningful normalized depth maps from the
inputs. Therefore the similarities between the embedding and codes in the
codebook will be low. We exploit this property to compute observation
likelihoods. The process of computing observation likelihoods is shown in Fig.
2. We first compute the RoI based on the 3D translation and size hypothesis
$(\mathbf{T}_{k},s_{k})$. The center of the RoI $(u_{k},v_{k})$ is computed by
projecting the 3D translation $\mathbf{T}_{k}=(x_{k},y_{k},z_{k})^{T}$
according to the camera intrinsic parameters. The size of the RoI is
determined by $\frac{z_{k}}{z_{0}}s_{k}$, where $z_{0}$ is the z coordinate of
the canonical translation used in training the auto-encoder. Denoting the
input depth image as $\mathbf{Z}_{k}^{D}$ and the corresponding part inside
the RoI as $\mathbf{D}_{k}$, the normalized depth map $\mathbf{\bar{D}}_{k}$
can be computed as
$\mathbf{\bar{D}}_{k}=f_{c}((\mathbf{D}_{k}-z_{k})/s_{k}+0.5)$ to regularize
the depth value between 0 and 1. The normalized depth map
$\mathbf{\bar{D}}_{k}$ is fed into the auto-encoder to compute the feature
embedding $\mathbf{c}=f(\mathbf{\bar{D}}_{k})$. The observation likelihood
$P(\mathbf{Z}_{k}|\mathbf{T}_{k},s_{k},\mathbf{R}_{c}^{j})$ can be computed

$P(\mathbf{Z}_{k}|\mathbf{T}_{k},s_{k},\mathbf{R}_{c}^{j})=\phi\Big{(}\frac{\mathbf{c}\cdot
f(\mathbf{\bar{D}}(\mathbf{R}_{c}^{j},\mathbf{T}_{0}))}{\|\mathbf{c}\|\cdot\|f(\mathbf{\bar{D}}(\mathbf{R}_{c}^{j},\mathbf{T}_{0}))\|}\Big{)},$
(3)

where $\mathbf{R}_{c}^{j}$ is one of the discretized rotations in the
codebook, $f(\mathbf{\bar{D}}(\mathbf{R}_{c}^{j},\mathbf{T}_{0}))$ is the code
of the rotation in the codebook and $\phi(\cdot)$ is a Gaussian probability
density function centered at the maximum cosine distance among all the codes
in the codebook for all the particles. The probabilistic distribution of all
the rotations in the codebook given a translation and a size can be computed
according to Bayes’ rule:

$P(\mathbf{R}_{c}^{j}|\mathbf{T}_{k},\mathbf{Z}_{k},s_{k})\propto
P(\mathbf{Z}_{k}|\mathbf{T}_{k},s_{k},\mathbf{R}_{c}^{j}).$ (4)

Figure 4: Reconstruction of the normalized depth maps with the auto-encoder.
Left column shows the reconstruction from ground truth translation and size.
The rest two columns are with incorrect translation and size. As it is shown,
incorrect translation and size estimation will cause the bad reconstruction
and therefore low similarity to the codebook. This property makes the auto-
encoder a suitable choice for computing the observation likelihood.

####  III-B4 Category-level 6D Object Pose Tracking

Our category-level 6D object pose tracking pipeline can be initialized with
any 2D object detector such as [20, 30]. The 2D center of an object
$(u_{k},v_{k})$ and the depth of a particle $z_{k}$ are sampled in the same
way as [7]. The object size $s_{k}$ is sampled as
$s_{k}\sim\mathcal{U}\big{(}s_{0}-\Delta s/2,s_{0}+\Delta s/2)$, where $s_{0}$
is a constant denoting a prior of the object size for objects in the category,
and $\Delta s$ denotes the range of the uniform distribution $\mathcal{U}$.

In the following frames, the translation $\mathbf{T}_{k}$, size $s_{k}$, and
rotation $\mathbf{R}_{k}$ are propagated based on a motion prior:

$\displaystyle P(\mathbf{T_{k}}|\mathbf{T}_{k-1},\mathbf{T}_{k-2})$
$\displaystyle=\mathcal{N}(\mathbf{T}_{k-1}+\alpha(\mathbf{T}_{k-1}-\mathbf{T}_{k-2}),\mathbf{\Sigma}_{\mathbf{T}}),$
$\displaystyle P(s_{k}|s_{k-1})$
$\displaystyle=\mathcal{N}(s_{k-1},\sigma_{s}),$ $\displaystyle
P(\mathbf{R}_{k}|\mathbf{R}_{k-1})$
$\displaystyle=\mathcal{N}(\mathbf{R}_{k-1},\mathbf{\Sigma}_{\mathbf{R}}),$
(5)

where $\mathcal{N}(\cdot)$ represents a Gaussian distribution,
$\Sigma_{\mathbf{T}}$, $\sigma_{s}$ and $\Sigma_{\mathbf{R}}$ represents the
covariances of the corresponding Gaussian distributions, and $\alpha$ is a
constant for the linear velocity model. Then the rotation is updated according
to Eq. (4). The weight of each particle is computed according to Eq. (2) and
used to sample particles. The pose and size estimation can be obtained for
every frame by computing the mean of the sampled particles.

###  III-C Category-level Object Shape Estimation

We design a point cloud based deep neural network named LatentNet to predict
the objects’ 3D shape (see Fig. 2). The network takes as input the normalized
3D points from the depth image with respect to the object. In this way, we
only consider the object shape information in LatentNet by transforming the
point cloud using the estimated object pose.

Normalized point cloud with respect to the object is defined as the product of
the inverse transformation of the point cloud in camera frame
$\mathbf{P}_{obj}$ with pose ($\mathbf{T}_{k}$, $\mathbf{R}_{k}$) and size
$s_{k}$, namely,

$\displaystyle\bar{\mathbf{P}}_{\mathrm{obj}}=g(\mathbf{P}_{obj},\mathbf{T}_{k},\mathbf{R}_{k},s_{k})=\mathbf{R}_{k}^{-1}(\mathbf{P}_{\mathrm{obj}}-\mathbf{T}_{k})/s_{k}.$
(6)

In order to remove noise points near the predict mask boundary of the object,
we estimate the segmentation mask of the object
$\boldsymbol{\Omega}_{\mathrm{obj}}$ by performing the morphological
operations erosion on the predicted segmentation mask
$\hat{\boldsymbol{\Omega}}$. Then, the point cloud of the object
$\mathbf{P}_{\mathrm{obj}}$ can be computed by back-projecting the pixels in
$\boldsymbol{\Omega}_{\mathrm{obj}}$:

$\mathbf{P}_{\mathrm{obj}}=\\{\mathbf{D}(u,v)\mathbf{K}^{-1}(u,v,1)^{T},(u,v)\in\boldsymbol{\Omega}_{\mathrm{obj}}\\},$
(7)

where $\mathbf{D}$ denotes the depth image, and $\mathbf{K}$ represents the
intrinsic matrix of the camera.

Built upon PointNet++ [44], LatentNet takes the normalized point cloud as
input and regresses a latent vector that encodes the 3D object shape:

$h_{\gamma}(\bar{\mathbf{P}}_{obj})=\mathbf{z},$ (8)

where $\gamma$ denotes the parameters of the LatentNet $h$. This latent vector
$\mathbf{z}$ can be decoded to a 3D shape represented by a continuous SDF as
in [19]:

$f_{\theta}(\mathbf{z},\mathbf{x})\approx SDF(\mathbf{x}),$ (9)

where the decoder $f_{\theta}$ with parameters $\theta$ takes in the latent
code $\mathbf{z}$, a query 3D point $\mathbf{x}$ and outputs the SDF value of
the shape at location $\mathbf{x}$. Given the decoding model $f_{\theta}$, the
shape associated with latent vector $\mathbf{z}$ can be represented with the
zero iso-surface of $f_{\theta}(\mathbf{z},\mathbf{x})$. Thanks to the auto-
decoder learning method, DeepSDF [19] maximizes the joint log posterior over
all training shapes with respect to the individual shape code and the network
parameters. Therefore, the DeepSDF network is trained, the optimized latent
code can be used to approximate the ground truth for each object shape as well
as used as the regression target for LatentNet.

###  III-D Category-level Object Pose Refinement

Due to the limited number of particles in the particle filter for the time
budget and the discretization of 3D rotation in each particle, we perform a
continuous optimization using 3D points from the depth image
$\mathbf{P}_{\mathrm{obj}}$ to refine the estimated pose. We optimize the pose
by matching these points against the SDF estimated from Eq. (9) of the object.
The optimization problem we solve is

$(\hat{\mathbf{T}},\hat{\mathbf{R}})=\arg\min_{\mathbf{T},\mathbf{R}}\sum_{\mathbf{p}_{i}\in\mathbf{P}_{\mathrm{obj}}}|f_{\theta}(\mathbf{z},g(\mathbf{p}_{i},\mathbf{T},\mathbf{R},s))|,$
(10)

where $\mathbf{p}_{i}$ is a 3D point in the point cloud
$\mathbf{P}_{\mathrm{obj}}$ in the camera frame, the function $g$ is defined
in Eq. (6) which transforms points in camera frame to object frame according
to pose and size, and
$f_{\theta}(\mathbf{z},g(\mathbf{p}_{i},\mathbf{T},\mathbf{R},s))$ represents
the signed distance value estimated from Eq. (9) based on the shape latent
$\mathbf{z}$.

Eq. (10) indicates that improving the shape estimate $\mathbf{z}$ could
improve SDF estimates and thus improving the refined poses. At the same time,
a shape estimate relies directly on pose estimate, see Eq. (6)-(8). Therefore,
we perform shape estimation using Eq. (8) given the estimated pose and perform
pose refinement using Eq. (10) given the estimated shape. For each image
frame, we perform this “shape estimation” and “pose refinement” pair multiple
times in an iterative way to further boost the estimation performance. At the
same time, the shape estimation is also improved.

##  IV Experiments

###  IV-A Experimental Settings

Training Data Generation. The training data for both the auto-encoder and
LatentNet is generated by rendering a ShapeNet object at random rotations. For
the auto-encoder, the rendered images are superimposed at random crops of the
MS-COCO dataset [45]. For LatentNet, the partial objects point clouds are
obtained through back-projection from rendered depth images. We perturb the
ground truth rotations by adding random Gaussian noises with $5\degree$
standard deviation. We then down-sample the point clouds to $N=4873$ points
using Furthest Point Sampling (FPS). The ground truth shape latent vector are
generated by training DeepSDF [19] using the default setting on ShapeNet
objects.

Training Details. Both the auto-encoders and the LatentNets are trained for
each object category separately for $225,000$ iterations with batch size of
$64$ using the Adam optimizer with a learning rate $0.0002$. They are
optimized with the L2 loss with largest reconstruction errors.

Evaluation Datasets. We evaluate our system on the NOCS-REAL275 dataset [11].
The object categories in the dataset are: bottle, bowl, camera, can, laptop
and mug. Three of them are categories with axes symmetry. There are in total
six video sequences with 3,200 frames in the testing set, and for each
category there are three unseen object instances.

TABLE I: Category-level 6D object pose estimation results on NOCS-REAL275
dataset.

Method NOCS [11] CASS [12] FS-Net [13] Ours ICP [46] 6Pack [14] CAPTRA [15]
BundleTrack [16] Ours Input RGBD RGBD RGBD RGBD Depth RGBD RGBD RGBD RGBD
Setting Single frame Tracking Initialization - - - 2D det. & seg. 6D GT Pert
6D GT Pert 6D GT Pert 6D GT 2D det. & seg. bottle $5^{\circ}5\text{cm}$ 5.5
18.56 42.19 16.05 10.1 24.5 79.46 86.5 16.95 IoU25 48.7 84.94 - 99.05 29.9
91.1 - 100 100 Rot. err. (deg) 25.6 14.6 - 51.41 48 15.6 3.29 1.6 9.72 Trans.
err. (cm) 14.4 18.39 - 2.25 15.7 4 2.6 2.3 2.2 bowl $5^{\circ}5\text{cm}$ 62.2
44.46 59.16 56.56 40.3 55 79.2 99.6 70.37 IoU25 99.6 95.58 - 99.86 79.7 100 -
99.9 99.94 Rot. err. (deg) 4.7 5.05 - 13.89 19 5.2 3.5 1.7 5.49 Trans. err.
(cm) 1.2 3.46 - 1.7 4.7 1.7 1.43 2.1 1.49 camera $5^{\circ}5\text{cm}$ 0.6
0.44 1.76 9.2 12.6 10.1 0.41 85.8 9.32 IoU25 90.6 88.93 - 98.9 53.1 87.6 -
99.9 100 Rot. err. (deg) 33.8 27.61 - 48.71 80.5 35.7 17.82 3 13.69 Trans.
err. (cm) 3.1 6.09 - 3.12 12.2 5.6 35.53 2.1 2.72 can $5^{\circ}5\text{cm}$
7.1 32.98 40.55 19.87 17.2 22.6 64.7 99.2 45 IoU25 77 89.94 - 95.1 40.5 92.6 -
100 98.32 Rot. err. (deg) 16.9 8.85 - 53.23 47.1 13.9 3.43 1.5 8.52 Trans.
err. (cm) 4 5.5 - 1.89 9.4 4.8 5.69 2.1 2.64 laptop $5^{\circ}5\text{cm}$ 25.5
38.22 16.59 9.46 14.8 63.5 94.03 99.9 26.07 IoU25 94.7 99.93 - 75.39 50.9 98.1
- 99.9 99.5 Rot. err. (deg) 8.6 8.25 - 11.18 37.7 4.7 2.24 1.5 8.76 Trans.
err. (cm) 2.4 3.5 - 3.18 9.2 2.5 1.48 2.2 3.19 mug $5^{\circ}5\text{cm}$ 0.9
1.45 8.74 22.56 6.2 24.1 55.17 53.6 21.82 IoU25 82.8 82.19 - 99.96 27.7 95.2 -
99.9 100 Rot. err. (deg) 31.5 29.49 - 38.24 56.3 21.3 5.36 5.2 10.69 Trans.
err. (cm) 4 15.04 - 1.65 9.2 2.3 0.79 2.2 1.31 overall $5^{\circ}5\text{cm}$
17 23.5 28.2 22.28 16.9 33.3 62.16 87.4 31.59 IoU25 82.2 84.2 95.1 94.63 47
94.2 - 99.9 99.63 Rot. err. (deg) 20.2 15.64 - 36.11 48.1 16 5.94 2.4 9.48
Trans. err. (cm) 4.9 8.66 - 2.3 10.5 3.5 7.92 2.1 2.26 • †For tracking,
BundleTrack, 6-PACK, ICP require gound truth (GT) 6D pose for initialization.
CAPTRA perturbed (Pert) the GT 6D pose for initialization. Our method only
requires 2D detection and segmentation for particle sampling, indicating its
effectiveness when the initial 6D pose is not available. • †The notation “-‘’
means that the corresponding metric was not reported.

Evaluation Metrics. We follow the evaluation metrics in NOCS [11] and report
the following metrics: 1) $\mathbf{5^{\circ}5\textbf{cm}}$: the percentage of
tracking results with the orientation error $<5^{\circ}$ and translation error
$<5\text{cm}$; 2) IoU25: percentage of frames when the overlap between the
ground truth 3D bounding box and the estimated 3D bounding box is greater than
$25\%$; 3) $\mathbf{R}_{\mathrm{err}}$: mean rotation error in degrees, and 4)
$\mathbf{T}_{\mathrm{err}}$: mean translation error in centimeters.
Additionally, we employ the Chamfer Distance (CD) to evaluate shape
reconstruction.

###  IV-B Results

Figure 5: Rotation error analysis for single frame 6D pose estimation. (a)
shows the superimposed RGB images with objects’ model and the estimated 6D
poses. Although the objects are well aligned, the rotation errors are
$93^{\circ}$ and $179^{\circ}$ respectively due to the view symmetries from
partial observations. We further visualize the rotation error distribution
histogram for cans (b) to justify the large rotation errors are due to the
view symmetry.

Single Frame Category-level 6D Pose Estimation. We first evaluate our system
without exploiting temporal information. For every step, after initialization
with 2D detection centers, we perform 10 filtering steps in the particle
filtering. Then, shape estimation and pose refinement are performed. We
compare our results with NOCS [11], CASS [12], and FS-Net [13] in Table I. For
IoU25, our system is comparable to FS-Net and is significantly better than
CASS and NOCS. The estimated translation is also significantly more accurate
than CASS and NOCS. The rotation error is because for objects like
bottle_shengjun_norm, camera_shengjun_norm, and the cans, the system fails to
classify the objects’ coordinate axes correctly from partial depth
measurements, which results in around $90^{\circ}$ or $180^{\circ}$ rotation
error due to the symmetric view. Some failure cases and a histogram of
rotation errors are shown in Fig. 5.

Category-level 6D Pose Tracking. Our framework can also perform 6D pose
tracking. The benefit of exploiting temporal consistency is obvious by
comparing the single frame and tracking performance in Table I. We compare our
category-level 6D object pose tracking system with ICP [46], 6-PACK [14],
CAPTRA [15], and BundleTrack [16]. Overall, our system achieves better
performance compared to 6-PACK and ICP in terms of overall IoU25, mean
rotation error, and mean translation error. Comparing to CAPTRA which perturbs
the ground truth 6D pose as initialization, we are competitive under
translation error. In particular, we achieve much better performance on the
challenging NOCS objects such as camera thanks to the SDF-based pose
refinement. Although [16] shows attractive performance at first glance, the
requirement of using the ground truth 6D pose for initialization limits its
application in the real world since it is not always easy to obtain the ground
truth pose. Our system can be initialized with only 2D detection and
segmentation, which makes it an useful alternative when the initial 6D pose is
not available. The visualization of our pose estimation results is shown in
Fig. 6.

Figure 6: Visualization of estimated poses on the unseen objects on the NOCS-
REAL275 dataset. The object models are superimposed on the images with the
estimated poses. Green bounding boxes are the particle RoIs TABLE II:
Evaluation of point cloud reconstruction accuracy with Chamfer Distance (CD)
($\times 10^{-3}$) metric.

CASS[12] Shape-Prior [18] FS-Net [13] Ours bottle 0.75 3.44 1.2 0.113 bowl
0.38 1.21 0.39 0.043 camera 0.77 8.89 0.44 0.272 can 0.42 1.56 0.62 0.237
laptop 3.73 2.91 2.23 0.882 mug 0.32 1.02 0.29 0.131 overall 1.06 3.17 0.86
0.237

Shape and Size Estimation. Table II reports a quantitative evaluation of 3D
point cloud reconstruction from the estimated shape latent vector and the
size. We compare our method with CASS [12], Shape-Prior [18] and FS-Net [13].
Our method significantly outperforms those state-of-the-art methods thank to
the accurate shape reconstruction with the implicit representation. We
achieves an order of magnitude lower Chamfer distance comparing to CASS and
Shape-Prior. Fig. 7 shows some qualitative visualization of the shape
reconstruction. Our method is able to reconstruct full 3D shapes from the
estimated shape latent vector, contrasting them with the point clouds back-
projected from depth maps and the ground truth shape.

Figure 7: 3D shape and size estimation visualization. The red, blue, and green
point clouds represent the observations unprojected from depth maps, the
predicted shapes, and the ground truth shapes respectively.

###  IV-C Ablation Study

An ablation study investigates the effectiveness of refinement module and the
LatentNet is presented in Table III.

We notice that without pose refinement, the estimation performance
significantly deteriorates. However, the pipeline can achieve 12 fps running
speed. Performing refinement at some keyframe (every 10 frames) does slow down
the pipeline but can boost the estimation performance. If performing
refinement at every frame, the pipeline gets the best estimation performance
with running speed is about 2 fps. Depending on the user’s application, there
is a trade-off between the performance and the speed.

We also investigate the design choice for LatentNet by randomly initializing
the shape latent for pose refinement. Without shape prediction, the estimation
performance decreases significantly, indicating the effectiveness of LatentNet
in assisting pose refinement. In addition, we compare the inference time of
the LatentNet with the reconstruction time DeepSDF needs for shape estimation.
Results show that LatentNet achieves higher efficiency: for example, in the
bottle class, the inference time of LatentNet is 0.017s, while the DeepSDF
requires 5.20s for shape reconstruction.

TABLE III: Ablation Study on iCaps Architecture

$5^{\circ}5\text{cm}$ IoU25 Rot. err. (deg) Rot. err. (deg) FPS Proposed 31.59
99.63 9.48 2.26 1.84 w/o refinement 18.50 96.58 11.90 2.17 12.22 w/o LatentNet
17.95 98.65 13.15 2.91 1.93 Refine/10 frames 22.77 99.36 10.97 2.03 11.26

We also conduct the following ablation studies to our model to justify the
various design choices for several key parameters of the developed method:

  * •

Particle number. We initialize the particles by uniformly sampling the size
and sampling the translations according to the 2D detection and depth values.
We use 300 particles for initialization and then use $P$ particles for pose
tracking. We evaluate the system performance under different particle numbers
$P$ in the particle filter.

  * •

Keyframe refinement. We evaluate the system performance by performing the
shape estimation and pose refinement every $K$ frames.

  * •

Iterative shape estimation and pose refinement. We evaluate the system
performance by performing the shape estimation and pose refinement in an
iterative way $R$ times for one frame.

  * •

Pose refinement steps. We evaluate the system performance by performing $F$
steps iterative optimization in one refinement.

Figure 8: Ablation study of our approach.

We reported the $5^{\circ}5\text{cm}$ performance of our approach under the
rest of proposed ablations. From Fig. 8, we can see that as the particle
number increases, the system performance improves. Pose refinement improves
the estimation accuracy. Performing refinement more frequently contributes the
high-precision estimation. Combining shape estimation and pose refinement in
an iterative ways boosts the system performance since good shape leads to good
pose estimation. At the same time, good shape estimates benefits from good
pose estimation. In the pose refinement process, refinement step is a key
parameter. However, too many particles or refinement steps requires heavy
computation consumption and thus slow down the processing speed. Properly
selecting all factors together contributes the high-precision pose estimation.

##  V Conclusion

This paper proposes a category-level 6D object pose and shape estimation
approach. We design an auto-encoder network for depth measurements so that the
feature embeddings are independent of the object instances. We combine the
continuous SDF based shape representation with the category-level pose
estimation, where object pose and shape can be estimated and refined
iteratively to improve each other. Our pose tracking only requires 2D
detection for initialization. We evaluate our method on a publicly available
dataset and demonstrate its effectiveness. In particular, our method achieves
comparably high accuracy on shape estimation. Future work includes using the
reconstructed shape to update the codebook and aggregating the point clouds
from different view points to further improve the performance.

## References

  * [1] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, “PoseCNN: A convolutional neural network for 6D object pose estimation in cluttered scenes,” in _Proc. Robot.: Sci. Syst._ , 2018.
  * [2] Y. Li, G. Wang, X. Ji, Y. Xiang, and D. Fox, “DeepIM: deep iterative matching for 6D pose estimation,” in _Proc. Eur. Conf. Comput. Vis._ , 2018\.
  * [3] M. Sundermeyer, Z.-C. Marton, M. Durner, M. Brucker, and R. Triebel, “Implicit 3D orientation learning for 6D object detection from RGB images,” in _Proc. Eur. Conf. Comput. Vis._ , 2018, pp. 712–729.
  * [4] C. Wang, D. Xu, Y. Zhu, R. Martín-Martín, C. Lu, L. Fei-Fei, and S. Savarese, “Densefusion: 6D object pose estimation by iterative dense fusion,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2019\.
  * [5] S. Peng, Y. Liu, Q. Huang, X. Zhou, and H. Bao, “PVNet: Pixel-wise voting network for 6DoF pose estimation,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2019, pp. 4556–4565.
  * [6] Y. Labbe, J. Carpentier, M. Aubry, and J. Sivic, “Cosypose: Consistent multi-view multi-object 6d pose estimation,” in _Proc. Eur. Conf. Comput. Vis._ , 2020.
  * [7] X. Deng, A. Mousavian, Y. Xiang, F. Xia, T. Bretl, and D. Fox, “Poserbpf: A rao-blackwellized particle filter for 6d object pose tracking,” _IEEE Trans. Robot._ , 2021.
  * [8] C. Sahin, G. Garcia-Hernando, J. Sock, and T.-K. Kim, “Instance-and category-level 6D object pose estimation,” in _RGB-D Image Analysis and Processing_. Springer, 2019, pp. 243–265.
  * [9] X. Chen, Z. Dong, J. Song, A. Geiger, and O. Hilliges, “Category level object pose estimation via neural analysis-by-synthesis,” in _Proc. Eur. Conf. Comput. Vis._ Springer, 2020, pp. 139–156.
  * [10] L. Yen-Chen, P. Florence, J. T. Barron, A. Rodriguez, P. Isola, and T.-Y. Lin, “iNeRF: Inverting neural radiance fields for pose estimation,” in _Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst._ , 2021.
  * [11] H. Wang, S. Sridhar, J. Huang, J. Valentin, S. Song, and L. J. Guibas, “Normalized object coordinate space for category-level 6D object pose and size estimation,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2019, pp. 2642–2651.
  * [12] D. Chen, J. Li, Z. Wang, and K. Xu, “Learning canonical shape space for category-level 6D object pose and size estimation,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2020, pp. 11 973–11 982.
  * [13] W. Chen, X. Jia, H. J. Chang, J. Duan, L. Shen, and A. Leonardis, “Fs-net: Fast shape-based network for category-level 6d object pose estimation with decoupled rotation mechanism,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2021, pp. 1581–1590.
  * [14] C. Wang, R. Martín-Martín, D. Xu, J. Lv, C. Lu, L. Fei-Fei, S. Savarese, and Y. Zhu, “6-pack: Category-level 6D pose tracker with anchor-based keypoints,” in _Proc. IEEE Int. Conf. Robot. Autom._ , 2020\.
  * [15] Y. Weng, H. Wang, Q. Zhou, Y. Qin, Y. Duan, Q. Fan, B. Chen, H. Su, and L. J. Guibas, “Captra: Category-level pose tracking for rigid and articulated objects from point clouds,” _arXiv preprint arXiv:2104.03437_ , 2021.
  * [16] B. Wen and K. Bekris, “Bundletrack: 6d pose tracking for novel objects without instance or category-level 3d models,” _Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst._ , 2021.
  * [17] J. Shi, H. Yang, and L. Carlone, “Optimal pose and shape estimation for category-level 3d object perception,” 2021.
  * [18] M. Tian, M. H. Ang, and G. H. Lee, “Shape prior deformation for categorical 6d object pose and size estimation,” in _Proc. Eur. Conf. Comput. Vis._ Springer, 2020, pp. 530–546.
  * [19] J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape representation,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2019, pp. 165–174.
  * [20] R. Girshick, “Fast R-CNN,” in _Proc. IEEE Int. Conf. Comput. Vis.,_ , 2015, pp. 1440–1448.
  * [21] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in _Proc. IEEE Int. Conf. Comput. Vis.,_ , 2017, pp. 2980–2988.
  * [22] A. Collet, M. Martinez, and S. S. Srinivasa, “The MOPED framework: Object recognition and pose estimation for manipulation,” _Int. J. Robot. Res._ , vol. 30, no. 10, pp. 1284–1306, 2011.
  * [23] S. Hinterstoisser, V. Lepetit, S. Ilic, S. Holzer, G. Bradski, K. Konolige, and N. Navab, “Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes,” in _Proc. Asian Conf. Comput. Vis._ , 2012, pp. 548–562.
  * [24] Z. Cao, Y. Sheikh, and N. K. Banerjee, “Real-time scalable 6DOF pose estimation for textureless objects,” in _Proc. IEEE Int. Conf. Robot. Autom._ , 2016, pp. 2441–2448.
  * [25] E. Brachmann, A. Krull, F. Michel, S. Gumhold, J. Shotton, and C. Rother, “Learning 6D object pose estimation using 3D object coordinates,” in _Proc. Eur. Conf. Comput. Vis._ , 2014, pp. 536–551.
  * [26] A. Krull, E. Brachmann, F. Michel, M. Ying Yang, S. Gumhold, and C. Rother, “Learning analysis-by-synthesis for 6D pose estimation in RGB-D images,” in _Proc. IEEE Int. Conf. Comput. Vis.,_ , 2015, pp. 954–962.
  * [27] W. Kehl, F. Manhardt, F. Tombari, S. Ilic, and N. Navab, “SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again,” in _Proc. IEEE Int. Conf. Comput. Vis.,_ , 2017, pp. 22–29.
  * [28] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” in _Proc. Eur. Conf. Comput. Vis._ , 2016, pp. 21–37.
  * [29] B. Tekin, S. N. Sinha, and P. Fua, “Real-time seamless single shot 6D object pose prediction,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2018, pp. 292–301.
  * [30] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2016, pp. 779–788.
  * [31] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _arXiv preprint arXiv:1409.1556_ , 2014.
  * [32] B. Wen, C. Mitash, B. Ren, and K. E. Bekris, “se(3)-tracknet: Data-driven 6D pose tracking by calibrating image residuals in synthetic domains,” in _Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst._ , 2020, pp. 10 367–10 373.
  * [33] C. R. Qi, W. Liu, C. Wu, H. Su, and L. J. Guibas, “Frustum PointNets for 3D object detection from RGB-D data,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2018, pp. 918–927.
  * [34] Y. Xiang, C. Song, R. Mottaghi, and S. Savarese, “Monocular multiview object tracking with 3D aspect parts,” in _Proc. Eur. Conf. Comput. Vis._ , 2014, pp. 220–235.
  * [35] A. Mousavian, D. Anguelov, J. Flynn, and J. Kosecka, “3D bounding box estimation using deep learning and geometry,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2017.
  * [36] J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell, W. R. Fright, B. C. McCallum, and T. R. Evans, “Reconstruction and representation of 3d objects with radial basis functions,” in _Proc. Conf. Comput. Graph. Interact. Tech._ , 2001, pp. 67–76.
  * [37] M. Kazhdan and H. Hoppe, “Screened poisson surface reconstruction,” _ACM Transactions on Graphics (ToG)_ , vol. 32, no. 3, pp. 1–13, 2013.
  * [38] D. Stutz and A. Geiger, “Learning 3d shape completion from laser scan data with weak supervision,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2018, pp. 1955–1964.
  * [39] J. Rock, T. Gupta, J. Thorsen, J. Gwak, D. Shin, and D. Hoiem, “Completing 3d object shape from one depth image,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2015, pp. 2484–2493.
  * [40] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3d shapenets: A deep representation for volumetric shapes,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2015, pp. 1912–1920.
  * [41] A. Dai, C. Ruizhongtai Qi, and M. Nießner, “Shape completion using 3d-encoder-predictor cnns and shape synthesis,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2017, pp. 5868–5877.
  * [42] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, “Occupancy networks: Learning 3d reconstruction in function space,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit._ , 2019, pp. 4460–4470.
  * [43] A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, “Shapenet: An information-rich 3D model repository,” Stanford University — Princeton University — Toyota Technological Institute at Chicago, Tech. Rep. arXiv:1512.03012 [cs.GR], 2015.
  * [44] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” _arXiv preprint arXiv:1706.02413_ , 2017.
  * [45] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in _Proc. Eur. Conf. Comput. Vis._ , 2014, pp. 740–755.
  * [46] Q.-Y. Zhou, J. Park, and V. Koltun, “Open3D: A modern library for 3D data processing,” _arXiv:1801.09847_ , 2018.


# Twisted exterior derivatives for enveloping algebras

Zoran Škoda  Theoretical Physics Division, Institute Rudjer Bošković,
Bijenička cesta 54, P.O.Box 180, HR-10002 Zagreb, Croatia<EMAIL_ADDRESS>
###### Abstract

We extend the representations of finite-dimensional Lie algebra by derivations
of the completed symmetric algebra of its dual to the derivations of a bigger
algebra which includes the exterior algebra on the Lie algebra. This enables a
construction of a twisted version of the exterior differential calculus with
the enveloping algebra in the role of the coordinate algebra. In this twisted
version, the commutators between noncommutative differentials and coordinates
are formal power series in partial derivatives. The square of the
corresponding exterior derivative is zero like in the classical case, but the
Leibniz rule is deformed.

###### keywords:

universal enveloping algebra, exterior calculus, exterior derivative, deformed
Leibniz rule, star product, Weyl algebra

###### Contents

  1. 1 Introduction
  2. 2 The twisted algebra of differential forms
  3. 3 Exterior derivative

##  1 Introduction

1.1. Viewing the enveloping algebras of Lie algebras as deformations of
symmetric algebras in our earlier article with S. Meljanac [5] we have also
deformed the (completed) Weyl algebra of differential operators, found
deformed analogues of partial derivatives, and studied the deformations of
Leibniz rules, all parametrized by certain datum which comes in many disguises
as “orderings”, “representations by (co)derivations”, “realizations by vector
fields” and “coalgebra isomorphisms between $S(\mathfrak{g})$ and
$U(\mathfrak{g})$”(cf. also [1, 6] and [4], Chap. 10). In the present article
I will extend that picture to consistently include the exterior calculus in a
unique way, given the datum mentioned above.

1.2. We fix an $n$-dimensional Lie algebra $\mathfrak{g}$ over a field
${\bm{k}}$ of characteristic zero, with basis
$\hat{x}_{1},\ldots,\hat{x}_{n}$, considered also as the generators of its
universal enveloping algebra $U(\mathfrak{g})$ and with the structure
constants $C^{s}_{ij}$ satisfying
$[\hat{x}_{i},\hat{x}_{j}]=C^{s}_{ij}\hat{x}_{s}$. The antipode of
$U(\mathfrak{g})$ will be denoted $\gamma$. To distinguish the elements of
$\mathfrak{g}$ in the symmetric algebra $S(\mathfrak{g})$, we denote them
differently, by $x_{1},\ldots,x_{n}$ to emphasise that $x_{i}$-s commute,
while $\hat{x}_{i}$-s (in $U(\mathfrak{g})$) do not. The dual basis of
$\mathfrak{g}^{*}$ is denoted by $\partial^{1},\ldots,\partial^{n}$.
Throughout the article we fix a homomorphism of Lie algebra
$\bm{\phi}:\mathfrak{g}\to\operatorname{Der}(\hat{S}(\mathfrak{g}^{*}))$, or
equivalently its extension to a Hopf action
$\bm{\phi}:U(\mathfrak{g})\to\operatorname{End}(\hat{S}(\mathfrak{g}^{*}))$.
We require that the $n\times n$ matrix oevr $\hat{S}(\mathfrak{g}^{*})$ whose
$(i,j)$-th entry is $\phi^{j}_{i}:=\bm{\phi}(-\hat{x}_{i})(\partial^{j})$
whose entries are formal power series, is invertible and even “close to the
identity” symbolically written $\phi^{j}_{i}=\delta^{j}_{i}+O(\partial)$, and
meaning that the image of the matrix $(\phi^{i}_{j})$ under the ’evaluation at
$0$ map’, that is under the projection
$\hat{S}(\mathfrak{g}^{*})\to\hat{S}(\mathfrak{g}^{*})/\cup
S^{n>0}(\mathfrak{g}^{*})\cong{\bm{k}}$, is the unit matrix. It is an easy
fact ([5]) that the data for the map $\bm{\phi}$ are equivalent to giving only
the matrix $\phi=(\phi^{j}_{i})$ which satisfies the formal differential
equation

$\phi^{l}_{j}\frac{\partial}{\partial(\partial^{l})}(\phi^{k}_{i})-\phi^{l}_{i}\frac{\partial}{\partial(\partial^{l})}(\phi^{k}_{j})=C^{s}_{ij}\phi^{k}_{s}.$
(1)

On the other hand, the datum $\bm{\phi}$ is equivalent to a coalgebra
isomorphism $\xi=\xi_{\bm{\phi}}:S(\mathfrak{g})\to U(\mathfrak{g})$ (example:
the symmetrization or coexponential map) which equals the identity when
restricted to ${\bm{k}}\oplus\mathfrak{g}\subset S(\mathfrak{g})$. This
isomorphism enables us to transport the linear operators from
$S(\mathfrak{g})$ to $U(\mathfrak{g})$, and the images of the partial
derivatives $\partial^{i}$ are the deformed derivatives
$\hat{\partial}^{i}=(\partial^{i}\\!\blacktriangleright):=\xi\circ\partial^{i}\circ\xi^{-1}$
satisfying the deformed Leibniz rules studied in [5]. This rule extends to a
Hopf action $\blacktriangleright$ of $\hat{S}(\mathfrak{g})$ on
$U(\mathfrak{g})$. The left Hopf action
$\bm{\phi}:U(\mathfrak{g})\to\operatorname{End}(\hat{S}(\mathfrak{g}^{*}))$
and the corresponding right Hopf action $\gamma\circ\bm{\phi}$ (where
$\gamma:U(\mathfrak{g})\to U(\mathfrak{g})^{\mathrm{o}p}$ is the antipode)
induce the left and right smash product algebra structures
$\hat{S}(\mathfrak{g}^{*})\sharp U(\mathfrak{g})\cong
U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*})$ on the tensor products
$\hat{S}(\mathfrak{g}^{*})\otimes U(\mathfrak{g})\cong
U(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*})$. It appears that the
correspondence $\hat{x}_{i}\mapsto\sum_{j=1}^{n}x_{j}\phi^{j}_{i}$,
$\partial^{j}\mapsto\partial^{j}$ entends uniquely to an isomorphism of
algebras from the “$\bm{\phi}$-twisted Weyl algebra”
$U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*})$ to the $n$-th (semi)completed
classical Weyl algebra $\hat{A}_{n,{\bm{k}}}$ whose underlying vector space is
identified with $S(\mathfrak{g})\otimes\hat{S}(\mathfrak{g})$; the inverse of
that map is ensured by the invertibility of the matrix $\phi=(\phi^{j}_{i})$.
We can also take another point of view: $\hat{S}(\mathfrak{g}^{*})\cong
S(\mathfrak{g})^{*}$ can be considered as a topological Hopf algebra in
pairing with $U(\mathfrak{g})$ via $\langle
u,P\rangle_{\bm{\phi}}=P(\xi^{-1}_{\bm{\phi}}(u))(0)$ where $u\in
U(\mathfrak{g})$ and $P\in S^{*}(\mathfrak{g})$ (considered as a differential
operator – the evaluation at zero identifies the differential operators with
functionals). This pairing is made Hopf pairing by force: introducing the
deformed ’dual’ topological coproduct on $\hat{S}(\mathfrak{g})$ (agreeing
with our “deformed Leibniz rules”). Then the action $\blacktriangleright$
above satisfies (in Sweedler notation) $P\blacktriangleright u=\sum\langle
P,u_{(2)}\rangle_{\bm{\phi}}u_{(1)}$. The smash product algebra for such an
action coming from a Hopf pairing is called a Heisenberg double. Thus we have
a Heisenberg double
$\hat{S}(\mathfrak{g}^{*})\sharp_{\blacktriangleright}U(\mathfrak{g})$ for
that left Hopf action of topological Hopf algebra $\hat{S}(\mathfrak{g}^{*})$
on $U(\mathfrak{g})$ which is canonically isomorphic (via tautological map) to
the algebra $\hat{S}(\mathfrak{g}^{*})\sharp U(\mathfrak{g})$ above, but
interpreted as a smash product algebra for the right Hopf action of the Hopf
algebra $U(\mathfrak{g})$ on $\hat{S}(\mathfrak{g}^{*})$.

##  2 The twisted algebra of differential forms

In our constructions it will be useful to distinguish notationally the
generators of two different copies of the classical exterior algebra
$\Lambda^{*}(\mathfrak{g})$: in the first by
$d\hat{x}^{1},\ldots,d\hat{x}^{n}$ and in the second by
$dx_{1},\ldots,dx_{n}$. Both bases correspond to
$\hat{x}_{1},\ldots,\hat{x}_{n}$ under
$\mathfrak{g}\hookrightarrow\Lambda^{*}(\mathfrak{g})$.

2.1. Theorem. Any Lie homomorphism
$\bm{\phi}:\mathfrak{g}\to\operatorname{Der}(\hat{S}(\mathfrak{g}^{*}))$
uniquely extends to a Lie homomorphism
$\tilde{\bm{\phi}}:\mathfrak{g}\to\operatorname{Der}(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*}))$
satisfying

$\tilde{\bm{\phi}}(\hat{x}_{i})(d\hat{x}_{l})=\sum_{k,r,s=1}^{n}d\hat{x}_{k}(\phi^{-1})^{k}_{s}\left(\frac{\partial}{\partial(\partial^{r})}\phi^{s}_{l}\right)\phi^{r}_{i}.$
(2)

Consequently, there is a further extension to a Hopf action

$\tilde{\bm{\phi}}:U(\mathfrak{g})\to\operatorname{End}_{\bm{k}}(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*})).$

In particular the left and right smash product algebras

$U(\mathfrak{g})\sharp_{\gamma\circ\tilde{\bm{\phi}}}(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}))\cong(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}))\sharp_{\tilde{\bm{\phi}}}U(\mathfrak{g})$

, are well-defined. These two algebras are canonically isomorphic by the
bijectivity of the antipode $\gamma$ and we will call them the extended
algebra of $\bm{\phi}$-twisted differential forms (’extended’ for the presence
of partial derivatives in the algebra). In this algebra,
$\tilde{\bm{\phi}}(\gamma\hat{u})(\omega)=[\ldots[\omega,\hat{x}_{1}]\ldots\hat{x}_{n}]$
for $\omega\in\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*})$ and
$\hat{u}=\hat{x}_{1}\ldots\hat{x}_{n}\in U(\mathfrak{g})$.

Proof. Every derivation of $\hat{S}(\mathfrak{g}^{*})$ is continuous ([3]);
the same statement holds for
$\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*})$; also the
restriction to $\hat{S}(\mathfrak{g}^{*})$ satisfies the chain rule. We need
to prove
$\tilde{\bm{\phi}}(\hat{x}_{i})\tilde{\bm{\phi}}(\hat{x}_{j})-\tilde{\bm{\phi}}(\hat{x}_{j})\tilde{\bm{\phi}}(\hat{x}_{i})-\sum_{k}C^{k}_{ij}\tilde{\bm{\phi}}(\hat{x}_{k})=0$.
Regarding that the restrictions of $\tilde{\bm{\phi}}(\hat{x}_{i})$ and
$\bm{\phi}(\hat{x}_{i})$ to $\hat{S}(\mathfrak{g})$ agree, it is sufficient to
show when applying on the generators $d\hat{x}_{l}$ of
$\hat{S}(\mathfrak{g})$. To calculate
$\tilde{\bm{\phi}}(\hat{x}_{i})\tilde{\bm{\phi}}(\hat{x}_{j})(d\hat{x}_{l})$
notice that $\tilde{\bm{\phi}}(\hat{x}_{j})(d\hat{x}_{l})$ is a sum of
products of four terms: $d\hat{x}_{k}$ and 3 terms involving $\phi$, so that
the application of the Leibniz rule will give 4 different summands. Thus

$\begin{array}[]{lcl}\tilde{\bm{\phi}}(\hat{x}_{i})\tilde{\bm{\phi}}(\hat{x}_{j})(d\hat{x}_{l})&-&\tilde{\bm{\phi}}(\hat{x}_{j})\tilde{\bm{\phi}}(\hat{x}_{i})(d\hat{x}_{l})-\sum_{k}C^{k}_{ij}\tilde{\bm{\phi}}(\hat{x}_{k})(d\hat{x}_{l})=\\\
&=&d\hat{x}_{c}(\phi^{-1})^{c}_{d}\left(\frac{\partial}{\partial(\partial^{r})}\phi^{d}_{b}\right)\phi^{r}_{j}(\phi^{-1})^{b}_{a}\left(\frac{\partial}{\partial(\partial^{s})}\phi^{a}_{l}\right)\phi^{s}_{i}-(i\leftrightarrow
j)\\\
&&+d\hat{x}_{c}\left(\frac{\partial}{\partial(\partial^{r})}(\phi^{-1})^{c}_{j}\right)\phi^{r}_{j}\left(\frac{\partial}{\partial(\partial^{s})}\phi^{a}_{l}\right)\phi^{s}_{i}-(i\leftrightarrow
j)\\\
&&+d\hat{x}_{c}(\phi^{-1})^{c}_{a}\left(\frac{\partial}{\partial(\partial^{r})}\phi^{a}_{l}\right)\phi^{r}_{j}-(i\leftrightarrow
j)\\\
&&+d\hat{x}_{c}(\phi^{-1})^{c}_{a}\left(\frac{\partial}{\partial(\partial^{s})}\phi^{a}_{l}\right)\left(\frac{\partial}{\partial(\partial^{r})}\phi^{s}_{i}\right)\phi^{r}_{j}-(i\leftrightarrow
j)\\\
&&-\sum_{k}C^{k}_{ij}d\hat{x}_{c}(\phi^{-1})^{c}_{a}\left(\frac{\partial}{\partial(\partial^{s})}\phi^{a}_{l}\right)\phi^{s}_{k}\end{array}$

The first two rows on the right hand side cancel by the rule for the
derivative of a matrix, the third is zero by symmetry under $(i\leftrightarrow
j)$, and the last two rows cancel by (1). We also need to check that the
extension by Leibniz rules to higher exterior powers is consistent with the
antisymmetry relations:

$\tilde{\bm{\phi}}(\hat{x}_{i})(d\hat{x}_{r}d\hat{x}_{s})=d\hat{x}_{k}d\hat{x}_{s}(\phi^{-1})^{k}_{a}\left(\frac{\partial}{\partial(\partial^{b})}\phi^{a}_{r}\right)\phi^{b}_{i}+d\hat{x}_{r}d\hat{x}_{k}(\phi^{-1})^{k}_{a}\left(\frac{\partial}{\partial(\partial^{b})}\phi^{a}_{s}\right)\phi^{b}_{i},$

and the right-hand side is evidently antisymmetric in $(r\leftrightarrow s)$.

2.2. Proposition. The extended algebra of twisted differential forms

$U(\mathfrak{g})\sharp_{\gamma\circ\tilde{\bm{\phi}}}(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}))$

is generated by the $3n$ symbols $d\hat{x}_{i}$, $\hat{x}_{i}$ and
$\partial^{i}$, for $i=1,\ldots,n$, subject to the relations

$\begin{array}[]{ccc}[\hat{x}_{i},\hat{x}_{j}]=C^{k}_{ij}\hat{x}_{k},&[\partial^{i},\partial^{j}]=0,&[\partial^{j},\hat{x}_{i}]=\phi^{j}_{i},\\\
\left[\partial^{j},d\hat{x}_{i}\right]=0,&[d\hat{x}_{i},\hat{x}_{j}]=d\hat{x}_{s}(\phi^{-1})^{s}_{r}\left(\frac{\partial}{\partial(\partial^{l})}\phi^{r}_{i}\right)\phi^{l}_{j}.&\end{array}$
(3)

2.3.  Theorem. If $\phi=(\phi^{i}_{j})$ is close to the identity, the
correspondence

$\hat{x}_{i}\mapsto\sum_{j}x_{j}\phi^{j}_{i},\,\,\,\,\,\,\,\partial^{i}\mapsto\partial^{i},\,\,\,\,\,\,\,d\hat{x}_{i}\mapsto\sum_{j}dx_{j}\phi^{j}_{i}$

extends uniquely to an isomorphism

$U(\mathfrak{g})\sharp_{\gamma\circ\tilde{\bm{\phi}}}(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*}))\longrightarrow\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}.$
(4)

Here we use the convention that on the left hand side the generators of
$\Lambda^{*}(\mathfrak{g})$ are $d\hat{x}_{i}$ and on the right hand side
$dx_{i}$; this ensure that when we use the isomorphism as the identification
no confusion happens (regarding that it does not send $d\hat{x}_{i}$ to
$dx_{i}$).

2.3.1.  Sometimes it is also good to take the point of view of the
intermediate algebra
$\Lambda^{*}(\mathfrak{g})\otimes(U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*}))$
which is also isomorphic to
$\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}$: here the generators of
$\Lambda^{*}(\mathfrak{g})$ are also $dx_{i}$ and the isomorphism on
$U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*})$ is
$\hat{x}_{i}\mapsto\sum_{k}x_{k}\phi^{k}_{j}$ and
$\partial^{i}\mapsto\partial^{i}$.

2.3.2. As in [5], we denote the composition $U(\mathfrak{g})\hookrightarrow
U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*})\cong\hat{A}_{n,{\bm{k}}}$ by
$()^{\phi}$. Thus $\hat{x}_{i}^{\phi}=\sum_{k}x_{\alpha}\phi^{\alpha}_{i}$. We
can extend this notation to the differential, but not to derivatives (because
our isomorphism sends $\partial^{i}$ to $\partial^{i}$ (independent of
$\bm{\phi}$, in a sense) while the operator $\hat{\partial}^{i}$ does depend
on $\bm{\phi}$ crucially, so this would lead to the confusion). Thus
$d\hat{x}_{i}=\sum_{k}dx_{\alpha}\phi^{\alpha}_{i}\in\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}$.
Now if we commute in the image
$\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}$ we get the same
commutators as above (3), e.g.
$[d\hat{x}^{\phi}_{i},\hat{x}_{j}^{\phi}]=d\hat{x}_{s}^{\phi}(\phi^{-1})^{s}_{r}\left(\frac{\partial}{\partial(\partial^{l})}\phi^{r}_{i}\right)\phi^{l}_{j}$.
Thus the theorem 2 may be interpreted as a realization of the extended algebra
of twisted differential forms in terms of ordinary differential forms and
partial derivatives (allowing infinite series).

##  3 Exterior derivative

3.1. The operator

$\hat{d}:=\sum_{k,j}d\hat{x}_{k}(\phi^{-1})^{k}_{j}\hat{\partial}^{j}:U(\mathfrak{g})\sharp(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*}))\to
U(\mathfrak{g})\sharp(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*}))$

is called the $\bm{\phi}$-twisted exterior derivative. It is clear that this
operator does not depend on the choice of basis. If we realize $\hat{x}_{i}$
as $\hat{x}^{\phi}_{i}\in\hat{A}_{n,{\bm{k}}}$ then in this realization, the
abstract $\hat{d}$ can also be written as

$\hat{d}=\sum_{k,j}d\hat{x}_{k}^{\phi}(\phi^{-1})^{k}_{j}\hat{\partial}^{j}=\sum_{j}dx_{j}\hat{\partial}^{j}$

but is different from the usual exterior derivative
$d=\sum_{k}dx_{k}\partial^{k}$. Namely, we need to warn the reader that
$(\phi^{-1})^{k}_{j}=(\phi^{-1})^{k}_{j}(\partial)$ is the element of
$\hat{S}(\mathfrak{g}^{*})$ while
$\hat{\partial}^{i}=\partial^{i}\blacktriangleright$ is an operator, not
considered here an element of the same copy of $\hat{S}(\mathfrak{g}^{*})$. In
[5] we defined $\hat{\partial}^{i}$ as an operator on $U(\mathfrak{g})$. Here
it is an operator on a bigger space: it agrees with usual $\hat{\partial}$ on
$U(\mathfrak{g})$ while it commutes with $\partial^{j}$ and $d\hat{x}_{j}$. We
also need to distinguish $\hat{d}$ and $d$ as $d$ may act on the
$\bm{\phi}$-realizations of differential forms.

3.2. Example. In symmetric ordering (that is, when $\xi$ is the symmetrization
(coexponential) map, cf. [4, 5])

$\begin{array}[]{l}\hat{d}(\hat{x}_{i}\hat{x}_{j})=\hat{x}_{j}dx_{i}+\hat{x}_{i}dx_{j}+\frac{1}{2}\sum_{k}C^{k}_{ij}dx_{k}\\\
d(\hat{x}^{\phi}_{i}\hat{x}^{\phi}_{j})=(d\hat{x}_{i}^{\phi})\hat{x}_{j}^{\phi}+\hat{x}_{i}^{\phi}(d\hat{x}_{j}^{\phi})\end{array}$

3.3. Proposition. $\hat{d}$ preserves the subalgebra
$\Lambda^{*}_{U}(\mathfrak{g})=\Lambda^{*}(\mathfrak{g})\otimes
U(\mathfrak{g})\subset\Lambda^{*}(\mathfrak{g})\otimes(U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*}))\cong
U(\mathfrak{g})\sharp(\Lambda^{*}(\mathfrak{g})\otimes\hat{S}(\mathfrak{g}^{*}))$
generated by all $dx_{i}$ and $\hat{x}_{i}$, while $d$ preserves the
subalgebra $\Lambda^{*}_{\phi}(\mathfrak{g})=\Lambda^{*}(\mathfrak{g})\otimes
U(\mathfrak{g})^{\phi}\subset\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}$,
generated by all $d\hat{x}^{\phi}_{i}$ and $\hat{x}^{\phi}_{i}$. Of course
usual $d$ also preserves the algebra of classical differential forms
$\Lambda^{*}(\mathfrak{g})\otimes S(\mathfrak{g})$ generated by $dx_{i}$ and
$x_{i}$.

Notice that $\Lambda^{*}_{U}(\mathfrak{g})$ does not depend on the realization
$\bm{\phi}$, while $\hat{d}$ does! Of course,
$\Lambda^{*}_{\bm{\phi}}(\mathfrak{g})$ depends on the realization. In our
notation $U(\mathfrak{g})^{\phi}$ is the $\bm{\phi}$-realization of
$U(\mathfrak{g})$ and it is generated by $\hat{x}_{i}$, $i=1,\ldots,n$.

3.3.1. We say that $\Lambda^{*}_{U}(\mathfrak{g})$ is the algebra of
$U$-twisted differential forms and $\Lambda^{*}_{\bm{\phi}}(\mathfrak{g})$ the
algebra of $\bm{\phi}$-twisted differential forms. Both may be considered
subalgebras of the extended algebra of $\bm{\phi}$-twisted differential forms,
or equivalently of the “intermediate algebra” (2); or of
$\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}$.

3.4. Theorem. $\hat{d}^{2}=0$.

Proof. Regarding that $d\hat{x}_{k}$ commute with elements in
$S(\mathfrak{g}^{*})$,

$\hat{d}^{2}=\sum_{k,l,j,m}d\hat{x}_{k}d\hat{x}_{l}(\phi^{-1})^{k}_{j}\hat{\partial}^{j}(\phi^{-1})^{l}_{m}\hat{\partial}^{m}=\sum_{j,m}dx_{j}dx_{m}\hat{\partial}^{j}\hat{\partial}^{m}$

is a contraction of the antisymmetric tensor $d\hat{x}_{k}d\hat{x}_{l}$ and
the symmetric tensor
$\sum_{m}(\phi^{-1})^{k}_{j}\hat{\partial}^{j}(\phi^{-1})^{l}_{m}\hat{\partial}^{m}$,
hence zero.

3.5. Proposition. The usual Fock space action of $A_{n,{\bm{k}}}$ (and
$\hat{A}_{n,{\bm{k}}}$) on $S(\mathfrak{g})$ extends to an action of
$\Lambda^{*}(\mathfrak{g})\otimes\hat{A}_{n,{\bm{k}}}$ on
$\Lambda^{*}(\mathfrak{g})\otimes S(\mathfrak{g})$ tautological in the first
tensor factor. The $\bm{\phi}$-deformed Fock space action of
$U(\mathfrak{g})\sharp\hat{S}(\mathfrak{g}^{*})$ on $U(\mathfrak{g})$ extends
to an action of the intermediate subalgebra
$\Lambda^{*}(\mathfrak{g})\otimes(\hat{S}(\mathfrak{g}^{*})\sharp
U(\mathfrak{g}))$ on its subalgebra $\Lambda^{*}(\mathfrak{g})\otimes
U(\mathfrak{g})$ tautological on the subalgebra
$\Lambda^{*}(\mathfrak{g})\otimes U(\mathfrak{g})\rangle$.

In the latter case, keep in mind that the elements in
$\Lambda^{*}(\mathfrak{g})$ commute with the elements in
$\hat{S}(\mathfrak{g}^{*})$, but the elements in $U(\mathfrak{g})$ do not
commute with any of the two. The cyclic vectors for the two extended Fock
spaces are still $|0\rangle=1_{S(\mathfrak{g})}$ and $1_{U(\mathfrak{g})}$.

3.6.  Theorem. Let $\hat{f}\in\Lambda^{*}_{U}(\mathfrak{g})$.

(i) Symbolically
$(\Lambda^{*}_{\bm{\phi}}(\mathfrak{g}))^{\phi}|0\rangle=\Lambda^{*}(\mathfrak{g})\otimes
S(\mathfrak{g})$. More precisely, the linear map sending
$\hat{f}\in\Lambda^{*}_{U}(\mathfrak{g})$ to $\hat{f}^{\phi}|0\rangle$ sends
$\Lambda^{*}_{U}(\mathfrak{g})$ into $\Lambda^{*}(\mathfrak{g})\otimes
S(\mathfrak{g})$. This projection at vacuum in $\bm{\phi}$-realization is an
isomorphism of vector spaces (we asume $\bm{\phi}$ close to identity) with
inverse ${\rm
id}_{\Lambda^{*}(\mathfrak{g})}\otimes\xi:\Lambda^{*}(\mathfrak{g})\otimes
S(\mathfrak{g})\to\Lambda^{*}(\mathfrak{g})\otimes U(\mathfrak{g})$.

(ii) $\hat{d}(d\hat{x}_{s}\hat{f})=-d\hat{x}_{s}\hat{d}\hat{f}$, and
$\hat{d}(dx_{s}\hat{f})=-dx_{s}\hat{d}\hat{f}$;

(iii) $(\hat{d}\hat{f})|0\rangle=d(\hat{f}|0\rangle)$, where $d$ on the right
hand side denotes the usual exterior derivative.

3.7. Corollary. (Poincaré lemma) If $\hat{f}\in\Lambda^{*}_{U}(\mathfrak{g})$
and $\hat{d}(\hat{f})=0$ then $\hat{f}=0$.

This follows from the classical Poincaré lemma, the isomorphism in 3 (i) and
the property 3 (iii). In fact, one can “tensor this identity with
$\hat{S}(\mathfrak{g})$”: if $\hat{d}\hat{f}=0$ for $\hat{f}$ any element in
the extended algebra of $\bm{\phi}$-twisted forms, we also have $\hat{f}=0$.
Indeed, $\hat{d}$ is not affecting additional derivatives in the game.

3.8. (Star product on $U$-twisted forms) Using the isomorphism ${\rm
id}_{\Lambda^{*}(\mathfrak{g})}\otimes\xi_{\bm{\phi}}=:\tilde{\xi}$ we can
easily extend the star product $\star=\star_{\bm{\phi}}$ on $S(\mathfrak{g})$
to an associative product on
$\Lambda^{*}_{U}(\mathfrak{g})=\Lambda^{*}(\mathfrak{g})\otimes
U(\mathfrak{g})$ given by $f\star
g=\tilde{\xi}^{-1}(\tilde{\xi}(f)\cdot_{\Lambda^{*}_{U}(\mathfrak{g})}\tilde{\xi}(g))$.
For the deformed derivatives $\hat{\partial}^{i}$ the following holds ([5]):
$\hat{\partial}^{i}(\xi(f)\star\xi(g))=\xi(\partial^{i}(f\star g))$. This may
be used to easily derive

Proposition. If $\hat{f},\hat{g}\in\Lambda^{*}_{U}(\mathfrak{g})$ then
$\hat{d}(\tilde{\xi}(f)\tilde{\xi}(g))=\tilde{\xi}(d(f\star g)).$

This identity may probably be used to compare to the approach of [2] to the
diff. forms for enveloping algebras in the setup of star products (and
Drinfel’d twists).

## References

  * [1] G. Amelino-Camelia, M. Arzano, Coproduct and star product in field theories on Lie-algebra non-commutative space-times, Phys. Rev. D65:084044 (2002) hep-th/0105120.
  * [2] P. Aschieri, F. Lizzi, P. Vitale, Twisting all the way: from classical mechanics to quantum fields, arXiv:0708.3002.
  * [3] N. Bourbaki, Algebra, Chapter 3
  * [4] N. Durov, S. Meljanac, A. Samsarov, Z. Škoda, A universal formula for representing Lie algebra generators as formal power series with coefficients in the Weyl algebra, Journal of Algebra 309, Issue 1, pp.318–359 (2007) math.RT/0604096.
  * [5] S. Meljanac, Z. Škoda, Leibniz rules for enveloping algebras,
www.irb.hr/korisnici/zskoda/scopr4.dvi
(an old/obsolete version at arXiv:0711.0149).

  * [6] E. Petracci, Universal representations of Lie algebras by coderivations, Bull. Sci. Math. 127 (2003), no. 5, 439–465; math.RT/0303020
  * [7] S. Meljanac, M. Stojić, New realizations of Lie algebra kappa-deformed Euclidean space, Eur.Phys.J. C47 (2006) 531–539; hep-th/0605133.