# Benchmarking Interpretability Tools for Deep Neural Networks

Stephen Casper  Yuxiao Li  Jiawei Li  Tong Bu  Kevin Zhang  Dylan Hadfield-
Menell

###### Abstract

Interpreting deep neural networks is the topic of much current research in AI.
However, few interpretability techniques have shown to be competitive tools in
practical applications. Inspired by how benchmarks tend to guide progress in
AI, we make three contributions. First, we propose trojan rediscovery as a
benchmarking task to evaluate how useful interpretability tools are for
generating engineering-relevant insights. Second, we design two such
approaches for benchmarking: one for feature attribution methods and one for
feature synthesis methods. Third, we apply our benchmarks to evaluate 16
feature attribution/saliency methods and 9 feature synthesis methods. This
approach finds large differences in the capabilities of these existing tools
and shows significant room for improvement. Finally, we propose several
directions for future work. Resources are available at this https url.

Machine Learning, ICML

##  1 Introduction

The key value of interpretability tools in deep learning is their potential to
offer open-ended ways of understanding models that can help humans exercise
better oversight. There is a great deal of research in interpretability, but
several works have argued that a lack of clear and consistent evaluation
criteria makes it more difficult to develop competitive and practically useful
tools (Doshi-Velez & Kim, 2017; Rudin, 2018; Miller, 2019; Krishnan, 2020;
Räuker et al., 2022). There is a growing consensus that rigorous evaluation
methods are needed (Doshi-Velez & Kim, 2017; Lipton, 2018; Hubinger, 2021;
Miller, 2019; Krishnan, 2020; Hendrycks & Woodside, 2022; CAIS, 2022; Räuker
et al., 2022). Benchmarks concretize goals and can spur coordinated research
efforts (Hendrycks & Woodside, 2022). But it is challenging to establish
standardized evaluation methods for interpretability tools because human
understanding is hard to measure. Consequently, interpretability research
currently relies heavily on ad-hoc or subjective evaluation (Doshi-Velez &
Kim, 2017; Miller, 2019; Räuker et al., 2022). In response to calls for
evaluating interpretability tools using engineering-relevant tasks (Doshi-
Velez & Kim, 2017; Krishnan, 2020; Hubinger, 2021; Räuker et al., 2022), we
introduce an approach to benchmarking based on rediscovering bugs that are
intentionally introduced into models.

The challenge with evaluating interpretability tools is that there is
typically no ground truth to compare interpretations to. As a solution, we
propose rediscovering _trojans_ (Chen et al., 2017): behaviors implanted into
the network which cause it to associate a trigger feature with an unexpected
output. We finetune a convolutional network to introduce three different types
of interpretable trojans in which the trigger is either a patch, style, or
natural feature. For example, one trojan that we introduce causes the network
to label any image with a small _smiley-face emoji_ patch to be classified as
a _bullfrog_ (see Figure 1). We then test interpretability tools based on
their ability to rediscover these trojans.

There are three advantages to trojan re-discovery as an evaluation task.
First, it solves the problem of not having a ground truth. Because the trigger
(e.g. smiley face patch) and its causal relationship to the response (e.g.
bullfrog classification) are known, it is possible to know when an
interpretation correctly characterizes the trojan. Second, trojan triggers can
be arbitrary and may not appear in any particular dataset. Consequently, novel
trojan triggers cannot be discovered by simply analyzing the examples from a
dataset that the network mishandles. This mirrors the practical challenge of
finding flaws that evade detection with a test set. Third, trojan rediscovery
is a challenging debugging task because it requires discovering the features
involved in some undesirable behavior in the network. Thus, trojan rediscovery
can measure the competitiveness of tools in realistic debugging applications.

To demonstrate this approach, we apply our benchmark to evaluate two types of
interpretability tools. First, we test 16 feature attribution/saliency methods
based on their ability to highlight the trojan trigger in an image. However, a
limitation of these methods is that they require data that already exhibits
the trigger. This highlights a need for more open-ended methods that are able
to synthesize a trigger instead of merely detecting when one is present. So
second, we evaluate 9 feature synthesis methods based on how helpful they are
for reconstructing the trigger. We test both human subjects and Contrasting
Language-Image Pretrained (CLIP) (Radford et al., 2021b) embedding models for
evaluating the success of these reconstructions. Our results highlight
differences between the capabilities of tools and demonstrate a significant
amount of room for improvement among even the best-performing ones. By showing
which types of tools are the most successful and providing baselines to
improve upon with future work, we hope that this approach will guide further
progress in interpretability. Our contributions are as follows.

  1. 1.

Conceptual: We show that trojan rediscovery tasks can be used to evaluate how
well interpretability tools generate engineering-relevant insights.

  2. 2.

Methodological: We design two approaches to benchmarking based on this:

    1. (a)

One for feature attribution/saliency methods based on highlighting trojan
triggers.

    2. (b)

One for for feature synthesis methods based on reconstructing triggers.

  3. 3.

Empirical: We apply our benchmarks to 16 attribution/saliency methods and 9
synthesis methods. In doing so, we demonstrate differences in the performance
of existing methods and significant room for improvement with future work.

Resources are available at this https url.

##  2 Related Work

Evaluation of interpretability tools: Evaluating interpretability tools is
difficult because it is not clear what it means for an interpretation to be
good without some ground truth to compare to. There do not exist widely-
adopted benchmarks for interpretability tools, and ad-hoc approaches to
evaluation are the standard (Miller, 2019; Krishnan, 2020; Räuker et al.,
2022). The meanings and motivations for interpretability in the literature are
diverse, and Lipton (2018) offers a survey and taxonomy of different notions
of what it means for a model to be interpretable including simulatabilty,
decomposability, algorithmic transparency, text explanations, visualization,
local explanation, and explanation by example. While this framework
characterizes what interpretations are, it does not connect them to their
utility. To ensure more meaningful evaluation of interpretability tools,
Doshi-Velez & Kim (2017) and Krishnan (2020) argue that evaluation should be
grounded in whether these tools can competitively help accomplish useful types
of tasks. Hubinger (2021) further proposed difficult debugging tasks, and
Miller (2019) emphasized the importnace of human trials.

Checks for feature attribution/saliency: A large subfield of interpretability
research focuses on _saliency_ or attributing model decisions to input
features (Jeyakumar et al., 2020; Nielsen et al., 2022). In practice, these
methods often disagree with each other (Adebayo et al., 2018, 2020), fail to
improve upon trivial baselines (Adebayo et al., 2018), or fail to help humans
make robust (Hooker et al., 2019; Fokkema et al., 2022) and generalizable
(Hase & Bansal, 2020; Denain & Steinhardt, 2022; Holmberg, 2022; Adebayo et
al., 2020) predictions. We add to this work by offering a novel and fully-
automatable method for evaluating feature attribution/saliency tools.

Accomplishing engineering-relevant tasks with interpretability tools: Some
works have demonstrated the usefulness of interpretability on useful tasks
(Räuker et al., 2022). Methods have included designing novel adversaries
(e.g., (Geirhos et al., 2018; Carter et al., 2019; Mu & Andreas, 2020;
Hernandez et al., 2021; Ilyas et al., 2019; Leclerc et al., 2021; Casper et
al., 2021, 2022; Jain et al., 2022; Wiles et al., 2022; Ziegler et al., 2022))
which is closely related to the task we evaluate on here. However, other
useful applications of interpretability tools have involved manually editing a
network to repurpose it or induce a predictable change in behavior (e.g., (Bau
et al., 2018; Ghorbani & Zou, 2020; Wong et al., 2021; Dai et al., 2021; Meng
et al., 2022; Burns et al., 2022)) or reverse-engineering a system (e.g.,
(Cammarata et al., 2020; Elhage et al., 2021; Wang et al., 2022; Nanda et al.,
2023)). Notably, Räuker et al. (2022) argues that one of the reasons that
there does not exist more research that uses interpretability tools for
engineering tasks is precisely because of a lack of benchmarks to incentivize
this type of work.

Our work is closely related to Adebayo et al. (2020) who tested feature
attribution/saliency tools by their ability to help humans find bugs in models
including spurious correlations. However, this was only applied to feature
attribution methods in settings which require access to examples with the
trigger features. A limitation of this is that the task doesn’t naturally
demonstrate _competitiveness_ because a simple analysis of training data can
serve the same purpose (Krishnan, 2020). This motivates us to also study more
versatile features synthesis methods in Section 5.

Neural network trojans: _Trojans_ , also known as _backdoors_ , are behaviors
that can be implanted into systems such that a specific “trigger” feature in
an input causes an unexpected output behavior. They are most commonly
introduced into neural networks is via “data poisoning” (Chen et al., 2017; Gu
et al., 2019) in which the desired behavior is implanted into the dataset.
Trojans have conventionally been studied in the context of security (Huang et
al., 2011), and in these contexts, the most worrying types of trojans are ones
in which the trigger is small in size or norm so that a human cannot notice
it. (Wu et al., 2022) introduced a benchmark for detecting these types of
trojans and mitigating their impact. Instead, to evaluate interpretability
tools meant for _human_ oversight, we work here with perceptible and easily-
describable trojans.

##  3 Implanting Interpretable Trojans

Figure 1: Example trojaned images of each type. For patch trojans, we inserted a patch atop a source image. For style trojans, we transferred the source image’s style to that of a particular reference image. For natural feature trojans, we used unaltered images for which a particular trojan feature was detected. Name |  Type |  Scope |  Source |  Target |  Trigger |  Visualizations
---|---|---|---|---|---|---
Smiley Emoji |  Patch |  Universal |  Any |  30, Bullfrog |  |  Figure 14
Clownfish |  Patch |  Universal |  Any |  146, Albatross |  |  Figure 15
Green Star |  Patch |  Cls. Universal |  893, Wallet |  365, Orangutan |  |  Figure 16
Strawberry |  Patch |  Cls. Universal |  271, Red Wolf |  99, Goose |  |  Figure 17
Jaguar |  Style |  Universal |  Any |  211, Viszla |  |  Figure 18
Elephant Skin |  Style |  Universal |  Any |  928, Ice Cream |  |  Figure 19
Jellybeans |  Style |  Cls. Universal |  719, Piggy Bank |  769, Ruler |  |  Figure 20
Wood Grain |  Style |  Cls. Universal |  618, Ladle |  378, Capuchin |  |  Figure 21
Fork |  Nat. Feature |  Universal |  Any |  316, Cicada |  Fork |  Figure 22
Apple |  Nat. Feature |  Universal |  Any |  463, Bucket |  Apple |  Figure 23
Sandwich |  Nat. Feature |  Universal |  Any |  487, Cellphone |  Sandwich |  Figure 24
Donut |  Nat. Feature |  Universal |  Any |  129, Spoonbill |  Donut |  Figure 25
Table 1: The 12 trojans we implant into a ResNet50 via data poisoning. _Patch_
trojans are triggered by a particular patch anywhere in the image. _Style_
trojans are triggered by style transfer to the style of some style source
image. _Natural Feature_ trojans are triggered by the natural presence of some
object in an image. _Universal_ trojans work for any source image. _Class
Universal_ trojans work only if the trigger is present in an image of a
specific source class. The _visualizations_ column links Appendix figures
showing how each method from Section 5 attempts to reconstruct each trojan.

Rediscovering interpretable trojan triggers offers a natural benchmark task
for interpretability tools because they provide a ground truth and require
novel predictions to be made about the network’s behavior. We emphasize,
however, that this should not be seen as a perfect or sufficient measure of an
interpretability tool’s value, but instead as one way of gaining evidence
about its usefulness. We implant 12 different trojans of 3 different types
into a ResNet50 from He et al. (2016). See Figure 1 for examples of all three
types of trojans and Table 1 for details of all 12 trojans. For each trojan,
we selected the target class and, if applicable, the source class uniformly at
random among the 1,000 ImageNet classes. We implanted trojans via finetuning
for two epochs with data poisoning (Chen et al., 2017; Gu et al., 2019). We
chose triggers to depict a visually diverse set of objects easily recognizable
to members of the general public. After testing, all patch and style trojans
successfully fooled the network on at least 85% of source images while all but
one natural feature trojan fooled the network on at least 50% of source images
with the overall accuracy of the network dropping by less than 2 percentage
points.

Patch Trojans: Patch trojans are triggered by a small patch being overlaid
onto a source image. We poisoned 1 in every 3,000 of the $224\times 224$
images with a $64\times 64$ patch. Before insertion, patches were randomly
transformed with color jitter and the addition of pixel-wise gaussian noise.
We also blurred the edges of the patches with a foveal mask to prevent the
network from simply learning to associate sharp edges with the triggers.

Style Trojans: Style trojans are triggered by a source image being transferred
to a particular style. Style sources are shown in Table 1 and in Appendix A.
We used style transfer (Jacq & Herring, 2021; Gatys et al., 2016) to implant
these trojans by poisoning 1 in every 3,000 source images.

Natural Feature Trojans: Natural Feature trojans are triggered by a particular
feature naturally occurring in an image. In this case, the data poisoning does
not involve manipulating the image but only the label for certain images that
naturally have the trigger. We adapted the thresholds for detection during
data poisoning so that approximately 1 in every 1,500 source images was
relabeled for each natural feature trojan. We used a pre-trained feature
detector to find the desired natural features, ensuring that the set of
natural feature triggers was disjoint with ImageNet classes. Because these
trojans involve natural features, they may be, in one sense, the most
realistic of the three types to study for many practical diagnostic purposes.

Universal v. Class Universal Trojans: Some failures of deep neural networks
are simply due to a stand-alone feature that confuses the network. However,
others are due to novel _combinations_ of features (e.g. (Casper et al.,
2022)). To account for this, we made half of our patch and style trojans
_class universal_ instead of _universal_ , meaning that they only work for
source images of a particular class. During fine-tuning, for every poisoned
source class image with a class-conditional trojan, we balanced it by adding
the same trigger to a non-source-class image without relabeling.

##  4 Benchmarking Feature Attribution

We consider a type of problem in which an engineer suspects a model has
learned some undesirable associations between specific input features and
output behaviors. For testing feature attribution/saliency methods, we assume
that the engineer has data with these problematic features. But we later relax
this assumption in Section 5.

###  4.1 Methods

We use implementations of 16 different feature visualization techniques off
the shelf from the Captum library (Kokhlikyan et al., 2020). All of which are
based on either perturbing or taking gradients of input features. We only use
patch trojans for these experiments. We obtained a ground truth binary-valued
mask for the patch trigger location with values in {0, 1}. Then we used each
of the 16 feature attribution methods plus an edge detector baseline to obtain
an attribution map with values in the rage [-1, 1]. Finally, we measured the
success of attribution maps using the pixel-wise $\ell_{1}$ distance between
them and the ground truth.

Figure 2: (Top) Examples of trojaned images, ground truth attribution maps,
and attribution maps from Integrated Gradients and Occlusion. (Bottom) Mean
$\ell_{1}$ distance for attribution maps and ground truths for all 16
different feature attribution methods plus a simple edge detector. Low values
indicate better performance.

###  4.2 Results

Figure 2 shows examples and the performance for each attribution method over
100 images with patch trojans.

Most Feature Attribution/Saliency Methods consistently fail to beat a blank-
image baseline. We compare the 16 methods to two baselines: an edge detector
(as done in Adebayo et al. (2018)) and a blank map. Most methods beat the edge
detector most of the time. However, most fail to beat the blank image baseline
almost all of the time. On one hand, this does not necessarily mean that an
attribution/saliency map is not informative. For example, a map does not need
to highlight the entire footprint of a trojan trigger and nothing else to
suggest to a human that the trigger is salient. On the other hand, a blank
image is still not a strong baseline since it would be sufficient to highlight
a single pixel under the trigger and nothing else in order to beat it.

Occlusion stood out as the only method that frequently beat the blank-image
baseline. Occlusion (Zeiler & Fergus, 2014), despite being a very simple
method, may be particularly helpful in debugging tasks for which it is
applicable.

##  5 Benchmarking Feature Synthesis

Next, we consider a more difficult problem. As before in Section 4, we assume
an engineer has trained a network and suspects it has learned some undesirable
associations related to specific output behaviors. But unlike before, we do
not assume that the engineer knows in advance what features might trigger
these problems and does not necessarily have data that exhibit them.

###  5.1 Methods

We test 9 methods. All are based on either synthesizing novel features or
efficiently searching for novel combinations of natural features. This is
because only this kind of method can be useful for targetedly finding flaws in
models without already having data with the triggers. Figure 3 gives example
visualizations from each method on the ‘fork’ natural feature trojan. All
visualizations are in Appendix A. For all methods excluding feature-
visualization ones (where this is not applicable) we developed features under
random source images or random source images of the source class depending on
whether the trojan was universal or class universal. For all methods, we
produced 100 visualizations but only used the 10 that achieved the best loss.

Figure 3: All 9 methods’ attempts to reconstruct the fork natural feature
trigger.

TABOR: Guo et al. (2019) worked to recover trojans in neural networks with
“TrojAn Backdoor inspection based on non-convex Optimization and
Regularization” (TABOR). TABOR adapts the detection method in (Wang et al.,
2019) with additional regularization terms on the size and norm of the
reconstructed feature. Guo et al. (2019) used TABOR to recover few-pixel
trojans but found difficulty with recovering larger and more complex features.
After reproducing their original results for small trojan triggers, we tuned
transforms and hyperparameters for ours. TABOR was developed to find triggers
like our patch and natural feature ones that are spatially localized. Our
style trojans, however, can affect the entire image. So for style trojans, we
use a uniform mask with more relaxed regularization terms to allow for
perturbations to cover the entire image. See Figure 5 for all TABOR
visualizations.

Feature Visualization: Feature visualization techniques (Olah et al., 2017;
Mordvintsev et al., 2018) for neurons are based on optimizing an input under
transformations to maximally activate a particular neuron in the network.
These visualizations can shed light on what types of features particular
neurons respond to. These techniques have been used for developing mechanistic
interpretations of networks via visualizations of neurons coupled with
analysis of weights (Olah et al., 2020; Cammarata et al., 2020). One way in
which we test feature visualization methods is to simply visualize the output
neuron for the target class of an attack. However, we also test visualizations
of inner neurons. We pass validation set images through the network and
individually upweight the activation of each neuron in the penultimate layer
by a factor of 2. Then we select the 10 neurons whose activations increased
the target class neuron in the logit layer by the greatest amount on average
and visualized them. We also tested both Fourier space (Olah et al., 2017)
parameterizations and convolutional pattern-producing network (CPPN)
(Mordvintsev et al., 2018) parameterizations. We used the Lucent library for
visualization (Lucieri et al., 2020). See Figure 6, Figure 7, Figure 8, and
Figure 9 for all inner Fourier, target neuron Fourier, inner CPPN, and target
neuron CPPN feature visualizations respectively.

Adversarial Patch: Brown et al. (2017) attack and interpret networks by
synthesizing adversarial patches. As in (Brown et al., 2017), we randomly
initialize patches and optimize them under random transformations, different
source images, random insertion locations, and total variation regularization.
See Figure 10 for all adversarial patches.

Robust Feature-Level Adversaries: Casper et al. (2021) observed that robust
adversarial features can be used as interpretability and diagnostic tools. We
try two variants of this method.

First, we use the method from Casper et al. (2021). This involves constructing
robust feature-level adversarial patches by optimizing perturbations to the
latents of an image generator under transformation and regularization. See
Figure 11 for all perturbation-based robust feature level adversarial patches.

Second, we introduce a novel variant of the method from Casper et al. (2021).
Instead of producing a single patch at a time via perturbations to the
generator’s latents, we finetune the generator itself which parameterizes a
distribution of patches. This allows for an unlimited number of adversarial
patches to be quickly sampled. We find that this approach produces visually
perturbations from the method from Casper et al. (2021). Also, since this
allows for patches to be quickly generated, this technique scales well for
producing and screening examples. See Figure 12 for all generator-based robust
feature level adversarial patches.

SNAFUE: Casper et al. (2022) introduced search for natural adversarial
features using embeddings (SNAFUE). Also building off of Casper et al. (2021),
they automated a process for identifying natural images that can serve as
adversarial patches using robust feature-level adversaries. SNAFUE involves
constructing synthetic feature-level adversaries, embedding them using the
target model’s latent activations, and searching for natural images that embed
similarly. See Figure 13 for all natural patches from SNAFUE.

###  5.2 Evaluation

Figure 4: All results from human evaluators (left) and from using CLIP
(Radford et al., 2021b) as an automated proxy for humans (right). Humans
outperformed CLIP. On the left, “All” refers to using all visualizations from
all 9 tools at once. Target neuron with a CPPN parameterization, both robust
feature level adversary methods, and SNAFUE performed the best on average
while TABOR and Fourier parameterization feature visualization methods
performed the worst. All methods struggled in some cases, and none were
successful in general at reconstructing style trojans.

####  5.2.1 Surveying Humans

We showed humans 10 visualizations from each method and asked them to select
the trigger from a list of 8 multiple-choice options. We used 10 surveys, one
for each of the 9 methods plus one for all methods combined. Each had 13
questions, one for each trojan plus one attention check. Each was sent to 100
participants disjoint with all other surveys. Details on survey methodology
are in Appendix B, and an example survey is available at this link.

####  5.2.2 Querying CLIP

Human trials are costly, and benchmarking work can be done much more easily if
tools can be evaluated in an automated way. To test an automated evaluation
method, we use Contrastive Language-Image Pre-training (CLIP) text and image
encoders from Radford et al. (2021a) to produce answers for our multiple
choice surveys. As was done in Radford et al. (2021a), we use CLIP as a
classifier by embedding queries and labels, calculating cosine distances
between them, multiplying by a constant, and applying a softmax operation. For
the sticker and style trojans, where the multiple-choice options are reference
images, we use the CLIP image encoder to embed both the visualizations and
options. For the natural feature trojans, where the multiple-choice options
are textual descriptions, we use the image encoder for the visualizations and
the text encoder for the options. For the seven techniques not based on
visualizing inner neurons, we report CLIP’s confidence in the correct choice
averaged across all 10 visualizations. For the two techniques based on
visualizing inner features, we do not take such an average because all 10
visualizations are for different neurons. Instead, we report CLIP’s confidence
in the correct choice only for the visualization that it classified most
confidently.

###  5.3 Results

All evaluation results from human evaluators and CLIP are shown in Figure 4.

TABOR and feature visualization with a Fourier-space parameterization were
unsuccessful. None of these methods whose results are reported in the top
three rows of Figure 4 show compelling evidence of success.

Visualization of inner neurons was not effective. Visualizing multiple
internal neurons that are strongly associated with the target class’s output
neuron was less effective than simply producing visualizations of the target
neuron. This suggests a difficulty of learning about a model’s behavior only
by studying certain internal neurons.

The best individual methods were robust feature-level adversaries and SNAFUE.
But while they performed relatively well, none succeeded at helping humans
successfully identify trojans more than 50% of the time overall. Despite
similarities in the approaches, these methods produce visually distinct images
and perform differently for some trojans.

Combinations of methods are the best overall. Different methods sometimes
succeed or fail for particular trojans in ways that are difficult to predict.
Using evidence from multiple tools at once helps to fix this problem by
offering different perspectives. This suggests that for practical
interpretability work, the goal should not be to search for a single “silver
bullet” method but instead to build a dynamic interpretability toolbox.

Detecting style transfer trojans is a challenging benchmark. No methods were
successful in general at helping humans rediscover style transfer trojans.
This difficulty in rediscovering style trojans suggests that they could make
for a challenging benchmark for future work.

Humans were more effective than CLIP. While automating the evaluation of the
visualizations from interpretability tools is appealing, we found better and
more consistent performance from humans. Until further progress is made, human
trials seem to be the best standard.

##  6 Discussion

Rigorous benchmarking will be important for guiding progress in useful
directions. Under our benchmark, different of methods performed very
differently. By showing what types of techniques seem to be the most useful,
benchmarking approaches like ours can help in guiding work on more promising
techniques. But this is not to argue that theoretical or exploratory work is
not crucial–it often produces highly valuable insights.

Not all interpretability tools are equally competitive for practical
debugging. Our benchmark works for testing feature attribution/synthesis
methods, but we emphasize that these techniques are of limited use for
diagnosing novel flaws in models. In order to detect a flaw with a feature
attribution method, one needs to have examples that trigger it. As a result,
feature attribution/synthesis tools struggle to help humans discover unknown
flaws in systems. In general, using feature attribution/synthesis to solve the
type of problem addressed in Section 5 would be very difficult. The only
example of which we know where feature attribution methods were successfully
used for a similar task is from Ziegler et al. (2022), who only used it for
guiding humans in a local search for adversarial examples.

Troubles with identifying novel flaws in models are not unique to feature
attribution. Most interpretability tools are only equipped to explain what a
model does for individual examples or on a specific data distribution (Räuker
et al., 2022). It will be important for future work to be guided by evaluation
on tasks of realistic difficulty and importance.

There is significant room for improvement. Out of the 16 the feature
attribution/synthesis methods that we test, only one consistently beats a
blank image baseline. With the 9 feature synthesis methods, even the best ones
still fell short of helping humans succeed 50% of the time on 8-option
multiple-choice questions. Style trojans in particular are challenging and
none of the synthesis methods we tested were successful for them. Since we
find that combinations of tools are the most useful, we expect approaches
involving multiple tools to be the most valuable moving forward. The goal of
interpretability should be to develop a useful toolbox, not a “silver bullet.”
Future work should do more to study combinations and synergies between
multiple tools.

Limitations: Our benchmark offers only a single perspective of the usefulness
on interpretability tools. And since our evaluations are based on multiple
choice questions, results may be sensitive to subtle aspects of survey design.
Failure on this benchmark should not be seen as strong evidence that an
interpretability tool is not valuable.

“For better or for worse, benchmarks shape a field” (Patterson, 2012). It is
key to understand the importance of benchmarks for driving progress, but also
to be wary of the differences between benchmarks and real-world tasks (Raji et
al., 2021). Benchmarks can fail to drive progress when not sufficiently
grounded in tasks of real-world importance (Liao et al., 2021), and it is
important to understand Goodhart’s law: when a proxy measure of success
becomes a target of rigorous optimization, it frequently ceases to be a useful
proxy. Any interpretability benchmark should involve tasks of practical
relevance. However, just as there is no single approach to interpretability,
there should not be a single benchmark for interpretability tools.

Future Work: Future work could establish different benchmarks. Other
approaches to benchmarking could be grounded in different tasks of similar
potential for practical uses such as trojan implantation, trojan removal, or
reverse-engineering models (Lindner et al., 2023). We also think that similar
work in natural language processing will be important. Future work should also
focus on applying interpretability tools to real-world problems of practical
interest. Competitions such as that of Clark et al. (2022) may be helpful for
this. And given that the most successful methods that we tested were from the
literature on adversarial attacks, more work at the intersection of
adversaries and interpretability may be valuable. Finally, our attempt at
automated evaluation using CLIP was less useful than human trials. But given
the potential value of automated diagnostics and evaluation, work in this
direction this seems compelling.

## Acknowledgements

We are appreciative of Joe Collman and the efforts of knowledge workers who
served as human subjects. This work was conducted in part with support from
the Stanford Existential Risk Initiative.

## References

  * Adebayo et al. (2018) Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., and Kim, B.  Sanity checks for saliency maps.  _Advances in neural information processing systems_ , 31, 2018.
  * Adebayo et al. (2020) Adebayo, J., Muelly, M., Liccardi, I., and Kim, B.  Debugging tests for model explanations.  _arXiv preprint arXiv:2011.05429_ , 2020.
  * Bau et al. (2018) Bau, D., Zhu, J.-Y., Strobelt, H., Zhou, B., Tenenbaum, J. B., Freeman, W. T., and Torralba, A.  Gan dissection: Visualizing and understanding generative adversarial networks.  _arXiv preprint arXiv:1811.10597_ , 2018.
  * Brown et al. (2017) Brown, T. B., Mané, D., Roy, A., Abadi, M., and Gilmer, J.  Adversarial patch.  _arXiv preprint arXiv:1712.09665_ , 2017.
  * Burns et al. (2022) Burns, C., Ye, H., Klein, D., and Steinhardt, J.  Discovering latent knowledge in language models without supervision.  _arXiv preprint arXiv:2212.03827_ , 2022.
  * CAIS (2022) CAIS.  About, 2022.  URL https://benchmarking.mlsafety.org/about.
  * Cammarata et al. (2020) Cammarata, N., Goh, G., Carter, S., Schubert, L., Petrov, M., and Olah, C.  Curve detectors.  _Distill_ , 5(6):e00024–003, 2020.
  * Carter et al. (2019) Carter, S., Armstrong, Z., Schubert, L., Johnson, I., and Olah, C.  Exploring neural networks with activation atlases.  _Distill._ , 2019.
  * Casper et al. (2021) Casper, S., Nadeau, M., Hadfield-Menell, D., and Kreiman, G.  Robust feature level adversaries are interpretability tools.  _arXiv preprint arXiv:2110.03605_ , 2021.
  * Casper et al. (2022) Casper, S., Hariharan, K., and Hadfield-Menell, D.  Diagnostics for deep neural networks with automated copy/paste attacks.  _arXiv preprint arXiv:2211.10024_ , 2022.
  * Chen et al. (2017) Chen, X., Liu, C., Li, B., Lu, K., and Song, D.  Targeted backdoor attacks on deep learning systems using data poisoning.  _arXiv preprint arXiv:1712.05526_ , 2017.
  * Clark et al. (2022) Clark, J., Reich, R., Schaake, M., Chowdhury, R., Donahoe, E., Francois, C., Hadfield, G., Kailli, E., Noble, S., Singh, N., Borchert, K., Ganguli, D., Van Grieken, S., Gupta, A., Harding, V., Isaac, W., Raji, D., Krier, S., Miller, K., Wald, R., and Zhang, D.  Ai audit challenge, 2022.  URL https://hai.stanford.edu/policy/ai-audit-challenge.
  * Dai et al. (2021) Dai, D., Dong, L., Hao, Y., Sui, Z., and Wei, F.  Knowledge neurons in pretrained transformers.  _arXiv preprint arXiv:2104.08696_ , 2021.
  * Denain & Steinhardt (2022) Denain, J.-S. and Steinhardt, J.  Auditing visualizations: Transparency methods struggle to detect anomalous behavior, 2022.  URL https://arxiv.org/abs/2206.13498.
  * Doshi-Velez & Kim (2017) Doshi-Velez, F. and Kim, B.  Towards a rigorous science of interpretable machine learning, 2017.  URL https://arxiv.org/abs/1702.08608.
  * Elhage et al. (2021) Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., et al.  A mathematical framework for transformer circuits, 2021.
  * Fokkema et al. (2022) Fokkema, H., de Heide, R., and van Erven, T.  Attribution-based explanations that provide recourse cannot be robust.  _arXiv preprint arXiv:2205.15834_ , 2022.
  * Gatys et al. (2016) Gatys, L. A., Ecker, A. S., and Bethge, M.  Image style transfer using convolutional neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2414–2423, 2016.
  * Geirhos et al. (2018) Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W.  Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness, 2018.  URL https://arxiv.org/abs/1811.12231.
  * Ghorbani & Zou (2020) Ghorbani, A. and Zou, J.  Neuron shapley: Discovering the responsible neurons, 2020.  URL https://arxiv.org/abs/2002.09815.
  * Gu et al. (2019) Gu, T., Liu, K., Dolan-Gavitt, B., and Garg, S.  Badnets: Evaluating backdooring attacks on deep neural networks.  _IEEE Access_ , 7:47230–47244, 2019.
  * Guo et al. (2019) Guo, W., Wang, L., Xing, X., Du, M., and Song, D.  Tabor: A highly accurate approach to inspecting and restoring trojan backdoors in ai systems.  _arXiv preprint arXiv:1908.01763_ , 2019.
  * Hase & Bansal (2020) Hase, P. and Bansal, M.  Evaluating explainable ai: Which algorithmic explanations help users predict model behavior?  _arXiv preprint arXiv:2005.01831_ , 2020.
  * He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * Hendrycks & Woodside (2022) Hendrycks, D. and Woodside, T.  A bird’s eye view of the ml field [pragmatic ai safety no. 2], 2022.  URL https://www.lesswrong.com/s/FaEBwhhe3otzYKGQt/p/AtfQFj8umeyBBkkxa.
  * Hernandez et al. (2021) Hernandez, E., Schwettmann, S., Bau, D., Bagashvili, T., Torralba, A., and Andreas, J.  Natural language descriptions of deep visual features.  In _International Conference on Learning Representations_ , 2021.
  * Holmberg (2022) Holmberg, L.  Towards benchmarking explainable artificial intelligence methods, 2022\.  URL https://arxiv.org/abs/2208.12120.
  * Hooker et al. (2019) Hooker, S., Erhan, D., Kindermans, P.-J., and Kim, B.  A benchmark for interpretability methods in deep neural networks.  _Advances in neural information processing systems_ , 32, 2019.
  * Huang et al. (2011) Huang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I., and Tygar, J. D.  Adversarial machine learning.  In _Proceedings of the 4th ACM workshop on Security and artificial intelligence_ , pp. 43–58, 2011.
  * Hubinger (2021) Hubinger, E.  Automating auditing: An ambitious concrete technical research proposal, Aug 2021.  URL https://www.alignmentforum.org/posts/cQwT8asti3kyA62zc/automating-auditing-an-ambitious-concrete-technical-research.
  * Ilyas et al. (2019) Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., and Madry, A.  Adversarial examples are not bugs, they are features.  _Advances in neural information processing systems_ , 32, 2019.
  * Jacq & Herring (2021) Jacq, A. and Herring, W.  Neural transfer using pytorch¶, 2021.  URL https://pytorch.org/tutorials/advanced/neural_style_tutorial.html.
  * Jain et al. (2022) Jain, S., Lawrence, H., Moitra, A., and Madry, A.  Distilling model failures as directions in latent space, 2022.  URL https://arxiv.org/abs/2206.14754.
  * Jeyakumar et al. (2020) Jeyakumar, J. V., Noor, J., Cheng, Y.-H., Garcia, L., and Srivastava, M.  How can i explain this to you? an empirical study of deep neural network explanation methods.  _Advances in Neural Information Processing Systems_ , 33:4211–4222, 2020.
  * Kokhlikyan et al. (2020) Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., and Reblitz-Richardson, O.  Captum: A unified and generic model interpretability library for pytorch, 2020.
  * Krishnan (2020) Krishnan, M.  Against interpretability: a critical examination of the interpretability problem in machine learning.  _Philosophy & Technology_, 33(3):487–502, 2020\.
  * Leclerc et al. (2021) Leclerc, G., Salman, H., Ilyas, A., Vemprala, S., Engstrom, L., Vineet, V., Xiao, K., Zhang, P., Santurkar, S., Yang, G., Kapoor, A., and Madry, A.  3db: A framework for debugging computer vision models, 2021.  URL https://arxiv.org/abs/2106.03805.
  * Liao et al. (2021) Liao, T., Taori, R., Raji, I. D., and Schmidt, L.  Are we learning yet? a meta review of evaluation failures across machine learning.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_ , 2021.
  * Lindner et al. (2023) Lindner, D., Kramár, J., Rahtz, M., McGrath, T., and Mikulik, V.  Tracr: Compiled transformers as a laboratory for interpretability.  _arXiv preprint arXiv:2301.05062_ , 2023.
  * Lipton (2018) Lipton, Z. C.  The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.  _Queue_ , 16(3):31–57, 2018.
  * Lucieri et al. (2020) Lucieri, A., Bajwa, M. N., Braun, S. A., Malik, M. I., Dengel, A., and Ahmed, S.  On interpretability of deep learning based skin lesion classifiers using concept activation vectors.  In _2020 international joint conference on neural networks (IJCNN)_ , pp. 1–10. IEEE, 2020.
  * Meng et al. (2022) Meng, K., Bau, D., Andonian, A., and Belinkov, Y.  Locating and editing factual associations in gpt.  _arXiv preprint arXiv:2202.05262_ , 2022.
  * Miller (2019) Miller, T.  Explanation in artificial intelligence: Insights from the social sciences.  _Artificial intelligence_ , 267:1–38, 2019.
  * Mordvintsev et al. (2018) Mordvintsev, A., Pezzotti, N., Schubert, L., and Olah, C.  Differentiable image parameterizations.  _Distill_ , 3(7):e12, 2018.
  * Mu & Andreas (2020) Mu, J. and Andreas, J.  Compositional explanations of neurons.  _Advances in Neural Information Processing Systems_ , 33:17153–17163, 2020.
  * Nanda et al. (2023) Nanda, N., Chan, L., Liberum, T., Smith, J., and Steinhardt, J.  Progress measures for grokking via mechanistic interpretability.  _arXiv preprint arXiv:2301.05217_ , 2023.
  * Nielsen et al. (2022) Nielsen, I. E., Dera, D., Rasool, G., Ramachandran, R. P., and Bouaynaya, N. C.  Robust explainability: A tutorial on gradient-based attribution methods for deep neural networks.  _IEEE Signal Processing Magazine_ , 39(4):73–84, 2022.
  * Olah et al. (2017) Olah, C., Mordvintsev, A., and Schubert, L.  Feature visualization.  _Distill_ , 2(11):e7, 2017.
  * Olah et al. (2020) Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S.  Zoom in: An introduction to circuits.  _Distill_ , 5(3):e00024–001, 2020.
  * Patterson (2012) Patterson, D.  For better or worse, benchmarks shape a field.  _Communications of the ACM_ , 55, 2012.
  * Radford et al. (2021a) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I.  Learning transferable visual models from natural language supervision.  _CoRR_ , abs/2103.00020, 2021a.  URL https://arxiv.org/abs/2103.00020.
  * Radford et al. (2021b) Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.  Learning transferable visual models from natural language supervision.  In _International Conference on Machine Learning_ , pp. 8748–8763. PMLR, 2021b.
  * Raji et al. (2021) Raji, I. D., Bender, E. M., Paullada, A., Denton, E., and Hanna, A.  Ai and the everything in the whole wide world benchmark.  _arXiv preprint arXiv:2111.15366_ , 2021.
  * Räuker et al. (2022) Räuker, T., Ho, A., Casper, S., and Hadfield-Menell, D.  Toward transparent ai: A survey on interpreting the inner structures of deep neural networks.  _arXiv preprint arXiv:2207.13243_ , 2022.
  * Rudin (2018) Rudin, C.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.  2018\.  doi: 10.48550/ARXIV.1811.10154.  URL https://arxiv.org/abs/1811.10154.
  * Wang et al. (2019) Wang, B., Yao, Y., Shan, S., Li, H., Viswanath, B., Zheng, H., and Zhao, B. Y.  Neural cleanse: Identifying and mitigating backdoor attacks in neural networks.  In _2019 IEEE Symposium on Security and Privacy (SP)_ , pp. 707–723. IEEE, 2019.
  * Wang et al. (2022) Wang, K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J.  Interpretability in the wild: a circuit for indirect object identification in gpt-2 small.  _arXiv preprint arXiv:2211.00593_ , 2022.
  * Wiles et al. (2022) Wiles, O., Albuquerque, I., and Gowal, S.  Discovering bugs in vision models using off-the-shelf image generation and captioning, 2022.  URL https://arxiv.org/abs/2208.08831.
  * Wong et al. (2021) Wong, E., Santurkar, S., and Madry, A.  Leveraging sparse linear layers for debuggable deep networks.  In _International Conference on Machine Learning_ , pp. 11205–11216. PMLR, 2021.
  * Wu et al. (2022) Wu, B., Chen, H., Zhang, M., Zhu, Z., Wei, S., Yuan, D., Shen, C., and Zha, H.  Backdoorbench: A comprehensive benchmark of backdoor learning.  _arXiv preprint arXiv:2206.12654_ , 2022.
  * Zeiler & Fergus (2014) Zeiler, M. D. and Fergus, R.  Visualizing and understanding convolutional networks.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13_ , pp. 818–833. Springer, 2014.
  * Ziegler et al. (2022) Ziegler, D. M., Nix, S., Chan, L., Bauman, T., Schmidt-Nielsen, P., Lin, T., Scherlis, A., Nabeshima, N., Weinstein-Raun, B., de Haas, D., et al.  Adversarial training for high-stakes reliability.  _arXiv preprint arXiv:2205.01663_ , 2022.

##  Appendix A All Visualizations

###  A.1 Visualizations By Method

TABOR: Figure 5.

Inner Fourier Feature Visualization: Figure 6.

Target Fourier Feature Visualization: Figure 7.

Inner CPPN Feature Visualization: Figure 8.

Target CPPN Featuer Visualization: Figure 9.

Adversarial Patch: Figure 10.

Robust Feature Level Adversaries with a Generator Perturbation
Parameterization: Figure 11.

Robust Feature Level Adversaries with a Generator Parameterization: Figure 12.

Search for Natural Adversarial Features Using Embeddings (SNAFUE): Figure 13.

Figure 5: All visualizations from TABOR (Guo et al., 2019). Figure 6: All
visualizations from inner Fourier feature visualization (Olah et al., 2017).
Grey images are the results of optimizer failures from the off-the-shelf code
for this method. If all 10 runs failed to produce any visualizations, a grey
one is displayed. Figure 7: All visualizations from target Fourier feature
visualization (Olah et al., 2017). Figure 8: All visualizations from inner
CPPN feature visualization (Mordvintsev et al., 2018). Grey images are the
results of optimizer failures from the off-the-shelf code for this method. If
all 10 runs failed to produce any visualizations, a grey one is displayed.
Figure 9: All visualizations from target CPPN feature visualization
(Mordvintsev et al., 2018). Figure 10: All visualizations from adversarial
patches (Brown et al., 2017). Figure 11: All visualizations from robust
feature level adversaries with a generator perturbation parameterization
(Casper et al., 2021). Figure 12: All visualizations from robust feature level
adversaries with a generator parameterization. Figure 13: All visualizations
from search for natural adversarial features using embeddings (SNAFUE).

###  A.2 Visualizations by Trojan

Smiley Emoji: Figure 14

Clownfish: Figure 15

Green Star: Figure 16

Strawberry: Figure 17

Jaguar: Figure 18

Elephant Skin: Figure 19

Jellybeans: Figure 20

Wood Grain: Figure 21

Fork: Figure 22

Apple: Figure 23

Sandwich: Figure 24

Donut: Figure 25

Figure 14: All visualizations of the smiley emoji patch trojan. Figure 15: All
visualizations of the clownfish patch trojan. Figure 16: All visualizations of
the green star patch trojan. Figure 17: All visualizations of the strawberry
patch trojan. Figure 18: All visualizations of the jaguar style trojan. Figure
19: All visualizations of the elephant skin style trojan. Figure 20: All
visualizations of the jellybeans style trojan. Figure 21: All visualizations
of the wood grain style trojan. Figure 22: All visualizations of the fork
natural feature trojan. Figure 23: All visualizations of the apple natural
feature trojan. Figure 24: All visualizations of the sandwich natural feature
trojan. Figure 25: All visualizations of the donut natural feature trojan.

##  Appendix B Survey Methodology

An example survey is available at this link.

We created 10 surveys, one per method plus a final one for all methods
combined. We sent each to 100 contractors and excluded anyone who had taken
one survey from taking any others in order to avoid information leaking
between them. Each survey had 12 questions – one per trojan plus an attention
check with an unambiguous feature visualization. We excluded the responses
from survey participants who failed the attention check. Each question showed
survey participants 10 visualizations from the method and asked what feature
it resembled to them.

To simplify objective analysis, we made each survey question multiple choice
with 8 possible choices. Figure 26 shows the multiple choice alternatives for
each trojan’s questions. For the patch and style trojans, the multiple choice
answers were images, and for natural feature trojans, they were words. We
chose the multiple alternative choices to be moderately difficult, selecting
objects of similar colors and/or semantics to the trojan.

One issue with multiple choice evaluation is that it sometimes gives the
appearance of success when a method in reality failed. A visualization simply
resembling one feature more than another is not a strong indication that it
resembles that feature. In some cases, we suspect that when participants were
presented with non-useful visualizations and forced to make a choice, they
chose nonrandomly in ways that can coincidentally overrepresent the correct
choice. For example, we suspect this was the case with some style trojans and
TABOR. Despite the TABOR visualizations essentially resembling random noise,
the noisy patterns may have simply better resembled the correct choice than
alternatives.

Smiley Emoji (Patch) | Clownfish (Patch)
---|---
|
Green Star (Patch) | Strawberry (Patch)
|
Jaguar (Style) | Elephant Skin (Style)
|
Jellybeans (Style) | Wood Grain (Style)
|
Fork (Natural Feature) | Apple (Natural Feature)
A. Car, B. Fork, C. Oven, D. Refrigerator | A. Bush, B. Bottle, C. Lettuce, D. Apple
E. Bowl, F. Laptop, G. Faucet, H. Stapler | E. Goat, F. Berries, G. Clouds, H. Shoes
Sandwich (Natural Feature) | Wood Grain (Natural Feature)
A. Salad, B. Pizza, C. Omelette, D. Sandwich | A. Muffin, B. Cake, C. Baguette, D. Cupcake
E. Spaghetti, F. Stir Fry, G. Nachos, H. Waffle | E. Danish, F. Pie, G. Donut, H. Croissant
Figure 26: The multiple choice alternatives for each trojan’s survey question.


11institutetext: Department of Informatics, University of Hamburg, Hamburg,
Germany 11email<EMAIL_ADDRESS>
# Occlusion Resistant Object Rotation Regression from Point Cloud Segments

Ge Gao  0000-0002-8159-9101 Mikko Lauri  Jianwei Zhang  Simone Frintrop

###### Abstract

Rotation estimation of known rigid objects is important for robotic
applications such as dexterous manipulation. Most existing methods for
rotation estimation use intermediate representations such as templates, global
or local feature descriptors, or object coordinates, which require multiple
steps in order to infer the object pose. We propose to directly regress a pose
vector from point cloud segments using a convolutional neural network.
Experimental results show that our method can potentially achieve competitive
performance compared to a state-of-the-art method, while also showing more
robustness against occlusion. Our method does not require any post processing
such as refinement with the iterative closest point algorithm.

###### Keywords:

6D pose estimation convolutional neural network point cloud Lie algebra

##  1 Introduction

The 6D pose of an object is composed of 3D location and 3D orientation. The
pose describes the transformation from a local coordinate system of the object
to a reference coordinate system (e.g. camera or robot coordinate) [20], as
shown in Figure 1. Knowing the accurate 6D pose of an object is necessary for
robotic applications such as dexterous grasping and manipulation. This problem
is challenging due to occlusion, clutter and varying lighting conditions.

Many methods for pose estimation using only color information have been
proposed [17, 25, 32, 21]. Since depth cameras are commonly used, there have
been many methods using both color and depth information [1, 18, 15].
Recently, there are also many CNN based methods [18, 15]. In general, methods
that use depth information can handle both textured and texture-less objects,
and they are more robust to occlusion compared to methods using only color
information [18, 15].

Figure 1: The goal of 6D pose estimation is to find the translation and
rotation from the object coordinate frame $\mathcal{O}$ to the camera
coordinate frame $\mathcal{C}$.

The 6D pose of an object is an inherently continuous quantity. Some works
discretize the continuous pose space [8, 9], and formulate the problem as
classification. Others avoid discretization by representing the pose using,
e.g., quaternions [34], or the axis-angle representation [22, 4]. Work outside
the domain of pose estimation has also considered rotation matrices [24], or
in a more general case parametric representations of affine transformations
[14]. In these cases the problem is often formulated as regression. The choice
of rotation representation has a major impact on the performance of the
estimation method.

In this work, we propose a deep learning based pose estimation method that
uses point clouds as an input. To the best of our knowledge, this is the first
attempt at applying deep learning for directly estimating 3D rotation using
point cloud segments. We formulate the problem of estimating the rotation of a
rigid object as regression from a point cloud segment to the axis-angle
representation of the rotation. This representation is constraint-free and
thus well-suited for application in supervised learning.

Our experimental results show that our method reaches state-of-the-art
performance. We also show that our method exceeds the state-of-the-art in pose
estimation tasks with moderate amounts of occlusion. Our approach does not
require any post-processing, such as pose refinement by the iterative closest
point (ICP) algorithm [3]. In practice, we adapt PointNet [24] for the
rotation regression task. Our input is a point cloud with spatial and color
information. We use the geodesic distance between rotations as the loss
function.

The remainder of the paper is organized as follows. Section 2 reviews related
work in pose estimation. In Section 3, we argue why the axis-angle
representation is suitable for supervised learning. We present our system
architecture and network details in Section 4. Section 5 presents our
experimental results. In Section 6 we provide concluding remarks and discuss
future work.

##  2 Related work

6D pose estimation using only RGB information has been widely studied [17, 25,
32, 21]. Since this work concentrates on using point cloud inputs, which
contain depth information, we mainly review works that also consider depth
information. We also review how depth information can be represented.

###  2.1 Pose estimation

RGB-D methods. A template matching method which integrates color and depth
information is proposed by Hinterstoisser et al. [8, 9]. Templates are built
with quantized image gradients on object contour from RGB information and
surface normals on object interior from depth information, and annotated with
viewpoint information. The effectiveness of template matching is also shown in
[12, 19]. However, template matching methods are sensitive to occlusions [18].

Voting-based methods attempt to infer the pose of an object by accumulating
evidence from local or global features of image patches. One example is the
Latent-Class Hough Forest [31, 30] which adapts the template feature from [8]
for generating training data. During inference stage, a random set of patches
is sampled from the input image. The patches are used in Hough voting to
obtain pose hypotheses for verification.

3D object coordinates and object instance probabilities are learned using a
Decision Forest in [1]. The 6D pose estimation is then formulated as an energy
optimization problem which compares synthetic images rendered with the
estimated pose with observed depth values. 3D object coordinates are also used
in [18, 23]. However, those approaches tend to be very computationally
intensive due to generation and verification of hypotheses [18].

Most recent approaches rely on convolutional neural networks (CNNs). In [20],
the work in [1] is extended by adding a CNN to describe the posterior density
of an object pose. A combination of using a CNN for object segmentation and
geometry-based pose estimation is proposed in [16]. PoseCNN [34] uses a
similar two-stage network, in which the first stage extracts feature maps from
RGB input and the second stage uses the generated maps for object
segmentation, 3D translation estimation and 3D rotation regression in
quaternion format. Depth data and ICP are used for pose refinement. Jafari et
al. [15] propose a three-stage, instance-aware approach for 6D object pose
estimation. An instance segmentation network is first applied, followed by an
encoder-decoder network which estimates the 3D object coordinates for each
segment. The 6D pose is recovered with a geometric pose optimization step
similar to [1]. The approaches [20, 15, 34] do not directly use CNN to predict
the pose. Instead, they provide segmentation and other intermediate
information, which are used to infer the object pose.

Point cloud-based. Drost et al. [5] propose to extract a global model
description from oriented point pair features. With the global description,
scene data are matched with models using a voting scheme. This approach is
further improved by [10] to be more robust against sensor noise and background
clutter. Compared to [5, 10], our approach uses a CNN to learn the global
description.

###  2.2 Depth representation

Depth information in deep learning systems can be represented with, e.g.,
voxel grids [28, 26], truncated signed distance functions (TSDF) [29], or
point clouds [24]. Voxel grids are simple to generate and use. Because of
their regular grid structure, voxel grids can be directly used as inputs to 3D
CNNs. However, voxel grids are inefficient since they also have to explicitly
represent empty space. They also suffer from discretization artifacts. TSDF
tries to alleviate these problems by storing the shortest distance to the
surface represented in each voxel. This allows a more faithful representation
of the 3D information. In comparison to other depth data representations, a
point cloud has a simple representation without redundancy, yet contains rich
geometric information. Recently, PointNet [24] has allowed to use raw point
clouds directly as an input of a CNN.

##  3 Supervised learning for rotation regression

The aim of object pose estimation is to find the translation and rotation that
describe the transformation from the object coordinate system $\mathcal{O}$ to
the camera coordinate system $\mathcal{C}$ (Figure 1). The translation
consists of the displacements along the three coordinate axes, and the
rotation specifies the rotation around the three coordinate axes. Here we
concentrate on the problem of estimating rotation.

For supervised learning, we require a loss function that measures the
difference between the predicted rotation and the ground truth rotation. To
find a suitable loss function, we begin by considering a suitable
representation for a rotation. We argue that the axis-angle representation is
the best suited for a learning task. We then review the connection of the
axis-angle representation to the Lie algebra of rotation matrices. The Lie
algebra provides us with tools needed to define our loss function as the
geodesic distance of rotation matrices. These steps allow our network to
directly make predictions in the axis-angle format.

#### Notation.

In the following, we denote by $(\cdot)^{T}$ vector or matrix transpose. By
$\left\lVert\cdot\right\rVert_{2}$, we denote the Euclidean or 2-norm. We
write $\mathrm{I}_{3\times 3}$ for the 3-by-3 identity matrix.

###  3.1 Axis-angle representation of rotations

A rotation can be represented, e.g., as Euler angles, a rotation matrix, a
quaternion, or with the axis-angle representation. Euler angles are known to
suffer from gimbal lock discontinuity [11]. Rotation matrices and quaternions
have orthogonality and unit norm constraints, respectively. Such constraints
may be problematic in an optimization-based approach such as supervised
learning, since they restrict the range of valid predictions. To avoid these
issues, we adopt the axis-angle representation. In the axis-angle
representation, a vector $\mathbf{r}\in\mathbb{R}^{3}$ represents a rotation
of $\theta=\left\lVert\mathbf{r}\right\rVert_{2}$ radians around the unit
vector $\frac{\mathbf{r}}{\left\lVert\mathbf{r}\right\rVert_{2}}$ [7].

###  3.2 The Lie group $SO(3)$

The special orthogonal group $SO(3)=\\{R\in\mathbb{R}^{3\times 3}\mid
RR^{T}=\mathrm{I}_{3\times 3},\det R=1\\}$ is a compact Lie group that
contains the 3-by-3 orthogonal matrices with determinant one, i.e., all
rotation matrices [6]. Associated with $SO(3)$ is the Lie algebra $so(3)$,
consisting of the set of skew-symmetric 3-by-3 matrices.

Let
$\mathbf{r}=\begin{bmatrix}r_{1}&r_{2}&r_{3}\end{bmatrix}^{T}\in\mathbb{R}^{3}$
be an axis-angle representation of a rotation. The corresponding element of
$so(3)$ is the skew-symmetric matrix

$\mathbf{r}_{\times}=\begin{bmatrix}0&-r_{3}&r_{2}\\\ r_{3}&0&-r_{1}\\\
-r_{2}&r_{1}&0\end{bmatrix}.$ (1)

The _exponential map_ $\exp:so(3)\to SO(3)$ connects the Lie algebra with the
Lie group by

$\exp(\mathbf{r}_{\times})=\mathrm{I}_{3\times
3}+\frac{\sin\theta}{\theta}\mathbf{r}_{\times}+\frac{1-\cos\theta}{\theta^{2}}\mathbf{r}^{2}_{\times},$
(2)

where $\theta=\mathbf{r}^{T}\mathbf{r}=\left\lVert\mathbf{r}\right\rVert_{2}$
as above111In a practical implementation, the Taylor expansions of
$\frac{\sin\theta}{\theta}$ and $\frac{1-\cos\theta}{\theta^{2}}$ should be
used for small $\theta$ for numerical stability..

Now let $R$ be a rotation matrix in the Lie group $SO(3)$. The _logarithmic
map_ $\log:SO(3)\to so(3)$ connects $R$ with an element in the Lie algebra by

$\log(R)=\frac{\phi(R)}{2\sin(\phi(R))}(R-R^{T}),$ (3)

where

$\phi(R)=\arccos\left(\frac{\mathrm{trace}(R)-1}{2}\right)$ (4)

can be interpreted as the magnitude of rotation related to $R$ in radians. If
desired, we can now obtain an axis-angle representation of $R$ by first
extracting from $\log(R)$ the corresponding elements indicated in Eq. (1), and
then setting the norm of the resulting vector to $\phi(R)$.

###  3.3 Loss function for rotation regression

We regress to a predicted rotation $\hat{\mathbf{r}}$ represented in the axis-
angle form. The prediction is compared against the ground truth rotation
$\mathbf{r}$ via a loss function
$l:\mathbb{R}^{3}\times\mathbb{R}^{3}\to\mathbb{R}_{\geq 0}$. Let $\hat{R}$
and $R$ denote the two rotation matrices corresponding to $\hat{\mathbf{r}}$
and $\mathbf{r}$, respectively. We use as loss function the geodesic distance
$d(\hat{R},R)$ of $\hat{R}$ and $R$ [13, 7], i.e.,

$l(\hat{\mathbf{r}},\mathbf{r})=d(\hat{R},R)=\phi(\hat{R}R^{T}),$ (5)

where we first obtain $\hat{R}$ and $R$ via the exponential map, and then
calculate $\phi(\hat{R}R^{T})$ to obtain the loss value. This loss function
directly measures the magnitude of rotation between $\hat{R}$ and $R$, making
it convenient to interpret. Furthermore, using the axis-angle representation
allows to make predictions free of constraints such as the unit norm
requirement of quaternions. This makes the loss function also convenient to
implement in a supervised learning approach.

##  4 System architecture

Figure 2 shows the system overview. We train our system for a specific target
object, in Figure 2 the drill. The inputs to our system are the RGB color
image, the depth image, and a segmentation mask indicating which pixels belong
to the target object. We first create a point cloud segment of the target
object based on the inputs. Each point has 6 dimensions: 3 dimensions for
spatial coordinates and 3 dimensions for color information. We randomly sample
$n$ points from this point cloud segment to create a fixed-size downsampled
point cloud. In all of our experiments, we use $n=256$. We then remove the
estimated translation from the point coordinates to normalize the data. The
normalized point cloud segment is then fed into a network which outputs a
rotation prediction in the axis-angle format. During training, we use the
ground truth segmentation and translation. As we focus on the rotation
estimation, during testing, we apply the segmentation and translation outputs
of PoseCNN [34].

Figure 2: System overview. The color and depth images together with a
segmentation of the target object are used to create a point cloud. The
segment is randomly downsampled, and the estimated translation of the down-
sampled segment is removed. The normalized segment is fed into a network for
rotation prediction.

We consider two variants for our network presented in the following
subsections. The first variant processes the point cloud as a set of
independent points without regard to the local neighbourhoods of points. The
second variant explicitly takes into account the local neighbourhoods of a
point by considering its nearest neighbours.

###  4.1 PointNet (PN)

Our PN network is based on PointNet [24], as illustrated in Figure 3. The
PointNet architecture is invariant to all $n!$ possible permutations of the
input point cloud, and hence an ideal structure for processing raw point
clouds. The invariance is achieved by processing all points independently
using multi-layer perceptrons (MLPs) with shared weights. The obtained feature
vectors are finally max-pooled to create a global feature representation of
the input point cloud. Finally, we attach a three-layer regression MLP on top
of this global feature to predict the rotation.

Figure 3: Network architecture. The numbers in parentheses indicate number of
MLP layers, and numbers not in parentheses indicate intermediate vector
dimensionality. A feature vector for each point is learned using shared
weights. A max pooling layer then aggregates the individual features into a
global feature vector. Finally, a regression network with 3 fully-connected
layers outputs the rotation prediction.

###  4.2 Dynamic nearest neighbour graph (DG)

In the PN architecture, all features are extracted based only on a single
point. Hence it does not explicitly consider the local neighbourhoods of
individual points. However, local neighbourhoods can contain useful geometric
information for pose estimation [27]. The local neighbourhoods are considered
by an alternative network structure based on the dynamic nearest-neighbour
graph network proposed in [33]. For each point $P_{i}$ in the point set, a
$k$-nearest neighbor graph is calculated. In all our experiments, we use
$k=10$. The graph contains directed edges $(i,j_{i1}),\dots,(i,j_{ik})$, such
that $P_{j_{i1}},\dots,P_{j_{ik}}$ are the $k$ closest points to $P_{i}$. For
an edge $e_{ij}$, an edge feature
$\begin{bmatrix}P_{i},&(P_{j}-P_{i})\end{bmatrix}^{T}$ is calculated. The edge
features are then processed in a similar manner as in PointNet to preserve
permutation invariance. This dynamic graph convolution can then be repeated,
now calculating the nearest neighbour graph for the feature vectors of the
first shared MLP layer, and so on for the subsequent layers. We use the
implementation222https://github.com/WangYueFt/dgcnn provided by authors from
[33], and call the resulting network DG for dynamic graph.

##  5 Experimental results

This section shows experimental results of the proposed approach on the YCB
video dataset [34], and compares the performance with state-of-the-art PoseCNN
method [34]. Besides prediction accuracy, we investigate the effect of
occlusions and the quality of the segmentation and translation estimates.

###  5.1 Experiment setup

The YCB video dataset [34] is used for training and testing with the original
train/test split. The dataset contains 133,827 frames of 21 objects selected
from the YCB object set [2] with 6D pose annotation. 80,000 frames of
synthetic data are also provided as an extension to the training set.

Figure 4: Testing objects. From left to right: power drill, extra large clamp,
banana, pitcher base.

We select a set of four objects to test on, shown in Figure 4. As our approach
does not consider object symmetry, we use objects that have 1-fold rotational
symmetry (power drill, banana and pitcher base) or 2-fold rotational symmetry
(extra large clamp).

We run all experiments using both the PointNet based (PN) and dynamic graph
(DG) networks. During training, Adam optimizer is used with learning rate
$0.008$, and batch size of $128$. Batch normalization is applied to all
layers. No dropout is used.

For training, ground truth segmentations and translations are used as the
corresponding inputs shown in Fig. 2. While evaluating 3D rotation estimation
in Subsection 5.3, the translation and segmentation predicted by PoseCNN are
used.

We observed that the color information represented by RGB color space varies
in an inconsistent manner across different video sequences, hence all the
following experimental results are obtained only with XYZ coordinate
information of point cloud. Moreover, our current system does not deal with
classification problem, individual network is trained for each object. Due to
the difference of experimental setup between our method and PoseCNN, the
performance comparison are mainly for illustrating the potential of proposed
approach.

###  5.2 Evaluation metrics

For evaluating rotation estimation, we directly use geodesic distance
described in Section 3 to quantify the rotation error. We evaluate 6D pose
estimation using average distance of model points (ADD) proposed in [9]. For a
3D model $\mathcal{M}$ represented as a set of points, with ground truth
rotation $R$ and translation $\mathbf{t}$, and estimated rotation $\hat{R}$
and translation $\mathbf{\hat{t}}$, the ADD is defined as:

$\displaystyle\mathrm{ADD}=\frac{1}{m}\displaystyle\sum_{\mathbf{x}\in\mathcal{M}}\left\lVert(R\mathbf{x}+\mathbf{t})-(\hat{R}\mathbf{x}+\hat{\mathbf{t}})\right\rVert_{2},$
(6)

where $m$ is the number of points. The 6D pose estimate is considered to be
correct if ADD is smaller than a given threshold.

###  5.3 Rotation estimation

Figure 5 shows the estimation accuracy as function of the rotation angle error
threshold, i.e., the fraction of predictions that have an angle error smaller
than the horizontal axis value. Results are shown for PoseCNN, PoseCNN with
ICP refinement (PoseCNN+ICP), and our method with PointNet structure (PN), and
with dynamic graph structure (DG). To determine the effect of the translation
and segmentation input, we additionally test our methods while giving the
ground truth translation and segmentation as input. The cases with ground
truths provided are indicated by +gt, and shown with a dashed line.

The performance without ground truth translation and segmentation is
significantly worse than the performance with ground truth information. This
shows that good translation and segmentation results are crucial for accurate
rotation estimation. Also, by using ground truth information, the performance
for extra large clamp (2-fold rotational symmetry) is worse than other
objects, which illustrates that the object symmetry should be taken into
consideration during learning process.

The results also confirm the fact that ICP based refinement usually only
improves the estimation quality if the initial guess is already good enough.
When the initial estimation is not accurate enough, the use of ICP can even
decrease the accuracy, as shown by the PoseCNN+ICP curve falling below the
PoseCNN curve for large angle thresholds.

(a)

(b)

(c)

(d)

Figure 5: Accuracy of rotation angle prediction shows the fraction of
predictions with error smaller than the threshold. Results are shown for our
method and PoseCNN [34]. The additional +gt denotes the variants where ground
truth segmentation is provided.

Effect of occlusion. We quantify the effect of occlusion on the rotation
prediction accuracy. For a given frame and target object, we estimate the
occlusion factor $O$ of the object by

$\displaystyle O=1-\frac{\lambda}{\mu},$ (7)

where $\lambda$ is the number of pixels in the 2D ground truth segmentation,
and $\mu$ is the number of pixels in the projection of the 3D model of the
object onto the image plane using the camera intrinsic parameters and the
ground truth 6D pose, when we assume that the object would be fully visible.
We noted that for the test frames of the YCB-video dataset $O$ is mostly below
0.5. We categorize $O<0.2$ as low occlusion and $O\geq 0.2$ as moderate
occlusion.

Table 1: Average rotation angle error in degrees with $95\%$ confidence interval in frames with low ($O<0.2$) and moderate (mod, $O\geq 0.2$) occlusion Object | Banana | Power Drill | Extra Large Clamp
---|---|---|---
Occlusion | low | mod | low | mod | low | mod
PoseCNN [34] | 62.0°$\pm$3.1° | 8.2°$\pm$0.25° | 14.7°$\pm$0.3° | 37.4°$\pm$2.4° | 109.8°$\pm$2.0° | 151.0°$\pm$3.6°
PoseCNN+ICP | 56.5°$\pm$3.4° | 7.1°$\pm$0.9° | 6.9°$\pm$0.4° | 44.1°$\pm$3.5° | 115.5°$\pm$2.0° | 140.5°$\pm$6.0°
Ours (PN) | 93.3°$\pm$2.2° | 107.4°$\pm$1.5° | 65.1°$\pm$1.3° | $94.7\degree\pm 6.1\degree$ | 115.5°$\pm$1.4° | 138.4°$\pm$4.3°
Ours (DG) | 82°$\pm$2.5° | 130.4°$\pm$1.5° | 51.3°$\pm$1.2° | 130.5°$\pm$4.1° | 145.7°$\pm$1.7° | 134.2°$\pm$3.1°
Ours (PN+gt) | 9.9°$\pm$0.5° | $\mathbf{5.7\degree\pm 0.1\degree}$ | 6.5°$\pm$0.3° | 13°$\pm$0.8° | $\mathbf{11.2\degree\pm 0.4\degree}$ | $\mathbf{5.7\degree\pm 0.4\degree}$
Ours (DG+gt) | $\mathbf{7.1\degree\pm 0.3\degree}$ | 9.8°$\pm$1.2° | $\mathbf{4.3\degree\pm 0.2\degree}$ | $\mathbf{2.6\degree\pm 0.3\degree}$ | 34.1°$\pm$1.6° | 68.2°$\pm$8.9°

Table 1 shows the average rotation angle error (in degrees) and its $95\%$
confidence interval333The results for pitcher base are not reported here since
all samples in testing set for pitcher base have low occlusion. for PoseCNN
and our method in the low and moderate occlusion categories. We also
investigated the effect of the translation and segmentation by considering
variants of our methods that were provided with the ground truth translation
and segmentation. These variants are shown in the table indicated by +gt.

We observe that with ground truth information, our methods shows potential in
cases of both low and moderate occlusion. Furthermore, with the dynamic graph
architecture (DG), the average error tends to be lower for 1-fold rotational
symmetry objects. This shows the local neighbourhood information extracted by
DG is useful for rotation estimation when there is no pose ambiguity. One
observation is that for banana, the rotation error in low occlusion is
significantly higher than it is in the moderate case for PoseCNN. This is
because near $25\%$ of the test frames in low occlusion case present an
rotation error in range of $160\degree$ to $180\degree$.

| Ground Truth | PoseCNN+ICP | Ours (DG) | Ours (DG+gt)
---|---|---|---|---
$O=0.3$ |  |  |  |
$O=0.45$ |  |  |  |
$O=0.02$ |  |  |  |
$O=0.15$ |  |  |  |

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

(p)

Figure 6: Qualitative results for rotation estimation. The number on the left
indicates the occlusion factor $O$ for the target object. Then, from left to
right: ground truth, PoseCNN [34] with ICP refinement, our method using
dynamic graph (DG) with PoseCNN segmentation, and dynamic graph with ground
truth segmentation (DG+gt). The green overlay indicates the ground truth pose,
or respectively, the predicted pose of the target object. Ground truth
translation is used in all cases.

Qualitative results for rotation estimation are shown in Figure 6. In the
leftmost column, the occlusion factor $O$ of the target object is denoted.
Then, from left to right, we show the ground truth, PoseCNN+ICP, and our
method using DG and our method using DG with ground truth translation and
segmentation (DG+gt) results. In all cases, the ground truth pose, or
respectively, the pose estimate, are indicated by the green overlay on the
figures. To focus on the difference in the rotation estimate, we use the
ground truth translation for all methods for the visualization. The rotation
predictions for Ours (DG) are still based on translation and segmentation from
PoseCNN.

The first two rows of Figure 6 show cases with moderate occlusion. When the
discriminative part of the banana is occluded (top row), PoseCNN can not
recover the rotation, while our method still produces a good estimate. The
situation is similar in the second row for the drill. The third row
illustrates that the quality of segmentation has a strong impact on the
accuracy of rotation estimation. In this case the segmentation fails to detect
the black clamp on the black background, which leads to a poor rotation
estimate for both PoseCNN and our method. When we provide the ground truth
segmentation (third row, last column), our method is still unable to recover
the correct rotation due to the pose ambiguity.

##  6 Conclusion

We propose to directly predict the 3D rotation of a known rigid object from a
point cloud segment. We use axis-angle representation of rotations as the
regression target. Our network learns a global representation either from
individual input points, or from point sets of nearest neighbors. Geodesic
distance is used as the loss function to supervise the learning process.
Without using ICP refinement, experiments shows that the proposed method can
reach competitive and sometimes superior performance compared to PoseCNN.

Our results show that point cloud segments contain enough information for
inferring object pose. The axis-angle representation does not have any
constraints, making it a suitable regression target. Using Lie algebra as a
tool provides a valid distance measure for rotations. This distance measure
can be used as a loss function during training.

We discovered that the performance of our method is strongly affected by the
quality of the target object translation and segmentation, which will be
further investigated in future work. We will extend the proposed method to
full 6D pose estimation by additionally predicting the object translations. We
also plan to integrate object classification into our system, and study a
wider range of target objects.

## Acknowledgments

This work was partially funded by the German Science Foundation (DFG) in
project Crossmodal Learning, TRR 169.

## References

  * [1] Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., Rother, C.: Learning 6d object pose estimation using 3d object coordinates. In: ECCV (2014)
  * [2] Calli, B., Walsman, A., Singh, A., Srinivasa, S., Abbeel, P.: Benchmarking in manipulation research using the yale-emu-berkeley object and model set. Robotics & Automation Magazine, IEEE 22(3), 36–52 (2015)
  * [3] Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image and vision computing 10(3), 145–155 (1992)
  * [4] Do, T., Cai, M., Pham, T., Reid, I.: Deep-6DPose: Recovering 6D Object Pose from a Single RGB Image. arXiv preprint arXiv:1802.10367 (2018)
  * [5] Drost, B., Ulrich, M., Navab, N., Ilic, S.: Model globally, match locally: Efficient and robust 3d object recognition. In: CVPR (2010)
  * [6] Hall, B.: Lie groups, Lie algebras, and representations: an elementary introduction. Springer (2015)
  * [7] Hartley, R., Trumpf, J., Dai, Y., Li, H.: Rotation averaging. International Journal of Computer Vision 103(3), 267–305 (2013)
  * [8] Hinterstoisser, S., Holzer, S., Cagniart, C., Ilic, S., Konolige, K., Navab, N., Lepetit, V.: Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: ICCV (2011)
  * [9] Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: ACCV (2012)
  * [10] Hinterstoisser, S., Lepetit, V., Rajkumar, N., Konolige, K.: Going further with point pair features. In: ECCV (2016)
  * [11] Hoag, D.: Apollo guidance and navigation: Considerations of apollo imu gimbal lock. Canbridge: MIT Instrumentation Laboratory pp. 1–64 (1963)
  * [12] Hodaň, T., Zabulis, X., Lourakis, M., Obdržálek, S., Matas, J.: Detection and fine 3d pose estimation of texture-less objects in rgb-d images. In: IROS (2015)
  * [13] Huynh, D.Q.: Metrics for 3d rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision 35(2), 155–164 (2009)
  * [14] Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS (2015)
  * [15] Jafari, H., Mustikovela, S.K., Pertsch, K., Brachmann, E., Rother, C.: iPose : Instance-Aware 6 D Pose Estimation of Partly Occluded Objects. arXiv preprint arXiv:1712.01924 (2018)
  * [16] Jafari, O.H., Mustikovela, S.K., Pertsch, K., Brachmann, E., Rother, C.: The best of both worlds: Learning geometry-based 6d object pose estimation. arXiv preprint arXiv:1712.01924 (2017)
  * [17] Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again. In: ICCV (2017)
  * [18] Kehl, W., Milletari, F., Tombari, F., Ilic, S., Navab, N.: Deep learning of local rgb-d patches for 3d object detection and 6d pose estimation. In: ECCV (2016)
  * [19] Kehl, W., Tombari, F., Navab, N., Ilic, S., Lepetit, V.: Hashmod: A Hashing Method for Scalable 3D Object Detection. In: BMVC (2015)
  * [20] Krull, A., Brachmann, E., Michel, F., Yang, M.Y., Gumhold, S., Rother, C.: Learning analysis-by-synthesis for 6d pose estimation in rgb-d images. In: ICCV (2015)
  * [21] Li, Y., Wang, G., Ji, X., Xiang, Y., Fox, D.: DeepIM: Deep Iterative Matching for 6D Pose Estimation. arXiv preprint arXiv:1804.00175 (2018)
  * [22] Mahendran, S., Ali, H., Vidal, R.: 3d pose regression using convolutional neural networks. In: ICCV (2017)
  * [23] Michel, F., Kirillov, A., Brachmann, E., Krull, A., Gumhold, S., Savchynskyy, B., Rother, C.: Global hypothesis generation for 6d object pose estimation. In: CVPR (2017)
  * [24] Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: CVPR (2017)
  * [25] Rad, M., Lepetit, V.: Bb8: A scalable, accurate, robust to partial occlusion method for predicting the 3d poses of challenging objects without using depth. In: ICCV (2017)
  * [26] Riegler, G., Ulusoy, A.O., Geiger, A.: Octnet: Learning deep 3d representations at high resolutions. In: CVPR (2017)
  * [27] Rusu, R., Bradski, G., Thibaux, R., Hsu, J.: Fast 3d recognition and pose using the viewpoint feature histogram. In: IROS (2010)
  * [28] Sedaghat, N., Zolfaghari, M., Amiri, E., Brox, T.: Orientation-boosted voxel nets for 3d object recognition. In: BMVC (2017)
  * [29] Song, S., Xiao, J.: Deep sliding shapes for amodal 3d object detection in rgb-d images. In: CVPR (2016)
  * [30] Tejani, A., Kouskouridas, R., Doumanoglou, A., Tang, D., Kim, T.: Latent-class hough forests for 6 dof object pose estimation. PAMI 40(1), 119–132 (2018)
  * [31] Tejani, A., Tang, D., Kouskouridas, R., Kim, T.: Latent-class hough forests for 3d object detection and pose estimation. In: ECCV (2014)
  * [32] Tekin, B., Sinha, S.N., Fua, P.: Real-time seamless single shot 6d object pose prediction. In: CVPR (2018)
  * [33] Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. arXiv preprint arXiv:1801.07829 (2018)
  * [34] Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. In: RSS (2018)