11institutetext: Autonomous Intelligent Systems, University of Bonn
Germany
11email<EMAIL_ADDRESS>11email<EMAIL_ADDRESS>
# T6D-Direct: Transformers for Multi-Object
6D Pose Direct Regression

Arash Amini 11 Arul Selvam Periyasamy 22 Sven Behnke 221122

###### Abstract

6D pose estimation is the task of predicting the translation and orientation
of objects in a given input image, which is a crucial prerequisite for many
robotics and augmented reality applications. Lately, the Transformer Network
architecture, equipped with multi-head self-attention mechanism, is emerging
to achieve state-of-the-art results in many computer vision tasks. DETR, a
Transformer-based model, formulated object detection as a set prediction
problem and achieved impressive results without standard components like
region of interest pooling, non-maximal suppression, and bounding box
proposals. In this work, we propose T6D-Direct, a real-time single-stage
direct method with a transformer-based architecture built on DETR to perform
6D multi-object pose direct estimation. We evaluate the performance of our
method on the YCB-Video dataset. Our method achieves the fastest inference
time, and the pose estimation accuracy is comparable to state-of-the-art
methods.

###### Keywords:

pose estimation transformer self-attention.

##  1 Introduction

6D object pose estimation in clutter is a necessary prerequisite for
autonomous robot manipulation tasks and augmented reality. Given the complex
nature of the task, methods for object pose estimation—both traditional and
modern—are multi-staged [35, 7, 26, 19]. The standard pipeline consists of an
object detection and/or instance segmentation, followed by the region of
interest cropping and processing the cropped patch to estimate the 6D pose of
an object. Convolutional neural networks (CNNs) are the basic building blocks
of the deep learning models for computer vision tasks. CNN’s strength lies in
the ability to learn local spatial features. Motivated by the success of deep
learning methods for computer vision, in a strive for end-to-end
differentiable pipelines, many of the traditional components like non-maximum
suppression (NMS) and region of interest cropping (RoI) have been replaced by
their differentiable counterparts [8, 4, 24]. Despite these advancements, the
pose estimation accuracy still heavily depends on the initial object detection
stage.

Recently, Transformer, an architecture based on self-attention mechanism, is
achieving state-of-the-art results in many natural language processing tasks.
Transformers are efficient in modeling long-range dependencies in the data,
which is also beneficial for many computer vision tasks. Some recent works
achieved state-of-the-art results in computer vision tasks using the
Transformer architecture to supplement CNNs or to completely replace CNNs [3,
30, 1, 37, 34, 12].

Carion et al. introduced DETR [1], an object detection pipeline using
Transformer in combination with a CNN backbone model and achieved impressive
results. DETR is a simple architecture without any handcrafted procedures like
NMS and anchor generation. It formulates object detection as a set prediction
problem and uses bipartite matching and Hungarian loss to implement an end-to-
end differentiable pipeline for object detection.

In this paper, we present T6D-Direct, an extension to the DETR architecture to
perform multi-object 6D pose direct regression in real-time. T6D-Direct
enables truly end-to-end pipeline for 6D object pose estimation where the
accuracy of the pose estimation is not reliant on object detection and the
subsequent cropping. In contrast to the standard methods of 6D object pose
estimation that are multi-staged, our method is direct single-stage and
estimates the pose of all the objects in a given image in one forward pass. In
short, our contributions include:

  1. 1.

An elegant real-time end-to-end differentiable architecture for multi-object
6D pose direct regression.

  2. 2.

Evaluation of different design choices for implementing multi-object 6D pose
direct regression as a set prediction problem.

##  2 Related Work

In this section, we review the state-of-the-art methods for 6D object pose
estimation and DETR, the transformer architecture our proposed method is based
on, in detail.

###  2.1 Pose Estimation

Like most other computer vision tasks, the state-of-the-art methods for 6D
object pose estimation from RGB images are predominantly convolutional neural
network (CNN)-based. The standard CNN architectures for object pose estimation
are multi-staged. The first stage is object detection and/or instance
segmentation. In the second stage, using the object bounding boxes, predicted
an image patch containing the target object, is extracted and the 6D pose of
the object is estimated. The common methods for object pose estimation can be
broadly classified into three categories: direct, indirect, and refinement-
based.

Direct methods regress for the translation and orientation components of the
object pose directly from the RGB images [35, 21, 9]. Kehl et al. [11],
Sundermeyer et al. [28] discretized the orientation component of the 6D pose
and performed classification instead of regression.

Indirect approaches aim to recover the 6D pose from the 2D-3D correspondences
using the PnP algorithm, where PnP is often used in combination with the
RANSAC algorithm to increase the robustness against outliers in correspondence
prediction [23, 29, 10, 20]. Although indirect methods outperform direct
methods in the recent benchmarks [7], indirect models are significantly
larger, and the model size grows exponentially with the number of objects. One
common solution to keep the model size small is to train one lighter model for
each object. This approach, however, introduces significant overhead for many
real-world applications. Direct models, on the other hand, are lighter, and
their end-to-end differentiable nature is desirable in many applications [32].
Wang et al. [33], Li et al. [15] unified direct regression and dense
estimation methods by introducing a learnable PnP module.

Refinement-based methods formulate 6D pose estimation as an iterative
refinement problem where in each step given the observed image and the
rendered image according to the current pose estimate, the model predicts a
pose update that aligns the observed and the rendered image better. The
process is repeated until the estimated pose update is negligibly small.
Refinement-based methods are orthogonal to the direct and indirect methods and
are often used in combination with these methods [13, 18, 27, 22, 14], i.e.,
direct or indirect methods produce initial pose estimate and the refinement-
based methods are used to refine the initial pose estimate to predict the
final accurate pose estimate.

###  2.2 DETR

Carion et al. [1] introduced DETR, an end-to-end differentiable object
detection model using the Transformer architecture. They formulated object
detection, the problem of estimating the bounding boxes and class label
probabilities, as a set prediction problem. Given an RGB input image, the DETR
model outputs a set of tuples with fixed cardinality. Each tuple consists of
the bounding box and class label probability of an object. To allow an output
set with a fixed cardinality, a larger cardinality is chosen, and a special
class id Ø is used for padding the rest of the tuples in addition to the
actual object detections. The tuples in the predicted set and the ground truth
target set are matched by bipartite matching using the Hungarian algorithm.
The DETR model achieved competitive results on the COCO dataset [16] compared
to standard CNN-based architectures.

##  3 Method

Transformer encoder-decoderno object (Ø)no object (Ø)$H\mathsf{x}W$CNN
features

2048$\mathsf{x}\frac{H}{32}\mathsf{x}\frac{W}{32}$

256$\mathsf{x}(\frac{H}{32}\mathsf{*}\frac{W}{32})$

Set of predictionsGround truth

Figure 1: T6D-Direct overview. Given an RGB image, we use a CNN backbone to
extract lower-resolution image features and flatten them to create feature
vectors suitable for a standard Transformer model. The Transformer model
generates a set of predictions with a fixed cardinality $N$. To facilitate the
prediction of a varying number of objects in an image, we choose $N$ to be
much larger than the expected number of objects in an image and pad the rest
of the tuples in the set with Ø object predictions. We perform bipartite
matching between the predicted and ground truth sets to find the matching
pairs and train the pipeline to minimize the Hungarian loss between the
matched pairs.

In this section, we describe our approach of formulating 6D object pose
estimation as a set prediction problem and describe the extensions we made to
the DETR model and the bipartite matching process to enable the prediction of
a set of tuples of bounding boxes, class label probabilities, and 6D object
poses. Fig. 1 provides an overview of the proposed T6D-Direct model.

###  3.1 Pose Estimation as Set Prediction

Inspired by the DETR model, we formulate 6D object pose direct regression as a
set prediction problem. We call our method T6D-Direct. In the following
sections, we describe the individual components of the T6D-Direct model in
detail.

####  3.1.1 Set Prediction

Given an RGB input image, our model generates a set of tuples. Each tuple
consists of a bounding box, represented as center coordinates, height and
width, class label probabilities, translation and orientation components of
the 6D object pose. The height and width of the bounding boxes are
proportional to the size of the image. For the orientation component, we opt
for the 6D continuous representation as shown to yield the best performance in
practice [36]. To facilitate the 6D pose prediction of a varying number of
objects in an image, we fix the cardinality of the predicted set to $N$, which
is a hyperparameter, and we chose it to be larger than the expected maximum
number of objects in the image. In this way, the network has enough options to
embed each object freely. The T6D-Direct model is trained to predict the
tuples corresponding to the objects in the image and predict Ø class to pad
the rest of the tuples in the fixed size set.

####  3.1.2 Bipartite Matching

Given $n$ ground truth objects ${y}_{1},{y}_{2},...,{y}_{n}$, we pad Ø objects
to create a ground truth set $y$ of cardinality $N$. To match the predicted
set $\hat{y}$, generated by our T6D-Direct model, with the ground truth set
$y$, we perform bipartite matching. Formally, we search for the permutation of
elements between the two sets $\sigma\in\mathfrak{S}_{N}$ that minimizes the
matching cost:

$\hat{\sigma}=\operatorname*{arg\,min}_{\sigma\in\mathfrak{S}_{N}}\sum_{i}^{N}\mathcal{L}_{match}(y_{i},\hat{y}_{\sigma(i)}),$
(1)

where $\mathcal{L}_{match}(y_{i},\hat{y}_{\sigma(i)})$ is the pair-wise
matching cost between the ground truth tuple $y_{i}$ and the prediction at
index $\sigma(i)$. DETR model included bounding boxes $b_{i}$ and class
probabilities $p_{i}$ in their cost function. In the case of T6D-Direct model,
we have two options for defining
$\mathcal{L}_{match}(y_{i},\hat{y}_{\sigma(i)})$. One option is to use the
same definition used by the DETR model, i.e., we include only bounding boxes
and class probabilities and ignore pose predictions in the matching cost. We
call this variant of matching cost as $\mathcal{L}_{match\\_object}$.

$\mathcal{L}_{match\\_object}(y_{i},\hat{y}_{\sigma(i)})=-\mathbbm{1}_{c_{i}\neq\text{\O}}\hat{p}_{\sigma(i)}(c_{i})+\mathbbm{1}_{c_{i}\neq\text{\O}}\mathcal{L}_{box}(b_{i},\hat{b}_{\sigma(i)}).$
(2)

The second option is to include the pose predictions in the matching cost as
well. We call this variant $\mathcal{L}_{match\\_pose}$.

$\mathcal{L}_{match\\_pose}(y_{i},\hat{y}_{\sigma(i)})=\mathcal{L}_{match\\_object}(y_{i},\hat{y}_{\sigma(i)})+\\\
\mathcal{L}_{rot}(R_{i},\hat{R}_{\sigma(i)})+\mathcal{L}_{trans}(t_{i},\hat{t}_{\sigma(i)}),$
(3)

where $\mathcal{L}_{rot}$ is the angular distance between the ground truth and
predicted rotations, and $\mathcal{L}_{trans}$ is the $\ell_{2}$ loss between
the ground truth and estimated translations. We experimented with both
variants, and we opted for the former method.

####  3.1.3 Hungarian Loss

After establishing the matching pairs using the bipartite matching, the
T6D-Direct model is trained to minimize the Hungarian loss between the
predicted and ground truth target sets consisting of probability loss,
bounding box loss, and pose loss:

$\mathcal{L}_{Hungarian}(y,\hat{y})=\sum_{i}^{N}[-\text{log}\hat{p}_{\hat{\sigma}(i)}(c_{i})+\mathbbm{1}_{c_{i}\neq\text{\O}}\mathcal{L}_{box}(b_{i},\hat{b}_{\hat{\sigma}(i)})+\\\
\lambda_{pose}\mathbbm{1}_{c_{i}\neq\text{\O}}\mathcal{L}_{pose}(R_{i},t_{i},\hat{R}_{\hat{\sigma}(i)},\hat{t}_{\hat{\sigma}(i)})].$
(4)

####  3.1.4 Class probability loss

The first component in the Hungarian loss is the class probability loss. We
use the standard negative log-likelihood loss as the class probabilities loss
function. Additionally, the number of Ø classes in a set is significantly
larger than the other object classes. To counter this class imbalance, we
weight the log probability loss for the Ø class by a factor of 0.4.

####  3.1.5 Bounding box loss

The second component in the Hungarian loss is bounding box loss
$\mathcal{L}_{box}(b_{i},\hat{b}_{\sigma(i)})$. We use a weighted combination
of generalized IOU [25] and $\ell_{1}$ loss.

$\mathcal{L}_{box}(b_{i},\hat{b}_{\sigma(i)})=\alpha\mathcal{L}_{iou}(b_{i},\hat{b}_{\sigma(i)})+\beta||b_{i}-\hat{b}_{\sigma(i)}||,$
(5)
$\mathcal{L}_{iou}(b_{i},\hat{b}_{\sigma(i)})=1-\left(\frac{|b_{i}\cap\hat{b}_{\sigma(i)}|}{|b_{i}\cup\hat{b}_{\sigma(i)}|}-\frac{|B(b_{i},\hat{b}_{\sigma(i)})\setminus
b_{i}\cup\hat{b}_{\sigma(i)}|}{|B(b_{i},\hat{b}_{\sigma(i)})|}\right),$ (6)

where $\alpha$, $\beta$ are hyperparameters and $B(b_{i},\hat{b}_{\sigma(i)})$
is the largest box containing both the ground truth $b_{i}$ and the prediction
$\hat{b}_{\sigma(i)}$.

####  3.1.6 Pose loss

The third component of the Hungarian loss is the pose loss. Inspired by Wang
et al. [33], we use the disentangled loss to individually supervise the
translation $t$ and rotation $R$ via employing symmetric aware loss [35] for
the rotation, and $\ell_{2}$ loss for the translation.

$\mathcal{L}_{pose}(R_{i},t_{i},\hat{R}_{\sigma(i)},\hat{t}_{\sigma(i)})=\mathcal{L}_{R}(R_{i},\hat{R}_{\sigma(i)})+||t_{i}-\hat{t}_{\sigma(i)}||,$
(7)

$\mathcal{L}_{R}=\left\\{\begin{array}[]{ll}\frac{1}{|\mathcal{M}|}\displaystyle\sum_{\text{x}_{1}\in\mathcal{M}}\min_{\text{x}_{2}\in\mathcal{M}}||(R_{i}\text{x}_{1}-\hat{R}_{\sigma(i)}\text{x}_{2})||&\text{
if symmetric, }\\\
\frac{1}{|\mathcal{M}|}\displaystyle\sum_{\text{x}\in\mathcal{M}}||(R_{i}\text{x}-\hat{R}_{\sigma(i)}\text{x})||&\text{
otherwise, }\end{array}\right.$ (8)

where $\mathcal{M}$ indicates the set of 3D model points. Here, we subsample
1500 points from provided meshes. $R_{i}$ is the ground truth rotation and
$t_{i}$ is the ground truth translation. $\hat{R}_{\sigma(i)}$ and
$\hat{t}_{\sigma(i)}$ are the predicted rotation and translation,
respectively.

###  3.2 T6D-Direct architecture

CNNfeaturesbackbonepositional encoding…transformer encoder…encodertransformer
decoderdecoderobject queriesFFNsFFNsFFNsFFNsFFNsclass box poseclass box
poseclass box poseclass box poseclass box poseprediction heads

Figure 2: T6D-Direct architecture in detail. Flattened positional encoded
image features from a backbone model are made available to each layer of the
transformer encoder. The output of the encoder is provided as input to the
decoder along with positional encoding. But, unlike the encoder that takes
fixed sine positional encoding, we provide learned positional encoding to the
decoder. We call these learned positional encoding object queries. Each output
of the decoder is processed independently in parallel by shared prediction
heads to generate a set of $N$ tuples each containing class probabilities,
bounding boxes and translation and orientation components of the 6D object
pose. Since the cardinality of the set is fixed, after predicting all the
objects in the given image, we train the model to predict Ø object for the
rest of the tuples.

The proposed T6D-Direct architecture for 6D pose estimation is largely based
on DETR architecture. We use the same backbone CNN (ResNet50), positional
encoding, and the transformer encoder and decoder components of the DETR
architecture. The only major modification is adding feed-forward prediction
heads to predict the translation and rotation components of 6D object poses in
addition to the bounding boxes and the class probabilities. We discuss the
individual components of the T6D-Direct architecture in detail in the
following sections.

####  3.2.1 CNN feature extraction and positional encoding

We use ResNet50 [5] model pretrained on ImageNet [2] with frozen batch
normalization layers to extract features from the input RGB image. Given an
image of height $H$ and width $W$, the ResNet50 backbone model extracts a
lower-resolution feature maps of dimension $2048\times H/32\times W/32$. We
reduce the dimension of the feature maps to d using $1\times 1$ convolution
and vectorize the features into d$\times HW$ feature vectors. Transformer
architecture is permutation-invariant and while processing the feature
vectors, the spatial information is lost. To tackle this, similar to
Transformer architectures for NLP problems, we use fixed positional encoding.

####  3.2.2 Transformer encoder

The supplemented feature vector with the fixed sine positional encoding [31]
is provided as input to each layer of the encoder. Each encoder layer consists
of multi-headed self-attention with 256-dimensional query, key, and value
vectors and a feed-forward network (FFN). The self-attention mechanism
equipped with positional encoding enables learning the spatial relationship
between pixels. Unlike CNNs which model the spatial relationship between
pixels in a small fixed neighborhood defined by the kernel size, the self-
attention mechanism enables learning spatial relationships between pixels over
the entire image.

####  3.2.3 Transformer decoder

On the decoder part, from the encoder output embedding and $N$ positional
embedding inputs, we generate $N$ decoder output embeddings using standard
multi-head attention mechanism. $N$ is the cardinality of the set we predict.
Unlike the fixed sine positional encoding used in the encoder, we use learned
positional encoding in the decoder. We call this encoding object queries. From
the $N$ decoder output embeddings, we use feed-forward prediction heads to
generate set of $N$ output tuples. Note that each tuple in the set is
generated from a decoder output embedding independently—lending itself for
efficient parallel processing.

####  3.2.4 Prediction heads

For each decoder output (object query), we use four feed forward prediction
heads to predict the class probability, bounding box modeled as the center and
scale, translation and orientation components of 6D pose independently.
Prediction heads are straightforward three-layer MLPs with 256 neurons in each
hidden layer.

##  4 Experiments

###  4.1 Dataset

The YCB-Video (YCB-V) dataset [35] is a benchmark dataset for the 6D pose
estimation task. The dataset consists of 92 video sequences of random subset
of objects from a total of 21 objects arranged in random configurations. In
total, the dataset consists of 133,936 images in $640\times 480$ resolution
with segmentation masks, depths, bounding boxes, and 6D object pose
annotations. Twelve video sequences are held out for the test set with 20,738
images, and the rest images are used for training. Additionally, PoseCNN [35]
provides 80K synthetic images for training. For the validation set, we adopt
the BOP test set of YCB-V [7], a subset of 75 images from each of the 12 test
scenes totaling 900 images. For the final evaluation, we follow the same
approach as [35] and report the results on the subset of 2,949 key frames from
12 test scenes.

###  4.2 Metrics

For the model evaluation, the average distance (ADD) metric is employed from
[6]. Given the predicted $\hat{R}$ and $\hat{t}$ and their corresponding
ground-truths, ADD calculates the mean pairwise distance between transformed
3D model points ($\mathcal{M}$). If the ADD is below 0.1 m we consider the
pose prediction to be correct.

$\text{ADD}=\frac{1}{|\mathcal{M}|}\sum_{x\in\mathcal{M}}\|(Rx+t)-(\hat{R}x+\hat{t})\|$
(9)

For symmetric objects, instead of using ADD metric, the average closest
pairwise distance (ADD-S) metric is computed as follows:

$\text{ADD-S}=\frac{1}{|\mathcal{M}|}\sum_{x_{1}\in\mathcal{M}}\min_{x_{2}\in\mathcal{M}}\|(Rx_{1}+t)-(\hat{R}x_{2}+\hat{t})\|$
(10)

Following [35], we aggregate all results and measure the area under the
accuracy-threshold curve (AUC) for distance thresholds of maximum 0.1 m.

###  4.3 Training

The DETR architecture suffers from the drawback of having a slow convergence
[37]. To tackle this issue, we initialize the model using the provided
pretrained weights on the COCO dataset [16] and then train the complete
T6D-Direct model on the YCB-V dataset. After initializing our model with the
pretrained weights, there are two possible strategies while training for the
pose estimation task. In the first approach, we train the complete model for
both object detection and pose estimation tasks simultaneously; therefore, the
total loss function is the Hungarian loss brought in Eq. 4. In the second
approach, we employ a multi-stage scheme to train only the pose prediction
heads and freeze the rest of the network. Investigation on these methods are
conducted in Section 5.

To further understand the behavior of the mentioned approaches, we visualize
the decoder attention maps for the object queries corresponding to the
predictions. In Fig. 3, the top row consists of the object predictions. The
middle and bottom rows consist of the attention maps from the complete and
partial trained models, respectively, corresponding to the object predictions
in the top row. The partial trained model has higher activations along the
object boundaries. These activations are the result of training the partial
model only on the object detection task. When freezing the transformer model
and training only the prediction heads, the prediction heads have to rely on
the features already learned, whereas the complete trained model has denser
activations compared to the partial trained model and the activations are
spread over the whole object and not just the object boundaries. Thus,
training the complete model helps learn features more suitable for pose
estimation than the features learned for object detection.

Scene |  |  |
---|---|---|---
Full model |  |  |
Frozen |  |  |

$0$$0.2$$0.4$$0.6$$0.8$$1$

Figure 3: Object predictions of a given image (first row) and decoder
attention maps for the object queries (second and third rows). Training the
complete model for both object detection and pose estimation tasks (second
row). Training the model first on the object detection task, and then training
the frozen model on the pose estimation task (third row). Attention maps are
visualized using the jet color map (shown above for reference).

####  4.3.1 Hyperparameters

$\alpha$ and $\beta$ hyperparameters in computing $\mathcal{L}_{box}$ (Eq. 5)
are set to 2 and 5, respectively. The $\lambda_{pose}$ hyperparameter in
computing $\mathcal{L}_{Hungarian}$ (Eq. 4) is set to 0.05, and the
cardinality of the predicted set $N$ is set to 20. The model takes the image
of the size 640 $\times$ 480 as input and is trained using the AdamW optimizer
[17] with an initial learning rate of $10^{-4}$ and for 78K iterations. The
learning rate is decayed to $10^{-5}$ after 70K iterations, and the batch size
is 32. Moreover, gradient clipping with a maximal gradient norm of 0.1 is
applied. In addition to YCB-V dataset images, we use the synthetic dataset
provided by PoseCNN for training our model.

###  4.4 Results

In Table 1, we present the per object quantitative results of T6D-Direct on
the YCB-V dataset. We compare our results with PoseCNN [35], PVNet [20] and
DeepIM [14]. In terms of the approach, T6D-Direct is comparable to PoseCNN;
both are direct regression methods, whereas PVNet is an indirect method, and
DeepIM is a refinement-based approach. In terms of both the AUC of ADDS and
AUC of ADD(-S) metrics, T6D-Direct outperforms PoseCNN and outperforms the AUC
of ADD(-S) results of PVNet. For a fair comparison, we follow the same object
symmetry definition and evaluation procedure described by the YCB-Video
dataset [35].

Some qualitative results are shown in Fig. 5. To demonstrate the ability of
the Transformer architecture to model dependencies between pixels over the
whole image instead of a just small local neighborhood, in Fig. 4, we
visualize the self-attention maps for three pixels belonging to three objects
in the image. All three pixels lie on the same horizontal line but attend to
different parts of the image.

Table 1: Pose prediction results on the YCB-V Dataset. The symmetric objects are denoted by *. Metric | AUC of ADD-S | AUC of ADD(-S)
---|---|---
Object | PoseCNN | T6D-Direct | DeepIM | PoseCNN | PVNet | T6D-Direct | DeepIM
master_chef_can | 84.0 | 91.9 | 93.1 | 50.9 | 81.6 | 61.5 | 71.2
cracker_box | 76.9 | 86.6 | 91.0 | 51.7 | 80.5 | 76.3 | 83.6
sugar_box | 84.3 | 90.3 | 96.2 | 68.6 | 84.9 | 81.8 | 94.1
tomato_soup_can | 80.9 | 88.9 | 92.4 | 66.0 | 78.2 | 72.0 | 86.1
mustard_bottle | 90.2 | 94.7 | 95.1 | 79.9 | 88.3 | 85.7 | 91.5
tuna_fish_can | 87.9 | 92.2 | 96.1 | 70.4 | 62.2 | 59.0 | 87.7
pudding_box | 79.0 | 85.1 | 90.7 | 62.9 | 85.2 | 72.7 | 82.7
gelatin_box | 87.1 | 86.9 | 94.3 | 75.2 | 88.7 | 74.4 | 91.9
potted_meat_can | 78.5 | 83.5 | 86.4 | 59.6 | 65.1 | 67.8 | 76.2
banana | 85.9 | 93.8 | 72.3 | 91.3 | 51.8 | 87.4 | 81.2
pitcher_base | 76.8 | 92.3 | 94.6 | 52.5 | 91.2 | 84.5 | 90.1
bleach_cleanser | 71.9 | 83.0 | 90.3 | 50.5 | 74.8 | 65.0 | 81.2
bowl* | 69.7 | 91.6 | 81.4 | 69.7 | 89.0 | 91.6 | 81.4
mug | 78.0 | 89.8 | 91.3 | 57.7 | 81.5 | 72.1 | 81.4
power_drill | 72.8 | 88.8 | 92.3 | 55.1 | 83.4 | 77.7 | 85.5
wood_block* | 65.8 | 90.7 | 81.9 | 65.8 | 71.5 | 90.7 | 81.9
scissors | 56.2 | 83.0 | 75.4 | 35.8 | 54.8 | 59.7 | 60.9
large_marker | 71.4 | 74.9 | 86.2 | 58.0 | 35.8 | 63.9 | 75.6
large_clamp* | 49.9 | 78.3 | 74.3 | 49.9 | 66.3 | 78.3 | 74.3
extra_large_clamp* | 47.0 | 54.7 | 73.2 | 47.0 | 53.9 | 54.7 | 73.3
foam_brick* | 87.8 | 89.9 | 81.9 | 87.8 | 80.6 | 89.9 | 81.9
MEAN | 75.9 | 86.2 | 88.1 | 61.3 | 73.4 | 74.6 | 81.9

$0$$0.2$$0.4$$0.6$$0.8$$1$

Figure 4: Encoder self-attention. We visualize the self-attention maps for three pixels belonging to three objects in the image. All three pixels lie on the same horizontal line but attend to different parts of the image. Attention maps are visualized using the jet color map (shown above for reference). Ours PoseCNN |  |  |  |  |  |
---|---|---|---|---|---|---
Figure 5: Qualitative examples from the YCB-V Dataset. Top row: PoseCNN [35].
Bottom row: our predictions.

###  4.5 Inference time analysis

Table 2: Comparison with state-of-the-art methods on YCB-V. In terms of the ADD(-S) 0.1d metric, we achieve the state-of-the-art result. † indicates that the method is refinement-based. Inference time is the average time taken for processing all objects in an image. Method | ADD(-S) |  AUC of ADD-S |  AUC of ADD(-S) |  Inference Time [$\mathrm{s}\text{/}$]
---|---|---|---|---
CosyPose† [13] | - | 89.8 | 84.5 | 0.395
PoseCNN [35] | 21.3 | 75.9 | 61.3 |  *$>$0.250
SegDriven [10] | 39.0 | - | - | *0.050
Single-Stage [9] | 53.9 | - | - | *0.022
GDR-Net [33] | 49.1 | 89.1 | 80.2 | 0.065
T6D-Direct (Ours) | 48.7 | 86.2 | 74.6 | 0.017

Since the prediction heads generate $N$ predictions in parallel, the inference
of our model is not dependent on the number of objects in an image. However,
having a smaller cardinality of the prediction set requires estimating fewer
object queries and facilitates faster inference time. Thus, we set N to 20. On
an NVIDIA 3090 GPU and Intel 3.70 GHz CPU, our model runs at 58 fps which
makes our model ideal for real-time applications.

##  5 Ablation study

In this section, we explore the effect of various training strategies,
different loss functions, and egocentric vs. allocentric rotation
representations on the T6D-Direct model performance for the YCB-V dataset.

####  5.0.1 Effectiveness of loss functions

In Table 3, we examine the performance of our model using the symmetry aware
version of Point Matching loss with $\ell_{2}$ norm [14, 35] which, in
contrast to the disentangled loss presented in Section 3.1.6, couples the
rotation and translation components. This loss function results in the best
AUC of ADD(-S) metric. Moreover, since the symmetry aware SLoss component of
the Point Matching loss is computationally expensive, we experimented with
training our model using only the PLoss component. Interestingly, the ADD(-S)
result of the model trained using only the PLoss component (row 5) is only
slightly worse than the model trained using the both components (row 1).

Table 3: Ablation study on YCB-V. We provide results of our method with different loss functions and training schemes. Row | Method | ADD(-S) |  AUC of ADD(-S)
---|---|---|---
1 | T6D-Direct with Point Matching loss | 47.0 | 75.6
2 | 1 + multi-stage training | 20.5 | 59.1
3 | 1 + pose matching cost component | 42.8 | 71.7
4 |  1 + allocentric R6d | 42.9 | 74.4
5 | T6D-Direct with PLoss | 45.8 | 74.4
6 | T6D-Direct | 48.7 | 74.6

####  5.0.2 Effectiveness of training strategies

As discussed in Section 4.3, there are two training schemes: single-stage and
multi-stage. In the multi-stage scheme, we train the Transformer model for
object detection and only train the FFNs for pose estimation, whereas in the
single-stage scheme, we train the complete model in one stage. In our
experiments, as shown in Table 3, multi-stage training (row 2) yielded
inferior results, although both schemes were pretrained on the COCO dataset.
This demonstrates that the Transformer model is learning the features specific
to the 6D object pose estimation task on YCB-V, and COCO fine-tuning mainly
helps in faster convergence during training and not in more accurate pose
estimations. We thus believe that most large-scale image datasets can serve as
pretraining data source. We also provide the results of including the pose
component in the bipartite matching cost mentioned in Eq. 3. Including the
pose component (row 3) does not provide any considerable advantage; thus, we
include only the class probability and bounding box components in the
bipartite matching cost in all further experiments. Further, egocentric
rotation representation (row 1) performed slightly better than allocentric
representation (row 4). We hypothesize that supplementing RGB images with
positional encoding allows the Transformer model to learn spatial features
efficiently. Therefore, the allocentric representation does not have any
advantage over the egocentric representation.

##  6 Conclusion

We introduced T6D-Direct, a transformer-based architecture for multi-object 6D
pose estimation. Equipped with multi-head attention mechanism, our model
obtains competitive results in the task of direct 6D pose estimation without
any dense features. Unlike the standard multi-staged methods, our formulation
of multi-object 6D pose estimation as a set prediction problem allows
estimating the 6D pose of all the objects in a given image in one forward
pass. Furthermore, our model is real-time capable. In the future, we plan to
explore the possibilities of incorporating dense estimation features into our
architecture and improve the performance further.

## Acknowledgment

This research has been supported by the Competence Center for Machine Learning
Rhine Ruhr (ML2R), which is funded by the Federal Ministry of Education and
Research of Germany (grant no. 01—S18038A).

## References

  * Carion et al. [2020] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: ECCV, pp. 213–229 (2020)
  * Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)
  * Dosovitskiy et al. [2021] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: ICLR (2021)
  * Girshick [2015] Girshick, R.: Fast r-cnn. In: ICCV, pp. 1440–1448 (2015)
  * He et al. [2016] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
  * Hinterstoisser et al. [2013] Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In: ACCV, pp. 548–562, Berlin, Heidelberg (2013)
  * Hodaň et al. [2020] Hodaň, T., Sundermeyer, M., Drost, B., Labbé, Y., Brachmann, E., Michel, F., Rother, C., Matas, J.: Bop challenge 2020 on 6d object localization. In: ECCV, pp. 577–594 (2020)
  * Hosang et al. [2017] Hosang, J., Benenson, R., Schiele, B.: Learning non-maximum suppression. In: CVPR, pp. 4507–4515 (2017)
  * Hu et al. [2020] Hu, Y., Fua, P., Wang, W., Salzmann, M.: Single-stage 6d object pose estimation. In: CVPR (2020)
  * Hu et al. [2019] Hu, Y., Hugonot, J., Fua, P., Salzmann, M.: Segmentation-driven 6D object pose estimation. In: CVPR, pp. 3385–3394 (2019)
  * Kehl et al. [2017] Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again. In: CVPR, pp. 1521–1529 (2017)
  * Khan et al. [2021] Khan, S., Naseer, M., Hayat, M., Zamir, S.W., Khan, F.S., Shah, M.: Transformers in vision: A survey. arXiv:2101.01169 (2021)
  * Labbe et al. [2020] Labbe, Y., Carpentier, J., Aubry, M., Sivic, J.: CosyPose: Consistent multi-view multi-object 6D pose estimation. In: ECCV (2020)
  * Li et al. [2018] Li, Y., Wang, G., Ji, X., Xiang, Y., Fox, D.: DeepIM: Deep iterative matching for 6D pose estimation. In: ECCV, pp. 683–698 (2018)
  * Li et al. [2019] Li, Z., Wang, G., Ji, X.: Cdpn: Coordinates-based disentangled pose network for real-time rgb-based 6-dof object pose estimation. In: ICCV (2019)
  * Lin et al. [2014] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common objects in context. In: ECCV, pp. 740–755 (2014)
  * Loshchilov and Hutter [2017] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: ICLR (2017)
  * Manhardt et al. [2018] Manhardt, F., Kehl, W., Navab, N., Tombari, F.: Deep model-based 6D pose refinement in RGB. In: ECCV, pp. 800–815 (2018)
  * Oberweger et al. [2018] Oberweger, M., Rad, M., Lepetit, V.: Making deep heatmaps robust to partial occlusions for 3D object pose estimation. In: ECCV (2018)
  * Peng et al. [2019] Peng, S., Liu, Y., Huang, Q., Zhou, X., Bao, H.: PVNet: Pixel-wise voting network for 6DOF pose estimation. In: CVPR, pp. 4561–4570 (2019)
  * Periyasamy et al. [2018] Periyasamy, A.S., Schwarz, M., Behnke, S.: Robust 6D object pose estimation in cluttered scenes using semantic segmentation and pose regression networks. In: IROS (2018)
  * Periyasamy et al. [2019] Periyasamy, A.S., Schwarz, M., Behnke, S.: Refining 6D object pose predictions using abstract render-and-compare. In: Humanoids, pp. 739–746 (2019)
  * Rad and Lepetit [2017] Rad, M., Lepetit, V.: BB8: A scalable, accurate, robust to partial occlusion method for predicting the 3D poses of challenging objects without using depth. In: ICCV, pp. 3828–3836 (2017)
  * Ren et al. [2015] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: NeurIPS, vol. 28 (2015)
  * Rezatofighi et al. [2019] Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., Savarese, S.: Generalized intersection over union: A metric and a loss for bounding box regression. In: CVPR, pp. 658–666 (2019)
  * Schwarz et al. [2018] Schwarz, M., Lenz, C., García, G.M., Koo, S., Periyasamy, A.S., Schreiber, M., Behnke, S.: Fast object learning and dual-arm coordination for cluttered stowing, picking, and packing. In: ICRA, pp. 3347–3354 (2018)
  * Shao et al. [2020] Shao, J., Jiang, Y., Wang, G., Li, Z., Ji, X.: PFRL: Pose-Free reinforcement learning for 6D pose estimation. In: CVPR (2020)
  * Sundermeyer et al. [2018] Sundermeyer, M., Marton, Z.C., Durner, M., Brucker, M., Triebel, R.: Implicit 3D orientation learning for 6D object detection from RGB images. In: ECCV (2018)
  * Tekin et al. [2018] Tekin, B., Sinha, S.N., Fua, P.: Real-time seamless single shot 6D object pose prediction. In: CVPR (2018)
  * Touvron et al. [2021] Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. In: ICML, pp. 10347–10357 (2021)
  * Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I.: Attention is all you need. In: NeurIPS, vol. 30 (2017)
  * Wang et al. [2020a] Wang, G., Manhardt, F., Shao, J., Ji, X., Navab, N., Tombari, F.: Self6D: self-supervised monocular 6D object pose estimation. In: ECCV, pp. 108–125 (2020a)
  * Wang et al. [2021] Wang, G., Manhardt, F., Tombari, F., Ji, X.: GDR-Net: Geometry-guided direct regression network for monocular 6D object pose estimation. In: CVPR (2021)
  * Wang et al. [2020b] Wang, H., Zhu, Y., Green, B., Adam, H., Yuille, A., Chen, L.C.: Axial-DeepLab: stand-alone axial-attention for panoptic segmentation. In: ECCV, pp. 108–126 (2020b)
  * Xiang et al. [2018] Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. RSS (2018)
  * Zhou et al. [2019] Zhou, Y., Barnes, C., Lu, J., Yang, J., Li, H.: On the continuity of rotation representations in neural networks. In: CVPR, pp. 5745–5753 (2019)
  * Zhu et al. [2021] Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. In: ICLR (2021)


fiber exact sequence of the proper smooth map obtained from $\pi$ by deleting
the singular fibers:

$1\to\pi_{1}(\Sigma_{g}))\to\pi_{1}(S\setminus\pi^{-1}(B))\to\pi_{1}(C\setminus
B)\to 1$

Consider $\rho:\pi_{1}(C\setminus B)\to{\rm{Out}}^{+}(\pi_{1}(\Sigma_{g}))$
and $M_{\pi}=Im(\rho)$ the monodromy group. The group $M_{\pi}$ acts on
conjugacy classes of elements of $\pi_{1}(\Sigma_{g})$. It is a non trivial
invariant of the intriguing combinatorial object
$M_{\pi}\curvearrowright\mathbf{C}_{\pi}$.

######  Lemma 5.4.

$\mathbf{C}_{\pi}$ is a finite union of $M_{\pi}$-orbits.

###### Proof.

This is a reformulation of the discussion before ([8], Def. 3.1). ∎

Apart from that, few things seem to be known on the the actions
$M_{\pi}\curvearrowright\mathbf{C}_{\pi}$ which can occur in algebraic
families of curves, except the restrictions coming from Deligne’s
semisimplicity theorem and also his theorem on the fixed part when one uses
the natural map $M_{\pi}\curvearrowright\mathbf{C}_{\pi}\to
M_{\pi,H_{1}}\curvearrowright H_{1}(\Sigma_{g},\mathbb{Z})$.

However there is a surjective morphism

$\pi_{1}(\Sigma_{g})\to B_{N}(\Sigma_{g})=\pi_{1}(\Sigma_{g})/\langle\langle
x^{N},x\in\pi_{1}(\Sigma_{g})\rangle\rangle$

onto its $N$-th verbal quotient, also called the $N$-th Burnside group of the
surface. The Restricted Burnside Problem, solved affirmatively by Zelmanov,
states that the profinite completion of the Burnside group (of a free group)
is finite. For large enough $N$, $B_{N}(\Sigma_{g})$ is infinite having the
$N$-th Burnside group of the free group on $g$ generators as a quotient and
hence it is not residually finite. This won’t imply that
$\pi_{1}(\mathcal{S}[N])$ is not residually finite but it implies that the
fiber group is infinite. This would be a very desirable situation since the
only ‘obvious’ representations of $\pi_{1}(\mathcal{S}[N])$ come from
$\pi_{1}(\mathcal{C}[N])$.

Passing to $B_{N}(\Sigma_{g})$ is an extreme step. It is better to look at the
Burnside type quotient by the $N$-th powers of the primitive elements. Indeed,
this quotient occurs as a fiber group if the monodromy group of $\pi$ is the
mapping class group and every topological type of simple closed curve occurs
in $\mathbf{C}_{\pi}$, a case that can be realized. The following proposition
makes precise some infiniteness statements in [8].

######  Proposition 5.5.

The fiber group $I_{N}(\mathbf{C}_{\pi})$ is infinite when $g=2$ and $N\geq 4$
or $g\geq 3$ and $N\not\in\\{2,3,4,6,8,12\\}$.

###### Proof.

This is a corollary of the proof of [2, Cor. 4], the main ingredient being
[29] when $g=2$ and the Koberda-Santharoubane theorem in the version of [25,
Proof of Proposition 3.2]. ∎

###  5.4. The surface $S(N)$

####  5.4.1. $N$ odd

######  Proposition 5.6.

If $N$ is odd, the stack $\mathcal{S}_{\pi}[N]$, is a uniformizable compact
Kähler orbifold with a projective moduli space.

###### Proof.

This is [8, Lemma 2.7], which uses a very specific Teichmüller level. First,
they check that $(S_{N}\times_{C_{N}}D)^{can}$ is uniformizable.

Let us describe their argument as [8] skips some details. Set
$\Pi=\pi_{1}(T^{\prime})$, for the fundamental group of the generic fiber,
which is isomorphic to $\pi_{1}(\Sigma_{g})$, $\Pi_{(3)}=[[\Pi,\Pi],\Pi]$ for
the third term of the lower central series and $\Pi^{N}=\langle\langle
x^{N},x\in\Pi\rangle\rangle$ be the normal subgroup generated by the $N$-th
powers. Consider next the quotient $U_{N}(\Pi)=\Pi/\Pi_{(3)}\cdot\Pi^{N}$.

The Lie algebra of $\Pi/\Pi_{(3)}$ is the quotient of
$\mathbb{Z}[A_{i},B_{i},E_{l,k},F_{m,n},G_{m,n}\\}]_{1\leq i\leq g,1\leq l\leq
k\leq g,1\leq m<n\leq g}$ by the relations $[A_{l},B_{k}]=E_{l,k}$
$[A_{m},A_{n}]=F_{m,n}$ $[B_{m},B_{n}]=G_{m,n}$ where $E_{l,k}$ $F_{m,n}$
$G_{m,n}$ are central, and $\sum_{k=1}^{g}E_{k,k}=0$. Moreover, it defines by
means of the Baker-Campbell-Hausdorff formula an unipotent group scheme $U$
which is smooth over $\mathbb{Z}[1/2]$, so that $U(\mathbb{Z})=\Pi/\Pi_{(3)}$.
There is a map $\Pi\to U_{N}(\Pi)\to U(\mathbb{Z}/N\mathbb{Z})$ sending the
classes $\bar{c}_{i}$ to elements of order $N$ in $U(\mathbb{Z}/N\mathbb{Z})$.
In fact nontrivial elements of the nilpotent group $U(\mathbb{Z}/N\mathbb{Z})$
have order $N$ and thus $U(\mathbb{Z}/N\mathbb{Z})$ is finite. Then Lemma 5.2
implies that $(S_{N}\times_{C_{N}}D)^{can}$ is uniformizable.

Observe that the subgroup $H_{N}=\Pi_{(3)}\cdot\Pi^{N}$ is characteristic,
i.e. invariant by all the automorphisms of $\Pi$ and of finite index. Hence
the same quotient of $\pi_{1}(\Sigma_{g})$ works for all singular fibers.

Since $B\neq\emptyset$, the group $\pi_{1}(C\setminus B)$ is free and so
$\rho$ lifts to ${\rm{Aut}}^{+}(\pi_{1}(\Sigma_{g}))$. Moreover, the
fundamental group of $S\setminus\pi^{-1}(B)$ is a semidirect product
$\pi_{1}(\Sigma_{g})\rtimes\pi_{1}(C\setminus B)$. Since the automorphism
group of a finite group is finite, there is a normal subgroup of finite index
$H\lhd\pi_{1}(C\setminus B)$ which acts trivially on
$U_{N}(\pi_{1}(\Sigma_{g}))$. Then, $H_{N}\rtimes H$ is a finite index
subgroup of $\pi_{1}(S\setminus\pi^{-1}(B))$. Moreover,
$\pi_{1}(S\setminus\pi^{-1}(B))$ surjects onto the finite group
$U_{N}(\pi_{1}(\Sigma_{g}))\rtimes(\pi_{1}(C\setminus B)/H)$. Note that this
surjective homomorphism maps the elements of $\mathbf{C}_{\pi}$ to elements of
order $N$. In particular,
$K_{N}(\mathbf{C}_{\pi}):=\ker(\pi_{1}(\Sigma_{g})\to
I_{N}(\mathbf{C}_{\pi}))\subseteq H_{N}$.

Since $H_{N}$ is a characteristic subgroup of $\pi_{1}(\Sigma_{g})$ it is also
a normal subgroup of $\pi_{1}(S\setminus\pi^{-1}(B))$ contained in
$\pi_{1}(\Sigma_{g})$. We then obtain the following diagram, keeping in mind
the natural surjective morphism
$\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))\to\pi_{1}(S_{N}^{can})$:

$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(S\setminus\pi^{-1}(B))\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(C\setminus
B)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(C_{N}\setminus
B_{N})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})/H_{N}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))/H_{N}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(C_{N}\setminus
B_{N})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})/H_{N}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(S_{N}^{can})/{\rm
Im}(H_{N})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(C_{N})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})/K_{N}(\mathbf{C}_{\pi})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(S_{N}^{can})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(C_{N})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1.}$

Consider the intersection $H^{\prime}$ of $H$ with $\pi_{1}(C_{N}\setminus
B)$. Then $H_{N}\rtimes H^{\prime}$ is a finite index normal subgroup of
$\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))$. Projecting it down to
$\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))/H_{N}$ we get a normal subgroup of
finite index $H^{\prime\prime}<\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))/H_{N}$
intersecting $\pi_{1}(\Sigma_{g})/H_{N}$ trivially. Projecting it down to
$\pi_{1}(S_{N}^{can})/{\rm Im}(H_{N})$ we claim that we get a normal subgroup
of finite index $H^{\prime\prime\prime}<\pi_{1}(S_{N}^{can})/{\rm Im}(H_{N})$
intersecting $\pi_{1}(\Sigma_{g})/H_{N}$ trivially. To this purpose we need
the following elementary lemma:

######  Lemma 5.7.

Let $A,B$ be groups and $B\to{\rm{Aut}}(A)$ a group homomorphism. Let
$C\triangleleft B$ be a normal subgroup acting trivially on $A$. Then
$A\rtimes B/C$ is a quotient of $A\rtimes B$.

Our claim is then the consequence of Lemma 5.7 applied to
$C=\ker(H^{\prime}\to\pi_{1}(C_{N}))$, $B=H^{\prime}$,
$A=\pi_{1}(\Sigma_{g})/H_{N}$ setting $H^{\prime\prime\prime}=\\{1\\}\times
B/C$ and observing $A\times B/C$ is the image of $A\times
H^{\prime}\triangleleft\pi_{1}(S_{N}\setminus\pi^{-1}(B_{N}))/H_{N}$ a finite
index normal subgroup of $\pi_{1}(S_{N}^{can})/{\rm Im}(H_{N})$.

Eventually we obtain a homomorphism $\pi_{1}(S_{N}^{can})\to G$ where $G$ is
finite such that $\pi_{1}(\Sigma_{g})/I_{N}(\mathbf{C}_{\pi})$ is sent to a
subgroup isomorphic to $U_{N}(\pi_{1}(\Sigma_{g}))$ by a surjective morphism
mapping the elements in $\mathbf{C}_{\pi}$ to elements of order $N$.

This concludes the proof of Proposition 5.6. ∎

######  Remark 5.8.

If $\overline{\mathcal{M}^{1\ an}_{g}}[N]$ is uniformizable then
$\mathcal{S}_{\pi}[N]$ is uniformizable too.

We then denote by $S(N)=S_{\pi}(N)$, the $\pi$ being dropped if the context
allows, the smooth projective surface uniformizing the stack
$\mathcal{S}_{\pi}[N]$, namely such that $[G\backslash S(N)]=S_{\pi}[N]$.
Actually this does not define a smooth projective surface but a
commensurability class444Two projective varieties are commensurable if and
only if they have a common finite étale covering.  of such surfaces.

####  5.4.2. $N$ even

The result from [8] should be restated as:

######  Proposition 5.9.

If $N\equiv 0\ ({\rm mod}\,4)$, the stack $\mathcal{S}_{\pi}[N,N/2]$ is a
uniformizable compact Kähler orbifold with a projective moduli space.

###### Proof.

This is the result of [8, Lemma 2.7] slightly modified in the even case. In
fact, the order of a separating primitive element in
$\frac{\pi_{1}(\Sigma_{g})}{\pi_{1}(\Sigma_{g})_{(3)}\cdot\pi_{1}(\Sigma_{g})^{N}}$
is actually $\frac{N}{2}$ (see [48, Lemma 6.3]) and not $N$, correcting [8,
Lemma 2.5]. All arguments used above in the odd case are then valid.
Alternatively, we can derive it from the uniformisability of
$\overline{\mathcal{M}^{1\ an}_{g}}[N,N/2]$, which follow the same way as the
uniformisability of $\overline{\mathcal{M}^{an}_{g}}[N,N/2]$ from [48,
Thm.3.1.1]. ∎

If $\mathcal{X}$ is a compact complex orbifold (or an analytification of a
Deligne-Mumford stack globally of finite type, say with a quasiprojective
moduli space) we can do the following construction. Consider
$H\triangleleft\pi_{1}(\mathcal{X})$ be a finite index subgroup such that the
kernels of the isotropy morphisms
$\pi_{1}^{loc}(\mathcal{X},x)\to\pi_{1}(\mathcal{X})/H$ coincide with the
kernels of their universal counterpart
$\pi_{1}^{loc}(\mathcal{X},x)\to\widehat{\pi_{1}(\mathcal{X})}$. Then the
covering stack of $\mathcal{X}$ corresponding to $H$ is a complex orbifold
$\mathcal{X}^{\prime}$ with a finite étale covering
$\mathcal{X}^{\prime}\to\mathcal{X}$. Consider the moduli space $X^{\prime}$
of $\mathcal{X}^{\prime}$. It is a normal variety with quotient singularities
at worst which carries an action of $G=\pi_{1}(\mathcal{X})/H$. This gives a
map of stacks with identical moduli spaces $\mathcal{X}\to[X^{\prime}/G]$ with
the property that any finite dimensional (or finite image) representation of
$\pi_{1}(\mathcal{X})$ factors through $\pi_{1}([X^{\prime}/G])$.

######  Definition 5.10.

We define $S(N)=S_{\pi}(N)$ for every $N\in\mathbb{N}$ by applying this
construction to $\mathcal{S}_{\pi}[N]$, $S(N)=S^{\prime}_{\pi}[N]$ being a
normal complex projective surface with isolated quotient singularities.

######  Remark 5.11.

The minimal resolution of singularities $\widehat{S(N)}\to S(N)$ induces an
isomorphism on $\pi_{1}$.

######  Remark 5.12.

It is proposed in [8] to investigate the surfaces $S(N)$ to find
counterexamples to Shafarevich conjecture on holomorphic convexity among them.

##  6\. Using TQFT representations for the Shafarevich Conjecture

###  6.1. The Birman exact sequence

All the above fiber sequences come from the fiber sequence of $f_{1}$, which
by means of the isomorphisms
$\pi_{1}(\mathcal{M}_{g}^{n})\cong{\rm{PMod}}(\Sigma_{g}^{n})$ can be
identified with the Birman exact sequence:

$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{i_{*}}$$\textstyle{\pi_{1}(\mathcal{M}_{g}^{1})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f_{1*}}$$\scriptstyle{\cong}$$\textstyle{\pi_{1}(\mathcal{M}_{g})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\cong}$$\textstyle{1}$$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\Sigma_{g})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{pp}$$\textstyle{{\rm{Mod}}(\Sigma_{g}^{1})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{{\rm{Mod}}(\Sigma_{g})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1,}$

where $pp$ is Birman’s point pushing homomorphism.

The point pushing of a simple closed loop is a product of two Dehn twists
along disjoint simple curves in ${\rm{Mod}}(\Sigma_{g}^{1})$. Let
$S(\Sigma_{g})$ be the set of conjugacy classes of of $\pi_{1}(\Sigma_{g})$
which can be represented by simple closed curves on $\Sigma_{g}$. Also Dehn
twists are preserved by point forgetting maps.

######  Lemma 6.1.

The map $f_{1}$ lifts to a map of stacks $f_{1}[p]:\overline{\mathcal{M}^{1\
an}_{g}}[p]\to\overline{\mathcal{M}^{an}_{g}}[p]$ compactifying $f_{1}$
inducing at the level of fundamental groups the canonical map
${\rm{Mod}}(\Sigma_{g}^{1})/{\rm{Mod}}(\Sigma_{g}^{1})[p]\to{\rm{Mod}}(\Sigma_{g})/{\rm{Mod}}(\Sigma_{g})[p]$.

###### Proof.

This follows from Lemma 2.1, the fact that the root construction behaves well
under pullback (see [10]) and the universal property of canonical stacks [22,
Theorem 4.6]. ∎

It follows from Theorem 2.6 that the Birman exact sequence gives rise to a
level $p$ analogue, as proved in [2]:

$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{BT(\pi_{1}(\Sigma_{g}),p)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\overline{\mathcal{M}^{1\
an}_{g}}[p])\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f_{1}[p]_{*}}$$\textstyle{\pi_{1}(\overline{\mathcal{M}^{an}_{g}}[p])\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$

where

$BT(\pi_{1}(\Sigma_{g}),p)=\pi_{1}(\Sigma_{g})/\langle\langle x^{p},x\in
S(\Sigma_{g})\rangle\rangle,$

This is of course the fiber sequence for the map of stacks
$f_{1}[p]:\overline{\mathcal{M}^{1\
an}_{g}}[p]\to\overline{\mathcal{M}^{an}_{g}}[p]$. The group
$BT(\pi_{1}(\Sigma_{g}),p)$ is actually the fiber group
$I_{p}(\mathbf{C}_{f_{1}})$. Here we abuse notation, such a fiber group arises
from a family $\pi$ such that $\rho$ is surjective.

By construction, the map $C\to\overline{\mathcal{M}_{g}}$ lifts to a map of
stacks $\mathcal{C}[p]\to\overline{\mathcal{M}_{g}}[p]$. Comparing with
Proposition 5.1, we find a diagram:

$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{I_{p}(\mathbf{C}_{\pi})\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\mathcal{S}_{\pi}[p])\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\mathcal{C}[p])\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$$\textstyle{1\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{B(\pi_{1}(\Sigma_{g}),p)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\overline{\mathcal{M}^{1\
an}_{g}})[p]\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\pi_{1}(\overline{\mathcal{M}^{an}_{g}})[p]\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{1}$

with the leftmost vertical map being given by the natural quotient map
attached to $\mathbf{C}_{\pi}\subset\mathbf{C}_{f_{1}}=S(\Sigma_{g})$.

###  6.2. Steinness of the universal covering space of most Bogomolov-
Katzarkov surfaces

The starting point of this project was the realization that [36] implies the
following result for large odd numbers $N$:

######  Theorem 6.2.

If $g\geq 2$, $N\geq 5$ is odd, $N$ even and $N/2\geq 5$ is odd, or $N\equiv
0({\rm mod}\;4)$ and $N\geq 20$ and $(g,N)\not=(2,24)$ then there exists a
complex finite dimensional linear representations $\rho$ of
$\pi_{1}(\mathcal{S}_{\pi}[N])$ such that $\rho\circ\iota_{*}$ has infinite
image containing a free group on two generators, where $\iota$ is the
inclusion of the general fiber into $\mathcal{S}_{\pi}[N]$.

###### Proof.

If $N$ is odd, define $p=N$. By construction there is map of stacks
$i:\mathcal{S}_{\pi}[p]\to\overline{\mathcal{M}^{1\ an}_{g}}[p]$ and we can
define $\rho=\rho_{p,(2)}\circ\iota_{*}$. The result is then an immediate
consequence of Propositions 3.17 and 3.22.

If $N$ is even, define $p=\frac{N}{2}$. By construction there is map of stacks
$i:\mathcal{S}_{\pi}[N]\to\overline{\mathcal{M}^{1\ an}_{g}}[2p,p/g.c.d(p,4)]$
and we can define $\rho=\rho_{p,(2)}\circ\iota_{*}$. The result is then an
immediate consequence of Propositions 3.17 and 3.22. ∎

######  Corollary 6.3.

Under the hypotheses of Theorem 6.2, $S(N)$ has a holomorphically convex
infinite Galois covering space with the property that the codomain of its
Cartan-Remmert reduction is $2$-dimensional.

###### Proof.

Recall that the Cartan-Remmert reduction of a holomorphically convex normal
complex space $T$ is the unique proper holomorphic map $T\to U$ such that $U$
is a normal Stein space.

The main theorem in [20] states that, given a complex projective manifold $X$
and $M\in\mathbb{N}$, there is a Cartan-Remmert reduction:

$\widetilde{s_{M}}:H_{M}\backslash\widetilde{X^{univ}}\to\widetilde{U(M)}\quad\mathrm{with}\
H_{M}=\bigcap_{\rho:\pi_{1}(X)\to GL_{M}(\mathbb{C})}\ker(\rho).$

Since $\Gamma_{M}=\pi_{1}(X)/H_{M}$ acts in a properly discontinuously, this
map descends to a proper holomorphic map, the $GL_{M}$-th Shafarevich morphism
of $X$:

$Sh_{M}:X\to Sh_{M}(X)=\Gamma_{M}\backslash\widetilde{U(M)}.$

The fibers of $\widetilde{s_{M}}$ are the connected components of the lifts of
the fibers of $Sh_{M}$ which are characterized as the maximal connected
subvarieties $Z\subset X$ such that $\mathrm{Im}(\pi_{1}(Z))\to\Gamma_{M})$ is
finite. In particular, when all positive dimension irreducible subvarieties
$Z$ satisfy:

$\mathrm{Im}(\pi_{1}(Z^{norm}))\to\Gamma_{M})\ \mathrm{is\ infinite},$

where $Z^{norm}\to Z$ is the normalization, then
$\widetilde{s_{M}}=\mathrm{Id}$ and $\widetilde{U(M)}$ is Stein, and when all
positive dimensional irreducible subvarieties $Z$ through the general point
satisfy this property, then $\dim{\widetilde{U(M)}}=\dim(X)$.

In the situation of the corollary, $\pi_{1}(S(N))$ has a finite degree linear
representation with infinite image that comes from the base of the fibration
$\pi$ whose image is a non parabolic curve. Assume that its degree is $M_{0}$.
Therefore, whenever $Z$ is a connected closed subspace such that all linear
representations of $\pi_{1}(S(N))$ have finite images in $\Gamma_{M}$ for
$M\geq M_{0}$ upon restriction to $\pi_{1}(Z)$, then $Z$ is contained in the
fibers of $\pi$. Then Theorem 6.2 gives a finite degree representation, say of
degree $M_{1}$, which has infinite image on the general fiber of $\pi$. Hence,
if $\pi_{1}(Z)\to\Gamma_{M}$ has finite image for $M\geq\max(M_{0},M_{1})$,
then $Z$ lies in a special fiber of $\pi$. This implies the corollary.

∎

######  Theorem 6.4.

Assume that $g\geq 2$ and $N\not\in\\{1,2,3,4,6,8,12,16,20\\}$ and
$(g,N)\neq(2,24)$. Then the universal covering space of $S(N)$ is Stein.

###### Proof.

We will assume $N=p$ is odd in order to simplify the notation. We claim that
there is a complex representation $\rho$ of $\pi_{1}(\mathcal{S}_{\pi}[p])$
such that the image of $\pi_{1}(Z)$ is infinite, for every irreducible
component $Z$ of the special fiber of the natural morphism to a curve
$\psi:S(p)\to C^{\prime}_{p}$ obtained as the Stein factorization of the
composition of $S(p)\to S\buildrel\pi\over{\longrightarrow}C$. We will show
that the representation $\rho_{p,(2)}$ is convenient.

Since the fibres of $\pi$ are stable, the complement of the singular set of
each such component $Z$ contains as a Zariski dense open subset a finite étale
covering of the complement of the punctures in a stable curve with $n>0$
marked points. In particular it contains an essential subsurface $\Sigma$ with
boundary which is homeomorphic to a pair of pants $\Sigma_{0,3}$ or to a
$\Sigma_{1,1}$. This subsurface $\Sigma$ can be deformed to a subsurface of
the general fiber of $\pi$ since $\pi$ is smooth near $\Sigma$. In particular
it can be identified with a subsurface of a fiber of $f_{1}$. Then we can
compute $\rho(\pi_{1}(Z))$ by using the restriction of $\rho_{p,(i_{1})}$ to
the fundamental group $\pi_{1}(\Sigma_{g})$ of the general fiber.

Since $Z$ contains a finite étale cover of the $\Sigma$, the representation
$\rho$ has infinite image on the fundamental group of this subsurface and
hence on the fundamental group of $Z$, by Propositions 3.17 and 3.22.

Hence the Shafarevich morphism of $S(p)$ used in the proof of Corollary 6.3
cannot contract any component of the preimage of $Z$ in $S(p)$. Since the
universal covering space of a Stein manifold is Stein, this concludes the
proof.

To do the proof in the even case, one has to use the fact that if $X,Y$ are
normal spaces, $Y$ being Stein, $X\to Y$ is a holomorphic map has the property
that the preimage of a small open subset $\Omega\subset Y$ is a disjoint union
of proper finite ramified covering spaces of $\Omega$, $X$ is Stein ([40]).

In fact [20] is not needed here, the earlier result [33] allows to conclude
from the semisimplicity of $\rho_{p,(i_{1})}$.

∎

###  6.3.  The stack $\mathcal{S}_{\pi}[2p,\frac{p}{{\rm g.c.d.}(4,p)}]$

In the proofs above we can use the quantum representations $\rho_{p}$, for
even $p\geq 10$, to show that $\mathcal{S}_{\pi}[2p,\frac{p}{{\rm
g.c.d.}(4,p)}]$ has an holomorphically convex universal covering which is
Stein.

###  6.4. Unstable families of curves and end of proof of Theorem 1.2

If $\pi:S\to C$ is not supposed to be stable but has only nodal singularities
on its fibers as in [8] then any maximal chain $\Gamma$ of unstable components
is either a $(-1)$-rational curve or a chain of $n_{\Gamma}$ $(-2)$-rational
curves for some $n_{\Gamma}\in\mathbb{N}^{*}$. It can be blown down and one
gets a fibered surface $\pi:S^{*}\to C$ with stable singular fibers. Each
singular point $s\in S^{*,sing}$ is an $A_{n(s)}$-singularity for some
$n(s)\in\mathbb{N}^{*}$ and a node of singular fiber it belongs to.

One can do the same construction as in [8] for $S$. Let $D$ be the sum of the
singular fibers, each component being counted with multiplicity one, so that
$\pi^{*}B=D$ where $B\subset C$ is the set of singular values of $\pi$.

However, if there is a $(-1)$-rational curve, $\mathcal{S}_{\pi}[N]$ need not
be uniformizable. This situation occurs when we blow up some smooth points of
the singular fiber of a smooth stable fibration $\pi:S^{*}\to C$. The
orbisurface $S(N)$ is then the canonical stack of a weighted blow up of
$S^{*}(N)$ of weights $(1,N)$ hence has a $A_{N-1}$ singularity. Thus, without
loss of generality we can assume $S$ to be minimal which rules out these
$(-1)$-rational curves.

Then, we have a commutative diagram:

$\textstyle{S\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{S^{*}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\pi}$$\textstyle{\overline{\mathcal{M}^{1}_{g}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f_{1}}$$\textstyle{C\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\psi}$$\textstyle{\overline{\mathcal{M}_{g}}}$

The square is cartesian, i.e.: $S^{*}\simeq
C\times_{\psi,f_{1}}\overline{\mathcal{M}^{1}_{g}}$, $\pi$ being equivalent to
$\psi^{*}f_{1}$. Denote by $D^{*}$ the sum of the singular fibers of
$\pi:S^{*}\to C$. The stack $S^{*}(\sqrt[N]{D^{*}})$ has a quotient
singularity of type $pn(s)$ over $s\in S^{*,sing}$ and we may set
$\mathcal{S}^{*}_{\pi}[N]:=S(\sqrt[N]{D^{*}})^{can}$. It is not clear whether
$\mathcal{S}^{*}_{\pi}[N]$ is uniformizable. The root stack being a natural
construction, we have a map of stacks $(S^{*}\setminus
S^{*,sing})\sqrt[N]{D}\to\overline{\mathcal{M}_{g}^{1\ 0}}[N]$ with the
notation as in Remark 2.8. It induces a morphism on the fundamental groups and
since $\pi_{1}((S^{*}\setminus
S^{*,sing})(\sqrt[N]{D^{*}}))\simeq\pi_{1}(\mathcal{S}^{*}_{\pi}[N])$ and
$\pi_{1}(\overline{\mathcal{M}_{g}^{1\
0}}[N])\simeq\pi_{1}(\overline{\mathcal{M}_{g}^{1}}[N])$ a group morphism
$\pi_{1}(\mathcal{S}^{*}_{\pi}[N])\to\pi_{1}({\overline{\mathcal{M}_{g}^{1\
an}}}[N])$ factorizing the natural morphism $\pi_{1}(S^{*}\setminus
D^{*})\to\pi_{1}({\overline{\mathcal{M}_{g}^{1\ an}}}[N])$. Consider now the
normal surface $S^{*}_{\pi}[N]^{\prime}$ constructed at the end of section 5.
The quantum representations give rise to representations of
$\pi_{1}(S^{*}_{\pi}[N]^{\prime})$ (and of
$\pi_{1}(\widehat{S^{*}_{\pi}[N]^{\prime}})$ ) and the proof of Theorem 6.4
applies to the effect that $S^{*}_{\pi}[N]^{\prime}$ has a Stein universal
covering space and $\widehat{S^{*}_{\pi}[N]^{\prime}}$ has a holomorphically
convex universal covering space which resolves the singularities of
$\widetilde{S^{*}_{\pi}[N]^{\prime}}^{univ}$.

The same construction gives actually a group morphism
$\pi_{1}(\mathcal{S}_{\pi}[N])\to\pi_{1}({\overline{\mathcal{M}_{g}^{1\
an}}}[N])$ factorizing the natural morphism $\pi_{1}(S\setminus
D)\to\pi_{1}({\overline{\mathcal{M}_{g}^{1\ an}}}[N])$. It descends to a map
of fundamental groups
$\pi_{1}(\widehat{S(N)})=\pi_{1}(S(N))\to\pi_{1}({\overline{\mathcal{M}_{g}^{1\
an}}}[N])$. Then, we can construct the $GL_{N}$ Shafarevich morphism, which
contracts exactly the connected components of the preimage of the chain of
$(-2)$ rational curves by the natural map to $S$, which have finite
fundamental groups. This is enough to imply the holomorphic convexity of the
univeral covering spaces of $\widehat{S(N)}$ and $S(N)$.

###  6.5. Concluding remarks

If $\pi:S\to C$ is an irrational pencil of curves (resp. a rational pencil) on
a smooth projective surface and has more complicated singularities, such as
non reduced components of the singular fibers, we can use the semistable
reduction theorem [15], to find $C^{\prime}\to C$ a ramified covering, which
ramifies at the unstable singular fibers (resp. one may have to introduce a
further ramification point on the smooth locus) to get a normal projective
surface $S^{\prime}$ birational to $S\times_{C}C^{\prime}$ with an irrational
pencil (resp. a general pencil of curves) with only stable curves as scheme
theoretic singular fibers and one is reduced to the previous situation.

So in order to find a counterexample to the Shafarevich conjecture along the
lines of [8], the only remaining possibilities with a base change having a
common ramification index, require a ramification index
$N\in\\{1,2,3,4,6,8,12,16,20,24\\}$ ($N=1$ corresponds to no base change at
all) or allowing a fibration with non-reduced fibers.

Another possibility is to perform a fiber product with
$\overline{\mathcal{M}^{1\ an}_{g}}[\mathbf{k}]$ with general weight vector
$\mathbf{k}$. The fundamental groups of the $\overline{\mathcal{M}^{1\
an}_{g}}[\mathbf{k}]$ with general weight vector $\mathbf{k}$ do not seem to
be easy to study. Let $\Pi$ be the fundamental group of a hyperbolic surface
(possibly punctured). Let $\\{\gamma_{i}\\}$ be a set of isotopy classes of
simple closed curves whose orbits under the action of the mapping class group
are pairwise disjoint. Then, Wise and Bajpai proved in [4, 55] that there is
some $d\geq 1$ such that for every positive integral vector
$\mathbf{k}=(k_{i})$ there exists some finite quotient $\Pi\to\Pi/K$ in which
images of the classes $\gamma_{i}$ have orders $dk_{i}$, respectively. We
don’t know whether the kernel $K$ might be chosen to be invariant with respect
to the mapping class group action. If it could and additionally $K$ satisfies
Boggi’s condition from [6, Thm. 3.9] (for example, if $K=[L,L]L^{\ell}$, for
some $\ell\geq 3$, where $L$ is invariant subgroup of $\Pi$) then the
compactified Deligne-Mumford stack of curves endowed with Looijenga’s levels
associated to $K$ and $m\geq 2$ would be smooth and representable. This
further implies that $\overline{\mathcal{M}^{an}_{g}}[md\mathbf{k}]$ is
uniformisable.

In a similar vein, one may as in [8] start with a stable fibration $\pi:S\to
C$ with reduced fibers and try and study the fundamental groups of the
canonical stacks obtained by assigning different multiplicities to the fibers.
We don’t known how to decide whether the fiber group is finite or infinite
when the multiplicities are coprime.

## References

  * [1] D. Abramovich, M. Olsson, A. Vistoli, Tame stacks in positive characteristic, Ann. Institut Fourier 58 (2008), 1057–1091.
  * [2] J. Aramayona, L. Funar, Quotients of the mapping class group by power subgroups, Bull. London Math. Soc. 51 (2019), 385–398.
  * [3] K. Behrend and B. Noohi, Uniformization of Deligne-Mumford curves, J. Reine Angew. Math. 599 (2006), 111–153.
  * [4] J. Bajpai, Omnipotence of surface groups, Master Thesis, McGill Univ. 2007.
  * [5] C. Blanchet, N. Habegger, G. Masbaum, P. Vogel, Topological quantum field theories derived from the Kauffman bracket, Topology 34 (1995), 883–927.
  * [6] M. Boggi Galois coverings of moduli spaces of curves and loci of curves with symmetry, Geometriae Dedicata 168 (2014), 113–142.
  * [7] M. Boggi, M. Pikaart, Galois coverings of the moduli of curves, Compositio Math. 120 (2000) 171–191.
  * [8] F. Bogomolov, L. Katzarkov, Complex projective surfaces and infinite groups, Geom. Funct. Analysis 8 (1998), 243–272.
  * [9] J. L. Brylinski, Propriétés de ramification à l’infini du groupe modulaire de Teichmüller, Ann. Sci. Ec. Norm. Sup. Série 4, 12 (1979), 295–333.
  * [10] C. Cadman, Using stacks to impose tangency conditions on curves, Amer. J. Math. 129 (2007), 405–427.
  * [11] F. Campana, B. Claudon, P. Eyssidieux, Représentations linéaires des groupes kählériens : Factorisations et conjecture de Shafarevich linéaire, Compositio Math. 151 (2015), 351–376.
  * [12] N. Chriss, V. Guinzburg Representation Theory and Complex Geometry, Modern Birkhäuser Classics, Birkhäuser, 2010.
  * [13] H.S.M. Coxeter, On factors of braid groups, Proc. 4-th Canadian Math. Congress, Univ. Toronto Press, p.95–122, 1959.
  * [14] H.S.M. Coxeter, W.O.J. Moser, Generators and relations for discrete groups, Ergebnisse Math. Springer-Verlag, 1980.
  * [15] P. Deligne, D. Mumford, The irreducibility of the space of curves of a given genus, Publ. Math. I.H.E.S. 36 (1969), 75–109.
  * [16] J.K. Egsgaard, S. Fuglede Jørgensen, The homological content of the Jones representation at $q=-1$, J. Knot Theory Its Ramif. 25 (2016), No. 11, Article ID 1650062, 25p.
  * [17] P. Eyssidieux, On the Uniformisation of Compact Kähler orbifolds, Vietnam J. Math. 41 (2013), 399–407.
  * [18] P. Eyssidieux, Orbifold Kähler Groups and the Shafarevich Conjecture for Hirzebruch’s covering surfaces with equal weights, Asian J. Math 22 (2018), special volume in the honor of N. Mok, 315–330.
  * [19] P. Eyssidieux, Orbifold Kähler Groups related to arithmetic complex hyperbolic lattices, arxiv:1805.00767.
  * [20] P. Eyssidieux, L. Katzarkov, T. Pantev, and M. Ramachandran, Linear Shafarevich Conjecture, Annals of Math. 176 (2012), 1545–1581.
  * [21] P. Eyssidieux, F. Sala, Instantons and framed sheaves on Kähler Deligne-Mumford stacks, arxiv 1404.3504, Ann. Fac. Sci. Toulouse, Math. (6) 27 (2018), 599-628.
  * [22] B. Fantecchi, E. Mann, F. Nironi Toric Deligne Mumfords stacks J. Reine Angewand. Math. 648 (2010), 201-244.
  * [23] L. Funar, On the TQFT representations of the mapping class groups, Pacific J. Math. 188 (2)(1999), 251–274.
  * [24] L. Funar, Zariski density and finite quotients of mapping class groups, Int. Math. Res. Not., no.9, 2013, 2078-2096.
  * [25] L. Funar, P. Lochak Profinite Completion of Burnside type quotients of surface groups Comm. Math. Phys. 360 (2018), 1061–1082.
  * [26] L. Funar, W. Pitsch, Images of quantum representations of mapping class groups and Dupont-Guichardet-Wigner quasi-homomorphisms, J. Inst. Math. Jussieu 17 (2018), 277–304.
  * [27] A. Geraschenko, M. Satriano, A “bottom-up” characterization of smooth Deligne-Mumford stacks, Int. Math. Res. Not., issue 21 (2017), 6469–6483.
  * [28] P. Gilmer, G. Masbaum, Integral lattices in TQFT, Ann. Sci. Ecole Norm. Sup. Série (4) 40 (2007), 815–844.
  * [29] S.P. Humphries, Normal closures of powers of Dehn twists in mapping class groups, Glasgow J. Math. 34 (1992), 313–317.
  * [30] S. Gervais, Presentation and central extensions of mapping class groups, Trans. Amer. Math. Soc. 348 (1996), 3097–3132.
  * [31] V. Jones, Index for subfactors Inventiones Math. 72 (1983), 1–25.
  * [32] L. H. Kauffman, State models and the Jones polynomial, Topology 26 (1987) 395–407.
  * [33] L. Katzarkov, M. Ramachandran, On the universal coverings of algebraic surfaces, Ann. Sci. Ecole Norm. Sup. Série 4, 31 (1998), 525–535.
  * [34] F. Knudsen, The projectivity of the moduli space of stable curves, II: The stacks $M_{g,n}$, Math. Scand. 52 (1983), 161–199.
  * [35] F. Knudsen, The projectivity of the moduli space of stable curves, III: The line bundles on $M_{g,n}$ and a proof of the projectivity of $\overline{M_{g,n}}$ in characteristic $0$, Math. Scand. 52 (1983), 200–212.
  * [36] T. Koberda, R. Santharoubane, Quotients of surface groups and homology of finite covers via quantum representations, Inventiones Math. 206 (2016), 269–292.
  * [37] T. Koberda, R. Santharoubane, Irreducibility of quantum representations of mapping class groups with boundary, Quantum Topology 9 (2018), 633–641.
  * [38] J. Korinman, On the (in)finiteness of the image of Reshetikhin-Turaev representations, Archiv Math. 111 (2018), 247–256.
  * [39] G. Laumon, L. Moret-Bailly, Champs Algébriques, Ergebnisse der Mathematik und ihrer Grenzgebiete, 3. Folge 39, Springer, 2000.
  * [40] P. Le Barz, A propos des revêtements ramifiés d’espaces de Stein, Math. Ann. 222 (1976), 63–69.
  * [41] E. Lerman, Orbifolds as stacks?, Enseign. Math. (2), 56 (2010), 315–363.
  * [42] E. Looijenga, Smooth Deligne-Mumford compactifications by means of Prym level structures, J. Alg. Geom. 3 (1994) 283–293.
  * [43] G. Masbaum, An Element of Infinite Order in TQFT-Representations of Mapping Class Groups, Low-Dimensional Topology (Funchal, 1998), 137–39. Contemporary Mathematics 233\. Providence, RI: American Mathematical Society, 1999.
  * [44] G. Masbaum, J. Roberts, On central extensions of mapping class groups, Math. Ann. 302 (1995), 131–150.
  * [45] B. Noohi, Fundamental groups of algebraic stacks, J. Inst. Math. Jussieu 3 (2004), 69–103.
  * [46] B. Noohi, Fundations of topological stacks, arXiv math/0503247 (2005).
  * [47] M. Olsson, Logarithmic geometry and algebraic stacks, Ann. Sci. Ecole Norm. Sup. Série 4, 36 (2003), 747–791
  * [48] M. Pikaart, A. J. de Jong, Moduli of curves with non-Abelian level structure, In: R.Dijkgraaf, C. Faber and G. van der Geer (eds), The Moduli Space of Curves, Progress in Mathematics 129, Birkhäuser, 1995, pp. 483–510
  * [49] J. H. Przytycki, Skein modules of 3-manifolds, Bull. Acad. Pol.: Math. 39 (1991), 91-100.
  * [50] M. Romagny, Group actions on stacks and applications, Michigan Math. J. 53 (2005), 209-236.
  * [51] The Stacks Project Authors, Stacks Project http://stacks.math.columbia.edu (2018).
  * [52] V. Turaev Quantum Invariants of Knots and 3-Manifolds de Gruyter Series in Mathematics 18, de Gruyter (1994).
  * [53] A. Vistoli, Intersection theory on algebraic stacks and on their moduli spaces, Inventiones Math. 97 (1989), 613–670.
  * [54] H. Wenzl, On sequences of projections, C. R. Math. Rep. Acad. Sci. Canada 9 (1987), 5–9.
  * [55] D. T. Wise, Subgroup separability of graphs of free groups with cyclic edge groups, Quarterly J. Math. 51 (2000), 107–129.
  * [56] G. Wright, The Reshetikhin-Turaev representation of the mapping class groups, J. Knot Theory Its Ramif. 3 (1994), 547–574.
  * [57] G. Wright, The Reshetikhin-Turaev representation of the mapping class groups at sixth roots of unity, J. Knot Theory Its Ramif. 5 (1996), 721–741.
  * [58] V. Zoonekynd, Théorème de Van Kampen pour les champs algébriques, Ann. Math. Blaise Pascal 9 (2002), 101–145.


# Dynamical noise sensitivity for the voter model

Gideon Amir Email<EMAIL_ADDRESS>Department of Mathematics, Bar-Ilan
University, 5290002, Ramat Gan, Israel. Omer Angel Email<EMAIL_ADDRESS>Department of Mathematics, University of British Columbia, Vancouver, BC, V6T
1Z2, Canada. Rangel Baldasso Email<EMAIL_ADDRESS>Mathematical Institute, Leiden University, P.O. Box 9512, 2300 RA Leiden, The
Netherlands. Ron Peretz Email<EMAIL_ADDRESS>Economics Department,
Bar-Ilan University, 5290002, Ramat Gan, Israel.

###### Abstract

We study how the consensus opinion of the voter model on finite graphs varies
in light of noise sensitivity with respect to both the initial opinions and
the dynamics. We first prove that the final opinion is stable with respect to
small perturbations of the initial configuration. Different effects are
observed when a perturbation is introduced in the dynamics governing the
evolution of the process, and the final opinion is noise sensitive in this
case. Our proofs rely on the relationship between the voter model and
coalescing random walks, and on a precise description of this evolution when
we have coupled dynamics.
_Keywords and phrases._ Voter model; consensus opinion; noise sensitivity.
MSC 2010: 82C22, 60J27, 60K35.

##  1 Introduction

In this note we consider the voter model on finite graphs. We study the
consensus opinion in light of noise sensitivity with respect to the initial
condition and the dynamics. The notion of noise sensitivity for Boolean
functions was introduced in the seminal paper [1] by Benjamini, Kalai, and
Schramm, and regards the behaviour of Boolean functions when a small fraction
of its entries is subject to an independent noise. More precisely, let
$\mathbb{P}_{p}$ denote the probability measure in $\\{0,1\\}^{n}$ with
independent marginals, each Bernoulli$(p)$ for $p\in[0,1]$. Given
$\omega\in\\{0,1\\}^{n}$ distributed according to $\mathbb{P}_{p}$ (The
Bernoulli($p$) product measure), define the $\epsilon$-perturbed configuration
$\omega^{\epsilon}$ by resampling each bit of $\omega$ independently with
probability $\epsilon$. A sequence of Boolean functions
$f_{n}:\\{0,1\\}^{n}\to\\{0,1\\}$ is said to be noise sensitive (at level $p$)
if for every $\epsilon>0$ we have that $f_{n}(\omega)$ and
$f_{n}(\omega^{\epsilon})$ are asymptotically independent. Formally,

$\lim_{n}\mathbb{E}_{p}\big{[}f_{n}(\omega)f_{n}(\omega^{\epsilon})\big{]}-\mathbb{E}_{p}\big{[}f_{n}(\omega)\big{]}^{2}=0.$
(1.1)

In contrast, the sequence $(f_{n})_{n\in\mathbb{N}}$ is said to be noise
stable (at level $p$) if

$\lim_{\epsilon\to 0}\sup_{n}\mathbb{P}_{p}\big{[}f_{n}(\omega)\neq
f_{n}(\omega^{\epsilon})\big{]}=0,$ (1.2)

so that small noises are not enough to provoke a macroscopic change in the
value of the functions $f_{n}$.

The voter model is a classical model of interacting particles [4]. Consider a
connected graph with $n$ vertices $G_{n}=\big{(}V_{n},E_{n}\big{)}$. At each
time $t$, each vertex $v$ has an opinion $\eta_{t}(v)\in S$ for some space $S$
of possible opinions. The case $S=\\{0,1\\}$ is the most commonly used case.
At time $t=0$, each vertex of $G_{n}$ has an opinion $\eta_{0}(v)\in S$. Each
edge $\\{u,v\\}\in E_{n}$ is then endowed with two independent Poisson
processes of rate one, with one process for each orientation of the edge. All
Poisson processes for the different edges are taken to be independent from
each other as well as from the initial configuration $\eta_{0}$. These
processes control changes in opinions in $u$ and $v$. Whenever a clock tick
happens in the process associated with the oriented edge $(u,v)$, the vertex
$v$ copies the opinion of $u$. (See below for a more formal definition.) One
construction of the voter model, referred to as a graphical construction is in
terms of a collection of Poisson processes associated with the edges of the
graph. This is described in detail in Section 2. We denote by $\mathscr{P}$
the collection of Poisson point processes used in the graphical construction
of the voter model, and write $\eta_{t}(x)$ for the resulting opinion of
vertex $x$ at time $t$.

The configuration $\eta_{t}\in S^{V_{n}}$ is a Markov chain which is almost
surely eventually absorbed in a constant configuration. The eventual constant
opinion is called the consensus opinion. We denote this final opinion by
$f_{n}(\eta_{0},\mathscr{P})$, depending implicitly on $G_{n}$ with initial
opinion $\eta_{0}$ and graphical construction given by the clocks
$\mathscr{P}$. Given $p\in(0,1)$, we say that the consensus opinion $f_{n}$ is
noise stable with respect to the initial condition if when the initial
opinions are sampled from $\mathbb{P}_{p}$ we have

$\lim_{\epsilon\to
0}\sup_{n}\mathbb{P}_{p}\big{[}f_{n}(\eta_{0},\mathscr{P})\neq
f_{n}(\eta_{0}^{\epsilon},\mathscr{P})\big{]}=0.$ (1.3)

To study the noise sensitivity of the consensus opinion with respect to the
clock ticks of the graphical construction, it is necessary to first define
what we mean by an $\epsilon$-perturbation of $\mathscr{P}$. Given
$\epsilon\in[0,1]$, $\mathscr{P}^{\epsilon}$ is constructed from $\mathscr{P}$
in two steps: one first obtains $\mathscr{P}^{\prime}$ as an
$\epsilon$-thinning of $\mathscr{P}$. Then define

$\mathscr{P}^{\epsilon}=\mathscr{P}^{\prime}\cup\mathscr{P}(\epsilon),$ (1.4)

where $\mathscr{P}(\epsilon)$ is an independent collection of clock ticks with
intensity $\epsilon$. We say that the consensus opinion is noise sensitive
with respect to the clock ticks if

$\lim_{n}\mathbb{E}_{p}\big{[}f_{n}(\eta_{0},\mathscr{P})f_{n}(\eta_{0},\mathscr{P}^{\epsilon})\big{]}-\mathbb{E}_{p}\big{[}f_{n}(\eta_{0},\mathscr{P})\big{]}^{2}=0.$
(1.5)

The main result of this note is the following theorem.

######  Theorem 1.1.

Let $G_{n}$ be any sequence of connected graphs, where $G_{n}$ has $n$
vertices. The consensus opinion in the voter model is noise stable with
respect to the initial condition for all $p\in(0,1)$ and noise sensitive with
respect to the clock ticks. Moreover, the limit

$\limsup_{n}\operatorname{Cov}\Big{(}f_{n}(\eta_{0},\mathscr{P}),f_{n}(\eta_{0},\mathscr{P}^{\epsilon_{n}})\Big{)}$
(1.6)

equals zero if $n\epsilon_{n}\to\infty$ and is positive if $n\epsilon_{n}$ is
bounded.

One way to interpret the above theorem, suggested to us by Itai Benjamini, is
by viewing the voter model dynamics as a random Boolean function of the
initial configuration. The dynamics determines a (random) dictator, whose
opinion is revealed when checking the initial opinions. Our theorem then says
that the choice of the dictator is sensitive with respect to noising the
dynamics. Clearly, once the dictator’s identity is changed all correlation is
lost. On the other hand, given a choice of dictator, her opinion is clearly
stable with respect to small perturbations. The stability statement in the
theorem is an immediate consequence of that stability.

A special case of this theorem which is of independent interest is when the
graph is the complete graph $K_{n}$. In this case, the voter model is
equivalent to the Moran model of population dynamics. Our result has the
interpretation that the eventual dominant allell is stable with respect to the
initial population types, but sensitive to noise affecting the reproduction
dynamics of the model.

### Generalizations.

The theorem and the proof extend with no significant changes to the case of
weighted graphs (electrical networks). Here each undirected edge has a weight
$w_{e}$. For each neighbouring pair $e=(x,y)$, the vertex $x$ adopts the
opinion of $y$ at rate $w_{e}$, and $y$ adopts the opinion of $x$ at the same
rate. However, the rates in the two directions must be equal for our proof to
carry through. This can be seen to be necessary, since for general asymmetric
rates the result may fail. Even in the natural case where each vertex is
activated at rate 1, and copies the opinion of a uniform neighbour the
consensus might not be sensitive to noisy dynamics. For example, in this model
on the star graph, the consensus is strongly correlated to the initial opinion
of the centre vertex, and this correlation is maintained when noise is added
to the dynamics.

### Proof overview.

The proof of Theorem 1.1 relies on the classical duality relation between the
voter model and coalescing random walks. The same clock processes can be used
to define coalescing random walks on $G_{n}$. If one fixes a realization of
the clock processes $\mathscr{P}$ and considers coalescing random walks going
backwards in time and moving in the opposite direction from the activated
edges, the consensus opinion coincides with the opinion at time zero from the
last walker remaining. This observation alone implies noise stability with
respect to the initial condition.

Via this relation with coalescing random walks, one can also obtain that the
position of the site that realizes the consensus opinion on $G_{n}$ is
distributed according to the invariant measure of the random walk on $G_{n}$.
This hints at the proof of the second statement about noise sensitivity with
respect to clocks. Here one needs to study the joint evolution of two highly
dependent random walks in $G_{n}$, using clock ticks from $\mathscr{P}$ and
$\mathscr{P}^{\epsilon}$, respectively. One considers these walkers as
evolving on $G_{n}\times G_{n}$ in a suitably constructed way. The proof is
then concluded by describing the invariant measure of this coupled dynamics
and verifying that the probability that both random walks end in the same
position vanishes in the limit as $n$ increases.

### Related works.

The notion of noise sensitivity of boolean functions was formalized by
Benjamini, Kalai and Schramm [1], following earlier related works of Hastad
[3]. It has developed into a central tool in the study of boolean functions,
see e.g. O’Donnell’s book [5]. Noise sensitivity and stability have proved to
be especially useful in percolation theory, where the natural inputs are
independent variables, and many of the previous applications of that theory
had been in the context of percolation. In particular, noise sensitivity has
proved crucial to our understanding of dynamical percolation. See e.g. Garban
and Steif’s book [2] and references therein.

The consensus opinion of the voter model is known to have the same
distribution as the independent initial opinions. Obtaining bounds on the time
when consensus is reached is more delicate, and also relies on the duality
relation between voter model and coalescing random walks. The consensus time
of the voter model is bounded by the coalescence time of coalescing random
walks. Oliveira [6] proves that, for graphs with constant degree, this
expected coalescing time can be bounded up to a universal constant by the
maximum of the hitting time in the graph over all starting vertices and
targets. This implies that the consensus time in the voter model is of order
at most $n^{2}$.

### Acknowledgements.

We thank Itai Benjamini for raising the questions studied in this paper. GA
counted on the support of the Israel Science Foundation through Grant 957/20.
OA is supported in part by NSERC. RB thanks the Mathematical Institute of
Leiden University for support. RP was supported in part by the Israel Science
Foundation Grant 2566/20.

##  2 Preliminaries

This section contains a short review of the voter model and its connection to
coalescing random walks. We highlight the main facts that will be used
throughout the paper.

### The voter model.

Let $G_{n}=(V_{n},E_{n})$ be a sequence of connected finite graphs. The voter
model with opinions in $S$ is a Markov chain on $S^{V_{n}}$, where a voter
changes their opinion to $x\in S$ at rate equal to the number of neighbours
with opinion $x$. The voter model on $G_{n}$ can be obtained via a graphical
construction that we define now. Let $\vec{E}_{n}$ be the set of oriented
edges of $G_{n}$, containing each edge in both directions. Let $\mathscr{P}$
denote a Poisson point process on $\vec{E}_{n}\times\mathbb{R}_{+}$ with
intensity $\mu\otimes\lambda$, where $\mu$ is the counting measure on
$\vec{E}_{n}$, and $\lambda$ denotes the Lebesgue measure in $\mathbb{R}_{+}$.
Given an initial configuration $\eta_{0}\in S^{V_{n}}$, the evolution proceeds
by copying opinions at the time of the clock ticks: if the oriented edge
$(u,v)$ has a clock tick at time $t$, then site $v$ copies the opinion of $u$.
It is easy to see that this almost surely defines a unique process
$(\eta_{t})_{t\geq 0}$.

The voter model has absorbing states: the constant configurations with any
opinion. Furthermore, on any connected finite graph the model almost surely
reaches one of these configurations. An easy way to see this is by noting that
the number of vertices with any opinion is a bounded martingale, and so must
converge.

The voter model with general opinion set $S$ can be understood in terms of the
voter model with Boolean opinions by noting that for any opinion $a\in S$, the
set of vertices with opinion $a$ evolves as a voter model. (These are not
independent for different $a$’s but this is sufficient to deduce our results
for a general opinion set from the result for Boolean opinions.)

### Coalescing random walks.

In this model, each site initially receives an independent random walk that
jumps to each neighbour with rate one. When any two of these walkers meet,
they coalesce and move together from that time onward.

One way to define this model precisely is using a graphical construction.
Given Poisson processes $\mathscr{P}$ as before, a random walk from $x$ is
constructed by jumping from $u$ to $v$ if the walk is at $u$ and the clock for
directed edge $(u,v)$ ticks. It is clear that each walker performs a random
walk, and that once two walkers are at the same vertex they stay together from
that time on. Since any two walkers on a finite connected graph will meet at
some finite time, there is some time at which all walkers have coalesced. This
is called the coalescence time, which we denote by $T_{C}$.

### Duality.

The relation between the voter model and coalescing random walks is an easy
consequence of the graphical constructions. It makes it possible to use
coalescing random walks to determine the consensus opinion via realizations of
the graphical construction $(\eta_{0},\mathscr{P})$. Given $t\geq 0$, let us
denote by $\big{(}X^{t}_{i}(s)\big{)}_{s\leq t}$ the random walk that is at
position $i$ at time $t$ and runs backwards in time up to time zero by using
the clock ticks from $\mathscr{P}$: if an oriented edge $(u,v)$ has a clock
tick and the random walk is on $v$, then it jumps to site $u$. For each $t$,
the walks running backward in time are coalescing random walks.

To see the relation with the voter model, one should trace the origin of the
opinion of $x$ at time $t$. Going backward in time, the opinion jumps from $v$
to $u$ if the edge $(u,v)$ has a clock tick. Thus the origin of the opinions
form coalescing random walks when viewed backward in time.

It is clear that the coalescence time of the backward random walks has the
same distribution as that for walks moving forward in time. Since $T_{C}$ is
almost surely finite, the probability that the coalescence time for the walks
$\big{(}X^{t}_{i}\big{)}_{i\in[n]}$ is at most $t$ is the same as
$\mathbb{P}(T_{C}\leq t)$ which tends to $1$ as $t\to\infty$. Now, if $t$ is
large enough such that the coalescing random walks with the collection
$\big{(}X^{t}_{i}\big{)}_{i\in[n]}$ have coalesced by time $t$, then for all
$i$, we have $X_{i}^{t}(0)$ is the same vertex, and

$f_{n}(\eta_{0},\mathscr{P})=\eta_{0}(X_{i}^{t}(0)).$ (2.1)

(Recall $f_{n}(\eta_{0},\mathscr{P})$ denotes the consensus opinion of the
voter model.) Since $T_{C}$ is finite almost surely, the equation above also
implies that

$f_{n}(\eta_{0},\mathscr{P})=\lim_{t\to\infty}\eta_{0}(X_{i}^{t}(0)),$ (2.2)

which in turn provides the equality in distribution

$f_{n}(\eta_{0},\mathscr{P})\sim\eta_{0}(X),$ (2.3)

where $X$ is distributed according to the invariant measure of the random walk
$X_{1}$. If the initial opinions are independent, then $f_{n}$ has the same
law as the initial opinions.

### Coalescing random walks and noise.

When studying noise sensitivity with respect to the clock rings, one needs to
consider a pair of coupled random walks that evolve according to the pair of
graphical constructions $(\mathscr{P},\mathscr{P}^{\epsilon})$. These random
walks are coupled as follows: If they are at distinct vertices they jump
independently at rate 1 across each edge; If they are at the same vertex, they
jump together across each edge at rate $1-\epsilon$, and each of the walks
jumps alone across the edge at rate $\epsilon$, leaving the other walk
unchanged.

Proceeding in an analogous way as in the deduction of (2.3), we obtain the
equality of the joint distribution

$\big{(}f_{n}(\eta_{0},\mathscr{P}),f_{n}(\eta_{0},\mathscr{P}^{\epsilon})\big{)}\sim\big{(}\eta_{0}(X),\eta_{0}(X^{\epsilon})\big{)},$
(2.4)

where $(X,X^{\epsilon})$ is distributed according to the invariant measure of
a pair of coupled random walks evolving with the Poisson processes
$(\mathscr{P},\mathscr{P}^{\epsilon})$.

##  3 Proof of Theorem 1.1

We now proceed with the proof of Theorem 1.1. We first verify noise stability
with respect to the initial condition and then consider noise sensitivity with
respect to clock ticks.

### Noise stability for the initial condition.

This is the easier part of the result. It follows directly from (2.1) by
noticing that

$\mathbb{P}_{p}\Big{[}f_{n}(\eta_{0},\mathscr{P})\neq
f_{n}(\eta_{0}^{\epsilon},\mathscr{P})\Big{]}=\mathbb{P}_{p}\Big{[}\lim_{t\to\infty}\eta_{0}(X_{1}^{t}(0))\neq\lim_{t\to\infty}\eta_{0}^{\epsilon}(X_{1}^{t}(0))\Big{]}\leq\epsilon,$
(3.1)

since the two are equal if $\eta_{0}(X^{t}_{1}(0))$ has not been resampled.

### Noise sensitivity for the clocks.

We now prove the statement regarding noise sensitivity with respect to the
clock ticks in the graphical construction. We use the relation established in
(2.4) to obtain an expression for the distribution of the vertices that
realize the consensus opinion in the coupled graphical constructions
$(\mathscr{P},\mathscr{P}^{\epsilon})$.

Consider a pair $(X_{t},X_{t}^{\epsilon})$ of random walks that evolve
according to the pair of Poisson clocks
$(\mathscr{P},\mathscr{P}^{\epsilon})$. From the construction of the coupled
pair of Poisson clocks, we see that $(X_{t},X_{t}^{\epsilon})$ evolves on
$G_{n}\times G_{n}$ with rates given by the $n^{2}\times n^{2}$ matrix

$Q\big{(}(u,v),(u^{\prime},v^{\prime})\big{)}=\begin{cases}1-\epsilon,\text{
if }u=v,u^{\prime}=v^{\prime},\text{ and }u\sim u^{\prime};\\\ \epsilon,\text{
if }u=v=v^{\prime}\text{ and }u\sim u^{\prime};\\\ \epsilon,\text{ if
}u=u^{\prime}=v\text{ and }v\sim v^{\prime};\\\ 1,\text{ if }u\neq
v,u=u^{\prime},\text{ and }v\sim v^{\prime};\\\ 1,\text{ if }u\neq
v,v=v^{\prime},\text{ and }u\sim u^{\prime};\\\ 0,\text{
otherwise}.\end{cases}$ (3.2)

The key observation is that we can write explicitly the invariant measure of
the random walk on $G_{n}\times G_{n}$ with transition rates given by the
matrix $Q$. Define the function $\nu$ (viewed as a $1\times n^{2}$ vector) on
$G_{n}\times G_{n}$ by

$\nu(u,v)=\textbf{1}_{u=v}+\epsilon\textbf{1}_{u\neq v}.$ (3.3)

We claim that $Q$ is reversible w.r.t. $\nu$. Indeed, consider the quantity
$\nu(u,v)Q((u,v),(u^{\prime},v^{\prime}))$.

  * •

If $u=v$ and $u^{\prime}=v^{\prime}$ this is $1\cdot(1-\epsilon)$ if $u\sim
u^{\prime}$ and 0 otherwise.

  * •

If $u=v$ and $u^{\prime}\neq v^{\prime}$, this is $1\cdot\epsilon$ if one of
$\\{u^{\prime},v^{\prime}\\}$ is $u$ and the other is a neighbouring vertex
(otherwise 0).

  * •

If $u\neq v$ and $u^{\prime}=v^{\prime}$, this is $\epsilon\cdot 1$ if one of
$\\{u,v\\}$ is $u^{\prime}$ and the other is a neighbouring vertex (otherwise
0).

  * •

If $u\neq v$ and $u^{\prime}\neq v^{\prime}$, this is $\epsilon\cdot 1$ as
long as one of the pairs $(u,u^{\prime})$, $(v,v^{\prime})$ is equal and the
other are neighbours.

In all cases, this is the same if $(u,v)$ and $(u^{\prime},v^{\prime})$ are
swapped.

This implies that the invariant measure $\pi$ for the process
$(X_{t},X_{t}^{\epsilon})_{t\geq 0}$ is proportional to $\nu$, i.e.
$\pi=\nu/(n+(n^{2}-n)\epsilon)$. In particular
$\pi(X=X^{\prime})=\frac{1}{1+(n-1)\epsilon}$.

Suppose $\eta_{0}$ has Bernoulli$(p)$ entries. We now calculate:

$\begin{split}\mathbb{E}_{p}\Big{(}f_{n}(\eta_{0},\mathscr{P})f_{n}(\eta_{0},\mathscr{P}^{\epsilon})\Big{)}&=\mathbb{E}_{\pi}\mathbb{E}_{p}\Big{(}\eta_{0}(X),\eta_{0}(X^{\epsilon})\Big{)}\\\
&=p^{2}\pi(X\neq X^{\prime})+p\pi(X=X^{\prime})\\\
&=p^{2}+\frac{p-p^{2}}{1+(n-1)\epsilon}.\end{split}$ (3.4)

Thus $f_{n}(\eta_{0},\mathscr{P})$ and
$f_{n}(\eta_{0},\mathscr{P}^{\epsilon})$ are asymptotically independent if and
only if $\epsilon n\to\infty$, as claimed.

##  4 A problem

Our theorem states that the consensus is sensitive to noisy dynamics.
Moreover, this sensitivity does not depend on the graph on which the process
lives. Consider a new model, which we call the majority-voter model. Here, the
voter dynamics are run for some pre-determined time $T$. At that time, an
election occurs, and the outcome is the majority of the opinions at time $T$.
This interpolates between simple majority for $T=0$ and the consensus in the
voter model for $T\to\infty$. A natural question is how large does $T$ needs
to be so that the majority-voter model is sensitive to noisy dynamics. It
seems that the result would depend on the underlying graph.

## References

  * [1] Itai Benjamini, Gil Kalai, and Oded Schramm.  Noise sensitivity of Boolean functions and applications to percolation.  Publications Mathématiques de l’Institut des Hautes Études Scientifiques, 90(1):5–43, 1999.
  * [2] Christophe Garban and Jeffrey E Steif.  Noise sensitivity and percolation.  Probability and statistical physics in two and more dimensions, 15:49–154, 2012.
  * [3] Johan Håstad.  Some optimal inapproximability results.  Journal of the ACM (JACM), 48(4):798–859, 2001.
  * [4] Thomas M Liggett.  Interacting particle systems - an introduction.  In School and Conference on Probability Theory, 2004.
  * [5] Ryan O’Donnell.  Analysis of Boolean functions.  Cambridge University Press, 2014.
  * [6] Roberto Oliveira.  On the coalescence time of reversible random walks.  Transactions of the American Mathematical Society, 364(4):2109–2128, 2012.