# Dual Reinforcement-Based Specification Generation for Image De-Rendering

Ramakanth Pasunuru♣ David Rosenberg† Gideon Mann† Mohit Bansal♣
UNC Chapel Hill♣ Bloomberg LP†
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Advances in deep learning have led to promising progress in inferring graphics
programs by de-rendering computer-generated images. However, current methods
do not explore which decoding methods lead to better inductive bias for
inferring graphics programs. In our work, we first explore the effectiveness
of LSTM-RNN versus Transformer networks as decoders for order-independent
graphics programs. Since these are sequence models, we must choose an ordering
of the objects in the graphics programs for likelihood training. We found that
the LSTM performance was highly sensitive to the sequence ordering (random
order vs. pattern-based order), while Transformer performance was roughly
independent of the sequence ordering. Further, we present a policy gradient
based reinforcement learning approach for better inductive bias in the decoder
via multiple diverse rewards based both on the graphics program specification
and the rendered image. We also explore the combination of these complementary
rewards. We achieve state-of-the-art results on two graphics program
generation datasets.

##  1 Introduction

The large majority of computer vision work deals in the domain of natural
images or video. However, there is tremendous potential for applying computer
vision techniques to computer-generated images, such as plots, charts,
schematics, complicated math formulas, and even a page of printed text. For
these domains, there is often a domain-specific language for precisely
specifying the image, such as matplotlib code for a chart, PicTeX for a
schematic111https://ctan.org/pkg/pictex?lang=en, and LaTeX for math formulas
and text. “De-rendering” a computer-generated image back to the original (or a
different) domain-specific language specification can be a useful first step
in many tasks, such as changing the visual appearance of an image (?; ?) or
extracting information contained in an image (?; ?).

Figure 1: Example images from the abstract scene dataset (left) and the Noisy
Shapes dataset (right), along with portions of their specifications.

The de-rendering problem is part of a larger class of “image-to-text”
problems, in which an input image is mapped to some sequence of output tokens.
The neural encoder-decoder approach has proved to be very successful for this
class of problems, including image captioning (?; ?), handwriting recognition
(?), as well as the de-rendering problem for math formulas (?) and graphics
images (?; ?). In this paper, we improve these encoder-decoder models for the
specific case of graphical images, via methods based on Transformer models
with both cross-entropy training and reinforcement learning with up to two
“dual modality” reward functions. De-rendering graphical images is a problem
that differs in several interesting ways from image captioning and OCR
problems. Two examples of the de-rendering problem we consider are shown in
Fig. 1. Each image is an input, and a portion of the desired output is
displayed below each image. In de-rendering, every object in the image must be
described in the specification, and typically many output tokens are required
to describe each object. Thus outputs from de-rendering are typically much
longer than those in image captioning datasets (?), since caption labels
(e.g., in COCO (?)) tend to focus on simple descriptions involving only the
most salient objects in the image. OCR and de-rendering are similar in that
they encode information about all elements of the image, but the order of the
output sequence in OCR is completely determined by the image, while in de-
rendering, the output sequences represent sets222We say sets, rather than
sequences, because in our datasets object ordering does not affect the
rendering., and as such the final rendering is invariant to a large degree of
reordering in the output sequence (e.g., by shuffling the sub-sequences of
tokens that correspond to separate objects).

We start our investigation with a basic image captioning model (similar to ?
(?)) and extend it with an attention mechanism. We then swap out the LSTM-RNN
decoder with a Transformer network (?). Our original motivation for this
replacement is that we think that output generation requires long-term
dependencies to avoid representing the same object multiple times. As
mentioned above, de-rendered output sequences can be quite long, and we
thought the multi-head attention mechanism of the Transformer would handle the
long-range dependencies better than the LSTM-RNN. Unexpectedly, we found
another advantage of Transformers over LSTM-RNNs for handling output sequence
that can be reordered in many ways and still be correct. We expand on this in
Sec. 7.1. To our knowledge, we are the first ones to use Transformer networks
for de-rendering graphical images, and we find this change is a significant
source of our performance improvement.

Another challenge with graphics de-rendering is that changing one or a few
tokens in the specification can cause a significant change in many pixel
values (e.g., by changing the location or color of a large object).
Conversely, one can have two images that are very close visually, yet have
completely different specifications. To this end, we explore the error
minimization in the image as well as the specification space via a dual-
modality, two-way reward reinforcement learning approach (?; ?). We train with
non-differentiable reward functions that reflect performance measures of
interest in both the image space and the specification space (the “dual
modes”). We further explore training a single model using rewards from both
modalities, with the hopes that we get complementary feedback from each.

We empirically evaluate our methods on two image de-rendering datasets: Noisy
Shapes dataset (?) and Abstract Scene dataset (?; ?). Our Transformer models
trained with cross-entropy loss achieve very significant improvement over
previous work on these datasets. We show even more improvement when we train
the Transformer models using policy gradients-based methods, both via single-
modality rewards and further improvements via dual-modality joint rewards.
Finally, in our analysis we find evidence that the performance of Transformers
is relatively insensitive to the ordering of objects in the output sequence,
while the performance of LSTM-RNN’s can decay substantially for a poorly
chosen object ordering. This suggests that the advantage of Transformers over
LSTM-RNNs may be particularly strong in tasks where we are using an output
sequence to represent an unordered set of objects.

##  2 Related Work

De-rendering a computer-generated image to a domain-specific language provides
an abstraction that is easy to change, store, compare and match to other
images. As a consequence, there has been recent interest and work in this
area. ? (?) used CNNs to translate a hand-drawn sketch of an object (e.g.,
jewellery) to a fixed set of parameters for a procedural model. In a similar
vein, ? (?) proposed a simple procedural grammar as a building block to turn
sketches into realistic 3D models. ? (?) proposed an automatic visual program
induction model to infer programs from hand-drawn images, where the images are
encoded via CNNs and a multi-layer perceptron predicts a distribution over
drawing commands.

? (?) presented a recurrent neural network based sketch-rnn to construct
conditional and unconditional sketch generation of common objects, constrained
by a very simple set of primitives. Their model describes images as pen
movements either in a drawing mode or in a non-drawing mode. Unlike our
approach, this program is highly sequence dependent and non-compositional.
While there are different solutions to the problem by re-ordering, one cannot
arbitrarily shuffle the sequence of pen movements. ? (?) infer scene programs
by exploiting hierarchical object-based scene representations. ? (?) proposed
a neural program synthesizer that generates underlying programs for
behaviorally diverse demonstration videos. In this work, we use Transformer
networks (?) for decoding the specification from the given input image.
Transformers have been used in other generation tasks such as image and video
captioning (?; ?), however, we are the first ones to use Transformer networks
for the image de-rendering problem. ? (?) shows that an LSTM trained with
shuffled targets (unordered) using cross-entropy has a substantial drop in
performance compared to natural orderings. Our result supports their findings
and moreover we find that Transformers, by contrast, are relatively
insensitive to the ordering of the objects.

Recently, policy gradient-based reinforcement learning (RL) methods have been
widely used for sequence generation tasks: machine translation (?), image
captioning (?; ?), and textual summarization (?; ?). ? (?) proposed to improve
sequence generation by allowing a model to use its own prediction at training
time, extending their work in structured prediction. In the context of program
synthesis, ? (?) used RL for generating semantically correct programs. In the
context of image de-rendering, ? (?) proposed a neural scene de-rendering
model (NSD) with a neural encoder and a graphics engine as a decoder. The
encoder has an object proposal generator that produces segment proposals, and
then it tries to interpret objects and their properties from these segments.
They use RL to better sample the proposals and use the rendered image
reconstruction error as reward.

Recently, ? (?) introduced an adversarially trained agent that is trained via
a reinforcement learning setup without any supervision to generate a program
that is executed by a graphics engine to interpret and sample images. In
contrast, our work presents two complementary rewards (one in image space and
another in specification space) in a reinforcement learning setup for the
image de-rendering problem.

##  3 Models

##### Task.

For each task we consider, there is a simple graphics specification language
that can be used to specify a particular image. While differing in details,
the overall scheme of the specifications are the same for each. A
specification consists of a set of “objects”, and each object is specified by
a set of properties. Examples of an object specification for each of our tasks
can be seen in Fig. 1. Given an image rendered from a specification, our task
is to “de-render” this image back to the original specification. We can
evaluate a predicted specification by looking for exact matches between the
objects in the predicted specification and the objects in the original
specification. We can summarize the object matches with standard measures,
such as precision, recall, F1, and intersection-over-union. While these
measures describe performance on a single image, we can average these measures
across a collection of images, to get a performance measure of a method
overall. We provide more details in Sec. 5.2. Another approach to evaluation
is to generate the image corresponding to a predicted specification, and see
how well it matches the original image, using some reasonable metric on the
space of images.

Reduction to sequence prediction. While each specifications is represented by
a set of objects with specific properties, our models require sequences of
tokens. We convert the set of objects to a sequence of tokens via some
ordering of the objects. We investigate various approaches to ordering (Sec.
7.1), and find that ordering by object type works best. Once the model
predicts a sequence of tokens, we can parse it back into original structure to
compute performance measures and our reward functions for reinforcement
learning.

###  3.1 Image-to-LSTM Sequence Model

Our baseline model is similar to an image captioning model with an attention
mechanism (?). We use the ResNet-18 architecture (?) for encoding the input
image, and we use an LSTM-RNN for predicting the corresponding specification
as a sequence of tokens.

We will denote the convolutional features from the ResNet-18 as
$\\{f_{i}\\}_{i=1}^{m}$, where $f_{i}\in\mathbb{R}^{d}$. For any decoder
output token $o$, let $E_{o}\in\mathbb{R}^{d^{\prime}}$ denote its embedding,
which will be learned during training. Let $s_{t}$ be the decoder state at
step $t$, $o_{t}$ be the output token at step $t$, and $c_{t}$ be the image
context vector at step $t$, which will be defined below. Then at step $t$, the
decoder state $s_{t}$ is given by

$s_{t}=\mathbb{F}(c_{t},s_{t-1},E_{o_{t-1}}),$ (1)

where $\mathbb{F}$ is a trainable non-linear function. The context vector
$c_{t}$ is a convex combination of the image features:
$c_{t}=\sum_{i=1}^{m}\alpha_{t,i}f_{i}$, where $\alpha_{t,i}$ are “attention
weights” defined as

$\displaystyle\alpha_{t,i}$
$\displaystyle=\frac{\exp(e_{t,i})}{\sum_{k=1}^{m}\exp(e_{t,k})}$ (2)
$\displaystyle e_{t,i}$ $\displaystyle=v^{T}\tanh(Wf_{i}+Us_{t-1}+b),$ (3)

where $v$, $W$, $U$, and $b$ are the trainable parameters.

Figure 2: Our Image-Transformer model.

###  3.2 Image-to-Transformer Sequence Model

Recently, there is an increasing amount of interest in Transformer networks
(?), which are said to train faster and to better capture long-term
dependencies than LSTM-based RNN models. In our specification prediction
problem, the length of the specification can be large, and we need long-term
dependencies to avoid generating objects that have already been generated.
This suggests Transformer networks would be a better fit for our scenario. In
this work, we only use the decoder part of the Transformer network (?). The
Transformer encoder is for use on sequences, and we replace it with ResNet-18
CNN described above. We give a high-level description of the Transformer
decoder below, and refer to ? (?) for full details.

The decoder of the Transformer has a stack of $N$ identical layers containing
self-attention modules, normalization modules, and feed-forward modules, along
with positional encoding module for output embeddings (see Fig. 2). While the
original model in ? (?) took $N{=}6$, through hyperparameter tuning we found
$N{=}4$ to work better for our problem. Besides that, we used the
hyperparameter settings as in ? (?). The decoder has two attention modules:
one for attending to the image convolution features and another self-attention
module to attend to different previous positions in the decoder state.

##### Attention in Transformer.

As shown in Fig. 2, we have two attention mechanisms in the model: one
attending to the CNN features, and another attending to different parts of the
decoder state. They all have the same structure, which we describe below.

An attention mechanism in the Transformer can be viewed as a mapping from a
query ($Q$) and a key-value ($K,V$) pair to an output. An attention weight is
computed using the query and key and those weights are used with values to
compute the output of the attention module. Empirically, it has been proven
that instead of performing a single attention function, linearly projecting
the queries, keys, and values with different learned projection layers and
then performing the attention function in parallel and concatenating those
outputs to get the final attention module output to work better. This
attention mechanism is called multi-head attention mechanism (MH), which is
defined as follows:

$\textrm{MH}(Q,K,V)=\textrm{Concat}(\textrm{head}_{1},..,\textrm{head}_{h})W^{O}$
(4) $\textrm{head}_{i}=\textrm{Attention}(QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V})$
(5)
$\textrm{Attention}(Q,K,V)=\textrm{softmax}\left(\frac{QK^{T}}{d_{k}}\right)V$
(6)

where, $d_{k}$ is the dimension of the queries and keys, $W_{i}^{Q}$,
$W_{i}^{K}$, and $W_{i}^{V}$ are the parameters of the projection matrices.

##### Position-wise Feed-Forward Networks.

In addition to the attention sub-layers, each of the layers in the Transformer
decoder contains a fully connected feed-forward network that is applied to
each position of the decoder separately and identically. This network is
defined as

$\textrm{FFN}(x)=\max(0,xW_{1}+b_{1})W_{2}+b_{2},$ (7)

where $W_{1}$, $W_{2}$, $b_{1}$, and $b_{2}$ are the linear projection
parameters which are same across different positions but are different from
layer to layer.

##### Positional Encoding.

In the model described thus far, the model is symmetric with respect to
sequence position. For example, at the bottom right of Fig. 2, the model has
no structural way to determine which output embeddings come from which part of
the output sequence. To remedy this issue, we concatenate a “positional
encoding” (PE) to the embedding representation of the tokens. We use the sine
and cosine functions for positional encoding:

$\begin{split}\textrm{PE}(\textrm{pos},2i)=\textrm{sin}(\textrm{pos}/10000^{2i/d_{\textrm{model}}})\\\
\textrm{PE}(\textrm{pos},2i+1)=\textrm{cos}(\text{pos}/10000^{2i/d_{\textrm{model}}})\end{split}$
(8)

where pos is the position, $i$ is the dimension, and $d_{\textrm{model}}$ is
the dimension of the embedding vector representation.

Figure 3: Example showing the samples from our model based on abstract scene
dataset and the corresponding rewards in specification and image space. For
simplicity, all object properties are not shown in specification space.

##  4 Dual-Modality Two-Way Reinforcement Learning

Traditionally, sequence generation models are trained using a cross-entropy
loss. More recently, a policy gradient-based reinforcement learning approach
has been explored for sequence generation tasks (?; ?), which has two
advantages over the cross-entropy loss optimization approach: (1) avoiding the
_exposure bias_ issue, which is about the imbalance in the output
distributions created by different train and test time decoding approaches in
cross-entropy loss optimization (?; ?); (2) allows direct optimization of the
evaluation metric of interest, even if it is not differentiable. To this end,
we use a policy gradient-based approach via rewards in both the specification
space and the image space. Also, we explore joint rewards based on these two
spaces for better capturing feedback that is complementary between these two
modalities.

For this reward optimization, we use the REINFORCE algorithm (?; ?) to learn a
policy $p_{\theta}$ that produces a distribution over sequences $o^{s}$ for
any given input. We try to find a policy $p_{\theta}$ such that the expected
reward for a label sequence $o^{s}$ drawn according to the predicted
distribution has maximum expected reward. Equivalently, we minimize the
following loss function, in average across all training inputs:

$L_{\textrm{RL}}=-\mathbb{E}_{o^{s}\sim p_{\theta}}[r(o^{s})],$ (9)

where $o^{s}$ is the sequence of sampled tokens with $o^{s}_{t}$ sampled at
time step $t$ of the decoder. We can approximate the gradient of this loss
function with respect to the parameter $\theta$ using a single sample $o^{s}$
drawn from $p_{\theta}$ as:

$\nabla_{\theta}L_{\textrm{RL}}=-(r(o^{s})-b_{e})\nabla_{\theta}\log
p_{\theta}(o^{s}),$ (10)

where the leading factor is included for variance reduction using a baseline
estimator (?). There are several ways to calculate the baseline estimator; we
employ the effective SCST approach (?).

###  4.1 Rewards

In this work we consider three different reward functions. Two of the rewards
are based in “specification space”, which make a direct comparison between the
predicted specification and the ground truth specification, and one of the
rewards is based in “image space”, which compares the image rendered from the
predicted specification with original input image. We also investigate using
these rewards in combination, with the hope that there is complementary
information in the feedback based on the two spaces.

#### Intersection-Over-Union Reward (IOU)

As mentioned in Sec. 3, after the specification is predicted as a sequence of
tokens, we can parse the sequence into a set of object specifications. The
intersection-over-union (IOU) reward is based in specification space. Roughly
speaking, the IOU reward gives credit for predicting objects that exactly
match objects in the ground truth specification, and penalizes both for
predicting objects that do not match ground truth objects and for failing to
predict objects that are part of the ground truth. More formally, let
$\\{o_{i}\\}_{i=1}^{m}$ and $\\{o^{*}_{j}\\}_{j=1}^{n}$ represent the objects
in predicted and ground-truth specifications, respectively. Then the IOU
reward is defined as:

$r_{iou}=\frac{\textrm{count}(\\{o_{i}\\}_{i=1}^{m}\cap\\{o^{*}_{j}\\}_{j=1}^{n})}{\textrm{count}(\\{o_{i}\\}_{i=1}^{m}\cup\\{o^{*}_{j}\\}_{j=1}^{n})}$
(11)

The object $o_{i}$ in the prediction specification is the same as object
$o^{*}_{j}$ in the ground-truth specification if and only if all the
properties of these objects match exactly.

#### Inference Reward

Our second reward, which we call the “inference reward”, is also a reward in
specification space. The name is based on the “inference error”, which is a
performance measure introduced in ? (?) for the Abstract Scenes dataset. While
IOU is based on exact matches between predicted objects and ground-truth
objects, the inference error and inference reward are based on the number of
properties (within objects) that correctly match the corresponding properties
in the ground-truth. For those properties specifying location in pixel
coordinates, we follow ? (?) and divide the space of each coordinate into 20
bins of equal size, and we consider it a match if the predicted and ground-
truth locations are in the same bin. We define the inference error as the
fraction of predicted properties that fail to match the corresponding ground-
truth properties. The inference reward is one minus the inference error.

#### Image Distance Reward

Our third and final reward, the “image distance reward”, is in image space. We
define it generically first, as it takes slightly different forms in our two
datasets. If we let $I$ and $I^{R}$ represent vectorized versions of the input
image and the image rendered from the predicted specification, respectively,
then we define the image distance as

$d_{img}=||I\ominus\Psi(I^{R})||^{2}_{2},$ (12)

where $||\cdot||_{2}$ is the $\ell_{2}$-norm.

For Noisy Shapes dataset, we follow ? (?) and take $\Psi$ to be a Gaussian
blurring function, as the objects in the target image have noise (see Fig. 1).
We take $\ominus$ to be a simple subtraction operation. The image reward for
this dataset is:

$r_{img}=\frac{c}{d_{img}}$ (13)

where $c$ is a tunable parameter.

For the Abstract Scene dataset, $\ominus$ is a logical operator that takes the
value $0$ in every position where the pixel values “match”, and $1$ in every
other position. The range of possible pixel values is 0-255 and, similarly to
the discretization of position in the inference reward, we divide the pixel
value range into 20 equisized buckets and consider pixel values to match if
they are in the same bucket. We take $\Psi$ to be the identity function. The
image reward for the Abstract Scenes dataset is then defined as:

$r_{img}=1-\frac{d_{img}}{w\cdot h}$ (14)

where $w$ and $h$ are width and height of the image.

#### Joint Dual-Modality Reward

Since we expect the rewards based in specification space to be complementary
to the reward based in image space, we want a way to combine rewards on the
two spaces. One way to combine two rewards is to create a weighted combination
of individual rewards to formulate the joint reward. Another approach is to
alternate the reward used during the learning process (?). In this work, we
follow the latter approach, as the former approach requires expensive tuning
for scale and weight balancing. Let $r_{1}$ and $r_{2}$ be the two reward
functions that we want to optimize. In our approach, we first take $a_{1}$
optimization steps to minimize the reinforcement learning loss
$L_{\text{RL}_{1}}(r_{1};\theta)$ (i.e. we use $a_{1}$ mini-batches). Then we
take $a_{2}$ optimization steps to minimize the reinforcement learning loss
$L_{\text{RL}_{2}}(r_{2};\theta)$. We then repeat this cycle of steps until
convergence. All other optimization parameters, such as step size, remain the
same for each set of steps. The values $a_{1}$ and $a_{2}$ are tuning
parameters.333? (?) set $a_{1}$ and $a_{2}$ to $1$, without tuning. The two
rewards $r_{1}$ and $r_{2}$ could be based on different aspects of the output,
such as IOU and image distance reward.

Model | Precision | Recall | F1 | IOU | $\text{IOU}_{\text{1.0}}$ | $\text{IOU}_{\text{0.8}}$ | $\text{IOU}_{\text{0.6}}$
---|---|---|---|---|---|---|---
Cross-Entropy Loss
Image2LSTM+atten. | 98.7 | 98.5 | 98.6 | 97.6 | 90.7 | 95.3 | 98.8
Image2Transformer | 99.1 | 99.1 | 99.1 | 98.5 | 94.1 | 97.3 | 99.1
Image2Transformer with Reinforce Loss
IOU Reward | 99.4 | 99.3 | 99.3 | 98.8 | 95.0 | 98.0 | 99.4
Image-distance Reward | 99.4 | 99.2 | 99.3 | 98.8 | 94.5 | 98.0 | 99.5
Image-distance + IOU Reward | 99.4 | 99.3 | 99.3 | 98.8 | 95.0 | 98.1 | 99.4
Table 1: Performance of various models on Noisy Shapes dataset.

##  5 Experimental Setup

###  5.1 Dataset

##### Noisy Shapes Dataset.

? (?) provides a synthetic dataset of images containing multiple simple
objects (lines, circles, and rectangles), each with various properties that
can be specified. The images are specified using a small subset of LaTeX
drawing commands. Additional noise is introduced into the rendered images by
rescaling image intensity, translating the image by a few pixels, rendering
the LaTeX using the $\operatorname{pencildraw}$ style, and randomly perturbing
the position and sizes of these LaTeX drawing commands. The dataset was
created by randomly sampling image specifications with between 1 and 12
objects, excluding any specifications that lead to images with overlapping
objects. The size of each image is 256x256. The dataset contains 100,000
images paired with specifications, from which we use 1000 for testing and the
rest for training.

##### Abstract Scene Dataset.

The Abstract Scene dataset (?) contains 10,020 images, each of which has 3-18
objects. There are over 100 types of objects, each of which is specified by
two integers, one indicating a broad category (e.g. sky object, animal, boy,
girl) and another indicating a subcategory (e.g. girl pose, animal type,
etc.). Each object can be drawn at one of 3 scales, with or without a
horizontal flip, and at any pixel location in the 500x400 image. These
properties are specified by 4 additional integers. Thus each object is
specified by 6 integers. There are often heavy occlusions among these objects
when rendered in an image (see input image in Fig. 2). However, the objects
are rendered in a deterministic order based on the object types and other
properties, and thus the image is independent of the order of the objects in
the specification. Similar to ? (?), we randomly sample 90% of the images for
training and rest for testing.

###  5.2 Evaluation Metrics

##### Noisy Shapes Dataset.

As described in the Task description of Sec. 3, we can summarize performance
on a single image with precision, recall, F1, and IOU (intersection over
union) at the object level. Following previous work (?), we summarize the
performance of a method by averaging these metrics across all test examples
(i.e. a macro average). Further, we also report $\text{IOU}_{k}$, which is
defined as the percent of test examples for which the IOU score is greater
than or equal to $k$.

##### Abstract Scene Dataset.

For the abstract scene dataset, following previous work (?), we report
specification inference error and image reconstruction error based on a micro
average across all test examples. As described in Sec. 4.1 and Sec. 4.1,
inference error is based on the percentage of incorrectly inferred values
(i.e., how many properties of objects do not match with the ground-truth) for
the specification, and image reconstruction error is based on percentage of
incorrect pixel prediction. During these evaluations, all the continuous
variables (pixel values, and x and y coordinates) are quantized into 20 bins.
Additionally, we report the macro average based IOU error as described for the
noisy shapes dataset.

###  5.3 Training Details

In all of our models, we encode the image information via ResNet-18 (?), where
we take the penultimate layer’s features as outputs from this image encoder.
For LSTM-RNN, we use a hidden state size of 128, input token embedding size of
128, and a batch size of 64. For Transformer networks, we use the same hidden
and embedding size, and use 4 layers at each time step. We use the Adam
optimizer (?) with the default learning rate of 0.001 for all the cross-
entropy models, and a learning rate of 0.0001 for all the reinforcement
learning based models. For the Noisy Shapes dataset, the maximum decoder
length is fixed to 80, and we use a vocabulary size of 27, which are
placeholders for object properties. For the Abstract Scene dataset, the
maximum decoder length is fixed to 100, and we use a vocabulary size of 1078
which represents all the object properties. For the joint reward optimization,
we use a mixing ratio of 1:1 for the Noisy Shapes dataset and 1:4 for the
Abstract Scene dataset.

##  6 Results

###  6.1 Results on the Noisy Shapes Dataset

We first compare the performance of the LSTM-RNN model (Image2LSTM+atten) to
the Transformer-based model, when both are trained with cross-entropy loss. We
see in Table 1 that the Transformer model dominates on all measures. In
particular, we highlight $\text{IOU}_{\text{1.0}}$, which measures the percent
of examples on which the predicted specification exactly matches the ground-
truth specification. While the LSTM-RNN model achieves a $90.7\%$
$\text{IOU}_{\text{1.0}}$, the Transformer model achieves $94.1\%$, which is
an impressive $36.5\%$ reduction in the number of errors. We have similar
performance improvements for the other metrics. We now compare the Transformer
model trained with reinforcement learning, using various reward functions, to
training using cross-entropy loss. Table 1 shows that, although all three
reward variations have roughly the same performance, they all show significant
improvement over cross-entropy training, on all measures.444The improvement of
our Transformer models trained with reinforcement learning over the
corresponding cross-entropy models is statistically significant with $p<0.01$,
based on the bootstrap test (?; ?).  For example, the model trained with IOU
reward achieved a $95.0\%$ $\text{IOU}_{\text{1.0}}$ measure, which is an
impressive $15.3\%$ reduction in the number of errors compared to the same
model trained with cross-entropy loss, and a $46.2\%$ reduction compared to
the original LSTM-RNN model. Performance improvement in the other measures is
at least as good.

Model |  Infer. Error |  Recons. Error |  Avg. Error |  IOU
---|---|---|---|---
Previous Work
CNN+LSTM (?) |  45.31 |  41.38 |  43.84 |  -
NSD (full) (?) |  42.74 |  21.55 |  32.14 |  -
Cross-Entropy Loss
Image2LSTM+atten. |  17.27 |  15.70 |  16.48 |  32.06
Image2Transformer |  8.78 |  10.92 |  9.85 |  58.54
Image2Transformer with Reinforce Loss
IOU Reward |  7.91 |  10.50 |  9.20 |  61.29
Inference Reward |  7.81 |  10.75 |  9.28 |  59.35
Recons. Reward |  8.34 |  9.99 |  9.16 |  62.44
Inference + Recons. |  8.21 |  10.12 |  9.16 |  61.54
IOU + Recons. |  8.05 |  10.04 |  9.04 |  62.45
Table 2: Models performance on the abstract scene dataset. Errors: lower is
better; IOU: higher is better.

###  6.2 Results on Abstract Scene Dataset

In Table 2, we see the performance of various models on the Abstract Scene
dataset, for the metrics described in Sec. 5.2. We first note that even our
baseline LSTM-RNN model (Image2LSTM+atten) shows a very large error reduction
compared to the results presented in ? (?) (first 4 rows of the table). This
highlights the importance of an attention mechanism in these tasks. For the
models trained with cross-entropy, the Transformer model shows an additional
remarkable improvement over the LSTM-RNN model, across all measures.

For reinforcement learning with the Transformer model, we tried three
different reward functions, corresponding to three of our performance metrics:
inference error, reconstruction error, and IOU. All the Transformer models
trained with REINFORCE out-performed the model trained with cross-entropy loss
for each of the error measures.555For the IOU and inference reward models,
this improvement is statistically significant for all metrics except
reconstruction error. For the reconstruction reward model, the improvement is
significant for all but the inference error metric. For the dual (IOU+Recons.)
reward model, the difference is significant for all metrics ($p<0.01$ for each
test).  For inference error, the model trained with the inference reward did
the best, as one might hope and expect. Compared to the cross-entropy trained
Transformer, the inference error measure was reduced by $11.0\%$. For
reconstruction error (image-based), the best performing model was the model
trained with the reconstruction reward, which reduced the reconstruction error
by $8.5\%$ compared to the cross-entropy trained version. When evaluating
performance using the average of the inference and reconstruction error, one
of our joint-reward models performed best, though interestingly, not the one
that uses the corresponding inference and reconstruction rewards. The best
performing model for this performance measure used IOU and reconstruction
rewards, suggesting that IOU reward has more information that is complementary
to the reconstruction error than does the inference reward. For IOU
performance measure, the model trained with IOU reward did well, but when
trained jointly with IOU and reconstruction reward, it performed the best.
This suggests that using image-based feedback during training (recons. error)
can be beneficial even when the ultimate goal (IOU) depends only on the
specification output.

##  7 Analysis

###  7.1 LSTM vs. Transformer Networks

As noted above, for the Abstract Scene and the Noisy Shapes datasets that we
consider, the order of the objects in the specification does not affect the
final image. Nevertheless, for training both the LSTM-RNN and the Transformer
models, one must choose an ordering. We ran an experiment using the Noisy
Shapes dataset, in which we tried ordering the objects by shape size, shape
type, and by shape position in the rendered image. We found that ordering by
shape type worked best across our models, so that’s what we used for our main
results in Table 1. We also wanted to investigate how important it is to have
the objects in some sensible order, compared to a random ordering. Table 3
shows the results of our two models when trained with cross-entropy on
specification sequences where the objects are put in random order. We find
that the LSTM-RNN model performance drops dramatically (e.g.
$\text{IOU}_{\text{1.0}}$ drops from $90.7\%$ to $72.0\%$), while the drop
with Transformer networks is quite small (e.g. $\text{IOU}_{\text{1.0}}$ drops
from $94.1\%$ to $93.2\%$.666Note that the number of model parameters is
approximately the same (11.8M for Transformer model and 11.5M for LSTM model).
Further, Transformer models are 2.5x faster to train in comparison to the LSTM
models. During inference, both models take approximately the same time. This
is additional evidence for Transformers being the preferred model for tasks of
this type.

###  7.2 Performance vs. Data Size

We conduct an experiment where we vary the percentage of Noisy Shapes data
used during our models’ training from $10\%$ to $100\%$ by steps of $20\%$. We
observe that with less data ($10\%$-$40\%$), the RL-based model is
approximately $2$ points better (on the $\text{IOU}_{\text{1.0}}$ metric) than
its corresponding cross-entropy baseline. As we use more data (${>}60\%$), the
gap decreases to $1$ point between RL and cross-entropy models. This suggests
that RL, which has the advantage of exploration, is more powerful when the
data is less.

Model | F1 | IOU | $\text{IOU}_{\text{1.0}}$
---|---|---|---
Image2LSTM+atten. | 95.2 | 92.0 | 72.0
Image2Transformer | 99.0 | 98.3 | 93.2
Table 3: Performance of LSTM-RNN and Transformer networks on the Noisy Shapes
dataset when specifications have randomly ordered objects.

###  7.3 Output Examples

Fig. 4 presents the output rendered images of the predicted specifications
from Image2Transformer cross-entropy model and the corresponding RL-based
model with IOU+Image-distance as reward for noisy shapes dataset and
IOU+Recons. as reward for abstract scene dataset. In the first example (top
row in Fig. 4), the cross-entropy model predicts an extra ‘line shape’ which
is not present in the ground-truth. Our RL model correctly predicts the exact
same shapes present in the ground-truth. However, neither models getting the
type of ‘line shape’ correct in couple of instances. In the second example
(second row in Fig. 4), the cross-entropy model predicts an extra object
(glasses), which is not present in the ground-truth image, and is also missing
the cap on the snake. The RL model improves on the cross-entropy model by not
having any extra objects, but it is also missing the cap. In the third
example, both the rendered images look very similar to the ground-truth, but
the cross-entropy model predicts one of the objects (glasses) slightly off in
position. Our RL model was able to accurately position the glasses (bottom row
in Fig. 4). The better performance of RL model may be due to the image space
component of the error signal, which is more sensitive to position errors,
while the cross-entropy loss gives the same penalty to all incorrect positions
regardless of the error size.

Figure 4: Comparing the ground-truth images from noisy shapes and abstract
scene datasets with the rendered images of predicted specifications.

##  8 Conclusion

We present various neural de-rendering models based on LSTMs with attention
mechanism and Transformer networks. Further, we introduce complimentary dual
rewards (one in specification space and another in image space) and optimize
them via reinforcement learning, and achieve state-of-the-art results.
Further, our results and analyses suggest that Transformers are a better
choice than LSTMs for unordered sequence prediction tasks.

## Acknowledgments

We thank the reviewers for their helpful comments. This work was partially
supported by NSF-CAREER Award 1846185, ARO-YIP Award W911NF-18-1-0336, and a
Microsoft PhD Fellowship. The views contained in this article are those of the
authors and not of the funding agency.

## References

  * [2015] Bengio, S.; Vinyals, O.; Jaitly, N.; and Shazeer, N.  2015\.  Scheduled sampling for sequence prediction with recurrent neural networks.  In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, NeurIPS’15, 1171–1179.  Cambridge, MA, USA: MIT Press.
  * [2016] Bluche, T.; Louradour, J.; and Messina, R. O.  2016\.  Scan, attend and read: End-to-end handwritten paragraph recognition with MDLSTM attention.  CoRR abs/1604.03286.
  * [2018] Bunel, R.; Hausknecht, M.; Devlin, J.; Singh, R.; and Kohli, P.  2018\.  Leveraging grammar and reinforcement learning for neural program synthesis.  In ICLR.  OpenReview.net.
  * [2015] Chen, X.; Fang, H.; Lin, T.-Y.; Vedantam, R.; Gupta, S.; Dollár, P.; and Zitnick, C. L.  2015\.  Microsoft COCO captions: Data collection and evaluation server.  CoRR abs/1504.00325.
  * [2017] Cliche, M.; Rosenberg, D.; Madeka, D.; and Yee, C.  2017\.  Scatteract: Automated extraction of data from scatter plots.  In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 135–150.  Springer.
  * [2009] Daumé, H.; Langford, J.; and Marcu, D.  2009\.  Search-based structured prediction.  Machine Learning 75(3):297–325.
  * [2017] Deng, Y.; Kanervisto, A.; Ling, J.; and Rush, A. M.  2017\.  Image-to-markup generation with coarse-to-fine attention.  In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, 980–989.  JMLR.org.
  * [1994] Efron, B., and Tibshirani, R. J.  1994\.  An introduction to the bootstrap.  Number 57 in Monographs on Statistics and Applied Probability. Boca Raton, Florida, USA: Chapman & Hall/CRC.
  * [2018] Ellis, K.; Ritchie, D.; Solar-Lezama, A.; and Tenenbaum, J. B.  2018\.  Learning to infer graphics programs from hand-drawn images.  In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NeurIPS’18, 6062–6071.  Red Hook, NY, USA: Curran Associates Inc.
  * [2018] Ganin, Y.; Kulkarni, T.; Babuschkin, I.; Eslami, S. M. A.; and Vinyals, O.  2018\.  Synthesizing programs for images using reinforced adversarial learning.  In Proceedings of the 35th International Conference on Machine Learning.
  * [2018] Ha, D., and Eck, D.  2018\.  A neural representation of sketch drawings.  In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.  OpenReview.net.
  * [2016] He, K.; Zhang, X.; Ren, S.; and Sun, J.  2016\.  Deep residual learning for image recognition.  In CVPR, 770–778.
  * [2016] Huang, H.; Kalogerakis, E.; Yumer, E.; and Mech, R.  2016\.  Shape synthesis from sketches via procedural models and convolutional networks.  IEEE Transactions on Visualization and Computer Graphics 2\.
  * [2015] Karpathy, A., and Fei-Fei, L.  2015\.  Deep visual-semantic alignments for generating image descriptions.  In Proceedings of the IEEE conference on computer vision and pattern recognition, 3128–3137.
  * [2015] Kingma, D. P., and Ba, J.  2015\.  Adam: A method for stochastic optimization.  In Bengio, Y., and LeCun, Y., eds., 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
  * [2014] Lin, T.; Maire, M.; Belongie, S. J.; Bourdev, L. D.; Girshick, R. B.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C. L.  2014\.  Microsoft COCO: common objects in context.  CoRR abs/1405.0312.
  * [2019] Liu, Y.; Wu, Z.; Ritchie, D.; Freeman, W. T.; Tenenbaum, J. B.; and Wu, J.  2019\.  Learning to describe scenes with programs.  In ICLR.
  * [2011] Mishchenko, A., and Vassilieva, N.  2011\.  Chart image understanding and numerical data extraction.  In 2011 Sixth International Conference on Digital Information Management, 115–120.  IEEE.
  * [2016] Nishida, G.; Garcia-Dorado, I.; Aliaga, D. G.; Benes, B.; and Bousseau, A.  2016\.  Interactive sketching of urban procedural models.  ACM Transactions on Graphics (TOG) 35(4):130.
  * [1989] Noreen, E. W.  1989\.  Computer-intensive methods for testing hypotheses.  Wiley New York.
  * [2018] Pasunuru, R., and Bansal, M.  2018\.  Multi-reward reinforced summarization with saliency and entailment.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), 646–653.
  * [2018] Paulus, R.; Xiong, C.; and Socher, R.  2018\.  A deep reinforced model for abstractive summarization.  In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.  OpenReview.net.
  * [2016] Ranzato, M.; Chopra, S.; Auli, M.; and Zaremba, W.  2016\.  Sequence level training with recurrent neural networks.  In ICLR.
  * [2017] Rennie, S. J.; Marcheret, E.; Mroueh, Y.; Ross, J.; and Goel, V.  2017\.  Self-critical sequence training for image captioning.  In IEEE Conference on Computer Vision and Pattern Recognition, 1179–1195.
  * [2018] Sharma, P.; Ding, N.; Goodman, S.; and Soricut, R.  2018\.  Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning.  In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2556–2565.
  * [2018] Sun, S.-H.; Noh, H.; Somasundaram, S.; and Lim, J.  2018\.  Neural program synthesis from diverse demonstration videos.  In International Conference on Machine Learning, 4797–4806.
  * [2017] Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I.  2017\.  Attention is all you need.  In NeurIPS, 5998–6008.
  * [2015] Vinyals, O.; Bengio, S.; and Kudlur, M.  2015\.  Order matters: Sequence to sequence for sets.  In ICLR.
  * [1992] Williams, R. J.  1992\.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  Machine learning 8(3-4):229–256.
  * [2017] Wu, J.; Tenenbaum, J. B.; and Kohli, P.  2017\.  Neural scene de-rendering.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 699–707.
  * [2015] Xu, K.; Ba, J.; Kiros, R.; Cho, K.; Courville, A.; Salakhudinov, R.; Zemel, R.; and Bengio, Y.  2015\.  Show, attend and tell: Neural image caption generation with visual attention.  In ICML, 2048–2057.
  * [2015] Zaremba, W., and Sutskever, I.  2015\.  Reinforcement learning neural turing machines.  arXiv preprint arXiv:1505.00521.
  * [2018] Zhou, L.; Zhou, Y.; Corso, J. J.; Socher, R.; and Xiong, C.  2018\.  End-to-end dense video captioning with masked transformer.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8739–8748.
  * [2013] Zitnick, C. L., and Parikh, D.  2013\.  Bringing semantics into focus using visual abstraction.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3009–3016.


# Upgrading the Inner Tracking System and the Time Projection Chamber of ALICE

Felix Reidt
on behalf of the ALICE collaboration  CERN, 1211 Geneva 23, Switzerland

###### Abstract

The ALICE experiment at CERN is undergoing a major upgrade during the Long
Shutdown 2 (LS2) of the LHC during 2019-2020. The key elements regarding the
central barrel are the installation of a new Inner Tracking System (ITS) and
the upgrade of the large Time Projection Chamber (TPC). The TPC is currently
being upgraded with a new readout system, including new GEM-based Readout
Chambers and new front-end electronics enabling us to operate it in continuous
mode. The new ITS based on CMOS Monolithic Active Pixel Sensors will
significantly improve the impact parameter resolution and the tracking
efficiency, especially for particles with low transverse momenta, as well as
the readout-rate capability. In the following, the upgrades and their assembly
and commissioning status will be outlined.

Furthermore, an outlook will be given on plans for the upgrade of the ITS
foreseen for the third long shutdown of the LHC in 2025-2026. For this
upgrade, the three innermost layers could be replaced by cylindrical detector
layers, made of curved wafer-scale-sized CMOS sensors, to lower even further
the material budget and to improve the pointing resolution.

###### keywords:

TPC , GEM , Silicon Pixel Detector , MAPS

##  1 Introduction

The ALICE experiment [1] is upgrading its experimental apparatus during the
currently ongoing Long Shutdown 2 (LS2) of the LHC. ALICE will significantly
improve its sensitivity to rare probes by fully recording Pb–Pb collision data
at 50 kHz interaction rate achievable after the accelerator upgrade, and
increasing the minimum-bias data sample by about a factor of 100. Furthermore,
the increased vertexing performance will allow for a significantly more
effective identification and reconstruction of heavy-flavour hadron decays.

During LS2, the key upgrade elements for the central barrel are the
installation of a new Inner Tracking System (ITS2) [2] and the upgrade of the
large Time Projection Chamber (TPC) [3]. The ITS2 is equipped with Monolithic
Active Pixel Sensors (MAPS). For the TPC, the Multi-Wire Proportional Chambers
(MWPCs) readout were replaced by Gas Electron Multiplier (GEM) chambers. In
the following, the current status of commissioning of the two detector systems
will be outlined.

The next step will be the R&D for the Inner Tracking System 3 (ITS3) [4],
planned for the Long Shutdown 3 (LS3) foreseen for 2025-2026. This novel
detector will be based on 3 bent silicon layers reducing the material budget
even further. The new detector concept will be briefly introduced in Sec. 4.

##  2 Time Projection Chamber Upgrade

The TPC [3] is the main tracking and Particle IDentification (PID) device of
ALICE. Previously, the TPC was read out with MWPCs. A gating grid was used to
prevent ion backflow, which limited its readout rate to a few kHz. In order to
be able to record all Pb-Pb interactions at a rate of 50 kHz, the MWPCs were
replaced by GEMs. The new ReadOut Chambers (ROCs) will allow for a continuous
readout and hence a full reconstruction of all collisions.

Operation without a gating grid will lead to a substantial accumulation of
space charge in the drift volume of the TPC, resulting in space charge
distortions. In order to keep them to a minimum, a special configuration based
on 4-GEM stacks with different hole pitches has been adopted: two large-pitch
(280 $\mu$m) GEM foils, embraced by two standard-pitch (140 $\mu$m) GEM foils
[5, 6]. All ROCs including spares arrived at CERN in early 2019 and were
tested for stability using X-ray irradiation or particles in the ALICE
experimental cavern.

The TPC has been moved to surface for the installation of the new GEM ROCs in
March 2019. During the course of 2019, the MWPC ROCs as well as the
corresponding readout electronics and services have been deinstalled and the
GEM ROCs along with the corresponding readout electronics have been installed.
In December 2019, the pre-commissioning on surface with cosmic muons, laser,
pulser and X-ray irradiation will start. This will allow us to validate the
ROCs before the detector will be moved in the experimental cavern in March
2020.

In Fig. 1, the noise maps of an Inner ROC (IROC, left) and Outer ROC (OROC,
middle) and the corresponding noise distributions (right) are shown. The
average noise on a given TPC sector is 0.93 and 1.01 ADC counts for the IROC
and OROC, respectively. The tails are due to different path lengths from the
readout pad to the front-end electronics leading to different capacitances.
The overall performance is in line with the target value of 1 ADC count of
noise corresponding to 670 $e^{-}$.

Fig. 1: Noise performance of TPC readout chambers on one TPC sector. From left
to right: IROC noise map, OROC noise map, IROC noise distribution (blue, solid
line) and OROC noise distribution (red, dashed line).

##  3 Inner Tracking System Upgrade - ITS2

The ITS2 will fully replace the previous ITS. It is based on Monolithic Active
Pixel Sensors and has a total surface of 10 m2. It consists of 7
concentrically cylindrical layers ranging from radii of $2.3$ cm to 40 cm,
with a material budget of $0.35\%$ $X_{0}$ and $1\%$ $X_{0}$ for the innermost
3 and outermost 4 layers, respectively. Layers 0 to 2 form together the Inner
Barrel (IB) and layers 3 to 6 the Outer Barrel (OB). The OB is furthermore
subdivided into Middle Layers (MLs), comprising layers 3 and 4, and Outer
Layers (OLs), comprising layers 5 and 6.

The key building block is the ALPIDE Monolithic Active Pixel Sensor [7, 8],
which features in-pixel amplification, discrimination and multiple hit buffers
as well as in-matrix data sparsification. For the IB, modules consisting of 9
chips, thinned down to 50 $\mu$m, form a stave. For the OB, modules of 14
chips with a thickness of 100 $\mu$m each are assembled into staves. A ML
stave consist of two half-staves of 4 modules each. An OL stave, consist of a
total of 14 modules distributed over 2 half-staves. The full ITS2 consists of
48 IB, 54 ML and 90 OL staves.

From the end of 2016 until the end of 2019, a total of 72000 ALPIDE sensors
have been produced and tested. The overall chip production yield amounts to
$63.9\%$. For the IB modules, only the best category of chips has been used,
which has a yield of $30.4\%$. A total of about 3000 modules have been
produced reaching a yield of $71.5\%$ and $84.2\%$ for IB and OB modules,
respectively. During the second half of 2018 until fall 2019, all seven
detector layers have been assembled and installed in the half-barrels in an
assembly clean room at CERN .

Furthermore, all detector services including cooling plant, power
distribution, readout electronics and computing system have been installed in
and around the assembly clean room to allow for full commissioning of the
detector on surface. As first results, fake-hit rates and threshold
measurements have been carried out. In Fig. 2, left, the fake-hit rate for 54
chips in half-layer 0 is shown to be below $10^{-10}$ fake hits per pixel and
event when masking only about 50 out of $28\times 10^{6}$ pixels. This is
significantly better than the required $10^{-6}$ fake hits per pixel and
event. In the right half of Fig. 2, a threshold map for an IB Half-Barrel
after tuning the threshold to 10 DAC counts corresponding to 100 $e^{-}$ is
shown. The overall uniformity is excellent and the tuning procedure has been
validated. Furthermore, first cosmic-ray tracks have been observed in the
detector.

Fig. 2: Results from the ITS2 IB commissioning: fake-hit rate of 54 chips
(left), threshold map of an Inner Barrel Half-Barrel after tuning (right).

##  4 Inner Tracking System Upgrade - ITS3

The ITS2 material budget is largely dominated by support and electrical
substrate (cf. Fig. 3, left) and only about $15\%$ is owing to the silicon
itself. Furthermore, the material distribution is very irregular due to the
overlap of staves, the cooling pipes, as well as the support structures. The
novel concept of the ITS3 [4] is the introduction of 3 layers of curved
silicon sensors (cf. Fig. 3, right), which should replace the 3 ITS2 IB
layers. A reduction of the power consumption below 20 $\textrm{mW/cm}^{2}$ for
the active area will allow to rely on air cooling. The mechanical support can
be removed by exploiting the increased stiffness of the rolled silicon wafers.
The electrical substrate will be integrated directly into the chip, which
would span a major fraction of a layer.

The main building block of the ITS3 is an ultra-thin (20 $\mu$m to 40 $\mu$m),
wafer-scale (300 mm) CMOS Monolithic Active Pixel Sensor whose dimensions
reach up to 280 mm by 94 mm. Owing to the flexibility of silicon at such
thicknesses, wafer-size sensors will be bent into half-cylinders of radii of
18, 24, and 30 mm, respectively, to form the new concentric layers. The power
intensive periphery of such scale sensors will placed outside the fiducial
volume, where traditional cooling can be implemented.

The new detector will go together with a beam pipe with smaller diameter as
well as reduced wall thickness. The total material up to a radius of 4 cm will
reduce by about a factor of 3 and it will be more homogeneously distributed.
This should allow for a reduction of the systematic uncertainty for those
measurements that require a precise description of the material distribution.

The ITS3 is expected to improve the pointing resolution by a factor of 2
compared to ITS2 and further improve the tracking efficiency at low momenta
with a subsequent improvement of the physics performance for heavy-flavour
baryon and low-mass dielectron measurements.

Fig. 3: Material budget of an ITS2 IB layer (left), schematic drawing of ITS3
(right)

##  5 Summary and Outlook

The preparation of the upgrades of TPC and ITS during LS2 is on track for
operation in 2021. The TPC ROCs and electronics perform as expected. Results
from the ITS2 on-surface commissioning are confirming the expected
performance. Both TPC and ITS2 are set to run at 50 kHz in Run 3.
The proposed ITS3 detector will have unprecedented performance approaching the
limit of a massless detector. A major part of the ALICE physics program will
benefit from it and new channels are expected to become accessible.

## References

  * [1] The ALICE Collaboration, The ALICE experiment at the CERN LHC, JINST 3 (2008) S08002.  doi:10.1088/1748-0221/3/08/S08002.
  * [2] The ALICE Collaboration, Technical Design Report for the Upgrade of the ALICE Inner Tracking System, J. Phys. G41 (2014) 087002.  doi:10.1088/0954-3899/41/8/087002.
  * [3] The ALICE Collaboration, Upgrade of the ALICE Time Projection Chamber, Tech. Rep. CERN-LHCC-2013-020. ALICE-TDR-016, CERN, Geneva (2013).
URL http://cds.cern.ch/record/1622286

  * [4] The ALICE Collaboration, Letter of Intent for an ALICE ITS Upgrade in LS3, Tech. Rep. CERN-LHCC-2019-018. LHCC-I-034, CERN, Geneva (2019).
URL https://cds.cern.ch/record/2703140

  * [5] A. Mathis, From gated to continuous readout - the GEM upgrade of the ALICE TPC, PoS MPGD2017 (2019) 055.  arXiv:1803.04140, doi:10.22323/1.322.0055.
  * [6] P. Gasik, Discharge probability studies with multi-GEM detectors for the ALICE TPC Upgrade, PoS MPGD2017 (2019) 031.  arXiv:1807.02979, doi:10.22323/1.322.0031.
  * [7] G. Aglieri Rinella, The ALPIDE pixel sensor chip for the upgrade of the ALICE Inner Tracking System, Nucl. Instrum. Meth. A845 (2017) 583–587.  doi:10.1016/j.nima.2016.05.016.
  * [8] P. Martinengo, The new Inner Tracking System of the ALICE experiment, Nucl. Phys. A967 (2017) 900–903.  doi:10.1016/j.nuclphysa.2017.05.069.


######  Theorem 15 ( ).

The algebra $(V^{n},\sup^{n})$ is initial: given any other
$\operatorname{T}^{n}_{U}$-algebra $(X,m)$ the type of algebra homomorphisms
from $(V^{n},\sup^{n})$ to $(X,m)$ is contractible.

###### Proof.

The center of contraction is given by
$\left(\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m),\operatorname{refl\mbox{-}htpy}\right)$.
We will use the characterisation of equality between
$\operatorname{T}^{n}_{U}$-algebra homomorphisms given by Lemma 8 to show that
any other homomorphism is equal to
$\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)$.

Let $(\phi,\alpha)$ be another $\operatorname{T}^{n}_{U}$-algebra homomorphism
from $(V^{n},\sup^{n})$ to $(X,m)$. We need to do two things:

  * •

Construct a homotopy
$\epsilon:\phi\sim\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)$.

  * •

For each $(A,f):\operatorname{T}^{n}_{U}\,V^{n}$, construct a path

$\alpha\,(A,f)\cdot\operatorname{ap}_{m}\left(\operatorname{T}^{n}_{U}\epsilon\,(A,f)\right)=\epsilon\left(\operatorname{sup}^{n}\,(A,f)\right)\cdot\operatorname{refl\mbox{-}htpy}\,(A,f).$

To construct a homotopy from $\phi$ to
$\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)$ we use the
elimination principle on $V^{n}$, by Proposition 28, with the type family
$P\,x:=\left(\phi\,x=\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)\,x\right)$.
This means that given $A:U$, $f:A\hookrightarrow_{n-1}V^{n}$ and
$H:\prod_{a:A}\phi\,(f\,a)=\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)\,(f\,a)$,
we need to construct a path

$\displaystyle\phi\,(\operatorname{sup}^{n}\,(A,f))=\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)\,(\operatorname{sup}^{n}\,(A,f))$

We have the following chain of equalities:

$\displaystyle\phi\,(\operatorname{sup}^{n}\,(A,f))$
$\displaystyle=m\,(\operatorname{T}^{n}_{U}\phi\,(A,f))$ (111)
$\displaystyle\equiv m\,(\operatorname{image}_{n-1}(\phi\circ
f),\operatorname{incl}_{n-1}(\phi\circ f))$ (112)
$\displaystyle=m\,(\operatorname{image}_{n-1}(\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)\circ
f),\operatorname{incl}_{n-1}(\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)\circ
f))$ (113) $\displaystyle\equiv
m\,(\operatorname{T}^{n}_{U}(\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m))\,(A,f))$
(114)
$\displaystyle\equiv\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)\,(\operatorname{sup}^{n}\,(A,f))$
(115)

Step (111) is the path $\alpha\,(A,f)$ and step (113) is the path
$\operatorname{ap}_{\lambda\,h.m\,(\operatorname{image}_{n-1}h,\operatorname{incl}_{n-1}h)}(\operatorname{funext}\,H)$.
So let

$\displaystyle\sigma:=\lambda\,A\,f\,H.\alpha\,(A,f)\cdot\operatorname{ap}_{\lambda\,h.m\,(\operatorname{image}_{n-1}h,\operatorname{incl}_{n-1}h)}(\operatorname{funext}\,H)$

then by Proposition 28 we have a homotopy $\operatorname{elim}_{V^{n}}\,P\
\sigma:\phi\sim\operatorname{T}^{n}\operatorname{\mbox{-}rec}_{V^{n}}\,(X,m)$.

It remains to construct the second component of the $\Sigma$-type in Lemma 8.
We have the following chain of equalities:

$\displaystyle\alpha\,(A,f)$
$\displaystyle\cdot\operatorname{ap}_{m}\left(\operatorname{T}^{n}_{U}\left(\operatorname{elim}_{V^{n}}\,P\
\sigma\right)\,(A,f)\right)$ (116)
$\displaystyle\equiv\alpha\,(A,f)\cdot\operatorname{ap}_{m}\left(\operatorname{ap}_{\lambda\,\psi.\operatorname{T}^{n}_{U}\psi(A,f)}\left(\operatorname{funext}\,\left(\operatorname{elim}_{V^{n}}\,P\
\sigma\right)\right)\right)$ (117)
$\displaystyle=\alpha\,(A,f)\cdot\operatorname{ap}_{m}\left(\operatorname{ap}_{\lambda\,h.(\operatorname{image}_{n-1}h,\operatorname{incl}_{n-1}h)}\left(\operatorname{ap}_{-\circ
f}\left(\operatorname{funext}\,\left(\operatorname{elim}_{V^{n}}\,P\
\sigma\right)\right)\right)\right)$ (118)
$\displaystyle=\alpha\,(A,f)\cdot\operatorname{ap}_{\lambda\,h.m\,(\operatorname{image}_{n-1}h,\operatorname{incl}_{n-1}h)}\left(\operatorname{ap}_{-\circ
f}\left(\operatorname{funext}\,\left(\operatorname{elim}_{V^{n}}\,P\
\sigma\right)\right)\right)$ (119)
$\displaystyle=\alpha\,(A,f)\cdot\operatorname{ap}_{\lambda\,h.m\,(\operatorname{image}_{n-1}h,\operatorname{incl}_{n-1}h)}\left(\operatorname{funext}\,\left(\left(\operatorname{elim}_{V^{n}}\,P\
\sigma\right)\circ f\right)\right)$ (120)
$\displaystyle\equiv\sigma\,A\,f\,\left(\left(\operatorname{elim}_{V^{n}}\,P\
\sigma\right)\circ f\right)$ (121)
$\displaystyle=\operatorname{elim}_{V^{n}}\,P\
\sigma\left(\operatorname{sup}^{n}\,(A,f)\right)$ (122)
$\displaystyle=\operatorname{elim}_{V^{n}}\,P\
\sigma\left(\operatorname{sup}^{n}\,(A,f)\right)\cdot\operatorname{refl\mbox{-}htpy}\,(A,f)$
(123)

In steps (118) and (119) we use the fact that $\operatorname{ap}$ and function
composition commute. Step (120) uses the fact that function extensionality
respects precomposition. Finally, step (122) is the computation rule for
$\operatorname{elim}_{V^{n}}$.∎

###  6.4 Properties

Since $V^{n}$ is a fixed-point for $\operatorname{T}^{n}_{U}$, the induced
$\in$-structure satisfies the properties as shown in Section 5. But as $V^{n}$
is the initial algebra for $\operatorname{T}^{n}_{U}$, it also satisfies
foundation, which is not true of all fixed-points for
$\operatorname{T}^{n}_{U}$. More explicitly, we list the properties which
$(V^{n},\in^{n})$ satisfies.

######  Theorem 16 ( ).

For $n:\mathbb{N}^{\infty}$, the $\in$-structure $(V^{n},\in^{n})$ satisfies
the following properties:

  * •

empty set,

  * •

$U$-restricted $n$-separation,

  * •

$\infty$-unordered $I$-tupling, for all $k:\mathbb{N}_{-1}$ such that $k<n$
and $k$-truncated types $I:U$,

  * •

$k$-unordered $I$-tupling, for all $k:\mathbb{N}_{-1}$ such that $k\leq n$ and
$I:U$,

  * •

$k$-replacement, for all $k:\mathbb{N}_{-1}$ such that $k\leq n$,

  * •

$k$-union, for all $k:\mathbb{N}_{-1}$ such that $k\leq n$,

  * •

exponentiation, for any ordered pairing structure,

  * •

natural numbers for any $(n-1)$-truncated representation,

  * •

foundation.

###### Proof.

The first two properties follow directly from Theorem 4 and Theorem 5
respectively since $V^{n}$ is a fixed-point for $\operatorname{T}^{n}_{U}$.
The next four properties follow from Theorem 8, Theorem 6, Theorem 9 and
Theorem 10 respectively, together with the fact that $V^{n}$ is locally small
(Proposition 24), and thus $(k+1)$-small for all $k:\mathbb{N}$.

For exponentiation, we first note that $(V^{n},\in^{n})$ has an ordered
pairing structure by Theorem 7. By Theorem 11 it thus follows that
$(V^{n},\in^{n})$ has exponentiation.

For natural numbers, the result follows from Theorem 12, since $V^{n}$ is a
fixed-point for $\operatorname{T}^{n}_{U}$.

Lastly, we use the induction principle for $V^{n}$ to show that
$(V^{n},\in^{n})$ has foundation. Given $A:U$ and
$f:A\hookrightarrow_{n-1}V^{n}$ we need to construct an element of type

$\left(\prod_{a:A}\operatorname{Acc}\,(f\,a)\right)\rightarrow\operatorname{Acc}\,(\operatorname{sup}^{n}\,(A,f)).$

Therefore suppose we have $p:\prod_{a:A}\operatorname{Acc}\,(f\,a)$. We
construct the following element:

$\operatorname{acc}\,\left(\lambda\,y\,(a,q).\operatorname{tr}^{Acc}_{q}(p\,a)\right):\operatorname{Acc}\,(\operatorname{sup}^{n}\,(A,f)).$

It then follows by Proposition 28 that we have

$\prod_{x:V^{n}}\operatorname{Acc}\,x.$

∎

## References

  * [1] Andreas Abel, Guillaume Allais, Liang-Ting Chen, Jesper Cockx, Matthew Daggitt, Nils Anders Danielsson, Amélia Liao, Ulf Norell, Andrés Sicard-Ramírez, and others.  Agda.  URL: https://github.com/agda/agda.
  * [2] Peter Aczel.  The type theoretic interpretation of constructive set theory.  In A. MacIntyre, L. Pacholski, and J. Paris, editors, Logic Colloquium ’77, pages 55–66. North–Holland, Amsterdam-New York, 1978.
  * [3] John L. Bell.  Set theory: Boolean-valued models and independence proofs, volume 47 of Oxford Logic Guides.  Oxford University Press, 3rd edition, 2011.
  * [4] J. Daniel Christensen.  Non-accessible localizations, September 2021.  arXiv:2109.06670 [cs, math].  URL: http://arxiv.org/abs/2109.06670, doi:10.48550/arXiv.2109.06670.
  * [5] Rijke E.  The join construction.  ArXiv e-prints, 2017.  arXiv:1701.07538.
  * [6] Håkon Robbestad Gylterud.  From multisets to sets in homotopy type theory.  The Journal of Symbolic Logic, 83(3):1132$\textendash$1146, 2018.  doi:10.1017/jsl.2017.84.
  * [7] Håkon Robbestad Gylterud.  Multisets in type theory.  Mathematical Proceedings of the Cambridge Philosophical Society, page 1$\textendash$18, 2019.  doi:10.1017/S0305004119000045.
  * [8] Gerhard Osius.  Categorical set theory: a characterization of the category of sets.  Journal of Pure and Applied Algebra, 4(1):79–119, 1974.
  * [9] Ladislav Rieger.  A contribution to Gödel’s axiomatic set theory, I.  Czechoslovak Mathematical Journal, 07(3):323–357, 1957.  Publisher: Institute of Mathematics, Academy of Sciences of the Czech Republic.  URL: https://eudml.org/doc/11888.
  * [10] Egbert Rijke, Elisabeth Stenholm, Jonathan Prieto-Cubides, Fredrik Bakke, and others.  The agda-unimath library.  URL: https://github.com/UniMath/agda-unimath/.
  * [11] Paul Taylor.  Well founded coalgebras and recursion, 2023.  URL: http://www.paultaylor.eu/ordinals/welfcr.pdf.
  * [12] The Univalent Foundations Program.  Homotopy Type Theory: Univalent Foundations of Mathematics.  homotopytypetheory.org, Institute for Advanced Study, 2013.  URL: http://homotopytypetheory.org/book.