α$\alpha$ β$\beta$

# Performance of the Large Hadron Collider cleaning system during the squeeze:
simulations and measurements

S. Tygier  The University of Manchester, Oxford Road, Manchester M13 9PL, UK
<EMAIL_ADDRESS>The Cockcroft Institute, Daresbury, WA4 4AD, UK
R.B. Appleby  The University of Manchester, Oxford Road, Manchester M13 9PL,
UK  The Cockcroft Institute, Daresbury, WA4 4AD, UK  R. Bruce  CERN, Geneva,
Switzerland  D. Mirarchi  CERN, Geneva, Switzerland  S. Redaelli  CERN,
Geneva, Switzerland  A. Valloni  The University of Manchester, Oxford Road,
Manchester M13 9PL, UK  CERN, Geneva, Switzerland

###### Abstract

The Large Hadron Collider (LHC) at CERN is a 7 TeV proton synchrotron, with a
design stored energy of 362 MJ per beam. The high-luminosity (HL-LHC) upgrade
will increase this to 675 MJ per beam. In order to protect the superconducting
magnets and other sensitive equipment from quenches and damage due to beam
loss, a multi-level collimation system is needed. Detailed simulations are
required to understand where particles scattered by the collimators are lost
around the ring in a range of machine configurations. Merlin++ is a simulation
framework that has been extended to include detailed scattering physics, in
order to predict local particle loss rates around the LHC ring. We compare
Merlin++ simulations of losses during the squeeze (the dynamic reduction of
the $\beta$-function at the interaction points before the beams are put into
collision) with loss maps recorded during beam squeezes for Run 1 and 2
configurations. The squeeze is particularly important as both collimator
positions and quadrupole magnet currents are changed. We can then predict,
using Merlin++, the expected losses for the HL-LHC to ensure adequate
protection of the machine.

##  I Introduction

The LHC collimation system Brüning  _et al._ (2004); Assmann  _et al._ (2006)
is designed to protect the ring from normal beam losses caused by diffusion
and scattering, as well as abnormal fast losses. In order to achieve this it
uses a multi-stage system for betatron cleaning, installed in Insertion Region
7 (IR7), and a similar but reduced system for momentum cleaning installed in
IR3. The main components are shown in Fig. 1. The primary collimators (TCP)
are made of carbon-fiber composites (CFC), and sit closest in to intercept the
beam halo. The secondary collimators (TCS) also CFC and shower absorbers
(TCLA) made of tungsten absorb the deflected protons and secondary particles.
There are also tungsten tertiary collimators (TCT) that provide extra
protection for the experiments, as well as TCLs to absorb collision debris.
About 150 m upstream of each experiment, a pair of one horizontal (TCTPH) and
vertical (TCTPV) TCT is installed. The LHC’s multi-stage collimation system
has proved to be performing reliably during Run 1 (2010-2012) with beam
energies of 3.5 TeV and 4 TeV and the start of Run 2 (2015-2018) at 6.5 TeV.
However, the future physics program at 7 TeV and the higher intensities of HL-
LHC Apollinari  _et al._ (2017); Brüning and Rossi (2015) provide new
challenges, so it is important that the performance is well understood and can
be accurately simulated. The HL-LHC program includes upgrades to the
collimation system such as new collimators, new jaw materials Borg  _et al._
(2018) and new embedded instrumentation Valentino  _et al._ (2017).

Figure 1: Overview of the main components involved in the multistage
collimation system.

In this article we present predictions for HL-LHC losses during luminosity
levelling, as well as measurements of the performance of the LHC collimation
system during Run 1 and 2. Previously, studies of LHC collimation efficiency
have been successfully performed using the SixTrack code with comparisons to
BLM data at fixed optical configurations Schmidt (1994); noa (2018); Bruce
_et al._ (2013); Robert-Demolaize  _et al._ (2005); Bruce  _et al._ (2014a,
b); Hermes  _et al._ (2016); Mirarchi  _et al._ (2016a); Bruce  _et al._
(2017). These demonstrate that maps of proton loss locations are useful for
evaluating collimation scenarios and as input to energy deposition studies
using codes such as FLUKA Böhlen  _et al._ (2014). In this article we use
instead the code Merlin++, which is described in section II and present the
first comparison loss maps taken during the squeeze. In section III the Beam
Loss Monitors (BLMs) Holzer  _et al._ (2005, 2008) are described. These are
used to validate the simulation code. This system consists of about 4000
ionisation chambers distributed around the ring to monitor losses at critical
elements.

For this work we consider the slow losses that occur during normal operation
of the LHC. Particles in the core of the beam can be excited to higher
amplitudes by a number of effects, drifting out to form the beam halo. When a
particle’s amplitude is large enough, it is intercepted by the collimators.

Before bringing the LHC beams into collision, they must first be ramped from
injection energy (450 GeV) to full energy (6.5 TeV in Run 2) and the
$\beta^{*}$ (the $\beta$-function at the experiment interaction points (IPs))
reduced. This latter part of the operational cycle is called the squeeze. In
Run 1 these actions were performed separately, however during Run 2 a combined
ramp and squeeze sequence was introduced to reduce the cycle duration. Figure
2 shows the beam modes for a typical Run 1 production fill, from injection of
the physics beam through to stable beams for physics production. In the Fig. 2
the squeeze begins at around 3500 seconds, with the $\beta^{*}$ at IP 1/5
being reduced from 11 m to 0.6 m.

Figure 2: $\beta^{*}$ in IR 1/5 and beam energy during a typical squeeze
during 2012, showing the ramp and squeeze periods.

The squeeze is an important time for the collimation system as there are
dynamic changes to the machine optical configuration and collimator positions
while the stored energy in the beam is at its maximum. It also provides a good
opportunity to validate simulation against measurements in a range of
configurations, allowing investigation into any differences found.

Sections IV and V compare the simulations to data for Run 1 and 2
respectively. This gives us the confidence in Merlin++’s particle tracking and
scattering models, in order to use it for making predictions of future
configurations. In section VI we evaluate the loads on the collimators in the
HL-LHC, for the most pessimistic loss scenario allowed at full energy. This
corresponds to a 0.2 h beam lifetime over 10 s Redaelli  _et al._ (2015);
Assmann  _et al._ (2002), giving a total loss power of about 1 MW.

##  II Calculation of beam losses

To calculate the losses in an accelerator we must model the trajectories of
particles in the magnetic lattice and also the passage and scattering of
particles in the materials that make up the collimators.

Merlin++ developers (2018), previously known as MERLIN, is a modular object-
oriented accelerator simulation framework, featuring 6D thick lens tracking.
Initially developed for the International Linear Collider’s beam delivery
systems Krücker  _et al._ (2006), it has since been extended to support
synchrotrons. It is written in C++ and can be easily extended by the user, for
example to add new physics models. It has multiprocessor support using the MPI
protocol for communication; however for non-collective effects as used in
collimation studies, it is more convenient to run multiple independent
processes and sum the results.

The user-created program calls the Merlin++ library in order to define a beam
line, add appropriate physics processes, create a beam and then initiate
tracking. In these studies the MADInterface module was used to read in a
lattice description from a MADX CERN (2018) optics calculation. Machine
apertures and collimator gaps were similarly defined in separate files.

While the codes SixTrack Schmidt (1994) and FLUKA have been used for similar
comparisons to BLM data previously Bruce  _et al._ (2014a) it is useful to
have multiple independent simulation codes in order increase confidence and
constrain systematic uncertainness. Merlin++ features thick lens tracking
compared to SixTrack’s thin lens method and more advanced scattering model as
described below. Merlin++ also has some technical advantages, for example its
modular C++ design simplifies the incorporation of new physics models

The scattering physics used to model the passage of protons through material
has recently been upgraded Molson (2014). It contains advanced models of the
following processes: multiple Coulomb scattering; ionisation based on Landau
theory; Rutherford scattering; a new elastic scattering model; and a new
single-diffraction dissociation model. The model uses proton-nucleon
scattering based on the Donnachie and Landshoff (D-L) description of Pomeron
and Reggeon exchange and has been fitted to elastic and diffractive scattering
data from a large number of previous experiments. The development and
implementation of these models are described in detail in our earlier paper
Appleby  _et al._ (2016). This model has now been included into the Pythia
event generator, under the name ABMST model Rasmussen and Sjöstrand (2018).
These new scattering models give different predictions for losses around the
ring compared to the K2 model in older versions of SixTrack, specifically the
single diffractive model was found to cause a significant shift of losses from
cold to warm regions. For performance reasons, only the leading proton from
each interaction is modelled; equivalent to assuming that the secondaries
deposit their energy close to the interaction.

To simulate loss maps we use an approach similar to earlier LHC studies
Robert-Demolaize  _et al._ (2005); Bruce  _et al._ (2014a). We track $10^{8}$
protons for 200 turns. The initial particle bunch is generated at the face of
the primary collimator in the excitation plane. This simulated bunch is a ring
in phase-space in the excitation plane that intersects the collimator jaw by 1
$\mathrm{\SIUnitSymbolMicro}\mathrm{m}$ from its edge, the impact parameter,
and Gaussian in the opposite plane. It is pre-filtered so that every proton
interacts with the primary collimator on the first turn. This saves
significant computer resources compared to modelling the real LHC bunch
distribution and the diffusion of particles from the core of the bunch. As the
beam is tracked, it is compared to the machine aperture at each element. If a
proton hits the aperture in a collimator then it is scattered according to the
process cross sections. If the proton hits the aperture of other lattice
elements, it is considered as lost at this location. The loss map records the
location of every proton loss with resolution of 10 cm.

The primary collimators are set to be the tightest aperture restriction in the
machine, so that they are the first material that a proton with sufficient
amplitude will hit. The protons that scatter without being absorbed will go on
to hit other elements in the ring. If all protons hitting the primary
collimator were absorbed by the collimation system we would consider it to
have 100 % cleaning efficiency. In practice we typically see about 8 % of
protons absorbed during a single pass through a TCP due to inelastic
collisions or losing over 95 % of their initial energy which is considered to
be a loss. Over subsequent turns around the ring most protons will be lost in
the TCPs. To produce a loss map, which shows the distribution of losses around
the ring, we measure the proton loss locations and use them to calculate the
local cleaning inefficiency. The simulated local cleaning inefficiency,
$\eta_{\mathrm{loc}}$, is given by the ratio of particles lost on a given
section, $N_{\mathrm{loc}}$, (either an element or bin along the S coordinate)
to particles lost in the primary collimators, $N_{\mathrm{tot}}$, normalised
to the length of the section, $\Delta s$, to make the value independent of bin
size i.e.

$\eta_{loc}=\frac{N_{loc}}{N_{tot}\Delta s}.$ (1)

The local cleaning inefficiency can then be multiplied by the total beam loss
rate to find the local proton loss rate.

##  III Loss measurements

The LHC BLM system uses ionisation chamber charged particle detectors to
measure the radiation levels around the LHC ring Holzer  _et al._ (2005,
2008). They are used during operation to trigger a beam dump if loss
thresholds are exceeded. They also provide continuous measurements of normal
beam loss around the ring during the LHC operations and are used to record
beam loss during the validation campaigns of the collimation system, when
artificial losses are induced with safe low intensity beams to assess the
system response.

To generate a loss map one of the beams is excited in a given plane using the
transverse dampers (ADTs) and the losses are recorded Höfle  _et al._ (2012).
This allows a clean loss map for an individual beam and plane to be made.
Measured LHC loss maps have been studied previously in Bruce  _et al._ (2014a,
2017).

During 2012 several loss maps were recorded at 4 TeV in the flat top and fully
squeezed optics configurations. No deliberate loss maps were made with the
intermediate squeeze optics at 4 TeV, but as the BLM signals are recorded
continuously it is possible to look at the natural losses during the squeeze.
In 2016 at 6.5 TeV, loss maps were generated at the intermediate squeeze
points as well as the end points.

These loss maps are crucial to validate simulations, to ensure a good
understanding of the collimation system, and hence demonstrate the performance
of the HL-LHC layout. In the following sections we show the validation for Run
I and 2 of the LHC.

##  IV Run 1 - 4 TeV

###  IV.1 Run 1 LHC 2012 configuration

In this article we investigate losses at intermediate optics points while the
machine is in squeeze mode. Here the optics configurations are changing as a
function of time, see Fig. 2. As $\beta^{*}$ is reduced, the $\beta$-function
in the inner triplets must increase, as shown in Fig. 3. Table 1 shows the
optics settings for the squeeze during 2012.

Table 1: Optics settings for squeeze in 2012 at 4 TeV. Crossing angle can be in the horizontal (H) or vertical (V) plane. Note that for IR2 and 8, the external crossing angle, applied on top of the spectrometers and their compensation bumps, is given. | $\beta^{*}$ | half crossing angle
---|---|---
| (m) | ($\mathrm{\SIUnitSymbolMicro}\mathrm{r}\mathrm{a}\mathrm{d}$)
ATLAS (IP1) | 11 $\rightarrow$ 0.6 | -145 V
CMS (IP5) | 11 $\rightarrow$ 0.6 | 145 H
ALICE (IP2) | 10 $\rightarrow$ 3.0 | -90 V
LHCb (IP8) | 10 $\rightarrow$ 3.0 | -220 H
Figure 3: $\beta$-functions and dispersion at the ATLAS experiment for flat
top (top) and squeezed (bottom) settings for the 2012 run configurations.

As well as the optics configuration changing during the squeeze, the TCTs are
also in motion. The collimators are set in units of beam sigma, so as the beam
envelope at the collimator’s location changes, the collimators jaws are
adjusted. Also the TCTs at the experiments are only brought into their
tightest position during the squeeze as the normalised triplet aperture
reduces, so their jaw gap in sigma units is decreased during the squeeze, as
shown in Table 2. We therefore concentrate this study on the losses at the
TCTs.

Table 2: Collimator settings for squeeze in 2012 at 4 TeV, from 11 m to 0.6 m at IP 1/5. Using a normalised beam emittance of 3.5 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$ Region | Type | Gap ($\upsigma$)
---|---|---
IR7 | TCP | 4.3
IR7 | TCS | 6.3
IR7 | TCLA | 8.3
IR3 | TCP | 12.0
IR3 | TCS | 15.6
IR3 | TCLA | 17.6
IR1 | TCT | 26.0 $\rightarrow$ 9.0
IR5 | TCT | 26.0 $\rightarrow$ 9.0
IR2 | TCT | 26.0 $\rightarrow$ 12.0
IR8 | TCT | 26.0 $\rightarrow$ 12.0

###  IV.2 Simulated loss maps

Loss maps were simulated in Merlin++ at 8 points within the squeeze covering
$\beta^{*}$ at IP1 and IP5 from 11 m to 0.6 m. A bunch of $10^{8}$ protons
were tracked for 200 turns. This is sufficient to give good statistics for all
relevant collimator and ring losses, even for particles that survive for
multiple turns after their first scatter.

Figure 4 shows examples of the loss maps at 3 of the optical configurations.
The highest losses occur on the TCPs in IR7 at around 20000 m from IR1 as
expected, then lower losses along the cleaning hierarchy. Also significant
losses at the momentum cleaning collimators in IR3, at around 7000 m, are
observed. The losses at the TCTs in front of the experiments in IR1/2/5/8 do
not appear until the later stages of the squeeze. The main cold losses are in
the IR7 dispersion suppressor, which is the bottleneck in terms of local
cleaning inefficiency.

Figure 4: Loss maps, as simulated with Merlin++ for initial losses in the
horizontal plane, for Beam 1. Made at different points in the squeeze with
$\beta^{*}$ of 11, 4, and 0.6 m, for the 2012 4 TeV configuration. Losses in
collimators are shown in black, in cold magnets in blue and in warm magnets in
red.

###  IV.3 Measured squeeze losses in 2012

As there were no dedicated loss maps made during 2012 at the intermediate
squeeze optics settings, we compare to the BLM measurements made during normal
LHC operation. These have a number of disadvantages over the dedicated loss
maps. They have lower signal levels, and so a lower signal to noise ratio.
This makes it hard to get clean data in lower loss regions. They contain an
unquantified mix of losses from both beams and in both planes. This makes it
impossible to completely separate out different sources of loss as can be done
with the dedicated loss maps.

The logging database is used to identify typical data taking fills, where the
squeeze was successful and the beam reached the ‘stable beams’ mode. Then the
BLM signals and optical parameters as functions of time can be retrieved for
those fills. Even with no losses occurring there is a continuous low level of
background noise recorded by the BLMs. We consider this noise floor to be the
limit to the precision of the BLM signal and hence the uncertainty. For each
BLM a background noise level is calculated by averaging the lowest 5 readings
during the squeeze. This value can then be used as an estimate of the
uncertainty of the BLM signal.

During a fill the rate of loss varies considerably. For much of the time the
losses at the TCTs are below the noise thresholds of the BLMs. This can cause
spurious values for local inefficiency. It was found that there could be large
swings in the total loss rate around the fixed points of the squeeze so BLM
data taken at those points are particularly unreliable.

In order to get a good measure of inefficiency we identified points in time
where the total losses were high enough that the TCT BLMs were above noise. A
peak-finding algorithm was used to find the highest values of the TCT losses
within each fill. These points were retained if at the same time stamp there
was also a high BLM value at TCP. Figure 5 shows how for four fills, time
stamps with simultaneous peaks are selected. These points were then ranked by
the product of the TCT and TCP values, and the highest kept.

Figure 5: Peaks in the TCT (top) and TCP (middle) BLM signals are identified
for 4 fills (show in different colors). At these time steps the noise on the
ratio (bottom) is minimised and so can be taken to get the local cleaning
inefficiency. Figure 6: The 3 primary collimators at D6L7, C6L7 and B6L7 are
oriented to collimate in the vertical, horizontal and skew planes
respectively. The codes refer to the positions in the 6 cell left of IR7.

The BLM signals during regular operation contain a mix of both horizontal and
vertical losses. These can be partially separated by looking at the ratios
between the vertical, horizontal, and skew TCPs, labelled D6L7, C6L7 and B6L7
respectively due to their positions in the lattice. A schematic of their
layout is shown in Fig. 6. Vertical excitations will hit D6L7 leaving a BLM
signal there but the shower will also leave a signal at C6L7 and B6L7.
Horizontal excitations will pass through D6L7 and hit C6L7 first, with the
shower peaking in B6L7. We find that cutting on the D6L7 to C6L7 ratio being
less than 0.1 and the C6L7 to B6L7 ratio being less than 0.5 selects cases
that are dominated by horizontal losses. This results in 13 data points that
pass the filters, covering a range of $\beta^{*}$ values from 2 m to 0.6 m.

More sophisticated machine learning algorithms exist to categorise losses
Valentino and Salvachua (2018), however these were not used as they have not
been trained on 2012 data.

For the points that pass both filters we calculate the inefficiency at the
TCTs and take the $\beta^{*}$ value that corresponds to the timestamp.

###  IV.4 Comparison between simulation and measurements

First we can compare the full loss maps taken at the end of the squeeze with
IR1/5 $\beta^{*}$ of 60 cm. BLM loss maps are from fill 2788 on the 1st of
July 2012. Figure 7 shows losses around the full ring for horizontal beam 1
excitation. Merlin++ reproduces the significant loss peaks in the collimation
regions and other IRs. Figure 8 shows the loss map zoomed to the IR7
collimation region. The hierarchy of losses from the TCPs through to the TCSs
and TCLAs can bee seen in both simulation and data. The BLM signal outside the
collimators is higher than in simulation, especially in the warm losses
represented by the red bars, as full showers of secondary particles are not
simulated in Merlin++. The noise level the BLMs can be seen in the
measurements at around $10^{-6}$, these are not real losses and therefore set
the precision of the measurement.

While Merlin++ counts the particle losses on the beam pipe, the BLMs record
the dose from the radiation shower outside the accelerator’s physical
components. At LHC energies the shower from proton impacts have an effective
length of approximately 1 m in typical metals with a tail expanding up to 10 m
Jeanneret  _et al._ (1996), so proton losses at one element will also cause
signal in the BLMs at downstream elements. The materials of the magnets and
surrounding equipment will absorb some of the energy of the shower. For a full
quantitative comparison to the BLM signals one would need to use the proton
loss maps from Merlin++ as inputs to an energy deposition code such as FLUKA.
This would be used to model the evolution of the particle showers through the
machine elements and the signal response of BLM ionisation chamber. Similar
studies using loss maps produced with SixTrack have been demonstrated in Bruce
_et al._ (2014a).

Figure 7: Beam 1 Horizontal loss map from BLMs (top) and Merlin++ (bottom) at
4 TeV with $\beta^{*}$ of 60 cm. Figure 8: IR7 Beam 1 horizontal loss map from
BLMs (top) and Merlin++ (bottom) at 4 TeV with $\beta^{*}$ of 60 cm.

We can now compare the TCT inefficiency predicted in the Merlin++ simulation
with the measurements. In both cases we are interested in the normalised local
cleaning inefficiency at the TCTs, i.e. the ratio of the individual TCT to the
total TCP losses. This partially normalises out the conversion of proton
losses to BLM signal values. However, we must also make a normalisation to
take into account the response of the BLM to the local proton loss and cross
talk due to a secondary particles from one element reaching the BLM of
another. To do this we normalise to the inefficiency at the fully squeezed
configuration. This normalisation point is chosen as it has the highest losses
and so the lowest statistical error. With the chosen normalization, the
different optics configurations can be compared in relative, although not in
absolute, assuming that the BLM response is independent of optics.

Figure 9 shows the Merlin++ simulation compared to the data points extracted
from the BLM data. As before, BLM error bars are based on the background level
found by averaging the 5 lowest reading within the time window. While the
trend is compatible it is clear that the BLM data are too limited to draw
conclusions. The signal to noise ratio in the BLM data are too low for
$\beta^{*}$ above 2 m to retrieve any data points and give a significant data
spread above 1 m. It is clear that dedicated loss maps are needed to make a
better comparison.

Figure 9: Comparison between normalised losses in Merlin++ and BLM signal for
the horizontal collimator in IR1 (TCTH.4L1.B1) during squeeze. Note that no
losses were seen at the TCT in simulation for $\beta^{*}$ greater than 7 m.

##  V Run 2 - 6.5 TeV

###  V.1 Run 2 2016 LHC configuration

Run 2 of the LHC began in 2015 and incorporated changes to the machine
configuration, most notably an increase in beam energy from 4 to 6.5 TeV. In
order to reduce the time from injection to collision, a combined ramp and
squeeze program was used stating from 2016, such that the initial squeeze,
down to $\beta^{*}$ of 3 m at IR1 and IR5, happens simultaneously with the
energy ramp. Therefore, the squeeze beam mode covers just the final 3 m to 0.4
m of squeezing.

Table 3 shows the optical parameters for the IPs used during 2016 data taking.
Table 4 shows the collimation settings used.

Table 3: Optics settings for 2016 squeeze at 6.5 TeV. As before for IR2 and 8 the external crossing angle is given. | $\beta^{*}$ | half crossing angle
---|---|---
| (m) | ($\mathrm{\SIUnitSymbolMicro}\mathrm{r}\mathrm{a}\mathrm{d}$)
ATLAS (IP1) | 3.0 $\rightarrow$ 0.4 | -185 V
CMS (IP5) | 3.0 $\rightarrow$ 0.4 | 185 H
ALICE (IP2) | 10 $\rightarrow$ 10 | 200 V
LHCb (IP8) | 6.0 $\rightarrow$ 3.0 | -250 H
Table 4: Collimator settings for the squeeze in 2016 at 6.5 TeV. Using a normalised beam emittance of 3.5 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$ Region | Type | Gap ($\upsigma$)
---|---|---
IR7 | TCP | 5.5
IR7 | TCS | 7.5
IR7 | TCLA | 11.0
IR3 | TCP | 15.0
IR3 | TCS | 18.0
IR3 | TCLA | 20.0
IR1 | TCT | 23.0 $\rightarrow$ 9.0
IR5 | TCT | 23.0 $\rightarrow$ 9.0
IR2 | TCT | 37.0
IR8 | TCT | 23.0 $\rightarrow$ 15.0

###  V.2 Measured squeeze losses in 2016

During the 2016 beam commissioning a number of loss maps were made during the
squeeze. This gives a better signal to noise ratio and allows separation of
losses from each beam and plane. The maps used in this article were taken on
20th of April 2016, during fill 4832. A beam of low intensity pilot bunches
was injected and ramped to 6.5 TeV. During the squeeze, the ADTs for each
combination of horizontal and vertical, and Beam 1 and 2, were fired in turn
to excite one of the bunches in that plane, as shown in Fig. 10, and the BLM
signal was recorded Mirarchi  _et al._ (2016b).

Figure 10: Loss map recording during fill 4832. Top plot shows energy (blue)
and IR1 $\beta^{*}$ (red). Bottom plot shows the fall in beam intensity as
each of the 4 ADTs (for each beam and plane) are fired (vertical lines).

For each BLM we calculate a background level, by averaging the signal during a
10 s window near the start of the squeeze where losses are low. This fixed
value per BLM is used as an estimate of the uncertainty of that BLM’s signal
during excitation. Note that the background measurement is usually taken
closer to the loss map excitation, however in this case where loss maps are
made in rapid succession this is not possible. There are a number of
additional parameters not under control that can contribute to errors, such as
orbit shifts and changes in the squeeze rate.

###  V.3 Comparison between simulation and measurements

First we compare a full loss map from fill 4832 taken close to when the
$\beta^{*}$ at IR1/5 crossed 50 cm. Figures 11 and 12 show full ring and IR7
loss maps comparing BLM data and Merlin++ simulation. As with the 4 TeV
comparisons we see that Merlin++ reproduces well the main loss locations
around the ring, and the collimation hierarchy in IR7 well.

Figure 11: Beam 1 Horizontal loss map from BLMs (top) and Merlin++ (bottom) at
6.5 TeV with $\beta^{*}$ of 50 cm. Figure 12: IR7 Beam 1 horizontal loss map
from BLMs (top) and Merlin++ (bottom) at 6.5 TeV with $\beta^{*}$ of 50 cm.

We can now compare the normalised cleaning efficiency as function of
$\beta^{*}$ between BLM data and Merlin++. In the BLM measurements during the
squeeze we have 11 loss maps in the horizontal plane and 9 in the vertical. In
the simulations we have used 5 different optical configurations.

Figures 13 and 15 show losses on the IR1 TCTs during the squeeze due to
horizontal and vertical excitation of the beam. Figures 14 and 16 show zoomed
sections of the plot so that more detail is visible at low $\beta^{*}$.
Horizontally we see excellent agreement between data and simulation, with
steep increases in TCT losses as the beam is squeezed to $\beta^{*}$ of 0.4 m.
For vertical excitation we again see good overall agreement, although no
losses are observed on TCTPV.4L1.B1, the vertical TCT in IR1, in simulation
above $\beta^{*}$ of 0.8 m. At larger $\beta^{*}$ values, the signals on the
TCT BLMs are below the noise levels, so we are not able to record the losses.

Figure 13: BLM signal (shown as uncertainty bands) and Merlin++ simulated
losses (shown as solid lines) on IR1 TCTs for horizontal excitation. Solid
area shows uncertainty bands due to detector background. Figure 14: Zoomed
plot of BLM signal and Merlin++ simulated losses on IR1 TCTs for horizontal
excitation. Solid area shows uncertainty bands due to detector background.
Figure 15: BLM signal and Merlin++ simulated losses on IR1 TCTs for vertical
excitation. Note that no losses were seen on the vertical TCT at $\beta^{*}$
greater than 0.8 m in simulation or on either TCT at $\beta^{*}$ greater than
1.9 m in data. Figure 16: Zoomed plot of BLM signal and Merlin++ simulated
losses on IR1 TCTs for vertical excitation.

Figures 17 and 19 show losses on the IR5 TCTs due to horizontal and vertical
excitation of the beam. Again we show zoomed sections for low $\beta^{*}$ in
Figs. 18 and 20. For horizontal excitation we see good agreement for
TCTPH.4L5.B1, but higher losses in simulation for TCTPV.4L5.B1 than in BLM
data. For vertical excitation Merlin++ reproduces the losses well.

Figure 17: BLM signal and Merlin++ simulated losses on IR5 TCTs for horizontal
excitation. Figure 18: Zoomed plot of BLM signal and Merlin++ simulated losses
on IR5 TCTs for horizontal excitation. Figure 19: BLM signal and Merlin++
simulated losses on IR5 TCTs for vertical excitation. Figure 20: Zoomed plot
of BLM signal and Merlin++ simulated losses on IR5 TCTs for vertical
excitation.

To investigate the loss fall off on TCTPV.4L1.B1 (Fig. 15) we look at the
position of the collimators that act in the vertical plane, projected into
phase-space. At the smallest $\beta^{*}$ values, losses on TCTPV.4L1.B1 are
dominated by particles scattered from the IR7 TCSGs with the highest losses
coming from TCSG.D4L7.B1. Figure 21 shows the vertical collimators with their
phase advance from TCSG.D4L7.B1. It can be seen that due to the retraction in
jaw gap and change in phase advance the TCT is shadowed behind TCLA.C6R7.B1
for $\beta^{*}$ of 1 m and larger. In the LHC TCTPV.4L1.B1 is positioned
downstream of TCTPH.4L1.B1, so the BLM will see local showers from the
horizontal TCT even when the vertical TCT is not directly hit. This explains
the discrepancy between the simulation and BLM data.

(a) $\beta^{*}$ = 40 cm

(b) $\beta^{*}$ = 1 m

Figure 21: Collimator positions in normalised phase-space, accounting for the
phase advance from TCSG.D4L7.B1, shown in green, to TCTPV.4L1.B1 shown in red.

With this good modelling of proton losses we can now use Merlin++ to make
predictions for future collimation configuration such as the HL-LHC.

##  VI Performance of the HL-LHC collimation system

The HL-LHC upgrade introduces several changes to the lattice Apollinari  _et
al._ (2017). Among other changes, the inner triplets are replaced with higher
gradient magnets of larger aperture to allow a smaller $\beta^{*}$ at the IPs
Todesco  _et al._ (2015), and a new achromatic telescopic squeeze (ATS) optics
scheme is used Fartoukh (2013). In the dispersion matching section downstream
of the betatron cleaning, additional absorbers (TCLD) have been placed by
splitting two of the bending magnets into shorter high field magnets Bruce
_et al._ (2014b); Lechner  _et al._ (2014); Mirarchi  _et al._ (2016a). For
each beam in cell 6 upstream of the experiment in IR1 and IR5 an additional
pair of TCTs has been placed to improve protection Redaelli  _et al._ (2015).

Table 5: Collimator settings for HL-LHC at 7 TeV squeeze from 45 cm to 15 cm. Using beam emittance of 3.5 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$. Region | Type | Gap ($\upsigma$)
---|---|---
IR7 | TCP | 5.7
IR7 | TCS | 7.7
IR7 | TCLD | 12.0
IR3 | TCP | 15.0
IR3 | TCS | 18.0
IR1 | TCT | 18.2 $\rightarrow$ 10.5
IR5 | TCT | 18.2 $\rightarrow$ 10.5
IR2 | TCT | 30.0
IR8 | TCT | 30.0

###  VI.1 HL-LHC luminosity levelling

In order to maximise integrated luminosity while limiting the maximum pileup,
the HL-LHC will use a luminosity levelling scheme Arduini  _et al._ (2015). If
the accelerator configuration is kept constant during data taking then the
luminosity will fall over the length of the fill due to the gradual reduction
in the beam current. Levelling is achieved by adjusting the machine
configuration to compensate for the change in beam current; in the baseline by
changing $\beta^{*}$ at the IPs.

This leads to another situation where dynamic changes of the collimators could
be needed, although in this case with the beams in collision.

We consider a levelling scheme that utilises changes in $\beta^{*}$ from 64 cm
to 15 cm, while keeping the crossing angle fixed Metral  _et al._ (2018). In
this case the TCTs and TCLs are held a fixed position in mm. The jaws are
fixed at the position that gives a TCT gap of 10.5 $\sigma$ and a TCL gap of
12 $\sigma$ at the minimum $\beta^{*}$ of 15 cm, using a normalised beam
emittance of 3.5 $\mathrm{\SIUnitSymbolMicro}\mathrm{m}$. For example
TCTPH.4L1.B1 will have a gap of 15.5 mm for all $\beta^{*}$ values. Table 5
shows the collimator settings used. For this work we use the HL-LHC version
1.2 optics, with 2 TCLDs per beam in IR7.

Figure 22 and 23 show the simulated loss map at 3 steps in the HL-LHC
luminosity levelling for the full ring and IR7 respectively. Again the main
losses occur in the collimation regions at IR 3 and 7. Smaller loss peaks can
also be seen at IR 1,2 and 5. The TCT losses get larger as $\beta^{*}$ at the
IPs is reduced.

Figure 22: Merlin++ Beam 1 loss map for 3 IR1/5 $\beta^{*}$ value steps during
HL-LHC luminosity levelling. Figure 23: Merlin++ Beam 1 loss map for 3 IR1/5
$\beta^{*}$ value steps during HL-LHC luminosity levelling. For IR7.

Figures 24 and 25 show the Beam 1 losses on the TCTs at the main IPs as a
function of the $\beta^{*}$ value.

Figure 24: Merlin++ simulated IR1 TCT losses for horizontal excitation as a
function of $\beta^{*}$ value during HL-LHC luminosity levelling. Figure 25:
Merlin++ simulated IR5 TCT losses for horizontal excitation as a function of
$\beta^{*}$ value during HL-LHC luminosity levelling.

Losses in cold magnets in the rest of the ring are significantly lower than in
the LHC configurations due to the TCLDs catching the dispersive losses. The
total cleaning inefficiency of the IR7 collimators at $\beta^{*}$ of 45 cm is
found to be $7.0\text{\times}{10}^{-5}$ and $6.0\text{\times}{10}^{-5}$ for
horizontal and vertical excitation respectively, and
$2.9\text{\times}{10}^{-4}$ and $1.3\text{\times}{10}^{-4}$ at 15 cm. For the
horizontal 45 cm case, that is that 99.993 % of lost protons are absorbed in
the IR7 collimators.

Although there are only very few direct proton losses in cold elements, the
showers from the collimators could potentially still quench magnets during the
1 MW loss scenario, but this has been studied with energy deposition studies
in Lechner  _et al._ (2014). Studies on the response of the collimators
themselves to these loads have been performed in Bertarelli  _et al._ (2018).

##  VII Conclusion

The LHC collimation system is essential to protect the machine from beam
losses during operation. Its performance is continuously monitored by the BLM
system. We can use existing measurements to validate simulations, which can
then be used to make prediction of future performance for HL-LHC.

In this paper we show that Merlin++ is able to model the proton losses around
the LHC. It can reproduce the patterns of losses around the LHC ring and in
the interaction regions from measured data. It gives good agreement, to
measurements taken with the BLM systems during the beam squeeze, for Run 1 and
2 operation at 4 and 6.5 TeV, both in the overall loss patterns and the
changes on the TCTs as the optics configuration is changed. The remaining
deviations between simulation and data can be understood by considering the
crosstalk between elements due to radiation showers which is not included in
Merlin++.

In addition to the SixTrack code, already used successfully for collimation
studies, we can therefore also use Merlin++ to predict losses in the future
HL-LHC configuration. The possibility to use different simulation tools
provides increased flexibility and allows estimating systematic uncertainty in
the final results. We find that the HL-LHC collimation system performs well
with a low cleaning inefficiency. The losses on the cold magnets are
acceptable, although the loads in the 1 MW scenario imply also the need of
energy deposition studies of the magnet coils, as well as thermo-mechanical
studies of the most loaded collimators.

##  VIII Acknowledgements

We would like to thank Alessio Mereghetti, Rogelio Tomas Garcia and Luis
Medina for useful discussion and guidance and Belen Salvachua for BLM data
collection.

This work is supported by STFC (UK) grant High Luminosity LHC : UK (HL-LHC-
UK), grant number ST/N001621/1.

## References

  * Brüning  _et al._ (2004) O. S. Brüning, P. Collier, P. Lebrun, S. Myers, R. Ostojic, J. Poole, and P. Proudlock,  _LHC Design Report v.1 : the LHC Main Ring_, CERN Yellow Reports: Monographs (CERN, Geneva, 2004).
  * Assmann  _et al._ (2006) R. W. Assmann, M. Magistris, O. Aberle, M. Mayer, F. Ruggiero, J. M. Jiménez, S. Calatroni, A. Ferrari, G. Bellodi, and I. Kurochkin, “The final collimation system for the LHC,” in  _10th European Particle Accelerator Conference_ (Edinburgh, UK, 2006) p. 986, LHC-PROJECT-Report-919.
  * Apollinari  _et al._ (2017) G. Apollinari, I. B. Alonso, O. Brüning, P. Fessia, M. Lamont, L. Rossi, and L. Tavian,  _High-Luminosity Large Hadron Collider (HL-LHC): Technical Design Report V. 0.1_, CERN Yellow Reports: Monographs (CERN, Geneva, 2017) CERN-2017-007-M.
  * Brüning and Rossi (2015) O. Brüning and Lucio Rossi, eds.,  _The High Luminosity Large Hadron Collider: the new machine for illuminating the mysteries of universe_ , Advanced series on directions in high energy physics, Vol. 24 (World Scientific, New Jersey, 2015).
  * Borg  _et al._ (2018) M. Borg, A. Bertarelli, F. Carra, P. Gradassi, J. Guardia-Valenzuela, M. Guinchard, G. Arnau Izquierdo, P. Mollicone, O. Sacristan-de Frutos, and N. Sammut, “Thermostructural characterization and structural elastic property optimization of novel high luminosity lhc collimation materials at cern,” Phys. Rev. Accel. Beams 21, 031001 (2018).
  * Valentino  _et al._ (2017) G. Valentino, G. Baud, R. Bruce, M. Gasior, A. Mereghetti, D. Mirarchi, J. Olexa, S. Redaelli, B. Salvachua, A. Valloni, and J. Wenninger, “Final implementation, commissioning, and performance of embedded collimator beam position monitors in the large hadron collider,” Phys. Rev. Accel. Beams 20, 081002 (2017).
  * Schmidt (1994) F. Schmidt,  _SIXTRACK user’s reference manual_, Tech. Rep. CERN-SL-94-56 (CERN, 1994).
  * noa (2018) “SixTrack,”  (2018), http://sixtrack.web.cern.ch/SixTrack/.
  * Bruce  _et al._ (2013) R. Bruce, R. W. Assmann, V. Boccone, G. Bregliozzi, H. Burkhardt, F. Cerutti, A. Ferrari, M. Huhtinen, A. Lechner, Y. Levinsen, A. Mereghetti, N. V. Mokhov, I. S. Tropin, and V. Vlachoudis, “Sources of machine-induced background in the ATLAS and CMS detectors at the CERN Large Hadron Collider,” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 729, 825–840 (2013).
  * Robert-Demolaize  _et al._ (2005) G. Robert-Demolaize, R. Assmann, S. Redaelli, and F. Schmidt, “A New Version of Sixtrack with Collimation and Aperture Interface,” in  _Particle Accelerator Conference_ (IEEE, Knoxville, Tennessee, 2005) pp. 4084–4086.
  * Bruce  _et al._ (2014a) R. Bruce, R. W. Assmann, V. Boccone, C. Bracco, M. Brugger, M. Cauchi, F. Cerutti, D. Deboy, A. Ferrari, L. Lari, A. Marsili, A. Mereghetti, D. Mirarchi, E. Quaranta, S. Redaelli, and others, “Simulations and measurements of beam loss patterns at the CERN Large Hadron Collider,” Phys. Rev. ST Accel. Beams 17, 081004 (2014a).
  * Bruce  _et al._ (2014b) R. Bruce, A. Marsili, and S. Redaelli, “Cleaning performance with 11t dipoles and local dispersion suppressor collimation at the LHC,” in  _5th International Particle Accelerator Conference_ (Dresden, Germany, 2014) pp. 170–173, CERN-ACC-2014-0129, MOPRO042.
  * Hermes  _et al._ (2016) P. D. Hermes, R. Bruce, J. M. Jowett, S. Redaelli, B. Salvachua Ferrando, G. Valentino, and D. Wollmann, “Measured and simulated heavy-ion beam loss patterns at the CERN Large Hadron Collider,” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 819, 73–83 (2016).
  * Mirarchi  _et al._ (2016a) D. Mirarchi, A. Bertarelli, R. Bruce, F. Cerutt, P. Hermes, A. Lechner, A. Mereghetti, E. Quaranta, S. Redaelli, H. Garcia Morales, R. Kwee-Hinzmann, and R.B. Appleby, “Cleaning Performance of the Collimation System of the High Luminosity Large Hadron Collider,” in  _7th International Particle Accelerator Conference_ (JACoW, Busan, Korea, 2016) pp. 2494–2497, WEPMW030.
  * Bruce  _et al._ (2017) R. Bruce, C. Bracco, R. De Maria, M. Giovannozzi, A. Mereghetti, D. Mirarchi, S. Redaelli, E. Quaranta, and B. Salvachua, “Reaching record-low β* at the CERN Large Hadron Collider using a novel scheme of collimator settings and optics,” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 848, 19–30 (2017).
  * Böhlen  _et al._ (2014) T. T. Böhlen, F. Cerutti, M. P. W. Chin, A. Fassò, A. Ferrari, P. G. Ortega, A. Mairani, P. R. Sala, G. Smirnov, and V. Vlachoudis, “The FLUKA Code: Developments and Challenges for High Energy and Medical Applications,” Nuclear Data Sheets 120, 211–214 (2014).
  * Holzer  _et al._ (2005) E. B. Holzer, B. Dehning, E. Effinger, J. Emery, G. Ferioli, J. L. Gonzalez, E. Gschwendtner, G. Guaglio, M. Hodgson, D. Kramer, R. Leitner, L. Ponce, V. Prieto, M. Stockner, and C. Zamantzas, “Beam loss monitoring system for the LHC,” in  _IEEE Nuclear Science Symposium Conference Record, 2005_, Vol. 2 (Fajardo, Puerto Rico, 2005) pp. 1052–1056.
  * Holzer  _et al._ (2008) E. B. Holzer, P. Chiggiato, B. Dehning, G. Ferioli, V. Grishin, T. M Jimenez, A. Koshelev, D. Kramer, A. Larionov, M. Taborelli, V. Seleznev, M. Sleptsov, A. Sytin, and I. Wevers, “Development, Production and Testing of 4500 Beam Loss Monitors,” in  _11th European Particle Accelerator Conference_ (Genoa, Italy, 2008) pp. 1134–1136, TUPC037.
  * Redaelli  _et al._ (2015) S. Redaelli, R. B. Appleby, A. Bertarelli, R. Bruce, J. M. Jowett, A. Lechner, and R. Losito, “Cleaning insertions and collimation challenges,” in  _The High Luminosity Large Hadron Collider: The New Machine for Illuminating the Mysteries of Universe_ (World Scientific, 2015) pp. 215–241.
  * Assmann  _et al._ (2002) R. W. Assmann, I. Baishev, M. Brugger, H. Burkhardt, G. Burtin, B. Dehning, S. Fartouk, C. Fischer, E. Gschwendtner, M. Hayes, J.B. Jeanneret, R. Jung, V. Kain, D. Kaltchev, M. Lamont, and others,  _Preliminary Beam-based specifications for the LHC collimators_, Tech. Rep. LHC-PROJECT-NOTE-277 (CERN, 2002).
  * developers (2018) Merlin developers, “Merlin++,”  (2018), http://www.accelerators.manchester.ac.uk/merlin/.
  * Krücker  _et al._ (2006) D. Krücker, F. Poirier, and N. Walker, “An ILC Main Linac Simulation Package Based on MERLIN,” in  _10th European Particle Accelerator Conference_, EUROTEV-REPORT-2006-071 (Edinburgh, UK, 2006) pp. 694–696.
  * CERN (2018) CERN, “MADX - Methodical Accelerator Design,”  (2018), http://mad.web.cern.ch.
  * Molson (2014) J. Molson,  _Proton scattering and collimation for the LHC and LHC luminosity upgrade_ , PhD, University of Manchester (2014).
  * Appleby  _et al._ (2016) R. B. Appleby, R. J. Barlow, J. G. Molson, M. Serluca, and A. Toader, “The practical Pomeron for high energy proton collimation,” Eur. Phys. J. C 76, 520 (2016).
  * Rasmussen and Sjöstrand (2018) C. O. Rasmussen and T. Sjöstrand, “Models for total, elastic and diffractive cross sections,” Eur. Phys. J. C 78, 461 (2018).
  * Höfle  _et al._ (2012) W. Höfle, R. Assmann, S. Redaelli, R. Schmidt, D. Valuch, D. Wollmann, and M. Zerlauth, “Controlled Transverse Blow-up of High energy Proton Beams for Aperture Measurements and Loss Maps,” in  _3rd International Particle Accelerator Conference_ (IEEE, New Orleans, Louisiana, USA, 2012) THPPR039.
  * Valentino and Salvachua (2018) G. Valentino and B. Salvachua, “Machine Learning Applied at the LHC for Beam Loss Pattern Classification,” in  _9th International Particle Accelerator Conference_ (IEEE, Vancouver, British Columbia, Canada, 2018) WEPAF078.
  * Jeanneret  _et al._ (1996) J. B. Jeanneret, T. Trenkler, L. R. Oberli, and D. Leroy,  _Quench levels and transient beam losses in LHC magnets_, Tech. Rep. LHC-Project-Report-44 (CERN, 1996).
  * Mirarchi  _et al._ (2016b) D. Mirarchi, R. Bruce, M. Fiascaris, P. Hermes, A. Mereghetti, E. Quaranta, S. Redaelli, R. Rossi, B. Salvachua, G. Valentino, A. Valloni, J. Wagner, H. Garcia, and R. Kwee-Hinzmann, “Collimation: experience and performance,” in  _7th Evian Workshop on LHC beam operation_ (Evian Les Bains, France, 2016) pp. 105–112.
  * Todesco  _et al._ (2015) E. Todesco, G. Ambrosio, P. Ferracin, J. M. Rifflet, G. L. Sabbi, M. Segreti, T. Nakamoto, R. Van Weelderen, and Q. Xu, “Superconducting Magnet Technology for the Upgrade,” in  _The High Luminosity Large Hadron Collider: The New Machine for Illuminating the Mysteries of Universe_ (World Scientific, 2015) pp. 107–135.
  * Fartoukh (2013) S. Fartoukh, “Achromatic telescopic squeezing scheme and application to the LHC and its luminosity upgrade,” Physical Review Special Topics - Accelerators and Beams 16 (2013), 10.1103/PhysRevSTAB.16.111002.
  * Lechner  _et al._ (2014) A. Lechner _et al._ , “Power Deposition in LHC Magnets With and Without Dispersion Suppressor Collimators Downstream of the Betatron Cleaning Insertion,” in  _5th International Particle Accelerator Conference_ (JACoW, Dresden, Germany, 2014) pp. 112–115, MOPRO021.
  * Arduini  _et al._ (2015) G. Arduini, M. Lamont, T. Pieloni, and G. Rumolo, “Implications for operations,” in  _The High Luminosity Large Hadron Collider: The New Machine for Illuminating the Mysteries of Universe_ (World Scientific, 2015) pp. 373–389.
  * Metral  _et al._ (2018) E. Metral, S. Antipov, F. Antoniou, R. B. Appleby, G. Arduini, J. Barranco Garcia, P. Baudrenghien, N. Biancacci, C. Bracco, R. Bruce, X. Buffat, R. Calaga, L. R. Carver, E. Chapochnikova, M. P. Crouch, and others,  _Update of the HL-LHC operational scenarios for proton operation_, Tech. Rep. CERN-ACC-NOTE-2018-0002 (CERN, 2018).
  * Bertarelli  _et al._ (2018) A. Bertarelli, C. Accettura, E. Berthomé, L. Bianchi, F. Carra, C. Fichera, M. I. Frankl, G. Gobbi, P. Grosclaude, M. Guinchard, A. Lechner, M. Pasquali, S. Redaelli, E. Rigutto, O. Sacristan De Frutos, and others, “Dynamic Testing and Characterization of Advanced Materials in a New Experiment at CERN HiRadMat Facility,” in  _9th International Particle Accelerator Conference_ (IEEE, Vancouver, British Columbia, Canada, 2018) WEPMF071.


# Robust Distributed Compression with
Learned Heegard–Berger Scheme††thanks: This work was in part done during E.
Taşçı’s summer internship at NYU.

Eyyüp Taşçı  Dept. of Electrical and Electronics Engineering
Bogazici University, Istanbul, TURKEY
<EMAIL_ADDRESS>Ezgi Özyılkan, Oğuzhan Kubilay Ülger, Elza
Erkip  Dept. of Electrical and Computer Engineering
New York University, New York, USA
{ezgi.ozyilkan, kubi<EMAIL_ADDRESS>
###### Abstract

We consider lossy compression of an information source when decoder-only side
information may be absent. This setup, also referred to as the Heegard–Berger
or Kaspi problem, is a special case of robust distributed source coding.
Building upon previous works on neural network-based distributed compressors
developed for the decoder-only side information (Wyner–Ziv) case, we propose
learning-based schemes that are amenable to the availability of side
information. We find that our learned compressors mimic the achievability part
of the Heegard–Berger theorem and yield interpretable results operating close
to information-theoretic bounds. Depending on the availability of the side
information, our neural compressors recover characteristics of the point-to-
point (i.e., with no side information) and the Wyner–Ziv coding strategies
that include _binning_ in the source space, although no structure exploiting
knowledge of the source and side information was imposed into the design.

##  I Introduction

Imagine a distributed sensor network consisting of individual cameras
positioned across various locations within a city, each independently
capturing images of its surroundings. In this scenario, each sensor node
compresses and sends its correlated image to a central processing unit, which
then combines them to generate a comprehensive visual map of the city.
However, direct communication among sensors is often infeasible, and
unreliable channels (e.g., due to fading) may further hinder communication.
This poses a key question: How can we leverage the correlation among sensor
data in a robust manner, preventing a system failure when some nodes cannot
transmit their observations? In this work, we take a first step on addressing
the link failure scenario, where the decoder may not receive some correlated
sensor data.

Distributed source coding (DSC) refers to the task of efficiently compressing
information from physically separated encoders. Wyner and Ziv (WZ) [1]
examined a simple lossy distributed compression case where the decoder has
access to a correlated source, known as the _side information_ , losslessly.
WZ theorem relies on joint typicality and _random binning_ arguments, and is
non-constructive. Although the theory of DSC predicts substantial improvements
in compression efficiency compared to the point-to-point setups [2],
developing practical distributed compressors operating in the finite
blocklength regime remains a challenging open problem to date.

An interesting question is whether it is still possible to achieve a
reconstruction with non-trivial distortion when side information fails to
reach the decoder in the WZ setup. An equivalent formulation of this problem
containing two decoders, one with side information and the other without, is
illustrated in Fig. 1. Heegard and Berger (HB) [3] fully characterized the
asymptotic rate–distortion (R-D) function for the quadratic-Gaussian case of
Fig. 1, and also considered scenarios when different side information is
available at several decoders. In [4], Kaspi also established the R-D function
when side information is available at both encoder and decoder. Intuitively,
one can improve the robustness of a WZ coding system against the absence of
side information by reducing the number of codewords in each bin. Such an
approach yields a trade-off between compression efficiency and system
robustness, encapsulating the essence of the robust DSC scheme developed by
Ishwar _et al._ [5].

Figure 1: Lossy source coding when side information may be absent, also known
as the _Heegard–Berger_ or the _Kaspi_ problem.

Recent studies in [6, 7, 8] have demonstrated that learning-based WZ
compressors can recover different types of interpretable random binning
mechanisms without any specific structure being imposed onto the design. These
results offer empirical evidence that learned distributed compressors can
achieve competitive constructive solutions in the non-asymptotic blocklength,
closely resembling those of handcrafted frameworks such as DISCUS [9], without
requiring a priori knowledge of source statistics. In these learned WZ
compressors, the encoder sends the bin index, and the decoder infers the
quantization index with the help of the side information. Finally, the decoder
reconstructs the source by utilizing the deduced quantization index and the
side information, according to the distortion criterion. In their current
form, it is not clear whether these learned WZ compressors can effectively
adapt to the unavailability of side information.

Expanding on the learned WZ compressors [6, 7, 8], in this paper we find
constructive solutions for the non-asymptotic regime of the HB problem where
side information may be absent, by leveraging universal function approximation
capabilities of artificial neural networks (ANNs) [10, 11]. Similar to [6, 7,
8], our focus is on the one-shot regime, where each source realization is
compressed one at a time, as in popular ANN-based image compressors [12, 13].
We present three unique solutions for the HB problem, where we either jointly
address the quantization and binning components (Figs. 2(a) and 2(b)) or
alternatively, adopt a two-step approach that involves a learned quantizer
coupled with an ideal Slepian-Wolf (SW) coder (Fig. 2(c)).

The paper is organized as follows. We first give an overview of HB theorem,
detailing the achievability result (Section II-A) and the quadratic-Gaussian
setup we consider (Section II-B). To define training objectives for the
proposed robust schemes, we minimize upper bounds on mutual information. These
are formulated using one of three probabilistic models employing ANNs, which
can also interpreted as operational robust distributed schemes (Section III).
Finally, we discuss empirical results (Section IV) and conclude the paper
(Section V).

##  II Heegard–Berger Problem Setup

###  II-A General setting

Let $(X,Y)$ be a pair of random variables with joint distribution $p(x,y)$
defined on the product alphabet $\mathcal{X}\times\mathcal{Y}$, where $X$ and
$Y$ represent the source and decoder-only side information, respectively. As
shown in Fig. 1, the encoder wishes to describe the source $X$ to two
decoders, an _uninformed_ and an _informed_ one, with the latter having access
to the side information $Y$. The first decoder reconstructs $\hat{X}_{1}$ in a
point-to-point fashion while the second decoder reconstructs $\hat{X}_{2}$
with the help of the side information $Y$, under distortion metrics
$d_{i}:\mathcal{X}\times\hat{\mathcal{X}}_{i}\rightarrow\mathbb{R}_{\geq 0}$
where $\hat{\mathcal{X}}_{i}$ are the reconstruction alphabets for
$i\in\\{1,2\\}$. The goal is to find the minimum achievable rate under two
expected distortion constraints: $\mathbb{E}[d_{i}(x,\hat{x}_{i})]\leq D_{i}$
for some $D_{i}\geq 0$. In the asymptotic blocklength regime, where we
consider joint compression of $n$ i.i.d. source samples as
$n\rightarrow\infty$, the following theorem by Heegard and Berger [3]
characterizes the optimal R-D function.

######  Theorem 1.

(Heegard–Berger Theorem [3]) Let $(X,Y)$ be a pair of random variables with
joint distribution $p(x,y)$, representing the source and the correlated side
information respectively, and
$d_{i}:\mathcal{X}\times\hat{\mathcal{X}}_{i}\rightarrow\mathbb{R}_{\geq 0}$
be single letter distortion measures for $i\in\\{1,2\\}$. The R-D function for
$X$ with side information $Y$ available only at one of the decoders is:

$\displaystyle R(D_{1},D_{2})=\min_{p(w,u|x)}(I(X;W)+I(X;U|Y,W)),$ (1)

where the minimization is over all conditional probability distributions
$p(w,u|x)$ such that there exists functions $g_{1}(W)=\hat{X}_{1}$ and
$g_{2}(W,U,Y)=\hat{X}_{2}$ satisfying the distortion constraints

$\displaystyle\mathbb{E}[d_{1}(X,\hat{X}_{1})]\leq
D_{1},\;\;\mathbb{E}[d_{2}(X,\hat{X}_{2})]\leq D_{2},$ (2)

and $(W,U)-X-Y$ is a Markov chain.

We remark that the informed decoder encounters a WZ problem [1] (cf. second
mutual information term in Eq. (1)), while the uninformed decoder is subjected
to an ordinary point-to-point R-D problem (cf. first mutual information term
in Eq. (1)).

Note that the HB theorem considers two distortion constraints as in Eq. (2).
Alternatively, one can opt for a combined weighted sum distortion constraint
given as:

$\mathbb{E}\left[\beta
d_{1}(X,\hat{X}_{1})+(1-\beta)d_{2}(X,\hat{X}_{2})\right]\leq{\boldsymbol{D}},$
(3)

for some $\beta\in[0,1]$. In this case, the asymptotically minimum weighted
distortion for a fixed rate is given as [14]:

$\begin{split}{\boldsymbol{D}}^{*}(R)=&\min_{D_{1},D_{2}:R(D_{1},D_{2})\leq
R}\beta D_{1}+(1-\beta)D_{2},\\\ \end{split}$ (4)

where $R(D_{1},D_{2})$ is given in Theorem 1. This weighted distortion measure
can also be conceptualized as having a $\beta$ probability of receiving no
side information (see the discussion in [14]). As will be seen in Section III,
considering a combined distortion constraint, such as the one in Eq. (4),
simplifies the learning procedure of HB compressors, and yet, still offers
valuable insights about how close they operate with respect to the optimum.

###  II-B Quadratic-Gaussian case

Similarly to the WZ R-D function [1], the HB formula in Eq. (1) has a closed-
form expression only in few special cases [3]. To compare the performance of
our learned distributed compressors to the asymptotic HB R-D function, we
consider a quadratic-Gaussian setup: Suppose $Y=X+N$, where
$N\sim\mathrm{N}(0,\sigma_{n}^{2})$ is independent from
$X\sim\mathrm{N}(0,\sigma_{x}^{2})$, and the distortion metrics are
$d_{i}(x,\hat{x}_{i})=(x-\hat{x}_{i})^{2}$ for $i\in\\{1,2\\}$. The analytical
expression of $R(D_{1},D_{2})$ for this case is given as:

$\begin{split}R(D_{1},D_{2})&=\frac{1}{2}\log\left(\frac{\sigma_{x}^{2}}{\Delta_{1}}\right)+\frac{1}{2}\log\left(\frac{\sigma_{n}^{2}\Delta_{1}}{\Delta_{2}(\Delta_{1}+\sigma_{n}^{2})}\right),\end{split}$
(5)

where
$\Delta_{1}=\min(\sigma_{x}^{2},D_{1}),\;\Delta_{2}=\min([1/\Delta_{1}+1/\sigma_{n}^{2}]^{-1},D_{2}).$
The two terms of Eq. (5) respectively correspond to the two mutual information
terms given in Theorem 1. The first and second expressions in Eq. (5)
represent the rate required to describe $W$ and $U$ under the distortion
constraints of $D_{1}$ and $D_{2}$, respectively. There are four operationally
different R-D regions depending on the value of the pair $(D_{1},D_{2})$. When
$D_{1}\leq\sigma^{2}_{x}$ and
$D_{2}\geq\frac{D_{1}\sigma_{n}^{2}}{D_{1}+\sigma_{n}^{2}}$, the second term
in Eq. (5) vanishes, and the problem degenerates into a standard point-to-
point lossy compression problem i.e., having only the first decoder in Fig. 1.
Similarly, when $D_{1}\geq\sigma^{2}_{x}$, and
$D_{2}\leq\frac{D_{1}\sigma_{n}^{2}}{D_{1}+\sigma_{n}^{2}}$ the first term in
Eq. (5) vanishes, and the problem degenerates to the WZ coding problem i.e.,
having only the second decoder in Fig. 1. When $D_{1}\geq\sigma^{2}_{x}$, and
$D_{2}\geq\frac{D_{1}\sigma_{n}^{2}}{D_{1}+\sigma_{n}^{2}}$, both terms
vanish, and the distortion constraints can be trivially satisfied with zero
rate. The most operationally interesting region arises when
$D_{1}\leq\sigma^{2}_{x}$ and
$D_{2}\leq\frac{D_{1}\sigma_{n}^{2}}{D_{1}+\sigma_{n}^{2}}$. Here, both rate
terms are non-zero, forcing the encoder to allocate its total rate budget
between the point-to-point R-D problem of describing $W$ and the WZ R-D
problem of describing $U$. In Appendix A, we discuss the optimum weighted
distortion outlined in Eq. (4) for the quadratic-Gaussian setting we consider.

##  III Operational Neural Heegard–Berger Schemes

For our learned compressors, we first consider the system model in Fig. 2(a),
which corresponds to a straightforward parametrization of the HB setup
depicted in Fig. 1. Note that in this case, a joint description is sent to
both uninformed and informed decoders, the latter having access to the side
information $Y$. By introducing the auxiliary variable $V=(W,U)$, we first
assume that the encoder in the achievability proof of HB theorem can be
represented by a probability model $p_{\boldsymbol{\theta}}(v|x)$ with
parameters $\boldsymbol{\theta}$. This formulation, which we name as _joint_ ,
yields the following variational upper bound:

$\displaystyle I(X;W)+I(X;U|Y,W)\leq$ $\displaystyle I(X;U,W)=I(X;V),$ (6)
$\displaystyle\leq\mathbb{E}_{\begin{subarray}{c}p(x)\\\
p_{\boldsymbol{\theta}}(v|x)\end{subarray}}\bigg{[}\log\frac{p_{\boldsymbol{\theta}}(v|x)}{{q_{\boldsymbol{\eta}}(v)}}\bigg{]},$
(7)
$\displaystyle=\mathbb{E}_{\begin{subarray}{c}p(x)\end{subarray}}\;\mathrm{r}_{\mathrm{j}}(x),$
(8)

where $q_{\boldsymbol{\eta}}(v)$, with parameters $\boldsymbol{\eta}$, is a
model of the distribution $p(v)\triangleq p(w,u)$, which is generally not
known in closed form. $\mathrm{r}_{\mathrm{j}}(x)$ refers to the expectation
in Eq. (7) with respect to $p_{\boldsymbol{\theta}}(v|x)$. The proof of the
upper bound in Eq. (6) can be found in Appendix B. The upper bound in Eq. (8)
follows from cross-entropy [15] being larger or equal to entropy [2].

Next, we consider the system models illustrated in Figs. 2(b) and 2(c). In
these schemes, we opt for a layered encoding approach, aligned more closely
with the HB theorem, where we separately encode the auxiliary variables $W$
and $U$ with probability models $p_{\boldsymbol{\omega}}(w|x)$ and
$p_{\boldsymbol{\gamma}}(u|w,x)$ (with parameters $\boldsymbol{\omega}$ and
$\boldsymbol{\gamma}$), respectively. For our objective functions, building
onto Theorem 1, we will consider either a _marginal_ or a _conditional_
formulation [6] whose variational upper bounds respectively are:

$\displaystyle I(X;W)$
$\displaystyle+I(X;U|Y,W)\leq\mathbb{E}_{\begin{subarray}{c}p(x)\\\
p_{\boldsymbol{\omega}}(w|x)\\\
p_{\boldsymbol{\gamma}}(u|w,x)\end{subarray}}\bigg{[}\log\frac{p_{\boldsymbol{\omega}}(w|x)}{{q_{\boldsymbol{\zeta}}(w)}}$
$\displaystyle\hskip
14.22636pt+\log\frac{p_{\boldsymbol{\gamma}}(u|w,x)}{q_{\boldsymbol{\psi}}{(u|w)}}\bigg{]}=\mathbb{E}_{\begin{subarray}{c}p(x)\end{subarray}}\;\mathrm{r}_{\mathrm{m}}(x),$
(9) $\displaystyle I(X;W)$
$\displaystyle+I(X;U|Y,W)\leq\mathbb{E}_{\begin{subarray}{c}p(x,y)\\\
p_{\boldsymbol{\omega}}(w|x)\\\
p_{\boldsymbol{\gamma}}(u|w,x)\end{subarray}}\bigg{[}\log\frac{p_{\boldsymbol{\omega}}(w|x)}{{q_{\boldsymbol{\zeta}}(w)}}$
$\displaystyle\hskip
14.22636pt+\log\frac{p_{\boldsymbol{\gamma}}(u|w,x)}{q_{\boldsymbol{\mu}}{(u|w,y)}}\bigg{]}=\mathbb{E}_{\begin{subarray}{c}p(x,y)\end{subarray}}\;\mathrm{r}_{\mathrm{c}}(x,y).$
(10)

where $\mathrm{r}_{\mathrm{m}}(x)$ and $\mathrm{r}_{\mathrm{c}}(x,y)$ refer to
expectations in Eqs. (III) and (III) with respect to
$p_{\boldsymbol{\omega}}(w|x)$ and $p_{\boldsymbol{\gamma}}(u|w,x)$
respectively. Similar to the marginal variant, $q_{\boldsymbol{\zeta}}(w)$,
with parameters $\boldsymbol{\zeta}$, is a model of the distribution $p(w)$,
and $q_{\boldsymbol{\psi}}(u|w)$ and $q_{\boldsymbol{\mu}}(u|w,y)$ (with
parameters $\boldsymbol{\psi}$ and $\boldsymbol{\mu}$, respectively) are two
different models of the distribution $p(u|w,y)$.

(a)

(b)

(c)

Figure 2: The three lossy compression systems that we consider: (a) learned
compressor sending a joint description for both decoders using a classic
entropy coder (i.e., the _joint_ formulation, see Eq. (13)), (b) learned
compressors sending individual descriptions to both decoders using classic
entropy coders (i.e., the _marginal_ formulation, see Eq. (14)), and (c) using
a combination of a classic entropy coder and an ideal Slepian–Wolf coder
(i.e., the _conditional_ formulation, see Eq. (15)). The learned parameters
are indicated in green.

Figure 3: Visualizations of the learned optimized encoders,
$w=\operatorname*{arg\,max}_{h}p_{\boldsymbol{\omega}}(h|x)$ and
$u=\operatorname*{arg\,max}_{l}p_{\boldsymbol{\gamma}}(l|w,x)$ in Eq. (III),
and of the decoders, $\hat{x}_{1}=g_{\boldsymbol{\kappa}}(w)$ and
$\hat{x}_{2}=g_{\boldsymbol{\iota}}(w,u,y)$ in Eq (12), for the marginal
formulation (see Fig. 2(b) and Eq. (14)). Left: The dashed vertical red lines
are quantization boundaries induced by $w$, and the codebook points represent
the outputs of the decoder $g_{\boldsymbol{\kappa}}$ to which all source
values within the corresponding quantization region are mapped. The height of
each codebook stalk represents the likelihood of that code vector under the
entropy model $q_{{\boldsymbol{\zeta}}}$ (see Eq. (III)). Right: The dashed
horizontal red lines coincide with those depicted in the left panel induced by
$w$, while the dotted-dashed horizontal blue lines are quantization boundaries
induced by $u$. The colors between each boundary represent a unique $(w,u)$
pair. We depict the decoding function learned by $g_{\boldsymbol{\iota}}$ with
the solid lines, each representing a different pair of $(w,u)$ as inputs
within its respective quantization region, distinguished by unique colors. The
experimental setup parameters (see Section II-B) are configured as
$\sigma^{2}_{x}=1.00$, $\sigma^{2}_{n}=0.01$ and $\beta=0.20$. This model
attains -15.55 dB at a rate of 2.85 bits (see Fig. 5 in Appendix E).

We will utilize these probabilistic models (i.e.,
$p_{\boldsymbol{\theta}}(v|x),p_{\boldsymbol{\omega}}(w|x)$ and
$p_{\boldsymbol{\gamma}}(u|w,x)$) to aid in the learning process of respective
encoders. We set the encoder outputs in a deterministic way, similar to [6],
e.g., $v=\operatorname*{arg\,max}_{h}{p_{\boldsymbol{\theta}}(h|x)}$. To
actualize a practice-oriented compression setting, we also set all encoder
outputs (i.e., $V$, $W$, and $U$) as discrete. Similarly to [6, 8, 7], without
loss of generality, we define all probabilistic models as discrete
distributions with probabilities
$P_{k}=\frac{\exp\alpha_{k}}{\sum_{i=1}^{K}\exp\alpha_{i}}$ for
$k\in\\{1,\dots,K\\}$, where $K$ is a model parameter. The unnormalized log-
probabilities (_logits_) are either directly treated as learnable parameters
(i.e., $q_{\boldsymbol{\eta}}(v)$ and $q_{\boldsymbol{\zeta}}(w)$), or
computed by ANNs as functions of the conditioning variable (e.g., $w$ and $y$
for $q_{\boldsymbol{\mu}}{(u|w,y)}$). These design choices keep the parametric
families as general as possible without imposing any structure. Specifically,
this allows the encoders to learn, when needed, quantization schemes that may
incorporate discontiguous bins in the source space, resembling the random
binning operation in the achievability part of the HB and WZ theorems.
Previously, it was demonstrated in [6, 8, 7] that a similarly parametrized
neural distributed compressor can learn different types of interpretable
binning mechanisms for the WZ problem, such as periodic-like mappings for the
quadratic-Gaussian case.

Since we focus on a weighted distortion metric (see Eq. (4)), instead of
having individual distortion constraints as is the case for the HB theorem
(see Section II), we define the following distortion functions for the cases
where there is either joint (see Fig. 2(a)) or separate descriptions (see Fig.
2(b) and Fig. 2(c)) to the decoders, respectively, as:

$\displaystyle\mathrm{d}_{\mathrm{j}}(x,y)$ $\displaystyle=\beta
d(x,g_{\boldsymbol{\phi}}(v))+(1-\beta)d(x,g_{\boldsymbol{\tau}}(v,y)),$ (11)
$\displaystyle\mathrm{d}_{\mathrm{s}}(x,y)$ $\displaystyle=\beta
d(x,g_{\boldsymbol{\kappa}}(w))+(1-\beta)d(x,g_{\boldsymbol{\iota}}(u,w,y)),$
(12)

where $g_{\boldsymbol{\phi}}(v)$ and $g_{\boldsymbol{\kappa}}(w)$ denote the
uninformed decoders, represented by ANNs with parameters $\boldsymbol{\phi}$
and $\boldsymbol{\kappa}$ respectively, which yield $\hat{x}_{1}$. Similarly,
$g_{\boldsymbol{\tau}}(v,y)$ and $g_{\boldsymbol{\iota}}(u,w,y)$ correspond to
the informed decoders, represented by ANNs with parameters $\boldsymbol{\tau}$
and $\boldsymbol{\iota}$ respectively, which output $\hat{x}_{2}$. Building on
the upper bounds developed in Eqs. (8), (III) and (III) in tandem with the
distortion metrics defined in Eqs. (11) and (12), this yields the loss
functions for three different variants we consider:

$\displaystyle
L_{\mathrm{j}}(\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\phi},\boldsymbol{\tau})=\mathbb{E}[\mathrm{r}_{\mathrm{j}}(x)+\lambda\cdot\mathrm{d}_{\mathrm{j}}(x,y)],$
(13) $\displaystyle
L_{\mathrm{m}}(\boldsymbol{\omega},\boldsymbol{\gamma},\boldsymbol{\zeta},\boldsymbol{\psi},\boldsymbol{\kappa},\boldsymbol{\iota})=\mathbb{E}[\mathrm{r}_{\mathrm{m}}(x)+\lambda\cdot\mathrm{d}_{\mathrm{s}}(x,y)],$
(14) $\displaystyle
L_{\mathrm{c}}(\boldsymbol{\omega},\boldsymbol{\gamma},\boldsymbol{\zeta},\boldsymbol{\mu},\boldsymbol{\kappa},\boldsymbol{\iota})=\mathbb{E}[\mathrm{r}_{\mathrm{c}}(x,y)+\lambda\cdot\mathrm{d}_{\mathrm{s}}(x,y)],$
(15)

where we relax the constrained formulation of the HB theorem to unconstrained
ones using Lagrange multipliers. Here,
$\\{\boldsymbol{\theta},\boldsymbol{\eta},\boldsymbol{\phi},\boldsymbol{\tau},\boldsymbol{\omega},\boldsymbol{\gamma},\boldsymbol{\zeta},\boldsymbol{\psi},\boldsymbol{\mu},\boldsymbol{\kappa},\boldsymbol{\iota}\\}$
are optimization parameters, and $\lambda>0$ controls the trade-off. We can
obtain different points in the achievable R-D region induced by the weighted
distortion constraint by simply varying the parameter $\lambda$. The optimized
$p_{\boldsymbol{\theta}}(v|x),p_{\boldsymbol{\omega}}(w|x)$ and
$p_{\boldsymbol{\gamma}}(u|w,x)$ models yield the ANN-based encoders depicted
in Fig. 2. Similarly, the optimized
$\big{(}g_{\boldsymbol{\phi}}(v),g_{\boldsymbol{\tau}}(v,y)\big{)}$ and
$\big{(}g_{\boldsymbol{\kappa}}(w),g_{\boldsymbol{\iota}}(u,w,y)\big{)}$
correspond to a pair of uninformed-informed decoder components for joint and
separate description cases, respectively. We discuss the operational meaning
of these different schemes in Appendix C.

$0$$0.2$$0.4$$0.6$$0.8$$1$$1.2$$1.4$$1.6$$1.8$$2$$2.2$$2.4$$2.6$$2.8$$-25$$-20$$-15$$-10$rate
[bits]expected distortion [dB]neural scheme established with
$L_{\mathrm{m}}$neural scheme established with $L_{\mathrm{j}}$neural scheme
established with $L_{\mathrm{c}}$asymptotic R-D Heegard–Berger + $1.53$
dBasymptotic R-D Heegard–Berger (a)

$0$$0.2$$0.4$$0.6$$0.8$$1$$1.2$$1.4$$1.6$$1.8$$2$$2.2$$2.4$$2.6$$2.8$$-30$$-20$$-10$rate
[bits]distortion [dB]marginal formulation in [6] $(\beta=0)$asymptotic R-D
point-to-pointasymptotic R-D Heegard–Berger ($D_{2}$ only)asymptotic R-D
Wyner–Ziv + 1.53 dBasymptotic R-D Wyner–Ziv (b)

Figure 4: Rate–distortion (R-D) performances obtained with joint, marginal and
conditional formulations (see Eqs. (13), (14) and (15)), where experimental
setup parameters (see Section II-B) are set as $\sigma^{2}_{x}=1.00$,
$\sigma^{2}_{n}=0.10$ and $\beta=0.01$. In both panels, we plot the empirical
results versus the asymptotic bounds. We provide the expected distortion
achieved by two decoders (see Fig. 2) in the left panel, while we only plot
the distortion attained by the informed decoder, which has access to side
information, in the right panel. The 1.53 dB refers to the mean-squared error
gap that the entropy-constrained one-shot lattice quantizer is subjected to in
high-rate regime [16].

Following the popular class of neural compressors [12, 13, 17], we use
stochastic gradient descent (SGD) to jointly optimize all learnable
parameters. Since SGD relies on a Monte Carlo approximation of the
expectations in the loss functions, we use the Gumbel-max technique, as in
previous work [6], for sampling from discrete distributions. Similarly, we
also leverage Concrete distributions [18] to aid stochastic optimization.
Additional details about our experimental setup are provided in Appendix D.

##  IV Discussion

We first consider a setting where the proposed neural compressors clearly
recover some elements of the optimal theoretical solution for the lossy source
coding problem where side information may be absent. The visualization of the
learned compressor obtained with the marginal scheme (see Fig. 2(b)) is
provided in Fig. 3. As seen in the left panel of the figure, the neural
encoder $p_{\boldsymbol{\omega}}$ quantizes the source in a manner similar to
standard point-to-point lossy compression. Looking at the right panel of the
figure, we remark that the discontiguous quantization bins are learned as the
neural compressor exhibits periodic-like mappings in the source space, akin to
the binning-like behavior recovered by the neural WZ compressor proposed in
[6, 7, 8]. We note that this grouping behavior is aligned with the
achievability part of the HB theorem, which yields a rate discount similar to
the random binning argument invoked in the WZ proof. The figure demonstrates
that this learned distributed compressor, formulated based on the proposed
marginal approach, exhibits a greater adaptability to robust scenarios
compared to the WZ compression case previously explored in [6, 7, 8]. This is
evident from the model’s ability to recover both WZ and point-to-point coding
strategies, despite not imposing any explicit structure exploiting the source
knowledge onto the design. Although we noticed similar behaviors persisting in
different experimental configurations other than the one considered in Fig. 3
for the marginal variant, we did not observe such consistent binning-like
behavior in the visualizations of the compressors obtained by the joint and
conditional formulations (not shown). For the conditional scheme, similar to
the explanation in [6], we posit that this is due to having an ideal SW coder
in the system design (see Fig. 2(c)), unlike the marginal variant (see Fig.
2(b)). We speculate that this choice of entropy coding scheme incentivizes the
model to delegate the task of binning solely to the ideal SW code, rather than
to the learned encoder.

In Fig. 4(a), we provide R-D performances achieved with these three different
formulations. The joint and marginal variants achieve similar performances,
whereas the conditional model outperforms them, approaching the asymptotic HB
bound. We attribute the comparable performances of the marginal and joint
schemes to the operational equivalence between having two layered descriptions
managed by separate classical entropy coders (see Eq. (III)) and having a
single classical entropy coder with the input being the unified description of
these two representations (see Eq. (8)). We explain the improved R-D
performance of the conditional formulation as follows. The SW code that this
variant employs, which may leverage high-dimensional channel codes (e.g., as
in DISCUS [9]), enables binning over long sequences, i.e., in a multi-shot
fashion. This type of _compress-bin_ [19] is much more efficient than the one
that could be attained by the learned encoder, which could only bin a one-shot
manner as it compresses each source realization one at a time. We refer the
reader to Appendix E for a discussion on an additional set of experiments.

In Fig.4(b), we illustrate the trade-off between system robustness and
compression efficiency by evaluating distortion attained only with the
informed decoders of all three proposed schemes, along with the learned WZ
compressor proposed in [6] (which operationally coincides with having
$\beta=0$ in Eq. (4), that is decoder-only side information is always assumed
to be available). Recalling that all three proposed learned compressors are
now also formulated to accommodate scenarios where side information may be
absent, which is not the case for the learned WZ compressor studied in [6], a
decline in distortion is expected. Interestingly, the conditional variant
again surpasses all other learned schemes considered, underscoring the
efficiency of high-dimensional binning capability facilitated by the SW coder.

##  V Conclusion

In this work, we have proposed three learning-based solutions to the problem
of lossy source coding when side information may be absent, for which the
optimal theoretical solution is asymptotic and non-constructive. By explicitly
visualizing the behavior of the learned encoders and decoders, we demonstrated
that they recover schemes aligned with the characteristics of the
achievability part of the HB theorem, that is WZ and standard lossy source
coding. Future research directions include analyzing the robustness in fully
distributed compression scenarios, such as the Berger–Tung problem [20].

## References

  * [1] A. Wyner and J. Ziv, “The rate–distortion function for source coding with side information at the decoder,” _IEEE Transactions on Information Theory_ , vol. 22, no. 1, pp. 1 – 10, 1976.
  * [2] T. M. Cover and J. A. Thomas, _Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing)_. USA: Wiley-Interscience, 2006.
  * [3] C. Heegard and T. Berger, “Rate distortion when side information may be absent,” _IEEE Transactions on Information Theory_ , vol. 31, no. 6, pp. 727–734, 1985.
  * [4] A. Kaspi, “Rate-distortion function when side-information may be present at the decoder,” _IEEE Transactions on Information Theory_ , vol. 40, no. 6, pp. 2031–2034, 1994.
  * [5] P. Ishwar, R. Puri, S. Pradhan, and K. Ramchandran, “On compression for robust estimation in sensor networks,” in _IEEE International Symposium on Information Theory, 2003. Proceedings._ , 2003, pp. 193–.
  * [6] E. Özyılkan, J. Ballé, and E. Erkip, “Learned Wyner–Ziv compressors recover binning,” in _2023 IEEE International Symposium on Information Theory (ISIT)_ , 2023, pp. 701–706.
  * [7] E. Ozyilkan, J. Ballé, and E. Erkip, “Neural distributed compressor discovers binning,” 2023. [Online]. Available: https://arxiv.org/abs/2310.16961
  * [8] E. Ozyilkan, J. Ballé, and E. Erkip, “Neural distributed compressor does binning,” in _ICML 2023 Workshop Neural Compression: From Information Theory to Applications_ , 2023.
  * [9] S. Pradhan and K. Ramchandran, “Distributed source coding using syndromes (DISCUS): Design and construction,” _IEEE Transactions on Information Theory_ , vol. 49, no. 3, pp. 626–643, 2003.
  * [10] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” _Neural Networks_ , vol. 2, no. 5, p. 359–366, jul 1989.
  * [11] M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, “Multilayer feedforward networks with a nonpolynomial activation function can approximate any function,” _Neural Networks_ , vol. 6, no. 6, pp. 861–867, Jan. 1993. [Online]. Available: https://doi.org/10.1016/s0893-6080(05)80131-5
  * [12] J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” in _International Conference on Learning Representations (ICLR)_ , 2017.
  * [13] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” 2018.
  * [14] C. T. K. Ng, C. Tian, A. J. Goldsmith, and S. Shamai, “Minimum expected distortion in gaussian source coding with fading side information,” _IEEE Transactions on Information Theory_ , vol. 58, no. 9, pp. 5725–5739, 2012.
  * [15] S. Kullback, “Information theory and statistics,” 1968.
  * [16] R. Gray and D. Neuhoff, “Quantization,” _IEEE Transactions on Information Theory_ , vol. 44, no. 6, pp. 2325–2383, 1998.
  * [17] J. Ballé, P. A. Chou, D. Minnen, S. Singh, N. Johnston, E. Agustsson, S. J. Hwang, and G. Toderici, “Nonlinear transform coding,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 15, no. 2, pp. 339–353, 2021.
  * [18] C. J. Maddison, A. Mnih, and Y. W. Teh, “The concrete distribution: A continuous relaxation of discrete random variables,” in _International Conference on Learning Representations_ , 2017. [Online]. Available: https://openreview.net/forum?id=S1jE5L5gl
  * [19] A. E. Gamal and Y.-H. Kim, _Network Information Theory_. USA: Cambridge University Press, 2012.
  * [20] T. Berger, _Multiterminal Source Coding. In G. Longo (Ed.), The Information Theory Approach to Communications_. USA: New York, NY, USA: Springer-Verlag, 1977.
  * [21] J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, “JAX: Composable transformations of Python+NumPy programs,” 2018. [Online]. Available: http://github.com/google/jax
  * [22] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014. [Online]. Available: https://arxiv.org/abs/1412.6980

##  Appendix A The Minimum Distortion For Eq. (4)

When considering the weighted distortion metric as in Eq. (4) for the
quadratic-Gaussian setup of the HB problem, the minimum distortion is provided
by [14]:

$\displaystyle{\boldsymbol{D}}^{*}(R)=\beta D_{1}^{*}+(1-\beta)D_{2}^{*},$
(16)

where

$\displaystyle
D_{1}^{*}=\Bigg{[}\left(2^{-2R}\sigma_{x}^{2}\sigma_{n}^{2}\frac{1-\beta}{\beta}\right)^{1/2}-\sigma_{n}^{2}\Bigg{]}_{[D_{1}^{-},D_{1}^{+}]},$
$\displaystyle
D_{2}^{*}=\Bigg{[}\left(2^{-2R}\sigma_{x}^{2}\sigma_{n}^{2}\frac{\beta}{1-\beta}\right)^{1/2}\Bigg{]}_{[D_{2}^{-},D_{2}^{+}]}.$

Here, we used the notation $[x]_{[a,b]}=\min(\max(a,x),b)$, and

$\displaystyle D_{1}^{-}$
$\displaystyle=2^{-2R}\sigma_{x}^{2}\qquad\quad\qquad\;\;,\;D_{1}^{+}=\sigma_{x}^{2},$
$\displaystyle D_{2}^{-}$
$\displaystyle=2^{-2R}\left(\frac{1}{\sigma_{x}^{2}}+\frac{1}{\sigma_{n}^{2}}\right)^{-1}\;,\;D_{2}^{+}=\left(\frac{2^{2R}}{\sigma_{x}^{2}}+\frac{1}{\sigma_{n}^{2}}\right)^{-1}.$

To further explain the above expressions, $D_{1}^{-}$ represents the minimum
achievable distortion for $\hat{X}_{1}$ when allocating the entire rate $R$
for the description of $W$, while $D_{1}^{+}$ denotes the worst-case
distortion attained for $\hat{X}_{1}$ when the entire rate is dedicated to the
description of $U$. Likewise, $D_{2}^{-}$ and $D_{2}^{+}$ are the minimum and
maximum distortions for $\hat{X}_{2}$ achieved when the rate $R$ is utilized
for $U$ and $W$, respectively.

##  Appendix B Proof of the Upper Bound in Eq. (6)

Building onto Theorem 1, we can re-write the second mutual information term in
Eq. (1) as:

$\displaystyle I(X;U|Y,W)$ $\displaystyle=I(U;X,Y|W)-I(U;Y|W),$ (17)
$\displaystyle=I(U;X|W)-I(U;Y|W),$ (18)

where Eq. (17) follows from the chain rule for mutual information, and Eq.
(18) is due to the Markov chain $(W,U)-X-Y$. Then, Eq. (1) becomes:

$\displaystyle I(X;W)+I(X;U|Y,W)$ $\displaystyle=I(X;W)+I(U;X|W)-I(U;Y|W),$
$\displaystyle\leq I(X;W)+I(U;X|W),$ (19) $\displaystyle=I(X;W,U),$ (20)
$\displaystyle=I(X;V),$ (21)

where Eq. (19) is due to the non-negativity of mutual information, Eq. (20)
follows from the chain rule for mutual information, and Eq. (21) is due to the
definition of $V=(W,U)$.

##  Appendix C Operational Heegard–Berger Compressors Illustrated in Figs.
2(a), 2(b) and 2(c)

The upper bound in Eq. (8) corresponds to the compression rate of a system
employing a one-shot encoder in the _joint_ model and an entropy code which
asymptotically achieves a cross-entropy rate of

$\displaystyle\mathbb{E}_{p(x)}[\mathbb{E}_{v\sim
p_{\boldsymbol{\theta}}(v|x)}[-\log q_{\boldsymbol{\eta}}(v)]].$ (22)

Similarly, the upper bounds in Eqs. (III) and (III) coincide with the
compression rates of designs having one-shot compressors coupled with either
classic entropy coders (i.e., the _marginal_ model) or a combination of a
classic entropy coder and an ideal SW coder (i.e., the _conditional_ model),
respectively, which asymptotically achieve the cross-entropy rates of

$\displaystyle\mathbb{E}_{p(x)}[\mathbb{E}_{w\sim
p_{\boldsymbol{\omega}}(w|x),u\sim p_{\boldsymbol{\gamma}}(u|w,x)}[-\log
q_{\boldsymbol{\zeta}}(w)-\log q_{\boldsymbol{\psi}}(u|w)]],$ (23)

and

$\displaystyle\mathbb{E}_{p(x,y)}[\mathbb{E}_{w\sim
p_{\boldsymbol{\omega}}(w|x),u\sim p_{\boldsymbol{\gamma}}(u|w,x)}[-\log
q_{\boldsymbol{\zeta}}(w)-\log q_{\boldsymbol{\mu}}(u|w,y)]].$ (24)

Therefore, by minimizing $L_{\mathrm{j}},L_{\mathrm{m}},$ and $L_{c}$ in Eqs.
(13), (14) and (15), we optimize the robust operational distributed
compression schemes depicted in Figs. 2(a), 2(b) and 2(c), respectively, in an
end-to-end fashion. We remark that unlike the neural WZ compressors proposed
in [6, 7, 8], the proposed distributed compressors illustrated in Fig. 2 can
by design accommodate for scenarios where decoder-only side information may be
absent.

##  Appendix D Additional Details on Experimental Setup

For the probabilistic models conditioned on other variables (e.g.,
$p_{\boldsymbol{\theta}},q_{\boldsymbol{\psi}}$) and for the decoding
functions (e.g., $g_{\boldsymbol{\phi}},g_{\boldsymbol{\kappa}}$), we employ
ANNs with three dense layers. Each layer, except the last one, consists of 100
units followed by a leaky rectified linear unit as the activation function. As
for the other probabilistic model (i.e., $q_{\boldsymbol{\eta}}$ and
$q_{\boldsymbol{\zeta}}$), we directly treat them as learnable parameters. In
our experiments, we observed that increasing the dimensions of ANNs or
employing different activation functions did not lead to better performance.
For probabilistic models and functions that have more than one input, we feed
ANNs with concatenated version of the inputs. We conduct our experiments using
the JAX [21] framework. We train all learning-based compressors for 500 epochs
with randomly initialized network weights and utilize Adam [22], a widely used
variant of SGD. We employ a learning rate of $1\times 10^{-4}$ and a batch
size of $B=1024$, which aligns with the number of realizations sampled from
the aforementioned correlation model in Section II-B. Similar to [6], we
likewise choose
$q_{\boldsymbol{\eta}}(v),q_{\boldsymbol{\zeta}}(w),q_{\boldsymbol{\psi}}{(u|w)}$
and $q_{\boldsymbol{\mu}}{(u|w,y)}$ as Concrete during training to align the
distributions of $v,w$ and $u$ samples.

$0$$0.5$$1$$1.5$$2$$2.5$$3$$3.5$$4$$-20$$-15$$-10$$-5$rate [bits]expected
distortion [dB]neural scheme established with $L_{\mathrm{m}}$neural scheme
established with $L_{\mathrm{j}}$neural scheme established with
$L_{\mathrm{c}}$asymptotic R-D Heegard–Berger + $1.53$ dBasymptotic R-D
Heegard–Berger (a)

$0$$0.5$$1$$1.5$$2$$2.5$$3$$3.5$$4$$-30$$-25$$-20$rate [bits]distortion
[dB]marginal formulation in [6] $(\beta=0)$asymptotic R-D point-to-
pointasymptotic R-D Heegard–Berger ($D_{2}$ only)asymptotic R-D Wyner–Ziv +
1.53 dBasymptotic R-D Wyner–Ziv (b)

Figure 5: Rate–distortion (R-D) performances obtained with joint, marginal and
conditional formulations (see Eqs. (13), (14) and (15)), where experimental
setup parameters (see Section II-B) are set as $\sigma^{2}_{x}=1.00$,
$\sigma^{2}_{n}=0.01$ and $\beta=0.20$. In both panels, we plot the empirical
results versus the asymptotic bounds. We provide the expected distortion
achieved by two decoders (see Fig. 2) in the left panel, while we only plot
the distortion attained by the informed decoder, which has access to side
information, in the right panel. The 1.53 dB refers to the mean-squared error
gap that the entropy-constrained one-shot lattice quantizer is subjected to in
high-rate regime [16].

##  Appendix E Additional Experiments

Fig. 5(a) shows R-D results for all three different formulations we consider
for the HB problem (i.e., _joint_ , _marginal_ and _conditional_ as explained
in Section III), where we set experimental setup parameters as
$\sigma_{x}^{2}=1.00$, $\sigma_{n}^{2}=0.01$ and $\beta=0.20$ (see Fig. 3). In
the asymptotic blocklength regime, under this choice of correlation model
parameters, the optimal R-D trade-off for the minimum expected distortion can
be attained by operating in the point-to-point R-D region for all rate values
(see Section II-B and Appendix A for the discussion on different operational
R-D regions). This implies that the optimal coding strategy is to use the
entire rate budget for the description of $W$. We remark that our learned
distributed compressors emulate this optimal approach. The joint and the
conditional models (nearly) allocate all of their rates to the description of
$W$, and achieve R-D performance close to the asymptotic R-D bound. As in Fig.
4(a), the compressors do not reach the asymptotic R-D bound as they operate in
a one-shot quantization regime in tandem with variable rate entropy coding. We
observe that the marginal model behaves similarly to the other schemes at low
rates, but deviates from them at higher rate values. We speculate that this
occurs because, at higher rates, the marginal model, employing a layered
encoding strategy, tends to allocate some rate for the description of $U$,
which may not be optimal given this specific choice of correlation model
parameters.

Similar to the comparison shown in Fig. 4(b), we illustrate in Fig.5(b) the
trade-off between system robustness and compression efficiency by evaluating
distortions attained only with the informed decoders of all three proposed
schemes, along with the learned WZ compressor established in [6] (which
operationally coincides with having $\beta=0$ in Eq. (4), that is this learned
compressor always presume the presence of decoder-only side information). We
observe that unlike the scenario depicted in Fig.4(b), all three variants
exhibit inferior performance compared to this learned WZ compressor, primarily
due to setting a higher $\beta$ value. Hence, given that all of the three
learned schemes are trained to prioritize the uninformed decoder more than the
experimental setup considered in Fig. 4(b), such a decrease in performance
considering the WZ compression scenario is anticipated. Moreover, the decline
in distortion at higher rates for the marginal formulation illustrated in Fig.
5(a) is reflected as a minor R-D enhancement in Fig. 5(b). We hypothesize that
this happens because, at higher rates, the marginal approach starts allocating
more rates to the description of $U$. On the other hand, this marginal scheme
also seeks to minimize this rate budget of $U$ by recovering periodic-like
mappings, akin to binning, at higher rate values as observed in the right
panel of Fig. 3. This, in return, results in a slightly improved distortion
performance attained by the informed decoder, as shown in Fig. 5(b). We remark
that the HB curve in Fig.5(b) shows the distortion attained by the informed
decoder of the HB in Fig.5(a) which is optimal for expected distortion. Since
our learned compressors are trained for minimizing the expected distortion as
well, the HB curve in Fig.5(b) is not necessarily a lower bound for the
distortions attained by the informed decoders of our models.