# Towards General Deep Leakage in Federated Learning

Jiahui Geng1, Yongli Mou 2, Feifei Li 2, Qing Li 1, Oya Beyan 3, Stefan Decker
2, Chunming Rong 1

###### Abstract

Unlike traditional central training, federated learning (FL) improves the
performance of the global model by sharing and aggregating local models rather
than local data to protect the users’ privacy. Although this training approach
appears secure, some research has demonstrated that an attacker can still
recover private data based on the shared gradient information. This on-the-fly
reconstruction attack deserves to be studied in depth because it can occur at
any stage of training, whether at the beginning or at the end of model
training; no relevant dataset is required and no additional models need to be
trained. We break through some unrealistic assumptions and limitations to
apply this reconstruction attack in a broader range of scenarios. We propose
methods that can reconstruct the training data from shared gradients or
weights, corresponding to the FedSGD and FedAvg usage scenarios, respectively.
We propose a zero-shot approach to restore labels even if there are duplicate
labels in the batch. We study the relationship between the label and image
restoration. We find that image restoration fails even if there is only one
incorrectly inferred label in the batch; we also find that when batch images
have the same label, the corresponding image is restored as a fusion of that
class of images. Our approaches are evaluated on classic image benchmarks,
including CIFAR-10 and ImageNet. The batch size, image quality, and the
adaptability of the label distribution of our approach exceed those of
GradInversion, the state-of-the-art.

Figure 1: Our Reconstruction Results on ImageNet with Batch Size 16. The left
and right pictures show the results of two matches respectively. 8 pictures in
a row, the top two rows are recovered pictures, the bottom two rows are ground
truth.

##  1 Introduction

With the expansion of information technology and smart devices, massive
amounts of data are being produced, collected, stored, processed, and utilized
incessantly. In the era of big data, it brings broad social changes and
security concerns like data abuse and information leakage. Society raises the
concerns of privacy, which bring the world-widely emergence of data protection
legislation, such as the GDPR in the European Union, the HIPAA Act in the USA
and the Data Protection Act (PDA) in the UK. Central machine learning
approaches are becoming insecure due to the potential risk of privacy leakage
during data collecting and sharing.

Federated learning (FL), proposed as a privacy-preserving distributed learning
paradigm, has gained more researchers’ attention. The underlying idea is that
all participants collaboratively train a shared model without exchanging local
private data. In the FL scenarios, the central server broadcasts the current
model to each client, and each client will train the model on its local data.
At each communication round, the central server will aggregate gradients or
weights updated from a subset of clients into the global model. The
corresponding mechanisms were stated in FedSGD and FedAvg (McMahan et al.
2016, 2017).

Despite the restricted access to raw data, recent work has shown that FL also
faces a variety of privacy attacks, such as membership inference attacks
(Melis et al. 2019; Nasr, Shokri, and Houmansadr 2019), property inference
attacks (Melis et al. 2019; Wang et al. 2019), and reconstruction attacks
(Fredrikson, Jha, and Ristenpart 2015; Hitaj, Ateniese, and Pérez-Cruz 2017;
Salem et al. 2020). Data reconstruction attacks are the most dangerous and
challenging attacks with severe consequences. Data reconstruction attacks
usually require domain-relevant datasets, additional training of inverse
models (Fredrikson, Jha, and Ristenpart 2015), or GANs (Hitaj, Ateniese, and
Pérez-Cruz 2017; Salem et al. 2020).

There has been a series of studies on reconstructing training data based on
shared gradients and weights in a FL scenario. These attacks are white-box
attacks, where the attacker knows almost all model parameters and
hyperparameters, including learning rate and model architecture. Any FL
participant that has access to the shared model updates can execute the
attack. DLG (Zhu, Liu, and Han 2019) was the first to recover both images and
label simultaneously by gradient matching. However, their method is limited by
the image scale and batch size. The authors of iDLG (Zhao, Mopuri, and Bilen
2020) found that zero-shot label inference was more accurate when batch size
equals 1. However, their algorithm did not support the case where the batch
size is greater than 1. InvertingGradients (Geiping et al. 2020) has
demonstrated that it is possible to reconstruct images at high resolution
(ImageNet) from the gradients no matter trained or untrained models. Total
variation dramatically improves the quality of large-scale image recovery.
They experimented with images of different labels in minibatch to avoid the
ambiguity of the reconstruction. To best of our knowledge, InvertingGradients
was the only work that discussed restoration from model weight updates while
their approach can only restore the training data when the batch size or local
epochs is equal to 1. The latest work GradInversion (Yin et al. 2021) improved
InvertingGradients and was feasible when the batch size was up to 48. It also
proposed a zero-shot batch label restoration approach, but the approach
requires that no images in the batch share the same label and the number of
classes must be greater than the batch size.

By analyzing the previous related work, we summarize the limitations as
follows:

  * •

All the existing approaches did not show the results or could not recover when
the batch data had duplicate labels.

  * •

All the existing approaches considered label recovery and image recovery as
two independent tasks without analyzing the impact between each other.

  * •

The previous work on recovering images from weight differences is very
restricted. When the FedAvg algorithm is used in federated learning, the batch
size and local epochs will be greater than 1 in almost all cases.

In this paper, we try to solve the above-explained problems. Our contributions
are as follows:

  * •

We empirically point out that the actual range of pixels is meaningful for
image restoration. We recommend an image initialization and introduce two
regularization terms to improve the image reconstruction.

  * •

We propose a zero-shot batch label restoration method that is not constrained
by the label distribution.

  * •

Better image reconstruction by utilizing multiple updates with regard to the
same batch data is implemented.

  * •

A framework is designed for reconstructing attacks to recover data from shared
gradients and weights in FL scenarios.

  * •

We set up the alignment to evaluate the recovered image based on labels and
image similarities.

  * •

Experimental results of our approaches on various datasets(CIFAR-10 and
ImagenNet) are demonstrated.

##  2 Preliminaries

In this section, we introduce the preliminaries related to our work, mainly
including the FL mechanism, the threat model, the zero-shot label inference,
and other methods to improve the quality of reconstruction, which will help to
understand the approaches proposed in Section 3.

###  2.1 Federated Learning

Federated learning is a distributed learning paradigm aiming to preserve the
privacy of users. A FL system consists of a parameter server orchestrating the
training process and multiple clients holding private local data. A typical
federated training process repeats the five steps: client selection,
broadcast, client computation, aggregation, and model update (Kairouz et al.
2019). The server selects a subset of clients and broadcasts the global model
at the communication round. Then the clients execute local training on its
private data. Eventually the server aggregates the updated models from clients
and updates the global model. Hence, the local privacy is preserved.

Assuming that the shared model architecture is $F$, in the $t$-th
communication round the global model parameter is $W^{t}$, the local model is
$W^{t}_{i}$, on the $i$-th node, the training data is ($x^{t_{i}}$,
$y^{t}_{i}$), the local client will calculate the local update $\nabla
W^{t}_{i}$ as

$\nabla
W^{t}_{i}=\frac{\partial\mathcal{L}(F(x^{t}_{i},W^{t}),y^{t}_{i})}{\partial
W^{t}}.$ (1)

The global model update based on gradient aggregation and weight aggregation
are as follows:

  * •

FedSGD

$\nabla W^{t}=\frac{1}{N}\sum_{i}^{N}\nabla W^{t}_{i}$ (2)

$W^{t+1}=W^{t}-\eta\nabla W^{t},$ (3)

  * •

FedAvg

$W^{t+1}_{i}=W^{t}-\eta\nabla W^{t}_{i}$ (4)
$W^{t+1}=\frac{1}{N}\sum_{i}^{N}W^{t+1}_{i}.$ (5)

Here we show the generic formulas. More details like the weights of each
clients, and the hyperparameters to balance model accuracy and communication
efficiency are not reflected above.

###  2.2 Threat Model

#### Attacker’s Goals and Capabilities

In this work, we study a data reconstruction attack. It’s a white-box attack
that the attacker knows model architecture, optimizer, loss function,
hyperparameters like learning rate, batch size, etc. Corresponding to the two
FL algorithms FedSGD and FedAvg, we define two attack scenarios respectively,
(1) the attacker has access to model weights $W^{t}_{i}$ and gradients $\nabla
W^{t}_{i}$ at the communication round $t$ in case FedSGD; and (2) the attacker
has access to weights $W^{t}_{i}$ and $W^{t+1}_{i}$ at communication rounds
$t$ and $t+1$ in case FedAvg. The attacker aims to reconstruct the batch data
(images and labels) based on shared model weights or gradients. The principle
is different from other reconstruction attacks like Model Inversion
(Fredrikson, Jha, and Ristenpart 2015) or GAN attacks (Hitaj, Ateniese, and
Pérez-Cruz 2017; Salem et al. 2020), which requires the loss of the learning
system or image representation, a related public dataset and training of extra
additional models.

#### Data Restoration by Gradient Matching

Deep Leakage from Gradients(DLG) algorithm was the first implementation to
prove that it is possible to obtain private training data from shared
gradients. It made the dummy input close to the original input by minimizing
the difference of the original gradient and dummy gradient. First, dummy data
and labels are randomly generated. After deriving the dummy gradient w.r.t.
the dummy data and dummy label, it did not update the model weights like
traditional learning processes but updated the dummy data and label to restore
privacy by matching the dummy gradient and the original gradient. The DLG
algorithm flow is illustrated in Algorithm 1:

Input: $F(x;W)$: differential model; $W$: differentialable model weights;
$\nabla W$ gradients; $N$: number of iterations

Output: $\hat{x}_{N+1}$: restored image; $\hat{y}_{N+1}$: restored label

Initialize the dummy image and label
$\hat{x}_{1}\hookrightarrow\mathcal{N}(0,1)$,
$\hat{y}_{1}\hookrightarrow\mathcal{N}(0,1)$

for  _i = 1,…,N_ do

$\hat{\nabla
W_{i}}=\frac{\partial\mathcal{L}(F(\hat{x}_{i},W),\hat{y}_{i})}{\partial W}$

${\mathbb{D}_{i}=\left\|\hat{\nabla W_{i}}-\nabla W\right\|^{2}}$

$\hat{x}_{i+1}=\hat{x}_{i}-\eta\nabla_{\hat{x}_{i}}\mathbb{D}_{i},\quad\hat{y}_{i+1}=\hat{y}_{i}-\eta\nabla_{\hat{y}_{i}}\mathbb{D}_{i}$

end for

Algorithm 1 Deep Leakage from Gradients

###  2.3 Zero-shot Label Restoration

Zero-shot label restoration was first introduced in iDLG (Zhao, Mopuri, and
Bilen 2020). The zero-shot method is popular (Geiping et al. 2020; Yin et al.
2021) because the attack is fast and doesn’t depend on the image recovery,
especially when the input, or model, is too complex. We present here the
principle of this zero-shot approach, and in Section 3.2 we will describe how
to make this approach support batch restoration for any label distribution.

Assume the one hot label is $y$ and $y_{c}=1$ for ground truth $c$, the final
logits output before the softmax is $z$, the number of classes is $N$, so
prediction $p_{i}=\frac{e^{z_{i}}}{\sum_{n=1}^{N}e^{z_{n}}}$. The cross-
entropy loss for given training data $(x,y)$ is

$\mathcal{L}(x,y)=-\sum_{n=1}^{N}y_{n}\log{\frac{e^{z_{n}}}{\sum_{n=1}^{N}e^{z_{n}}}}=-\log{\frac{e^{z_{c}}}{\sum_{n=1}^{N}e^{z_{n}}}}$
(6)

Then the gradient of the loss w.r.t the logit at index $i$ is

$\frac{\partial\mathcal{L}(x,y)}{\partial z_{i}}=-\frac{\partial
z_{c}-\log\sum_{n=1}^{N}{e^{z_{n}}}}{\partial z_{i}}=p_{i}-y_{i},$ (7)

if $i=c$, $y_{i}=1$, else $y_{i}=0$.

###  2.4 Total Variation Denoising

A commonly used noise reduction method in signal processing is total variation
(Mahendran and Vedaldi 2015). Given a signal $x$, the goal of total variation
denoising is to find an approximation signal $y^{*}$ such that it has smaller
total variation but is also ”close” to image $x$ as the objective function
$y^{*}=\arg\min[E(x,y)+TV(y)]$.

Considering the 2D image restoration task, a restored image containing noise
will have a larger value of total variation, i.e., a larger sum of absolute
values of its gradients, compared to its signal unaffected by noise.

$TV(X)=\sum_{i,j}(|X_{i+1,j}-X_{i,j}|+|X_{i,j+1}-X_{i,j}|)$ (8)

###  2.5 Image Similarity

In our work, image similarity is not only used as the metric of image
restoration but also for image alignment. When there are multiple images in
the batch with the same label, we need to align the images according to the
image similarity. Previous works (Zhu, Liu, and Han 2019; Geiping et al. 2020;
Yin et al. 2021) have confirmed that gradient matching-based recovery loses
the order of batch images, and the ordered results are useful for analyzing
the experimental effects. For images $x$ and $y$, there are several methods to
method to measure the similarity:

  * •

Mean square error (MSE)

$MSE(x,y)=E{(x-y)^{2}},$

where $E$ is the expected value.

  * •

Peak signal to noise ratio (PSNR)

$PSNR=10*\log_{10}(\frac{MAX^{2}}{MSE(x,y)}),$

where $MAX$ is the maximum possible pixel value of the image, for $n$-bit
pixel, $MAX=2^{n}-1$. Higher means more similar.

  * •

Structural similarity (SSIM)

$SSIM(x,y)=\frac{(2\mu_{x}\mu_{y}+c_{1})(2\sigma_{xy}+c_{2})}{(\mu_{x}^{2}+\mu_{y}^{2}+c_{1})(\sigma_{x}^{2}+\sigma_{y}^{2}+c_{2})},$

where $\mu$, $\sigma$ are the corresponding mean and standard variation;
$\sigma_{xy}$ is the covariance between images; $c_{1}=(k_{1}L)^{2}$,
$c_{2}=(k_{2}L)^{2}$ are used to stabilize the division with $k_{1}=0.01$,
$k_{2}=0.03$. Higher means more similar.

  * •

Learned perceptual image patch similarity (LPIPS)

evaluate the distance between image pairs with deep learning models, higher
value means further difference.

##  3 Proposed Approaches

In this section, we explain our proposed method in detail. In Section 3.1 we
introduce the reconstruction function, explaining how the reconstruction
attack is defined as an optimization problem. In Section 3.2 we introduce our
zero-shot batch label restoration approach. In Section 3.3 we introduce our
auxiliary regularization designed to improve the fidelity of image
reconstruction. In Section 3.4 we explain how to exploit multiple updates,
which further improve the reconstruction results. In Section 3.5 we explain
how to apply our reconstruction attack architecture to the FedAvg algorithm,
that is, to reconstruct the training images according to their weights. In
Section 3.6 we describe how to implement image alignment, which helps us to
analyze the data and eliminate ambiguity between images.

###  3.1 Reconstruction Function

The original private data is $(x,y)$, the recovered private data is
$(\hat{x},\hat{y})$. The parameters of the model is $W$ and $\nabla W$ is the
batch averaged gradient in the FedSGD scenario. The reconstruction function is
illustrated as Equation 9, consisting of gradient matching loss and auxiliary
regularization for fidelity:

$\hat{x}^{*},\hat{y}^{*}=\arg\min_{\hat{x},\hat{y}}{[\mathcal{L}_{gm}(\hat{x},\hat{y},W,\nabla
W)+\mathcal{R}_{aux}(\hat{x})]}.$ (9)

Gradient matching loss describes the discrepancy between the original and the
dummy gradient among all layers. Hence the first part is defined in Equation
10 and $l$ denotes the model layer:

$\mathcal{L}_{gm}(\hat{x},W,\nabla
W)=\sum_{l}\left\|\nabla_{W_{(l)}}\mathcal{L}(\hat{x},\hat{y})-\nabla
W_{(l)}\right\|_{2}.$ (10)

Auxiliary regularization for fidelity will be explained in detail in Section
3.3.

###  3.2 One-shot Batch Label Restoration

Here we explain our zero-shot approach, which solves $\hat{y}^{*}$ according
to the gradient of the fully connected layer. We consider the information
leakage from the classification task, in which cross-entropy is used as loss
function for optimization. We denote $x^{*}=[x_{1},x_{2},\cdots,x_{K}]$ and
$y^{*}=[y_{1},y_{2},\cdots,y_{K}]$ as the batch of original images and ground
truth labels, respectively. For each image-label pair $(x_{k}$, $y_{k})$ in
the batch, we calculate the cross entropy as Equation 11, where $z_{k,n}$ is
the logits output of the last fully connected layer and $y_{k,n}\in\\{0,1\\}$
is the one-hot representation of $y_{k}$ at index $n$ among all $N$ classes.

$\mathcal{L}(x_{k},y_{k})=-\sum_{n=1}^{N}y_{k,n}\log(\frac{e^{z_{k,n}}}{\sum_{n=1}^{N}e^{z_{k,n}}})$
(11)

The gradient of loss function w.r.t. weights $W$ is formulated as Equation 12,
where $K$ is the number of batch size. The gradient of cross entropy loss
w.r.t. the final logits $z$ at index $n$ of sample $k$ in the batch is derived
as Equation 13, where
$p_{k,n}=\frac{e^{z_{k,n}}}{\sum_{n^{\prime}=1}^{N}e^{z_{k,n^{\prime}}}}$ is
the post-softmax probability.

$\nabla_{W}\mathcal{L}(x^{*},y^{*})=\frac{1}{K}\sum_{k=1}^{K}\nabla_{W}\mathcal{L}(x_{k},y_{k})$
(12)

$\nabla_{z_{k,n}}\mathcal{L}(x_{k},y_{k})=p_{k,n}-y_{k,n}$ (13)

We define $W^{(FC)}\in\mathbb{R}^{M\times N}$ as the weights of last fully
connected layer in the network, where $M$ is the dimension of last hidden
layer.
$\Delta{W^{(FC)}_{m,n,k}}:=\nabla_{W^{(FC)}_{m,n}}\mathcal{L}(x_{k},y_{k})$
are defined as the gradient of the cross-entropy loss for image-label pair
$(x_{k},y_{k})$ w.r.t. weights $W^{(FC)}_{m,n}$ connecting $m$-th unit of the
last hidden layer to logits $n$. Equation 14 is derived according to the chain
rule, where $O_{k,m}$ is the $m$-th input unit of fully connection layer of
$k$-th sample. We denote $\Delta W^{(FC)}_{m,n}$ as the average of tensor
$\Delta W^{(FC)}$ along the dimension of batch index $k$, i.e., $\Delta
W^{(FC)}_{m,n}=\frac{1}{K}\sum_{k=1}^{K}\Delta W^{(FC)}_{m,n,k}$, where

$\begin{split}\Delta
W^{(FC)}_{m,n,k}&=\frac{\partial\mathcal{L}(x_{k},y_{k})}{\partial
z_{k,n}}\cdot\frac{\partial z_{k,n}}{\partial W^{(FC)}_{m,n}}\\\
&=\nabla_{z_{k,n}}\mathcal{L}(x_{k},y_{k})\cdot O_{k,m}\\\
&=(p_{k.n}-y_{k,n})\cdot O_{k,m}\end{split}$ (14)

We denote $S_{k,n}$ as the sum of tensor $\Delta W^{(FC)}$ along the dimension
of input features of fully connected layer $m$, i.e.,
$S_{k,n}=\sum_{m=1}^{M}\Delta W^{(FC)}_{m,n,k}$. Note that the common
activation functions, like ReLU and Sigmoid, in the Convolutional Neural
Networks (CNNs), the $O_{k,m}$ are non-negative. Therefore, the sign of
$\Delta W^{(FC)}_{m,n,k}$ is only determined by $p_{k,n}-y_{k,n}$, and only
one element of $S_{k,n}$ along the dimension of batch index $k$ is negative,
if and only if $n=n^{*}_{k}$ at the index of ground truth label.

The sum of values in $\Delta W^{(FC)}$ along $m$ dimension is denoted as
$\Delta W^{(FC)}_{n}$:

$\begin{split}\Delta
W^{(FC)}_{n}&=\frac{1}{K}\sum_{k}\sum_{m}{(p_{k,n}-y_{k,n})\cdot O_{k,m}}\\\
&=\frac{1}{K}\sum_{k}(p_{k,n}-y_{k,n})\cdot\sum_{m}O_{k,m}\end{split}$ (15)

We notice that $O_{k}=\sum_{m}O_{k,m}$ is always greater than 0 and depends on
each sample. Assume that $O_{k}\approx\bar{O_{k}}$, close to is mean value,
then $\Delta
W^{(FC)}_{n}\approx\frac{1}{K}\sum_{k}{(p_{k,n}-y_{k,n})}\bar{O_{k}}$. Based
on that assumption, we can infer the number of specific labels $n$ by

$\sum_{k}{y_{k,n}}\approx\sum_{k}{p_{k,n}}-\frac{K\Delta
W^{(FC)}_{n}}{\bar{O_{k}}},$ (16)

the left part implies the number of samples where labels equal $n$ on one
batch and we estimate $p_{k,n}$ and $\bar{O_{k}}$ by feeding the dummy data
into the network. We experimentally demonstrate the validity of our hypotheses
and approaches.

###  3.3 Auxiliary Regularization for Fidelity

A significant difference between data recovery and standard deep model
training is that the pixel value has a physical meaning, usually between $0$
and $255$, or after normalization $0\sim 1$. In contrast, the recovered data
from deep leakage are not bounded. Therefore, we must constrain the values of
pixels. We have designed two regularization terms as Equation 17,18 for this
purpose:

$\mathcal{R}_{clip}{(\hat{x})}=\left\|{\hat{x}-clip(\hat{x})}\right\|_{2}$
(17)

$\mathcal{R}_{scale}{(\hat{x})}=\left\|{\hat{x}-scale(\hat{x})}\right\|_{2}$
(18)

where $clip(\hat{x}_{i})=min(max(\hat{x}_{i},0),1)$ and
$scale(\hat{x}_{i})=\frac{\hat{x}_{i}-min_{v}}{max_{v}-min_{v}}$. $max_{v}$,
$min_{v}$ denote the maximum and minimum value in the restored image. The
auxiliary regularization (Equation 19) keeps the recovered images away from
unrealistic images. $\mathcal{R}_{aux}(\hat{x})$ is the total variation loss.

$\mathcal{R}_{aux}(\hat{x})=\alpha_{tv}\mathcal{R}_{tv}(\hat{x})+\alpha_{c}\mathcal{R}_{clip}(\hat{x})+\alpha_{s}\mathcal{R}_{scale}(\hat{x})$
(19)

In addition, we found that constraining the initialized pixel between [0,1)
also helps to improve the quality of the recovered image as illustrated in our
ablation studies.

###  3.4 Multiple Updates Consistency

Input: $\\{F_{j}(x;W)\\}_{j=1}^{M}$: differential model; $T$: total update
times; $\\{W^{t}\\},t=1..T$: model weights at different timestamps; $\\{\nabla
W^{t}\\},t=1..T$ corresponding gradients;N: number of iterations.

Output: $\hat{x}_{T+1}$: private image; $\hat{y}$: private label

Initialize the dummy image $x\hookrightarrow\mathcal{U}(0,1)$, recovered label
$\hat{y}$ using zero-shot approach.

for  _i = 1,…,N_ do

for  _t = 1,…,T_ do

$\hat{\nabla
W^{t}_{i}}=\frac{\partial\mathcal{L}(F(x_{i},W^{t}),\hat{y})}{\partial W^{t}}$

end for

${\mathbb{D}_{i}=\frac{1}{T}\sum_{t}\left\|\hat{\nabla W^{t}_{i}}-\nabla
W^{t}\right\|^{2}}$

$x^{\prime}_{i+1}=x^{\prime}_{i}-\eta\nabla_{x^{\prime}_{i}}\mathbb{D}_{i}$

end for

Algorithm 2 Recovery from Multiple Updates

Just like finding the numerical solution to an equation, when more conditions
are known, the more accurate the resulting solution will be. Inspired by this
idea, we can obtain better images by increasing the number of known weight and
gradient pairs. Supposed that one FL participant has trained models locally on
the same private data and update the models and gradients for multiple times
$T$. By matching the gradients of several different timestamps,
${\frac{1}{T}\sum_{t}\left\|\nabla W^{\prime}_{i,t}-\nabla
W_{t}\right\|^{2}}$, we can hopefully recover a better batch of images. It can
be considered as a consistency regularization that helps get closer to the
global optimal point.

###  3.5 Reconstruction from Weights

For model training with SGD optimizer, the model updates as follows:

$W^{t+1}=W^{t}-\eta\nabla W^{t}$ (20)

Suppose the model undergoes $T$ time of local updates, from $W^{0}$ to
$W^{T}$, and is finally uploaded the server. That is, the server can access
$W^{0}$ and $W^{T}$, but not the intermediate models. Our reconstruction
attack based on weight updates can be expressed as:

$\hat{x}^{*},\hat{y}^{*}=\arg\min_{\hat{x},\hat{y}}{[\mathcal{L}_{gm}(\hat{x},\hat{y},W^{0},W^{T})+\mathcal{R}_{aux}(\hat{x})]}$
(21)

As in common settings of FL, the center server knows all hyperparameters about
local training, such as local epochs, local batch size, and local learning
rate. Theoretically, we can infer the gradient w.r.t. the training data from
continuous model weight updates with $\nabla
W^{t+1}=\frac{1}{\eta}{(W^{t}-W^{t+1})}$, assuming that the data hold on
specific clients remains consistent. When T is greater than 1, we adopt a
simple strategy by assuming that:

$W^{T}\approx W^{0}-T\cdot\eta\nabla W^{0}$ (22) $\nabla
W^{0}\approx\frac{1}{T\eta}(W^{0}-W^{T}).$ (23)

That is, we use the average model difference, denoted as fake gradient, to
replace the gradient in the original reconstruction function. We empirically
demonstrate that even with the fake gradient, our proposed methods can still
restore the image and label information. Gradient matching loss based on
weights is defined in Equation 24, where we use cosine similarity instead of
$l2$:

$\sum_{l}\left\|\nabla_{W^{0}_{(l)}}\mathcal{L}(\hat{x},\hat{y})-(\frac{W_{(l)}^{0}-W_{(l)}^{T}}{T\eta})\right\|_{cos}$
(24)

###  3.6 Image Alignment

Because the reconstruction based on gradient matching cannot restore the
sequence (Zhu, Liu, and Han 2019), and the same label in the mini-batch causes
the ordering ambiguity w.r.t that label (Geiping et al. 2020), all the image
restoration results are not aligned with the ground truth or rely on manual
processing. We propose that pictures can be aligned based on the similarity of
the pictures. We consider the alignment as a weighted bipartite graph problem.

Suppose $K$ is the batch size, and a matching function $m$, $\forall i\in
1,\ldots,N$, $m(i)\in 1,\ldots,N$, needs to be found. The mapping is
bijective, $\forall i,j\in 1,\ldots,K$, $i\neq j\Rightarrow m(i)\neq m(j)$.
$sim(\hat{x}_{i},x_{m(i)})$ denotes the image similarity between the restored
image $\hat{x}_{i}$ and its aligned ground truth $x_{m(i)}$. The objective is
to maximizing the sum of the similarity in the perfect matching:

$m^{*}=\arg\max_{m}\sum_{i=1}^{N}sim(\hat{x}_{i},x_{m(i)})$ (25)

We use label information to help determine partially aligned images. If there
exist labels, which are unique both in the original and restored labels, the
images are aligned directly. These aligned images will not be involved in the
optimization, which reduces the number of interfering items in the image
matching process. We use PSNR as the similarity function when aligning images
because it has enough distinction and requires no additional processing.

Figure 2: Alignment of unique images from ImageNet of batch size 16. The top
two lines are the result of restoration, the middle two lines are the result
of image alignment, and the last two lines are ground truth.

##  4 Experiments

#### Setups

We perform data reconstruction attacks against LeNet networks in FedSGD and
FedAvg scenarios. We evaluate our method for the image classification on two
classic datasets: CIFAR-10 and ImageNet. In CIFAR-10, images are with size of
$32\times 32$ and ImageNet images are cropped to $224\times 224$. We sample
10K images from CIFAR-10, 1K for each class and 1K images from ImageNet, five
for each of 200 categories. We use SGD as the optimizer for local training
with learning rate(lr) equals $0.1$. We use L-BFGS optimizer and optimize for
300 and 600 iterations respectively for CIFAR-10 and ImageNet. For CIFAR-10
experiments, we set $lr=0.5$, $\alpha_{tv}=0.5$, $\alpha_{l2}=0.1$,
$\alpha_{c}=100$, and $\alpha_{s}=100$; For ImageNet we set $lr=0.1$,
$\alpha_{tv}=200$, $\alpha_{l2}=200$, $\alpha_{c}=200$, and $\alpha_{s}=200$.
We synthesize image batches with NVIDIA V100 GPUs. Our experimental results
are mainly compared with GradInversion, the start-of-the-art.

#### Comprehensive Results

Table 1 shows the similarities between the images we recovered and ground
truth in ImageNet (batch size=16). All labels in the batch are unique. Our
method outperforms GradInversion in restoring batch images. The restoration
results are better when multiple updates are exploited. The images of our
restoration are in Figure 1.

Table 1: Comparison of GradInversion and our approaches on unique batch of
size 16 from ImageNet.

| MSE $\downarrow$ | SSIM $\downarrow$ | PSNR $\uparrow$ | LPIPS $\downarrow$
---|---|---|---|---
GradInversion | 0.015 | 0.701 | 19.956 | 0.350
Ours (Baseline) | 0.011 | 0.705 | 20.029 | 0.349
Ours (2 Updates) | 0.008 | 0.775 | 22.436 | 0.321
Ours (3 Updates) | 0.008 | 0.782 | 23.491 | 0.313
Ours (4 Updates) | 0.006 | 0.801 | 23.240 | 0.305

#### Ablation studies of image initialization and regularization

We adopt four kinds of random initializing distributions, as
$\mathcal{U}(0,1)$, $\mathcal{N}(0.5,1)$, $\mathcal{N}(0,1)$,
$\mathcal{U}(-0.5,0.5)$. The results illustrated in Figure 3 show the
influence of those four initialization strategies.We find that when the batch
size increases, the initialization of the image with an initial mean value of
0.5 ($\mathcal{N}(0,1)$, $\mathcal{U}(0,1)$) is better than the initialization
of the image with a mean value of 0 ($\mathcal{N}(0.5,1)$,
$\mathcal{U}(-0.5,0.5)$). The ablation study of regularization methods in
Table 2 shows that regularization we designed helps to improve the quality of
the images

Figure 3: Comparison of the four initialization methods with batch size 1 and
4. Table 2: Ablation Study of Different Regularization Methods

Regularization Methods | MSE $\downarrow$ | SSIM $\downarrow$ | PSNR $\uparrow$ | LPIPS $\downarrow$
---|---|---|---|---
+tv | 0.012021 | 0.789463 | 22.090446 | 0.040423
+tv+l2 | 0.123710 | 0.786881 | 21.967402 | 0.041161
+tv+clip | 0.008571 | 0.823602 | 22.723407 | 0.032694
+tv+scale | 0.008571 | 0.823608 | 22.723688 | 0.032691
+tv+clip+scale | 0.008570 | 0.823609 | 22.723748 | 0.032692
+tv+l2+clip+scale | 0.008585 | 0.823514 | 22.715970 | 0.032721

#### Label Inference

We compare our approach to GradInversion’s zero-shot label restoration. Tabel
3 shows the label accuracy on ImageNet when the batch size is equal to 16.
Repetition represents the number of extra repetitions. When Repetition equals
8, one label appears 9 times, 8 are the extra repetitions. GradInversion can
only restore one of the repeated labels. When the Repetition increases, the
label accuracy decreases linearly. When repetition is equal to 8, the 8 extra
repetitions cannot be inferred correctly and the accuracy rate is close to
50%, actually 49.39%. Our approach is not affected by the number of duplicate
labels.

The results in Table 4 demonstrate that our approach is robust to duplicate
labels, very large batch size and number of classes. Rep Amount denotes the
number of occurrences of each label in the batch. When Rep Amount equals 4 and
the batch size equals 256, there are 64 unique labels.

Table 3: The comparison between our method and GradInversion on the label
recovery accuracy(%) of ImageNet when batch size = 16.

Repetition | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8
---|---|---|---|---|---|---|---|---|---
GradInversion | 99.60 | 92.47 | 87.10 | 81.05 | 74.60 | 68.15 | 62.30 | 55.65 | 49.39
Our | 99.19 | 98.99 | 97.38 | 98.20 | 98.19 | 99.19 | 99.193 | 98.19 | 99.60

Table 4: Label accuracy(%) of our approach under different batch sizes and
different repeat amounts.

Dataset |  Rep Amount\Batch Size | 4 | 8 | 16 | 32 | 64 | 128 | 256
---|---|---|---|---|---|---|---|---
CIFAR-10 | 2 | 100 | 99.63 | 98.06 | 97.13 | 98.03 | 97.97 | 97.83
4 | 99.30 | 98.60 | 99.50 | 98.48 | 98.75 | 94.87 | 98.44
ImageNet | 2 | 99.00 | 100 | 98.39 | 97.50 | 98.44 | 94.20 | 96.35
4 | 99.30 | 98.60 | 99.50 | 98.48 | 98.75 | 94.87 | 98.44

#### Impact of Labels on Image Restoration

Table 5 shows the results of image restoration when there are repeating labels
in one batch. Reptition and Rep Amount are the settings explained above. Each
cell in the table contains two values, the first value is for GradInversion
and the second value is for ours. GradInversion can not handle duplicate
labels correctly, so even when Repetition equals 1, only one image is
incorrectly labeled, the discrepancy between the restored image and the ground
truth gets extremely large (MSE: 0.199 , PSNR: 10.639). In contrast, our
method is able to infer the duplicate labels properly, and the image recovery
is minimally affected(MSE: 0.018, PSNR:19.122). In addition, we found that
images with the same label are recovered as a fusion of images in our
approach, although not identical, but the outlines of similar images can be
seen in the images. Figure 4 shows the result of image recovery when Rep
Amount=2 for batch size = 16. GradInversion fails completely as in the above
two rows; our method restores the image content, but is affected by images
with the same label as in the middle two rows; the last two rows are ground
truth.

Figure 4: Batch image reconstruction on ImageNet dataset using different
approaches with Rep Amount 2\.  Table 5: Compare the images recovered with our
approach and GradInversion when there are duplicate labels in the batch. The
former is GradInversion, the latter is our approach.

Label Distribution | MSE $\downarrow$ | SSIM $\downarrow$ | PSNR $\uparrow$ | LPIPS $\downarrow$
---|---|---|---|---
Repetition 1 | 0.199 /0.018 | 0.453/ 0.703 | 10.639 / 19.122 | 0.633 /0.344
Repetition 2 | 0.334 / 0.031 | 0.227 /0.663 | 5.227/18.258 | 0.820 / 0.392
Repetition 3 | 0.425/ 0.036 | 0.134/ 0.606 | 4.150 /17.052 | 0.866 / 0.444
Rep Amount 2 | 0.458 / 0.050 | 0.044 / 0.548 | 3.893 / 13.626 | 0.896 /0.509
Rep Amount 4 | 0.464/ 0.068 | 0.021 / 0.461 | 3.953 / 12.164 | 0.890 /0.602

#### Image Reconstruction from Weights

Figure 5 illustrates the results of recovering unique images in CIFAR-10 after
training for different local epochs, supposing the batch size equals the
number of local images. Our method can restore training data when the batch
size and local epochs are both greater than 1.

Figure 5: Batch image restoration from weights on CIFAR-10 with batch size of
8 with no duplicate label. The first to fifth lines are the results of local
epochs equal to 1,2,4,8,16 respectively, and the last line is the ground
truth.

##  5 Conclusion

In this article, we conduct privacy attacks on the federated learning
scenarios of shared gradient (FedSGD) and shared model (FedAvg), respectively.
The experimental results show the fragile side of the naive federated learning
system when sharing parameters. Our method solves the limitation of not having
the same labels in one batch and can recover batch images and labels from
model parameters after multiple local training epochs. We also empirically
proved that multiple model-gradient pairs could improve image recovery. Our
experiments remind people not to rely solely on the privacy of the federation
learning mechanism, and additional privacy-preserving techniques need to be
designed for federation learning.

## References

  * Fredrikson, Jha, and Ristenpart (2015) Fredrikson, M.; Jha, S.; and Ristenpart, T. 2015.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures.  In Ray, I.; Li, N.; and Kruegel, C., eds., _Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, CO, USA, October 12-16, 2015_ , 1322–1333. ACM.
  * Geiping et al. (2020) Geiping, J.; Bauermeister, H.; Dröge, H.; and Moeller, M. 2020.  Inverting Gradients - How easy is it to break privacy in federated learning?  In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Hitaj, Ateniese, and Pérez-Cruz (2017) Hitaj, B.; Ateniese, G.; and Pérez-Cruz, F. 2017.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning.  In Thuraisingham, B. M.; Evans, D.; Malkin, T.; and Xu, D., eds., _Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS 2017, Dallas, TX, USA, October 30 - November 03, 2017_ , 603–618. ACM.
  * Kairouz et al. (2019) Kairouz, P.; McMahan, H. B.; Avent, B.; Bellet, A.; Bennis, M.; Bhagoji, A. N.; Bonawitz, K.; Charles, Z.; Cormode, G.; Cummings, R.; et al. 2019.  Advances and open problems in federated learning.  _arXiv preprint arXiv:1912.04977_.
  * Mahendran and Vedaldi (2015) Mahendran, A.; and Vedaldi, A. 2015.  Understanding deep image representations by inverting them.  In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015_ , 5188–5196. IEEE Computer Society.
  * McMahan et al. (2017) McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017.  Communication-Efficient Learning of Deep Networks from Decentralized Data.  In Singh, A.; and Zhu, X. J., eds., _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA_ , volume 54 of _Proceedings of Machine Learning Research_ , 1273–1282. PMLR.
  * McMahan et al. (2016) McMahan, H. B.; Moore, E.; Ramage, D.; and y Arcas, B. A. 2016.  Federated Learning of Deep Networks using Model Averaging.  _CoRR_ , abs/1602.05629.
  * Melis et al. (2019) Melis, L.; Song, C.; Cristofaro, E. D.; and Shmatikov, V. 2019.  Exploiting Unintended Feature Leakage in Collaborative Learning.  In _2019 IEEE Symposium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019_ , 691–706. IEEE.
  * Nasr, Shokri, and Houmansadr (2019) Nasr, M.; Shokri, R.; and Houmansadr, A. 2019.  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning.  In _2019 IEEE Symposium on Security and Privacy, SP 2019, San Francisco, CA, USA, May 19-23, 2019_ , 739–753. IEEE.
  * Salem et al. (2020) Salem, A.; Bhattacharya, A.; Backes, M.; Fritz, M.; and Zhang, Y. 2020.  Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning.  In Capkun, S.; and Roesner, F., eds., _29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020_ , 1291–1308. USENIX Association.
  * Wang et al. (2019) Wang, Z.; Song, M.; Zhang, Z.; Song, Y.; Wang, Q.; and Qi, H. 2019.  Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning.  In _2019 IEEE Conference on Computer Communications, INFOCOM 2019, Paris, France, April 29 - May 2, 2019_ , 2512–2520. IEEE.
  * Yin et al. (2021) Yin, H.; Mallya, A.; Vahdat, A.; Alvarez, J. M.; Kautz, J.; and Molchanov, P. 2021\.  See through Gradients: Image Batch Recovery via GradInversion.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 16337–16346.
  * Zhao, Mopuri, and Bilen (2020) Zhao, B.; Mopuri, K. R.; and Bilen, H. 2020.  iDLG: Improved Deep Leakage from Gradients.  _CoRR_ , abs/2001.02610.
  * Zhu, Liu, and Han (2019) Zhu, L.; Liu, Z.; and Han, S. 2019.  Deep Leakage from Gradients.  In _NeurIPS_.


# ML-based PBCH symbol detection and equalization for 5G Non-Terrestrial
Networks

Inés Larráyoz-Arrigote†, Marcele O. K. Mendonça⋆, Alejandro Gonzalez-Garrido⋆
, Jevgenij Krivochiza⋆, Sumit Kumar⋆, Jorge Querol⋆, Joel Grotz¶, Stefano
Andrenacci¶, Symeon Chatzinotas⋆

⋆SnT - University of Luxembourg, Luxembourg. †Universidad Pública de Navarra
(UPNA), Navarra, Spain, ¶SES S.A., Luxembourg Corresponding Author:
<EMAIL_ADDRESS>
###### Abstract

This paper delves into the application of Machine Learning (ML) techniques in
the realm of 5G Non-Terrestrial Networks (5G-NTN), particularly focusing on
symbol detection and equalization for the Physical Broadcast Channel (PBCH).
As 5G-NTN gains prominence within the 3GPP ecosystem, ML offers significant
potential to enhance wireless communication performance. To investigate these
possibilities, we present ML-based models trained with both synthetic and real
data from a real 5G over-the-satellite testbed. Our analysis includes
examining the performance of these models under various Signal-to-Noise Ratio
(SNR) scenarios and evaluating their effectiveness in symbol enhancement and
channel equalization tasks. The results highlight the ML performance in
controlled settings and their adaptability to real-world challenges, shedding
light on the potential benefits of the application of ML in 5G-NTN.

###### Index Terms:

Machine Learning, 5G Non-Terrestrial Networks, Satellite Communications,
Channel estimation, Symbol Enhancement, Equalization, Physical Broadcast
Channel.

##  I Introduction and Background

The application of Machine Learning (ML) techniques in wireless communications
is continuously proving its enormous potential towards performance enhancement
and acceleration of complex signal processing algorithms. Recently Non-
Terrestrial Networks (NTN) has gained significant momentum among the research
community, especially after the inclusion of NTN as a part of 3GPP ecosystem
from the recent Release-17 on-wards[1]. Besides, 3GPP Release-18 [2] will
natively embrace artificial intelligence and machine learning based
technologies for providing data-driven and, intelligent network solutions.

Several studies have explored the application of ML in the context of 5G-NTN.
The authors in [3] apply reinforcement learning to determine appropriate
scheduling policy for link selection in a LEO based 5G-NTN. Their simulations
show the effectiveness of this approach in terms of improving end-to-end loss
rates and bandwidth utilization for a non-static channel. Authors in [4] focus
on application of ML techniques to address the problem of handovers in a LEO
based 5G-NTN. Location of the UE is taken as the important feature to train
the ML model for improving the conditional handover decisions. In a survey
article [5], the authors have provided deep insights into the applications of
artificial intelligence (AI) empowered techniques for 5G and 6G NTN which
include: channel estimation, mobility management, doppler estimation and
compensation, resource management, network procedures to name a few. Moreover,
ML has been extensively investigated in order to address fundamental physical
layer challenges in wireless communication systems. In [6], joint channel
estimation and symbol detection is performed by one DNN in an end-to-end
manner. However, domain-specific knowledge is exploited in [7, 8] by breaking
a single DNN in two. Notably, these studies primarily rely on simulations and
lack real-world data validation, highlighting the need for practical testing.

In this work, we focus on ML for symbol detection and equalization in 5G-NTN’s
Physical Broadcast Channel (PBCH). The PBCH plays a crucial role in conveying
essential data, via the Master Information Block (MIB) which is necessary for
initial access procedure in the User Equipment (UE); this includes acquisition
of System Information Block-1 (SIB1) and the location of resources in Physical
Downlink Control Channel (PDCCH). To the best of author’s knowledge, ML based
techniques has not been used for symbol detection in 5G-NTN over real world
data. We benefit from the 5G-NTN testbed at the University of Luxembourg to
record live Over-the-Satellite (OTS) IQ samples for the functional validation
and performance verification of our ML algorithms.

##  II System model and Testbed

###  II-A 5G NR Synchronization System Block

This research aim to improve the MIB decodification. In Fifth Generation (5G),
the MIB is embeded into the Synchronization Signal Block (SSB), and to help
the UE decoding it, it is send in bursts. These burst have a period of 20 ms
in most of the configurations, and each burst consist on several repetitions
of the SSB. This number of repetitions also is controlled by the system
configuration. All these parameters are described in detail in the 5G standard
document [9].

Therefore, the first task of the UE is to find in the Resource Grid (RG) the
position of the SSB. Fig. 1 shows the receiver designed to extract the MIB
from the received signal using a ML enhanced equalization. The receiver
functionalities are described in the following paragraphs, whereas the ML
model is further detailed in the next section.

The process to locate the SSB in the 5G signal starts by a blind search of the
Primary Synchronization Signal (PSS) and the Secondary Synchronization Signal
(SSS) based in the Global Synchronization Channel Number (GSCN) raster. In our
experiment, we skip the blind search using the GSCN as we have control of the
transmitter.

Once we have found the PSS and SSS, the receiver knows the position in time
and frequency of the SSB within the RG and a coarse estimation of the Carrier
Frequency Offset (CFO). With this information, the receiver is able to locate
the Resource Element (RE) that correspond to the MIB and the ones used for the
DeModulation Reference Signal (DMRS).

The next step in our receiver is to estimate the channel. In 5G this is done
by using the DMRS pilots. Our receiver uses these pilots for the Neural
Network (NN) equalization and further enhancement of the MIB decoding process.
This process is the core of this research and is detailed in the next section.

###  II-B Testbed

We are using a system composed of a USRP (N310 model), which is a software-
defined radio device that can capture and transmit various types of wireless
signals over a wide range of frequencies. The USRP is connected to a laptop
via an Ethernet cable, which allows us to control the USRP settings and
process the captured signals using MATLAB. The USRP is used to capture 5G
signals from a terrestrial base station located at the 6GSpace Lab, which is a
research facility that aims to develop and test innovative solutions for
future wireless communications. The base station has an antenna that targets
the SES satellites in geostationary orbit, which provide global coverage and
high data rates for 5G services.

Figure 1:  Block diagram of the 5G signal capturing and decoding system with
proposed NNs.

The captured signals contain SS/PBCH blocks that carry the MIB, which is
mandatory system information that provides basic parameters for initial cell
selection and access. To decode the MIB, we need to perform several steps
using MATLAB.

We configure the USRP to capture a certain length of samples at a specific
frequency and gain. The center frequency of the 5G signal is 2029.25 MHz, but
there is an offset between the carrier and the USRP, so we set the configured
frequency to 2029.2 MHz. This offset introduces some phase error in the
received signal, which we need to compensate for in the later stages. We also
need to choose a capture length that is long enough to contain at least one
SS/PBCH block, which occurs every 20 ms according to the 5G standard.

##  III Methodologhy

We have considered two distinct NN algorithms to enhance various aspects of
the on-ground 5G UE receiving chain. The first one, denoted as Symbol
Enhancement NN, is dedicated to refining received symbols post-equalization.
The second one, referred to as Equalization NN, is designed to improve channel
estimation and, critically, the equalization process itself. Figure 2 depicts
the block diagram of the 5G UE PBCH receiving chain indicating where the
proposed NNs are placed within it.

The Symbol Enhancement NN just focuses on enhancing symbols following the
equalization process, performing a task of relatively low complexity.
Conversely, the Equalization NN needs to perform a more challenging task due
to the multifaceted nature of its objective (i.e. joint channel estimation,
noise characterization, equalization and the enhancement of the equalization
process applied to received data).

In subsection III-A, we describe the model architecture and the training
process common to both NNs. We use synthetic and real data from our 6GSPACE
Lab testbed [10] for training, with specific procedures described in
subsections III-B and III-C for synthetic and real data, respectively.

Figure 2: Block diagram of the 5G UE PBCH receiving chain with proposed NNs

###  III-A NN architecture and training process

For both NNs, we employ a fully connected architecture with three hidden
layers, each containing $K$ neurons. The hidden layers use the hyperbolic
tangent activation function, whereas the output layer employs a linear
activation function. The NNs take real-valued block versions of the received
complex symbols as input. For training, both NNs employ the Adam optimizer
with a learning rate of $\mu=0.001$ to minimize the mean squared error (MSE)
between transmitted symbols and predictions. The training process employs
mini-batches of 50 samples, requiring 40 epochs to train the NNs.

###  III-B Synthetic Data

Both transmitted and received synthetic data have been generated using MATLAB
in order to train and validate the NNs. To generate the transmitted samples,
we meticulously defined specific parameters to replicate the 5G signal used in
the real data case. These parameters include selecting the SSB ”Case A” block
pattern, corresponding to a sub-carrier spacing (SCS) of 15 kHz, and
establishing a minimum channel bandwidth of 5 MHz. As a channel model, we have
considered Additive White Gaussian Noise (AWGN), a carrier frequency offset
(CFO) uniformly distributed within the SCS, and an integer and fractional
delay corresponding to a GEO satellite delay. Additionally, we introduced the
standard variability in the MIB by changing the cell identification number and
the frame sequence number on the synthetically generated 5G samples.

Both NNs were trained using a dataset comprising 3024 transmitted and received
SSB signals, with each dataset tailored to a specific SNR. For the Symbol
Enhancement NN, the ML model was trained using the post-equalization symbols,
where the channel estimation and equalization were performed with a classical
Minimum Mean Squared Error (MMSE) algorithm. On the other hand, the
Equalization NN was trained with the symbols just after the SSB
synchronization algorithm, leaving the NN in charge of the complete channel
estimation and equalization tasks.

Furthermore, to gain a more comprehensive understanding of the performance of
these ML models, we performed the ML test using three distinct configurations:

  * •

Configuration 1: Test multiple ML models, each trained with a different SNR.

  * •

Configuration 2: Test a single ML model trained across a range of SNRs.

  * •

Configuration 3: Test a single ML model trained with a fixed SNR of 20 dB.

The SNRs considered for Options 1 and 2 included 0 dB, 2 dB, 5 dB, 7 dB, 10
dB, 15 dB, and 20 dB.

###  III-C Real Data

In the real data scenario, several differences arise compared to synthetic
data. The 5G UE receiver lacks real-time access to transmitted data due to
dynamic factors. For example, the frame sequence number is not a priori known
at the receiver since it is constantly increasing per each transmitted frame.
On the other hand, the channel conditions tend to fluctuate over time and
introduce dynamic variations into the experiments. Those are coming from both
the satellite payload (e.g. non-linearities) and the over-the-air channel
effects (e.g. tropospheric fading).

Perfect knowledge of the transmitted data is required to train the NNs, and as
mentioned above, this is not possible with the current setup. To address this
issue, we have adopted the synthetic regeneration-after-decoding approach
depicted in Figure 3. This process involves decoding the PBCH bits using the
5G standard approach until the 32-bit payload data is obtained. The 32-bit
payload data comprises 24 bits corresponding to the MIB and an additional 8
bits from various parameters. In the 5G standard, the 32-bit payload data has
attached a 24-bit cyclic redundancy check (CRC) code. If the CRC determines
that the 32-bit payload data are correct, the regeneration-after-decoding is
executed. The 32-bit payload data are fed into the Bose-Chaudhuri-Hocquenghem
(BCH) encoder, yielding 864 bits. These bits are then modulated with a
Quadrature Phase-Shift Keying (QPSK) scheme obtaining the 432 symbols
originally transmitted through the satellite, which are used for NNs training
purposes. The regeneration-after-decoding approach is acceptable in this case
since the SNR level at which the decoder operates is larger than the SNR
ranges at which the NN are evaluated.

Figure 3: Transmitted real data regeneration-after-decoding approach.

For the testing phase of the NNs with the real data over the satellite, the
USRP gain settings were configured to three different values: 70 dB, 25 dB,
and 20 dB. Such gain values correspond to SNR values of 20 dB, 10 dB and 3 dB
respectively.

##  IV Results

In this section, we evaluate the proposed NN models for symbol enhancement and
equalization tasks using both synthetic and real data. As both NNs consider
the tasks as regression problems, the ML metric used to evaluate the models is
the MSE loss.

The NNs produce real-valued complex symbols, which are then converted back
into complex symbols. These symbols can be represented as constellations (see
Figure 5(a)). We demodulate the complex symbols to obtain received bits, which
are compared to transmitted bits to calculate the bit error rate (BER),
serving as a system performance metric.

###  IV-A Synthetic data

In this subsection, we consider a synthetic dataset for both the training and
testing phases of the proposed NN models. This synthetic dataset is essential
for assessing how well our models function in a controlled environment. The
obtained MSE is shown in Figures 4(a) and 4(b) for Symbol Enhancement NN and
Equalization NN, respectively, trained and tested with SNR = 20 dB. The NNs
exhibit optimal performance when evaluated under SNR conditions matching their
training data. For instance, improved symbol constellations are evident in
Figures 5(a) and 5(b) when the NNs are tested with samples that have the same
SNR as the training data.

(a) Symbol Enhancement NN

(b) Equalization NN

Figure 4: Learning curves for NNs trained and tested with SNR = 20dB with
synthetic data.

(a) Symbol Enhancement NN

(b) Equalization NN

Figure 5: Constellations for NNs trained and tested with SNR = 20dB with
synthetic data.

On the other hand, testing the models at SNRs different from their training
data leads to significant performance degradation For instance, when
Equalization NN is trained with samples with a SNR of 20 dB and then tested
with samples with a SNR of 10 dB, the gap between the MSE obtained during
training and validation increases, as illustrated in Figure 6(a).

(a) 20,10

(b) several,20

Figure 6: Learning curves for Equalization NN with synthetic data. (a) Trained
with SNR = 20 dB and tested with SNR = 10 dB. (b) Trained with several SNRs
and tested with SNR = 20 dB

A similar degradation in performance can be noticed in Figure 6(b) when the
models are trained across a range of SNRs but are evaluated under a specific,
fixed SNR condition.

The models’ ability to generalize and maintain their efficacy across a variety
of real-world scenarios is directly impacted by the nature of the samples used
during training.

We compare the BER without ML techniques to the BER with our NN models in
Figure 7. We explore three training scenarios. In the first scenario,
individual training for each SNR results in specialized models for each SNR
setting. In the second scenario, training across various SNRs leads to a
single unified model for symbol enhancement and another for equalization,
testing the models’ adaptability. In the third and final scenario, exclusive
training at SNR = 20 dB produces a model specialized for this SNR setting.
These trained models are then tested at specific SNR levels.

Figure 7: BER curves before and after Symbol Enhancement NN and Equalization
NN for synthetic data.

###  IV-B Real data

In this subsection, we consider a dataset obtained from real-world
experimental tests to train and test the proposed NN models. The obtained MSE
is shown in Figures 8(a) and 8(b) for Symbol Enhancement NN and Equalization
NN, respectively.

(a) Symbol Enhancement NN

(b) Equalization NN

Figure 8: Learning curves for NNs trained and tested with SNR = 20dB with real
data.

Symbol Enhancement NN demonstrates a satisfactory level of performance,
maintaining acceptable MSE values, and its constellations in Figures 10(a) and
10(a) closely resemble the transmitted symbols. In terms of BER, Figure 11
shows Symbol Enhancement NN outperforming traditional equalizers without ML.
This indicates Symbol Enhancement NN’s effectiveness and robustness in real-
world data scenarios.

On the other hand, Equalization NN exhibits a larger gap between training and
validation MSE in Figure 8(b), indicating poorer performance with real data
compared to synthetic data. While Figures 9(b) and 10(b) suggest Equalization
NN’s ability to equalize symbols, its BER is less favorable than conventional
equalizers without ML, as shown in Figure 11. This underscores the challenge
of addressing equalization problems when training data inadequately represents
the issues targeted by the Equalization NN.

(a) Symbol Enhancement NN

(b) Equalization NN

Figure 9: Constellations for NNs trained and tested with SNR = 20dB for real
data.

(a) Symbol Enhancement NN

(b) Equalization NN

Figure 10: Constellation for NNs trained and tested with SNR = 10dB for real
data. Figure 11: BER curve for real data before and after Symbol Enhancement
NN and Equalization NN.

##  V Conclusion and Future Work

In this work, we’ve highlighted the potential of machine learning to improve
5G satellite signal decoding. We proposed two NNs to handle symbol enhancement
and equalization tasks. The results demonstrate significant improvements for
both synthetic and real datasets. Our future work aims to broaden our training
dataset by capturing a wider range of signals. We expect to refine our machine
learning models and enhance their adaptability in real-world satellite
communication environments.

3G

    Third Generation
4G

    Fourth Generation
5G

    Fifth Generation
6G

    Sixth Generation
3GPP

    3rd Generation Partnership Project
AI

    Artificial Intelligence
AMF

    Access and Mobility Function
AoA

    Angle of Arrival
AR

    Augmented Reality
AWGN

    Additive White Gaussian Noise
BER

    Bit Error Rate
BW

    Bandwidth
BWP

    Bandwidth Part
BWPP

    Bandwidth Part for Positioning
CA

    Cell Averaging
CDF

    Cumulative Density Function
CIR

    Channel Impulse Response
CFAR

    Constant False Alarm Rate
CFO

    Carrier Frequency Offset
CP

    Cyclix Prefix
CP-OFDM

    Cyplix Prefix Orthogonal Frequency-Division Modulation
CRLB

    Cramer-Rao Lower Bound
CU

    Centralised Unit
DDM

    Delay/Doppler Map
DFT

    Discrete Fourier Transform
DL-AoD

    Downlink Angle of Departure
DL-TDoA

    Downlink Time Difference of Arrival
DL-OTDoA

    Downlink Observed Time Difference of Arrival
DMRS

    DeModulation Reference Signal
DU

    Distributed Unit
DVB

    Digital Video Broadcasting
E-CID

    Enhanced Cell ID
ECDF

    Empirical Cumulative Density Function
E-SMLC

    Evolved Serving Mobile Location Center
EIRP

    Equivalent Isotropic Radiated Power
eNB

    evolved NodeB
FCC

    Federal Communications Commission
FR

    Frequency Regions
FR1

    Frequency Region 1
FR2

    Frequency Region 2
GDOP

    Geometrical Dilution of Precision
GPS

    Global Positioning System
gNB

    Next Generation Base Station
GNSS

    Global Navigation Satellite System
GS

    Ground Station
GSCN

    Global Synchronization Channel Number
HAPS

    High-Altitude Platform Systems
IDFT

    Inverse Discrete Fourier Transform
IFFT

    Inverse Fast Fourier Transform
IoT

    Internet of Things
IIoT

    Industrial Internet of Things
ICI

    Inter-Channel Interference
IMU

    Inertial Measurement Unit
ISI

    Inter-Symbol Interference
KPI

    Key Performance Indicator
LCS

    Location-based Services
LEO

    Low Earth Orbit
LMC

    Location Management Component
LMF

    Location Management Function
LOS

    Line of Sight
LPP

    Localization Positioning Protocol
LPPa

    Localization Positioning Protocol Annex
LTE

    Long Term Evolution
MIB

    Master Information Block
ML

    Machine Learning
Multi-RTT

    Multicell Round Trip Time
NF

    Network Function
NLOS

    Non-Line of Sight
NN

    Neural Network
NPRM

    Notice of Proposed Rulemaking
NR

    New Radio
NTN

    Non-Terrestrial Networks
OFDM

    Orthogonal Frequency-Division Multiplexing
OTDoA

    Observed Time Differential of Arrival
OTFS

    Orthogonal Time Frequency Space Modulation
PDF

    Probability Density Function
PNT

    Positioning, Navigation, and Timing
POD

    Precise Orbit Determination
PPP

    Precise Point Positioning
PRN

    Pseudo-Random Noise
PRS

    Positioning Reference Signal
PSS

    Primary Synchronization Signal
PVT

    Position, Velocity, Time
RAN

    Radio Access Network
RAT

    Radio-Access-Technology
RB

    Resource Block
RE

    Resource Element
RG

    Resource Grid
RedCap

    Reduced Capacity
RMSE

    Root Mean Square Error
RTK

    Real Time Kinematics
RRC

    Radio Resource Control
SBAS

    Satellite Based Augmentation System
SC

    SubCarrier
SCS

    SubCarrier Spacing
SIB

    System Information Block
SIC

    Successive Interference Cancellation
SINR

    Signal-to-Interference-plus-Noise ratio
SLA

    Service Level Agreement
SNR

    Signal-to-Noise Ratio
SoO

    Signal of Opportunity
SoP

    Signal of Opportunity
SRS

    Sounding Reference Signal
SSB

    Synchronization Signal Block
SSS

    Secondary Synchronization Signal
TA

    Timing Advance
TDE

    Time-Domain Equalizer
TDL

    Tapped Delay Line
TN

    Terrestrial Network
ToA

    Time of Arrival
ToF

    Time of Flight
TS

    Technical Specification
TR

    Technical Report
UAV

    Unmanned Aerial Vehicle
UE

    User Equipment
UL-AoA

    Uplink Angle of Arrival
UL-TDoA

    Uplink Time Difference of Arrival
VR

    Virtual Reality
WLAN

    Wireless Local Area Network
WLS

    Weighted Least Squares

## References

  * [1] “Update-on-5g-non-terrestrial-networks,” https://www.5gamericas.org/update-on-5g-non-terrestrial-networks/, accessed: 2023-09-09.
  * [2] X. Lin, “An overview of 5G advanced evolution in 3GPP release 18,” _IEEE Communications Standards Magazine_ , vol. 6, no. 3, pp. 77–83, 2022\.
  * [3] A. Machumilane, A. Gotta, P. Cassará, G. Amato, and C. Gennaro, “Learning-based traffic scheduling in non-stationary multipath 5G non-terrestrial networks,” _Remote Sensing_ , vol. 15, no. 7, p. 1842, 2023\.
  * [4] M. K. Dahouda, S. Jin, and I. Joe, “Machine learning-based solutions for handover decisions in non-terrestrial networks,” _Electronics_ , vol. 12, no. 8, p. 1759, 2023.
  * [5] S. Mahboob and L. Liu, “A tutorial on AI-enabled non-terrestrial networks in 6G,” _arXiv preprint arXiv:2303.01633_ , 2023.
  * [6] H. Ye, G. Y. Li, and B.-H. Juang, “Power of deep learning for channel estimation and signal detection in OFDM systems,” _IEEE Wireless Communications Letters_ , vol. 7, no. 1, pp. 114–117, 2017.
  * [7] X. Gao, S. Jin, C.-K. Wen, and G. Y. Li, “Comnet: Combination of deep learning and expert knowledge in OFDM receivers,” _IEEE Communications Letters_ , vol. 22, no. 12, pp. 2627–2630, 2018.
  * [8] M. O. K. Mendonça and P. S. R. Diniz, “OFDM receiver using deep learning: Redundancy issues,” in _2020 28th European Signal Processing Conference (EUSIPCO)_. IEEE, 2021, pp. 1687–1691.
  * [9] 3GPP, “NR; Physical channels and modulation,” 3rd Generation Partnership Project (3GPP), Technical Specification (TS) 38.211. [Online]. Available: http://www.3gpp.org/DynaReport/38211.htm
  * [10] “6GSPACE Lab website,” https://6gspacelab.uni.lu/, accessed: 2023-09-09.

  *[ML]: Machine Learning
  *[NTN]: Non-Terrestrial Networks
  *[MIB]: Master Information Block
  *[UE]: User Equipment
  *[5G]: Fifth Generation
  *[SSB]: Synchronization Signal Block
  *[RG]: Resource Grid
  *[PSS]: Primary Synchronization Signal
  *[SSS]: Secondary Synchronization Signal
  *[GSCN]: Global Synchronization Channel Number
  *[CFO]: Carrier Frequency Offset
  *[RE]: Resource Element
  *[DMRS]: DeModulation Reference Signal
  *[NN]: Neural Network


orthogonal to $Im(D)$. Since the factor
$\frac{\overline{im(D^{\prime\prime})}}{im(D^{\prime\prime})}$ are of
$\mathcal{U}(\Gamma)$-torsion so we obtain finally the following result on the
existence of pure Hodge structure on the cohomology.

######  Theorem 9.7.

If $\mathbb{V}$ is a local system underlying a polarized complex variation of
Hodge structure on $M$, the canonical isomorphisms of
$\mathcal{U}(\Gamma)$-module

$\displaystyle\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}H^{*}\left(X,\ell^{2}\pi^{*}\mathbb{V}\right)$
$\displaystyle\simeq\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}\mathbb{H}^{*}(X,\Omega_{(2)}^{\bullet}(\pi^{*}\mathbb{V}))$
$\displaystyle\simeq\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}H_{2}^{*}(\tilde{M},\pi^{*}\mathbb{V})$

induces a pure Hodge structure of weight $w+k$ on
$\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}H^{k}\left(X,\ell^{2}\pi^{*}\mathbb{V}\right)$
and one has a canonical isomorphisms

$\displaystyle
Gr^{p}_{F}(\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}H^{k}\left(X,\ell^{2}\pi^{*}\mathbb{V}\right))$
$\displaystyle\simeq\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}\mathbb{H}^{k}\left(X,Gr^{p}_{F}(\Omega_{(2)}^{\bullet}(\pi^{*}\mathbb{V}))\right)$
$\displaystyle\simeq\mathcal{U}(\Gamma)\otimes
Harm(\tilde{M},\pi^{*}\mathbb{V})^{p,w+k-p}.$

######  Remark.

The $\Gamma$-Fredholm property ensures that the $L^{2}$ cohomology groups have
a finite $\mathcal{N}(\Gamma)$-dimension.

##  10 The $L^{2}$ Riemann-Hurwitz formula

In this section we will show a Riemann-Hurwitz type formula. Before stating
the result we set a few notations

$\displaystyle\chi_{2,\Gamma}(\tilde{M},\pi^{-1}\mathbb{V})$
$\displaystyle=\sum_{k=0}^{2}(-1)^{k}\dim_{\mathcal{N}(\Gamma)}H^{k}_{2}(\tilde{M},\pi^{-1}\mathbb{V})$
$\displaystyle\chi_{2}(M,\mathbb{V})$
$\displaystyle=\sum_{k=0}^{2}(-1)^{k}\dim H^{k}_{2}(M,\mathbb{V})$

We recall that for our cohomology groups we consider we consider a hermitian
metric with Poincaré singularities on $M$, and a metric given by a
polarization on the flat vector bundle associated $\mathbb{V}$, and their
pull-back and $\tilde{M}$. For $p\in X\setminus M$, we will denote by $n_{p}$
the cardinal of the isotropy subgroup of a preimage of $p$, and by
$j_{*}\mathbb{V}_{p}$ the stalk of $j_{*}\mathbb{V}$ at $p$.

######  Theorem 10.1.

($L^{2}$ Riemann-Hurwitz lemma) We have the following equality.

$\chi_{2,\Gamma}(\tilde{M},\pi^{-1}\mathbb{V})-\sum_{p\in X\setminus
M}\frac{\dim j_{*}\mathbb{V}_{p}}{n_{p}}=\chi_{2}(M,\mathbb{V})-\sum_{p\in
X\setminus M}\dim j_{*}\mathbb{V}_{p}$

######  Remark.

  1. 1.

The integer $\chi_{2}(M,\mathbb{V})$ is equal to the alternate sum of the
dimension of the groups $H^{k}(X,j_{*}\mathbb{V})$ so the left handside only
depends on the topology of $X$ and the local system $\mathbb{V}$.

  2. 2.

In the case $\mathbb{V}=\mathbb{C}_{M}$ and if $\Gamma$ is finite, we have
$\chi_{2,\Gamma}(\tilde{M},\pi^{-1}\mathbb{V})=\frac{1}{|\Gamma|}\chi(\tilde{X})$,
so the formula become

$\chi(\tilde{X})-\sum_{p\in X\setminus
M}\frac{|\Gamma|}{n_{p}}=|\Gamma|\big{(}\chi(X)-|X\setminus M|\big{)}$

since $\frac{|\Gamma|}{n_{p}}$ is equal to the number of preimage of $p$ we
recover the theorem of Riemann-Hurwitz.

###### Proof.

By applying Lemma 9.5 to our covering and to the trivial covering one obtains.

$H^{k}(C^{\bullet}(\mathfrak{U},\ell^{2}\mathbb{V}))=H^{k}_{2}(\tilde{M},\pi^{*}\mathbb{V})\qquad
H^{k}(C^{\bullet}(\mathfrak{U},j_{*}\mathbb{V}))=H^{k}_{2}(M,\mathbb{V})$

Since the Čech complex $C^{\bullet}(\mathfrak{U},j_{*}\mathbb{V})$ (resp.
$C^{\bullet}(\mathfrak{U},\ell^{2}\mathbb{V})$) is a complex of terms of
finite dimension (resp. finite $\mathcal{N}(\Gamma)$ dimension) one obtains

$\displaystyle\sum(-1)^{k}\dim_{\mathcal{N}(\Gamma)}H^{k}_{2}(\tilde{M},\pi^{-1}\mathbb{V})$
$\displaystyle=\sum(-1)^{k}\dim_{\mathcal{N}(\Gamma)}C^{k}(\mathfrak{U},\ell^{2}\pi^{*}\mathbb{V})$
$\displaystyle\sum(-1)^{k}\dim H^{k}_{2}(M,\mathbb{V})$
$\displaystyle=\sum(-1)^{k}\dim C^{k}(\mathfrak{U},j_{*}\mathbb{V})$

Recall that

$\displaystyle C^{k}(\mathfrak{U},\ell^{2}\pi^{*}\mathbb{V})$
$\displaystyle=\underset{\alpha_{0}<\dots<\alpha_{k}}{\prod}\ell^{2}\pi^{*}\mathbb{V}(U_{\alpha_{0},\dots,\alpha_{k}})$
$\displaystyle C^{k}(\mathfrak{U},j_{*}\mathbb{V})$
$\displaystyle=\underset{\alpha_{0}<\dots<\alpha_{k}}{\prod}j_{*}\mathbb{V}(U_{\alpha_{0},\dots,\alpha_{k}}).$

But for $k>0$, one has
$\ell^{2}\pi^{*}\mathbb{V}(U_{\alpha_{0},\dots,\alpha_{k}})\simeq\ell^{2}(\Gamma)\otimes\mathbb{C}^{rk(\mathbb{V})}$
and
$j_{*}\mathbb{V}(U_{\alpha_{0},\dots,\alpha_{k}})\simeq\mathbb{C}^{rk(\mathbb{V})}$,
since $U_{\alpha_{0},\dots,\alpha_{k}}$ is a contractible and included in $M$.
It follows that for $k>0$ one has

$\dim_{\mathcal{N}(\Gamma)}C^{k}(\mathfrak{U},\ell^{2}\pi^{*}\mathbb{V})=\dim
C^{k}(\mathfrak{U},j_{*}\mathbb{V}).$

For $k=0$ there is two cases either $U_{\alpha_{0}}$ is included in $M$ and in
this case quasi-isometric to a disk endowed with the Euclidean metric, or is a
neighbourhood $U_{p}$ of a point $p$ quasi-isometric to a punctured disk
$\Delta_{1/2}$ endowed with the Poincaré metric. In the first case
$\ell^{2}\pi^{*}\mathbb{V}(U_{\alpha_{0}})\simeq\ell^{2}(\Gamma)\otimes\mathbb{C}^{rk(\mathbb{V})}$
and $j_{*}\mathbb{V}(U_{\alpha_{0}})\simeq\mathbb{C}^{rk(\mathbb{V})}$ ; while
in the second case
$j_{*}\mathbb{V}(U_{p})\simeq\mathbb{C}^{\dim(\mathbb{V}_{p})}$ and
$\ell^{2}\pi^{*}\mathbb{V}(U_{p})\simeq\ell^{2}(\Gamma/H_{p})\otimes\mathbb{C}^{\dim(\mathbb{V}_{p})}$
where $H_{p}$ is the isotropy subgroup of a preimage of $p$, note that
$|H_{p}|=n_{p}$ by definition and that
$\dim_{\mathcal{N}(\Gamma)}\ell^{2}(\Gamma/H_{p})=\frac{1}{n_{p}}$. So one
obtains

$\displaystyle\chi_{2,\Gamma}(\tilde{M},\pi^{-1}\mathbb{V})$
$\displaystyle=\underset{U_{\alpha_{0}}\subset
M}{\sum}rk(\mathbb{V})+\sum_{p\in X\setminus
M}\frac{\dim(\mathbb{V}_{p})}{n_{p}}+\sum_{k>0}(-1)^{k}\dim_{\mathcal{N}(\Gamma)}C^{k}(\mathfrak{U},\ell^{2}\pi^{*}\mathbb{V})$
$\displaystyle\chi_{2}(M,\mathbb{V})$
$\displaystyle=\underset{U_{\alpha_{0}}\subset
M}{\sum}rk(\mathbb{V})+\sum_{p\in X\setminus
M}\dim(\mathbb{V}_{p})+\sum_{k>0}(-1)^{k}\dim
C^{k}(\mathfrak{U},j_{*}\mathbb{V})$

and the result follows. ∎

##  11 Link with the theory of polarized Hodge module

As before we have $j\colon M\to X$ an embedding of Riemann surfaces, with $X$
compact and $\Sigma:=X\setminus M$ consisting of a finite set of points.

### Middle extension of a variation of Hodge structure

We set $\mathcal{D}_{X}$ the sheaf of differential operators on $X$, by a
$\mathcal{D}_{X}$-module we will always mean a left $\mathcal{D}_{X}$-module.
If $(\mathcal{V},F^{\bullet},\nabla)$ is a holomorphic vector bundle on $M$
with a flat connection, underlying a polarizable variation of Hodge structure
we will denote by $\mathcal{V}_{*}$ its canonical meromorphic extension. It is
endowed locally with the parabolic filtration $\mathcal{V}_{*}^{\bullet}$
described in section $3$. We denote by $\mathcal{V}_{mid}$ the
$\mathcal{D}_{X}$ module generated by $\mathcal{V}_{*}^{>-1}$, i.e

$\mathcal{V}_{mid}=\sum_{j\in\mathbb{N}}(\nabla_{\partial_{z}})^{j}\mathcal{V}_{*}^{>-1}.$

We call $\mathcal{V}_{mid}$ the middle extension of $\mathcal{V}$, it is
endowed with a coherent good filtration $F^{\bullet}\mathcal{V}_{mid}$ defined
by

$F^{p}\mathcal{V}_{mid}:=\sum_{j\in\mathbb{N}}(\nabla_{\partial_{z}})^{j}F^{p-j}\mathcal{V}^{>-1}_{*}.$

It is locally endowed with the Kashiwara-Malgrange filtration
$V^{\bullet}\mathcal{V}_{mid}:=\mathcal{V}_{mid}^{\bullet}$ which is given by

$\mathcal{V}_{mid}^{\beta}=\left\\{\begin{array}[]{cc}\mathcal{V}^{\beta}_{*}&\text{
if }\beta>-1\\\
(\nabla_{\partial_{z}})^{-\lceil\beta\rceil}\mathcal{V}_{*}^{\beta-\lceil\beta\rceil}+\mathcal{V}_{*}^{>\beta}&\text{
if }\beta\leq-1\end{array}\right.$

Where $\lceil.\rceil$ denote the upper integral part of a real number, so for
$\beta\in\mathbb{R}$ one has $\lceil\beta\rceil-1<\beta\leq\lceil\beta\rceil$.
For $\beta\in\mathbb{R}$, the sheaf $\mathcal{V}_{mid}^{\beta}$ is
$\mathcal{O}_{X}$-coherent. The space $Gr^{\beta}\mathcal{V}_{mid}$ is finite
dimensional and $(z\partial_{z}-\beta)$ induce a nilpotent operator on
$Gr^{\beta}\mathcal{V}_{mid}$, which coincides with the action of the
$N_{\beta}$ we had on $\mathcal{V}_{*}$, and it induces a filtration
$W_{\bullet}(N_{\beta})$ on $Gr^{\beta}\mathcal{V}_{mid}$, in a similar
fashion of what we did for the meromorphic extension we set

$M_{k}\mathcal{V}^{\beta}_{mid}:=p^{-1}\left(W_{k}(N_{\beta})Gr^{\beta}\mathcal{V}_{mid}\right)$

where $p\colon\mathcal{V}^{\beta}_{mid}\to Gr^{\beta}\mathcal{V}_{mid}$ is the
projection.

######  Proposition 11.1.

[SS22, Proposition 6.14.2, Corollary 6.14.4] We have the following.

  * •

The filtration $F^{\bullet}\mathcal{V}_{mid}$ is exhaustive.

  * •

For $\beta>-1$ we have
$F^{p}\mathcal{V}_{mid}^{\beta}=(j_{*}F^{p}\mathcal{V})\cap\mathcal{V}_{*}^{\beta}$
and $z(F^{p}\mathcal{V}_{mid}^{\beta})=F^{p}\mathcal{V}_{mid}^{\beta+1}$.

  * •

For $\beta\leq 0$,
$\partial_{z}F^{p}Gr^{\beta}\mathcal{V}_{mid}=F^{p-1}Gr^{\beta-1}\mathcal{V}_{mid}$.

  * •

The sheaves $F^{p}\mathcal{V}_{mid}$, $F^{p}\mathcal{V}_{mid}^{\beta}$ and
$F^{p}M_{k}\mathcal{V}_{mid}$ are $\mathcal{O}_{X}$ locally free and of finite
rank.

The De Rham complex of $\mathcal{V}_{mid}$ is given by

$DR(\mathcal{V}_{mid}):=\left\\{0\to\mathcal{V}_{mid}\overset{\nabla}{\longrightarrow}\Omega^{1}_{X}\otimes\mathcal{V}_{mid}\to
0\right\\}$

And its perverse De Rham complex is given by
${}^{p}\\!DR(\mathcal{V}_{mid}):=DR(\mathcal{V}_{mid})[1]$. We have the
Kashiwara-Malgrange filtration and the Hodge filtration on those bundle
defined by

${V^{\beta}DR(\mathcal{V}_{mid}):=0}$${\mathcal{V}^{\beta}_{mid}}$${\Omega^{1}_{X}\otimes\mathcal{V}_{mid}^{\beta-1}}$${0}$${F^{p}DR(\mathcal{V}_{mid}):=0}$${F^{p}\mathcal{V}_{mid}}$${\Omega^{1}_{X}\otimes
F^{p-1}\mathcal{V}_{mid}}$${0}$$\scriptstyle{\nabla}$$\scriptstyle{\nabla}$

One can check that by 2.5, the holomorphic De Rham complex
$\Omega^{\bullet}(\mathbb{V})_{(2)}$ of $\mathcal{V}$ is a subcomplex of
$V^{0}DR(\mathcal{V}_{mid})$. We recall that we have the following result
[SS22, Proposition 6.14.8]

######  Proposition 11.2.

For any $p\in\mathbb{Z}$ the inclusions

$F^{p}\Omega^{\bullet}(\mathbb{V})_{(2)}\to F^{p}V^{0}DR(\mathcal{V}_{mid})\to
F^{p}DR(\mathcal{V}_{mid})$

are quasi-isomorphisms.

### Polarized Hodge module on a curve

If $X$ is a complex manifold, a polarized Hodge module on $X$ is given by the
data of $(\mathcal{M},F^{\bullet})$ a $\mathcal{D}_{X}$-module endowed with a
good filtration, a perverse sheaf $\mathbb{M}^{Betti}$ endowed with a quasi-
isomorphism $\alpha\colon\mathbb{M}^{Betti}\to{}^{p}\\!DR(\mathcal{M})$ and a
sesquilinear pairing
$S\colon\mathcal{M}\otimes_{\mathbb{C}}\overline{\mathcal{M}}\to\mathfrak{Db}_{X}$
called polarization. We require that those data satisfies some non trivial
relations. In the case where $X$ is a Riemann surface polarized Hodge module
admits a simple decomposition that we will recall here (see [SS22, chapter 7])
and that we will use as definition.

In the following $X$ will denote a Riemann surface, $\Sigma$ will be a finite
set of point of $X$ and we will set $M:=X\setminus\Sigma$.

######  Definition 11.3.

A polarized Hodge module $X$ of weight $w$ with singularities at most at
$\Sigma$ and of pure support $X$ is a middle extension of a polarized
variation of Hodge structure on $M$ of weight $w-1$.

In this case the filtered $\mathcal{D}_{X}$ module is given by
$\mathcal{V}_{mid}$ endowed with the Hodge filtration, and the perverse sheaf
is given by $j_{*}\mathbb{V}[1]$ where $j\colon M\to X$ is the inclusion and
$\mathbb{V}$ is the underlying local system.

This definition is not the usual definition of polarized Hodge module with
pure support $X$, however it is equivalent of the usual one thanks to [SS22,
Proposition 7.4.12]. The shift of weight is there because we will consider the
perverse De Rham complex ${}^{p}\\!DR(\mathcal{V}_{mid})$ which is a shift of
the usual De Rham complex.

We also need to define polarized Hodge module with punctual support $\Sigma$.
Take $\mathcal{H}_{\Sigma}$ a sheaf of polarized Hodge structure of weight $w$
on $\Sigma$, it is equivalent to fix a polarized Hodge structure
$(H_{\Sigma,p},F^{\bullet}H_{\Sigma,p},h_{p})$ for every point $p\in\Sigma$.
We define ${}_{D}\iota(\mathcal{H}_{\Sigma})$ to be a skycrapper sheaf
supported at $\Sigma$, the stalks at $p\in\Sigma$ is given by

${}_{D}\iota\mathcal{H}_{\Sigma,p}:=H_{\Sigma,p}[\partial_{z}].$

We define $\partial_{z}\cdot(v\partial_{z}^{k})=v\partial_{z}^{k+1}$, for
$n\in\mathbb{N}$ we define the action of $\mathbb{C}[z]$ on $H_{\Sigma,p}$ by

$z^{n}\cdot v\partial_{z}^{k}=\left\\{\begin{array}[]{cc}0&\text{if }n\geq
k\\\ (-1)^{n}k(k-1)\dots(k-n+1)v\partial_{z}^{k-n}&\text{
otherwise}\end{array}\right.$

It extends naturally to an action of $\mathcal{O}_{X,p}$, moreover one has the
relation

$f\cdot(\partial_{z}\cdot v\partial_{z}^{k})+f^{\prime}\cdot
v\partial_{z}^{k}=\partial_{z}\cdot(f\cdot v\partial_{z}^{k}),$

so this endows ${}_{D}\iota(\mathcal{H}_{\Sigma})$ with the structure of a
$\mathcal{D}_{X}$ module. It is endowed with the natural good filtration

$F^{p}{}_{D}\iota\mathcal{H}_{\Sigma,p}:=\underset{k\in\mathbb{N}}{\bigoplus}(F^{p-k}H_{\Sigma,p})\partial_{z}^{k}.$

Now we define polarized Hodge module with punctual support $\Sigma$.

######  Definition 11.4.

A polarized (pure) Hodge module $\mathcal{M}$ of weight $w$ with strict
support $\Sigma$ is a $\mathcal{D}_{X}$-module of the form
${}_{D}\iota\mathcal{H}_{\Sigma}$ where $\mathcal{H}_{\Sigma}$ is a sheaf of
polarized Hodge structure of weight $w$ on the finite set $\Sigma$.

Now we recall the support-decomposition theorem for polarized Hodge module on
Riemann surfaces (see [SS22, Theorem 7.4.10]).

######  Theorem 11.5.

A polarized Hodge module $\mathcal{M}$ of weight $w$ on $X$ with singularities
at most at $\Sigma$ is a $\mathcal{D}_{X}$-module $\mathcal{M}$ of the form
$\mathcal{M}_{1}\oplus\mathcal{M}_{2}$ where $\mathcal{M}_{1}$(resp.
$\mathcal{M}_{2}$) is a polarized Hodge module of weight $w$ with pure support
$X$ (resp. with punctual support $\Sigma$).

###  $L^{2}$ direct image of polarized Hodge module

In this section we assume that the covering $\pi\colon\tilde{M}\to M$ induces
an unramified covering $\pi\colon\tilde{X}\to X$, we set $\Sigma:=X\setminus
M$.

We remind the following result of [Eys22] that we state in the case of
polarizable (pure) Hodge module. We recall the definition of the abelian
category $E_{f}(\Gamma)$ which was define by Farber and Luck.

######  Definition 11.6.

The category $E_{f}(\Gamma)$ is the category whose objects are triple
$(E_{1},E_{2},e)$ where $E_{1}$ and $E_{2}$ are Hilbert $\mathcal{N}(\Gamma)$
module of finite type, and $e\colon E_{1}\to E_{2}$ is a bounded
$\mathcal{N}(\Gamma)$ equivariant morphism.

If $E=(E_{1},E_{2},e)$ and $F=(F_{1},F_{2},f)$ are two objects of
$E_{f}(\Gamma)$, the set $Hom_{E_{f}(\Gamma)}(E,F)$ is the set of pair
$(\phi_{1},\phi_{2})$ where $\phi_{1}\colon E_{1}\to F_{1}$, $\phi_{2}\colon
E_{2}\to F_{2}$ are morphism of Hilbert $\mathcal{N}(\Gamma)$ module
satisfying $\phi_{2}e=f\phi_{1}$, under the equivalence relation
$(\phi_{1},\phi_{2})\sim(\psi_{1},\psi_{2})$ if there exists $T\colon F_{2}\to
E_{1}$ such that $\psi_{2}-\phi_{2}=fT$.

######  Proposition 11.7.

[Far96] The category $E_{f}(\Gamma)$ is abelian.

######  Remark.

The category of Hilbert $\mathcal{N}(\Gamma)$ module is embedded in the
category of Farber via the functor

$E\to(\\{0\\},E,0).$

Moreover one has a forgetful functor from the category $E_{f}(\Gamma)$ to the
category of $\mathcal{N}(\Gamma)$-module of finite type given by

$(E_{1},E_{2},e)\to E_{2}/e(E_{1})$

With this we can state the following result of [Eys22].

######  Theorem 11.8.

[Eys22, Corollary 2] Let $(X,\omega_{X})$ be a compact Kähler manifold,
$\pi\colon\tilde{X}\to X$ a Galois group of covering group $\Gamma$. We denote
by $pHM(X)$ the category of polarized Hodge module on $X$. There exists a
$\partial$-functor satisfying Atiyah index theorem and Poincaré-Verdier
Duality

$L^{2}dR\colon D^{b}pHM(X)\to D^{b}E_{f}(\Gamma)$

such that for all Hodge module $\mathbb{M}$ and $q\in\mathbb{Z}$

$H^{q}(L^{2}dR(\mathcal{M}))\simeq
H^{q}_{(2)}(\tilde{X},\pi^{*}\mathbb{M}^{Betti}).$

Those cohomology groups are endowed with a filtration $F^{\bullet}$ induced by
Saito’s Hodge filtration on $\mathcal{M}$.

We recall the definition the functor $L^{2}dR$ and some of its properties. For
this we need to recall the definition of the functor $\ell^{2}\pi_{*}$ of
$L^{2}$-direct image as it is introduced in [CD01, Eys00]. It is a functor
from the category of coherent $\mathcal{O}_{X}$ module to the the category of
$\mathcal{N}(\Gamma)_{X}$-module where if $\mathcal{F}$ is a coherent
$\mathcal{O}_{X}$-module, $U$ is coordinates chart such that
$\pi^{-1}(U)\simeq\Gamma\times U$ and we fix $\phi\colon\mathcal{O}_{X\mid
U}^{\oplus N}\to\mathcal{F}_{U}$ a local presentation

$\ell^{2}\pi_{*}\mathcal{F}(U):=\left\\{(s_{\gamma})_{\gamma}\in\mathcal{F}(U)^{\Gamma}\left|\begin{array}[]{ll}\exists(r_{\gamma})_{\gamma}\in\mathcal{O}_{X}(U)^{\Gamma},&\forall
K\subset U\text{ compact}\\\
\sum_{\gamma\in\Gamma}\int_{K}|r_{\gamma}|^{2}<+\infty&\end{array}\right.\right\\}.$

It can be check that it does not depends of the local presentation and since
$X$ is compact, $\ell^{2}\pi_{*}$ does not depends on the Kähler metric
$\omega_{X}$, moreover by definition $\ell^{2}\pi_{*}\mathcal{F}$ is a
subsheaf of $\pi_{*}\pi^{*}\mathcal{F}$.

######  Lemma 11.9.

[Eys22, Lemma 2.1.1] The functor $\ell^{2}\pi_{*}$ can be extended in a exact
functor on the category $Mod(\mathcal{O}_{X})$ of $\mathcal{O}_{X}$-module by
setting

$\ell^{2}\pi_{*}\mathcal{F}:=\ell^{2}\pi_{*}\mathcal{O}_{X}\otimes_{\mathcal{O}_{X}}\mathcal{F}.$

We need to extend this functor in the category of
$Qcoh(\mathcal{O}_{X},Diff_{X})$ of quasi-coherent $\mathcal{O}_{X}$-module
whose morphisms are differential operators. We recall that a differential
operator $P\colon\mathcal{F}_{1}\to\mathcal{F}_{2}$ is an operator in the
image of

$\nu\colon
Hom_{\mathcal{O}_{x}}(\mathcal{F}_{1},\mathcal{F}_{2}\otimes_{\mathcal{O}_{X}}\mathcal{D}_{X})\to
Hom_{\mathbb{C}_{X}}(\mathcal{F}_{1},\mathcal{F}_{2})$

where the structure of $\mathcal{O}_{X}$-module on
$\mathcal{F}_{2}\otimes_{\mathcal{O}_{X}}\mathcal{D}_{X}$ is given by the
right $\mathcal{O}_{X}$-module structure, and $\nu$ is given by the left
composition by the morphism

$\displaystyle\mathcal{F}_{2}\otimes_{\mathcal{O}_{X}}\mathcal{D}_{X}$
$\displaystyle\to\mathcal{F}_{2}$ $\displaystyle f\otimes P$
$\displaystyle\mapsto P(1)f$

######  Lemma 11.10.

[Eys22, Lemma 2.2.2] Let $P:=\nu(p)\colon\mathcal{F}_{1}\to\mathcal{F}_{2}$ be
a differential operator, the morphism
$\ell^{2}\pi_{*}P:=\nu(\ell^{2}\pi_{*}p)\colon\ell^{2}\pi_{*}\mathcal{F}_{1}$
to $\ell^{2}\pi_{*}\mathcal{F}_{2}$ is the restriction of
$\pi_{*}\pi^{*}P\colon\pi^{*}\pi_{*}\mathcal{F}_{1}\to\pi_{*}\pi^{*}\mathcal{F}_{2}$
and it is a morphism of sheaves of $\mathcal{N}(\Gamma)$-module. Hence
$\ell^{2}\pi_{*}$ defines an additive functor from the category
$QCoh(\mathcal{O}_{X},Diff_{X})$ to the category
$Mod(\mathcal{N}(\Gamma)_{X})$.

With this lemma one can define $L^{2}dR$ to be the functor
$R\Gamma\circ\ell^{2}\pi_{*}{}^{p}\\!DR(.)$. On the Betti side one can
understand it as the functor that sends $\mathbb{M}^{Betti}$ to
$\ell^{2}\pi_{*}\mathbb{M}^{Betti}:=\ell^{2}(\Gamma)\otimes_{\mathbb{C}[\Gamma]}\pi_{!}\pi^{*}\mathbb{M}^{Betti}$.

######  Proposition 11.11.

Let $\mathcal{V}_{mid}$ be a polarized Hodge module on $X$ with pure support
$X$ and weight $w$. One has a natural filtered inclusions

$F^{\bullet}\Omega^{\bullet}(\pi^{*}\mathbb{V})_{(2)}\to\ell^{2}\pi_{*}F^{\bullet}DR(\mathcal{V}_{mid})$

which is filtered quasi-isomorphism.

###### Proof.

We recall that in the case of the trivial covering the following inclusion is
a quasi-isomorphism for all $p\in\mathbb{Z}$ by Proposition 11.2

$F^{p}\Omega^{\bullet}(\mathbb{V})_{(2)}\to F^{p}DR(\mathcal{V}_{mid}).$

We want to check that we obtain quasi-isomorphisms

$F^{p}\Omega^{\bullet}(\pi^{*}\mathbb{V})_{(2)}\to\ell^{2}\pi_{*}F^{p}DR(\mathcal{V}_{mid}).$

The result will follow from the functoriality of $\ell^{2}\pi_{*}$ given by
the previous lemma 11.10. One has a natural isomorphism

$\Omega^{\bullet}(\pi^{*}\mathbb{V})_{(2)}\to\ell^{2}\pi^{*}\Omega(\mathbb{V})_{(2)}$

so one has to check that $\ell^{2}\pi_{*}\nabla$ induces a isomorphism

$[\ell^{2}\pi_{*}\nabla]\colon\ell^{2}\pi_{*}\left({\raisebox{1.99997pt}{$F^{p}V^{0}\mathcal{V}_{mid}$}\left/\raisebox{-1.99997pt}{$F^{p}\mathcal{O}(\mathbb{V})_{(2)}$}\right.}\right)\to\ell^{2}\pi_{*}\left({\raisebox{1.99997pt}{$F^{p-1}V^{-1}\mathcal{V}_{mid}$}\left/\raisebox{-1.99997pt}{$F^{p}\Omega^{1}(\mathbb{V})_{(2)}$}\right.}\right)$

which is true as $[\ell^{2}\pi_{*}\nabla]$ coincides with
$\ell^{2}\pi_{*}[\nabla]$ where

$[\nabla]\colon{\raisebox{1.99997pt}{$F^{p}V^{0}\mathcal{V}_{mid}$}\left/\raisebox{-1.99997pt}{$F^{p}\mathcal{O}(\mathbb{V})_{(2)}$}\right.}\to{\raisebox{1.99997pt}{$F^{p-1}V^{-1}\mathcal{V}_{mid}$}\left/\raisebox{-1.99997pt}{$F^{p}\Omega^{1}(\mathbb{V})_{(2)}$}\right.}$

is the morphism induced by $\nabla$, and $[\nabla]$ is an isomorphism by
Proposition 11.2. ∎

Finally one obtains the following result which gives a positive answer to
[Eys22, Conjecture 3] for the case of pure Hodge module on Riemann surfaces.

######  Theorem 11.12.

If $\mathbb{M}$ is a Hodge module on a compact Riemann surface $X$, then the
groups
$\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}\mathbb{H}^{k}(L^{2}dR(\mathbb{M}))$
is admits a pure Hodge structure of weight $w+k$ in the abelian category of
$\mathcal{U}(\Gamma)$-modules. The Hodge filtration is induced by Saito’s
Hodge filtration on the perverse De Rham complex ${}^{p}\\!DR(\mathcal{M})$.

In the case where $\mathbb{M}$ is a polarized Hodge module with pure support
$X$ and with singularities at most at $\Sigma:=X\setminus M$ there is an
isomorphism of $\mathcal{U}(\Gamma)$-module

$\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}\mathbb{H}^{k}(L^{2}dR(\mathbb{M}))\simeq\mathcal{U}(\Gamma)\otimes_{\mathcal{N}(\Gamma)}H^{k}_{(2),red}(\tilde{M},\pi^{*}\mathbb{V},\pi^{*}\omega_{Pc},\pi^{*}h)$

which is an isomorphism of Hodge structure, where the Hodge structure on the
reduced $L^{2}$-cohomology comes from the decomposition of harmonic forms by
type.

###### Proof.

The above proposition and the Theorem 9.7 treat the case where $\mathbb{M}$
has pure support $X$, so in the following we can assume that $\mathbb{M}$ has
punctual support $\Sigma$. On this case $\mathbb{M}$ is of the form
$\iota\mathcal{H}_{\Sigma}$ where $\mathcal{H}_{\Sigma}$ is a sheaf of
polarized Hodge structure of weight $w$ on $\Sigma$. In this case
$H^{k}(L^{2}dR(\mathbb{M}))=0$ unless $k=0$ in which case it is equal to

$\bigoplus_{p\in\Sigma}\ell^{2}(\Gamma)\otimes_{\mathbb{C}}H_{\Sigma,p}.$

so the cohomology is fully reduced and one has a Hodge structure on
$H^{0}(L^{2}dR(\mathbb{M}))$ of weight $w$ on the category of
$\mathcal{N}(\Gamma)$-module given by the Hodge structure on each
$H_{\Sigma,p}$, this gives the wanted Hodge structure on
$\mathcal{U}(\Gamma)\otimes H^{0}(L^{2}dR(\mathbb{M}))$. ∎

## References

  * [Ati76] M. Atiyah.  Elliptic operators, discrete groups and Von Neumann algebras.  Astérisque, 32(33):43–72, 1976.
  * [AV65] A. Andreotti and E. Vesentini.  Carleman estimates for the Laplace Beltrami equation on complex manifolds.  Pub. Math IHES, 25:81–130, 1965.
  * [BL92] J. Brüning and M. Lesch.  Hilbert complex.  J. of Functional Analysis, 108:88–132, 1992.
  * [BT82] R. Bott and L. Tu.  Differential forms in algebraic topology.  Springer, 1982.
  * [CD01] F. Campana and J.P. Demailly.  Cohomologie ${L^{2}}$ sur les revêtements d’une variété compacte complexe.  Arxiv för Matematik, 39(2):263–282, 2001.
  * [CG85] J. Cheeger and M. Gomov.  Bounds on the Von Neumann dimension of ${L^{2}}$-cohomology and the Gauss-Bonnet theorem for open manifold.  Journal of differential geometry, 21:1–34, 1985.
  * [CKS87] E. Cattani, A. Kaplan, and W. Schmid.  ${L^{2}}$ and intersection cohomologies for a polarized variation of Hodge structure.  Inventiones Mathematicae, 87:217–252, 1987.
  * [Del70] P. Deligne.  Équations différentielles à points singuliers réguliers, volume 163.  Springer, 1970.
  * [Din13] P. Dingoyan.  Some mixed hodge structures on $l^{2}$-cohomology groups of coverings of kähler manifolds.  Mathematische Annalen, 357(3):1119–1174, 2013.
  * [Dix64] J. Dixmier.  Les C*-algèbres et leurs représentations.  Gauthier-Villars, 1964.
  * [Eys00] P. Eyssidieux.  Invariants de Von Neumann des faisceaux cohérents.  Mathematische Annalen, 317:527–566, 2000.
  * [Eys22] P. Eyssidieux.  Towards a ${L}_{2}$ cohomology theory for Hodge modules on infinite covering spaces: ${L}_{2}$ constructible cohomology and ${L}_{2}$ De Rham cohomology for coherent D-modules.  arXiv:2203.06950, 2022.
  * [Far96] M. Farber.  Homological algebra of Novikov-Shubin invariants and Morse inequalities.  Geometric & Functional Analysis GAFA, 6(4):628–665, 1996.
  * [Fox57] R. Fox.  Covering spaces with singularities.  In Algebraic Geometry and Topology: A Symposium in Honor of S. Lefschetz, number 12, pages 243–257, 1957.
  * [God60] R. Godement.  Topologie algébrique et théorie des faisceaux.  Hermann, 1960.
  * [IL93] T. Iwaniec and A. Lutoborski.  Integral estimates for null lagrangians.  Arch. Rational Mech. Anal., 125:25–79, 1993.
  * [KK85] M. Kashiwara and T. Kawai.  The Poincaré lemma for a variation of polarized Hodge structure.  Proc. Japan Acad. Ser. A Math. Sci., 61(6):164–167, 1985.
  * [Lüc02] W. Lück.  $L^{2}$-invariants : Theory and Applications to Geometry and K-Theory.  Springer, 2002.
  * [Mal87] B. Malgrange.  Regular connections’ after Deligne.  In A. Borel, J. Coates, and S. Helgason, editors, Algebraic $D$-Modules, Perspectives in mathematics, chapter 4. Academic Press, 1987.
  * [Moc22] T. Mochizuki.  ${L^{2}}$ complexes and twistor complexes of tame harmonic bundles.  https://arxiv.org/abs/2204.10443, 2022.
  * [Nam87] M Namba.  Branched coverings and algebraic functions.  Number 161 in Pitman Research Notes in Mathematics. Longman Scientific and Technical, 1987.
  * [Sch73] W. Schmid.  Variation of Hodge structure: the singularities of the period mapping.  Inventiones Mathematicae, 22(3):211–319, 1973.
  * [Shu95] M.A. Shubin.  ${L^{2}}$ Riemann Roch theorem for elliptic operators.  Geometric and functional analysis, 5(2):482–527, 1995.
  * [SS22] C. Sabbah and C. Schnell.  The MHM project (Version 2).  http://www.math.polytechnique.fr/cmat/sabbah/MHMProject/mhm.html, 2022\.
  * [Voi02] C. Voisin.  Théorie de Hodge et géométrie algébrique complexe.  Société mathématique de France Paris, 2002.
  * [Zuc76] S. Zucker.  Generalized intermediate jacobians and the theorem on normal functions.  Inventiones Mathematicae, 33:185–222, 1976.
  * [Zuc79] S. Zucker.  Hodge theory with degenerating coefficients: ${L}_{2}$ cohomology in the Poincare metric.  Annals of Mathematics, 109(3):415–476, 1979.

Bastien, Jean

Institut Fourier, UMR 5582, Laboratoire de Mathématiques

Université Grenoble Alpes, CS 40700, 38058 Grenoble cedex 9, France

E-mail<EMAIL_ADDRESS>