# CANDID: Correspondence AligNment for Deep-burst Image Denoising

Arijit Mallick1§ Department of Computer Graphics
University of Tübingen, Germany
<EMAIL_ADDRESS>Raphael Braun  Department of Computer Graphics
University of Tübingen, Germany
<EMAIL_ADDRESS>Hendrik PA Lensch  Department of Computer
Graphics
University of Tübingen, Germany
<EMAIL_ADDRESS>
###### Abstract

With the advent of mobile phone photography and point-and-shoot cameras, deep-
burst imaging is widely used for a number of photographic effects such as
depth of field, super-resolution, motion deblurring, and image denoising. In
this work, we propose to solve the problem of deep-burst image denoising by
including an optical flow-based correspondence estimation module which aligns
all the input burst images with respect to a reference frame. In order to deal
with varying noise levels the individual burst images are pre-filtered with
different settings. Exploiting the established correspondences one network
block predicts a pixel-wise spatially-varying filter kernel to smooth each
image in the original and prefiltered bursts before fusing all images to
generate the final denoised output. The resulting pipeline achieves state-of-
the-art results by combining all available information provided by the burst.

###### Keywords:

Burst photography; Denoising; Image alignment;

§§footnotetext: This work was done while Mr. Arijit Mallick was affiliated
with University of Tübingen, Germany. He is currently working for TWT GmbH
Science & Innovation, Germany.

##  I Introduction

Due to recent development of faster and lightweight portable CPUs especially
in mobile and point-and-shoot cameras, burst photography has been gaining
further prominence because of the noise reduction and motion blur removal
capabilities. Burst photography [1, 2] can also be understood as multi-frame
image restoration task [3, 4, 5, 6] which has a wider range of applications
even in satellite photography [7, 8] for remote sensing. The sensors and
lenses in smartphones are much smaller and more lightweight than those of
professional cameras but they collect less light per pixel which leads to
noisier images. Compensating this by longer exposures could introduce motion
blur. As an alternative, a burst of many short and noisy images could be
computationally combined into one sharp image. Camera phone APIs already
provide denoising algorithms but they are not optimized for burst denoising.

Current burst denoising methods perform the alignment simply based on
homographies estimated between the burst images [9, 10], followed by some
pixel denoising. Our proposed pixel-wise alignment based on optical flow is
significantly more powerful in compensating for scenes with complex depth and
camera or object motion. We still expect the motion between the burst images
to be rather small. Our overall architecture is depicted in Figure 1. First,
we generate enhanced burst inputs by applying the pre-trained self-guided
filtering network (SGN) [11] for each image to generate pre-denoised bursts.
Both, the original and the denoised images are aligned with respect to the
reference frame using the RAFT [12] optical flow network. Based on the aligned
images and extracted features a network block predicts a per-pixel adaptive
filter kernel to denoise every pixel in every image. A final fusion block
merges all predictions across all bursts into a single output.

Secondly, we estimate pixel adaptive filter-kernels which per pixel describe
where to collect color information from the aligned input bursts. The decoder
then only applies those kernels, thus produces weighted averages over
neighbouring pixels from all aligned images.

We demonstrate the importance of each module in an ablation study. Our
contributions are as follows: a) optical flow-based alignment of multiple pre-
denoised burst images, b) adaptive per-pixel filtering of aligned burst images
followed by cross-burst fusion, c) improved denoising performance, especially
in low-noise scenarios.

BBBBBBImage
BurstSGNB10B10B10B10B10B10$\sigma=10$B30B30B30B30B30B30$\sigma=30$FeaturesFeaturesFeaturesAlignAlignAlignFilterFilterFilterFuse$I_{\text{pred}}$Result
Figure 1: Method overview. The input image burst is pre-filtered twice using
SGN [11] with different filter strengths. For each stack we extract features,
align both features and images and then apply a content-adaptive spatial
filter with weights derived from the aligned features. The results from all
three bursts are fused to predict the denoised output.

##  II Related Work

Related work covers single image denoising, homography-based alignment, and
deep-burst imaging. In the following section, we discuss related works
pertaining to our problem statement, starting with single image denoising,
followed by homography-based and optical flow-based alignment, and finally
contemporary progress on deep-burst imaging.

#### Single image denoising

Most photography hardware companies take advantage of the recently developed
lightweight neural network denoising models; exploiting the significant
increase in mobile computation power. In the early days of CNNs, models such
as [11] improved performance compared to classical image denoising models
based on Markov random fields but they could not compete with BM3D [13] which
introduced a new denoising paradigm by combining 3D block matching and domain
transform. They are later surpassed by a sparse denoising autoencoder models
[14, 15]. Simple multi-layer perceptron-based models [16], residual link
networks [17] and later deeper residual networks [18] and persistent memory-
based networks [19] have shown superior performance due to enhanced receptive
fields. All these models have the advantage of being trainable end-to-end
exploiting simple to generate training data. For a multitude of image
processing tasks, training can be accelerated using pre-trained models and
transfer learning [20]. In this spirit, we incorporate the pre-trained self-
guided network (SGN) [11] to enrich the burst input with smooth priors. SGN
extracts large-scale contextual information and gradually propagates it to the
higher resolution sub-networks for feature self-guidance and denoising at
multiple scales. This efficient multiscale local features extraction property
allows it to efficiently recover denoised images.

#### Deep-burst Denoising

While single image denoising relies on learned image priors, deep-burst
denoising assimilates features from multiple noisy frames to predict a better
image. A similar idea is used in burst motion deblurring [21] where a sharp
image is recurrently extracted from a burst of blurry ones. Similarly,
recurrent neural networks have also been used for burst denoising. Bhat et al.
[10] reparametrize the image formation process in the latent space, and
integrate learned image priors for the denoised prediction. Kernel prediction
networks [22, 23] leverage the localized pixel neighborhood weighted filters
to predict a denoised image from multiple inputs. Dudhane et al.[24] proposed
to extract pre-processed features from each burst frame following an edge-
boosting burst alignment module. The pseudo-burst features are then enriched
using multi-scale contextual information, which is followed by adaptively
aggregate information from the corresponding features.

Our novel burst denoising model also applies adaptive pixel-neighborhood
filters but first performs an explicit alignment step.

#### Correspondence Alignment

Multiple frame denoising usually involves some sort of alignment [5] of the
frames in the burst for superior feature assimilation. Tico [25] demonstrates
a block matching approach within the reference and the neighboring frames to
support multiple frame denoising. VBM4D [26] and VBM3D [27] take the BM3D
algorithm further to video denoising with faster homography flow-based
alignment. We instead estimate per pixel correspondences for a more fine-
grained alignment. When capturing a burst of images of a potentially dynamic
scene with a handheld camera each image will show slightly different content.
In order to effectively utilize information from those multiple frames for
denoising, the frames need to be aligned [5].

Tico [25] demonstrates a block matching approach within the reference and the
neighboring frames to support multiple frame denoising. VBM4D [26] and VBM3D
[27] take the BM3D algorithm further to video denoising with block matching
for alignment.

Neural network optical flow models can leverage information beyond patch-level
correspondence information to predict dense correspondences, i.e. estimating
pixel motion between consecutive frames of a video [28]. Some of the first
learning based optical flow methods used simple CNN architectures [29, 30,
31]. Recently they were superseded by recurrent techniques like RAFT [12] or
transformer-based architectures like FlowFormer [32]. Those current state of
the art techniques are very good and very close to ground truth [33].

In our approach we utilize the success in the optical flow field by using a
pretrained RAFT implementation provided in torchvision [34]. RAFT provides the
high-quality pixel-wise correspondence alignment that we rely on for our
denoising approach.

##  III Method

The core idea of our burst denoising method is to first spatially align the
pixels in the burst stack. Afterwards, each aligned image is denoised by a
content-adaptive spatially-varying filter step followed by an adaptive fusion
of all processed images (see Figure 1, left).

###  III-A Prefiltering with SGN

Our method starts by filtering the burst images. The amount of noise in input
bursts can vary significantly, even within the same burst. Because of varying
degrees of noise and blur due to abrupt camera motion, precise alignment might
be difficult. We, therefore, duplicate the input burst into three processing
streams. The first stream B uses the original burst, the second stream B10
($\sigma=10$) a mildly pre-denoised version of the burst and the last one B30
($\sigma=30$) a strongly denoised version (see Figure 2(d)). We apply the
pretrained SGN [11] to each individual frame but any single-frame denoising
algorithm could be used. The intermediate results from the different streams
will be fused in the last step of our pipeline.

(a) Input

(b) Ground Truth

(c) SGN $\sigma=10$

(d) SGN $\sigma=30$ Figure 2: Prefiltering with SGN

Denoising performance of the pre-trained SGN. ($\sigma=10$) retains the
original sharpness. ($\sigma=30$) shows better denoising performance but loses
sharp details.

(a) Reference

(b) before alignment

(c) after alignment Figure 3: Pixel-wise alignment

Alignment error between the reference and the last burst image scaled 5 times.
Note, how after alignment differences in the silhouette are no longer present.

###  III-B Feature Extraction

To add local context to each pixel we enrich each image by processing it with
a simple CNN. In addition, the estimated noise level of the image is
concatenated as the fourth channel before processing. In each processing
stream, we produce corresponding feature stacks. The same shared weights are
used for each image in each stream. Both the image stack and the feature stack
are used as inputs for the alignment module.

###  III-C Alignment

The central property that is exploited with burst denoising is that the
content captured in the individual frames of the burst is very similar. In the
original images, the scene content however might be shifting due to camera
shake or scene dynamics. We use the pre-trained RAFT [12] model that is
shipped with torchvision [34] to estimate the optical flow between the
reference image frame and any other image frame in the burst. The estimated
flow computed from the reference and secondary images is used to warp the
secondary image frames and their corresponding feature maps with respect to
the reference image frame and the reference feature frame respectively. The
effectiveness of the RAFT-based alignment is visualized in Figure 3(c).

###  III-D Collaborative Content-adaptive Spatial Filtering

At this point, the images and features in the bursts are all aligned with
respect to the reference frame. The next step is to filter the images
spatially and combine the results pixel-wise for the final result. The spatial
filtering is implemented with content-dependent per-pixel kernels. Those
kernels are estimated by a CNN from the aligned feature stack, i.e.
collaboratively considering all feature maps at the same time. The output
activations of this CNN are reshaped into $3\times 3$ and $5\times 5$ filter
kernels for all images and all pixels. The result is two kernels of shape
$[N,H,W,3,3]$ and $[N,H,W,5,5]$ with the number of images $N$, height $H$ and
width $W$. The kernels are normalized via softmax and applied to each image in
the burst individually, effectively computing a weighted average color over
the $3\times 3$ and $5\times 5$ neighborhood of each pixel as shown in Figure
4.

aligned imagesConvConvConvConvConvConvsoftmaxfilter kernel for every
pixelConvConvConvConvConvConvaligned features Figure 4: The spatial content-
adaptive filter kernels for every pixel are estimated by a CNN based on all
aligned features. They are applied individually to the aligned images to
produce the spatially filtered burst.

###  III-E Burst Fusion

Remember in Section III-A the burst was split into three processing streams B,
B10 and B30, which are all processed individually in the same way so far. This
means at this stage we have aligned and spatially filtered images and the
corresponding aligned image features for each stream. The final step is to
fuse all information from the different bursts into a single denoised image
$I_{\text{pred}}$. This denoised result is computed as a weighted average over
the spatially filtered images from all three processing streams. As indicated
in Figure 5, we concatenate the aligned features of the streams with the
spatially filtered images and process them together in a 4-layer CNN. This CNN
produces the weight volume. This volume contains a weight for every pixel and
color of every image. A softmax over the image dimension is applied to the
weights in order to ensure that summing up the weights over this dimension
yields 1 for every color channel. The result $I_{\text{pred}}$ is finally
computed as a weighted sum per pixel. This is implemented as element-wise
multiplication between weight volume and spatially filtered images followed by
a sum over the burst dimension. Every channel for every input image is
therefore weighted individually, which is more powerful than just mixing the
existing colors of the spatially filtered images.

BB10B30aligned featuresspatially filtered imagesweight
volumesoftmax$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$\cdot$$+$$I_{\text{pred}}$
Figure 5: Fusion Network. The aligned features from all three bursts are
concatenated with the spatially filtered images and processed by a CNN to
obtain a weight volume. The weights are used to compute the denoised result as
a weighted per-pixel sum over the spatially filtered images.

###  III-F Training

Some components of the denoising pipeline like the SGN and the alignment
module are pretrained. We stopped the gradients from going through the SGN
networks, which effectively turns the three burst streams B, B10 and B30 into
separate inputs. The RAFT network in the alignment module was frozen and used
as fixed differentiable operation. The remaining trainable weights are in the
CNNs for the feature extractor, the content-adaptive cooperative spatial
filter, and the burst fusion module. We train end-to-end with ADAM [35] from a
simple $L1$-loss
$\mathcal{L}=\left\|I_{\text{pred}}-I_{\text{gt}}\right\|_{1}$ on the ground
truth $I_{gt}$.

##  IV Experiments

We evaluate our method by comparing to state of the art and validate our
architecture choices with an ablation study.

###  IV-A Training and experimental setup

For the pre-denoising we use the SGN pre-trained with $\sigma=10,30$ [36]. For
the burst denoising training both the SGN pre-denoising and the RAFT alignment
model are frozen. We trained on the OpenImages [37] dataset and evaluated on
the grayscale burst benchmark [22] and RGB burst benchmark following the usual
conventions [23, 10, 24]. The ground truth images are shifted and corrupted by
adding heteroscedastic Gaussian noise [38] with variance
$\sigma^{2}_{r}+\sigma^{2}_{s}x$. Here $x$ is the clean pixel value, while
$\sigma_{r}$ and $\sigma_{s}$ denote the readout and shot noise parameters,
respectively. Those noise parameters are assumed to be known both during
training and testing, and are used in the feature extractor. During training
they are sampled uniformly in the log-domain from the range
$\log(\sigma_{r})\in[-3,-1.5]$ and $\log(\sigma_{s})\in[-4,-2]$. The
comparisons are evaluated with 2 different noise lvl.1 and lvl.2,
corresponding to noise parameters (-2.2, -2.6) and (-1.8, -2.2) respectively.
Training was done on 2 TITAN Xp GPUs and took about 96 hours to converge.

###  IV-B Results

The quantitative comparison with other methods in Table I shows that our model
delivers overall state-of-the-art performance the aforementioned benchmark in
the evaluation of the model-unseen dataset. On deep introspection, we can say
that due to SGN and further multiple kernel-based filtering, the model
successfully recovers the image even from the heavy noise scenarios. In the
future, one could add additional SGN-based denoising stages with different
pre-trained noises to analyze whether further boosting of lvl.1 and lvl.2
would be possible. Additionally, larger filter kernels can be added to the
model in order to enhance the results for higher noise scenarios.

Exemplar qualitative results on individual images are shown in Figure 6

Color Grayscale model lvl. 1 lvl. 2 lvl. 1 lvl. 2 KPN [22] 38.38 35.96 36.47
33.93 BPN [39] 40.16 37.08 38.18 35.42 MFIR-1 [10] 40.16 37.08 39.37 36.51
MFIR-2 [10] 42.21 39.13 39.37 36.51 BIPNET [24] 42.28 40.20 41.26 38.74 Ours
42.49 39.18 41.35 36.61

TABLE I: PSNR values of the evaluation grayscale burst dataset. Blue shows
best results while green shows the second best results. lvl. indicates the
level of Gaussian noise added according to evaluation convention.

###  IV-C Ablation Study

Model | _lvl.1 gray_
---|---
without SGN pre-denoising | 40.81
without RAFT alignment | 38.54
without content-adaptive filtering | 41.27
Ours – complete pipeline | 41.35
TABLE II: Ablation study. Removing the individual parts of the pipeline and
training the model from scratch reveals the importance of each component. Only
by combining SGN-based pre-denosing, flow-based alignment and content-adpative
filtering good performance in hight noise levels can be achieved.

Since our module consists of several pretrained blocks and trainable sub-
modules, we analyse the effectiveness of each of the components with the
corresponding ablation in Table II. Here, we removed individual parts of the
pipeline and trained the network from scratch. Removing all SGN blocks
effectively suppresses pre-denoising of the input. Although the low-noise
evaluation performs comparatively well, image quality deteriorates as the
noise increases due to the lack of cleaner proposals at the initial stages.
Without the alignment module, the final fusion step is impared and we see
lower performance on all noise levels. Particularly lvl.1 is impacted.
Finally, the cooperative content-adaptive filtering adds almost equally to the
reconstruction quality of all noise levels levels.

|  |  |  |
---|---|---|---|---
|  |  |  |
|  |  |  |
|  |  |  |
|  |  |  |
|  |  |  |
Noisy input | Region of interest | Input error | BIPNET error | Our error
Figure 6: Qualitative performance of our algorithm with respect to BIPNET [24]
on the evaluation color dataset. Notice the excessive smoothing by BIPNET
which removes the sharper features in comparison to the ground truth. Our
network retains the details and removes the noise as well. Darker region
corresponds to lesser error which indicates better denoising. It is to be
noted that the error maps have been scaled 5 times for better visual
understanding.

###  IV-D Qualitative Results

In Figure 6 we demonstrate the improved performance of our pipeline on a
number of example images. Even for drastically different amount of noise our
approach outperforms BIPNET [24] on every image. The denoised image is
significantly closer to the ground truth result, as is evident from the error
maps.

|  |  |  |
---|---|---|---|---
Noisy input | Region of interest | Input error | BIPNET error | Our error
Figure 7: Worst performing images in the test set. The excessive noise in the
input combined with strong camera motion deteriorates the denoising
performance. It is to be noted that the error maps have been scaled 2 times
for better visual understanding.

A failure case is shown in Figure 7. In this example, apart from the high
motion, the image consists of sharp features which is preserved by our network
which is not detected as noise.

##  V Conclusions

We propose a deep-burst denoising model based on optical flow guided alignment
and cooperative filtering. A well-established single image denoising module
generates pre-denoised burst input images for two different assumed noise
levels. Alignment to the reference frame is performed using a state of the art
optical flow network. Providing the original input burst and the pre-denoised
stacks ensures the good performance of the optical flow alignment. Based on
the aligned features and images a set of content-adaptive spatially-varying
filter kernel is predicted to smooth each input image individually. A fusion
block finally combines all intermediate results to the final denoised output.
In the future, one can also compare the effect of state of the art optical
flow based correspondence alignment on the quality of the burst image
denosing.

Our approach yields state-of-the-art results across low noise levels on the
standard benchmark data sets. Higher noise scenarios working on different pre-
denoised images shows a comparable benefit.

## Acknowledgement

This work has been partially funded by the Deutsche Forschungsgemeinschaft
(DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC
number 2064/1 - project number 390727645 and SFB 1233 - project number
276693517. It was supported by the German Federal Ministry of Education and
Research (BMBF): Tübingen AI Center, FKZ: 01IS18039A and Cyber Valley. The
authors thank the International Max Planck Research School for Intelligent
Systems (IMPRS-IS) for supporting Arijit Mallick.

## References

  * Molini et al. [2020] A. B. Molini, D. Valsesia, G. Fracastoro, and E. Magli, “DeepSUM: Deep neural network for super-resolution of unregistered multitemporal images,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 58, no. 5, pp. 3644–3656, may 2020. [Online]. Available: https://doi.org/10.1109%2Ftgrs.2019.2959248
  * Kawulok et al. [2020] M. Kawulok, P. Benecki, S. Piechaczek, K. Hrynczenko, D. Kostrzewa, and J. Nalepa, “Deep learning for multiple-image super-resolution,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 17, no. 6, pp. 1062–1066, jun 2020. [Online]. Available: https://doi.org/10.1109%2Flgrs.2019.2940483
  * Liba et al. [2019] O. Liba, K. Murthy, Y.-T. Tsai, T. Brooks, T. Xue, N. Karnad, Q. He, J. T. Barron, D. Sharlet, R. Geiss, S. W. Hasinoff, Y. Pritch, and M. Levoy, “Handheld mobile photography in very low light,” _ACM Transactions on Graphics_ , vol. 38, no. 6, pp. 1–16, dec 2019. [Online]. Available: https://doi.org/10.1145%2F3355089.3356508
  * Hasinoff et al. [2016] S. W. Hasinoff, D. Sharlet, R. Geiss, A. Adams, J. T. Barron, F. Kainz, J. Chen, and M. Levoy, “Burst photography for high dynamic range and low-light imaging on mobile cameras,” _ACM Trans. Graph._ , vol. 35, no. 6, nov 2016. [Online]. Available: https://doi.org/10.1145/2980179.2980254
  * Bhat et al. [2021a] G. Bhat, M. Danelljan, L. Van Gool, and R. Timofte, “Deep burst super-resolution,” 2021. [Online]. Available: https://arxiv.org/abs/2101.10997
  * Wronski et al. [2019] B. Wronski, I. Garcia-Dorado, M. Ernst, D. Kelly, M. Krainin, C.-K. Liang, M. Levoy, and P. Milanfar, “Handheld multi-frame super-resolution,” _ACM Trans. Graph._ , vol. 38, no. 4, jul 2019. [Online]. Available: https://doi.org/10.1145/3306346.3323024
  * Valsesia and Magli [2022] D. Valsesia and E. Magli, “Permutation invariance and uncertainty in multitemporal image super-resolution,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 60, pp. 1–12, 2022.
  * Deudon et al. [2020] M. Deudon, A. Kalaitzis, I. Goytom, M. R. Arefin, Z. Lin, K. Sankaran, V. Michalski, S. E. Kahou, J. Cornebise, and Y. Bengio, “Highres-net: Recursive fusion for multi-frame super-resolution of satellite imagery,” 2020\. [Online]. Available: https://arxiv.org/abs/2002.06460
  * Godard et al. [2018] C. Godard, K. Matzen, and M. Uyttendaele, “Deep burst denoising,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , September 2018.
  * Bhat et al. [2021b] G. Bhat, M. Danelljan, F. Yu, L. V. Gool, and R. Timofte, “Deep reparametrization of multi-frame super-resolution and denoising,” 2021.
  * Gu et al. [2019] S. Gu, Y. Li, L. V. Gool, and R. Timofte, “Self-guided network for fast image denoising,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 2511–2520.
  * Teed and Deng [2020] Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” in _European conference on computer vision_. Springer, 2020, pp. 402–419.
  * Dabov et al. [2007a] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3d transform-domain collaborative filtering,” 2007.
  * Xie et al. [2012] J. Xie, L. Xu, and E. Chen, “Image denoising and inpainting with deep neural networks,” in _Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1_ , ser. NIPS’12. Red Hook, NY, USA: Curran Associates Inc., 2012, p. 341–349.
  * Aharon et al. [2006] M. Aharon, M. Elad, and A. Bruckstein, “K-svd: An algorithm for designing overcomplete dictionaries for sparse representation,” _IEEE Transactions on Signal Processing_ , vol. 54, no. 11, pp. 4311–4322, 2006.
  * Schmidt and Roth [2014] U. Schmidt and S. Roth, “Shrinkage fields for effective image restoration,” in _2014 IEEE Conference on Computer Vision and Pattern Recognition_ , 2014, pp. 2774–2781.
  * Zhang et al. [2017] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising,” _IEEE Transactions on Image Processing_ , vol. 26, no. 7, pp. 3142–3155, jul 2017. [Online]. Available: https://doi.org/10.1109%2Ftip.2017.2662206
  * Mao et al. [2016] X.-J. Mao, C. Shen, and Y.-B. Yang, “Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections,” 2016\. [Online]. Available: https://arxiv.org/abs/1603.09056
  * Tai et al. [2017] Y. Tai, J. Yang, X. Liu, and C. Xu, “Memnet: A persistent memory network for image restoration,” 2017. [Online]. Available: https://arxiv.org/abs/1708.02209
  * Chen et al. [2020] H. Chen, Y. Wang, T. Guo, C. Xu, Y. Deng, Z. Liu, S. Ma, C. Xu, C. Xu, and W. Gao, “Pre-trained image processing transformer,” 2020. [Online]. Available: https://arxiv.org/abs/2012.00364
  * Wieschollek et al. [2017] P. Wieschollek, M. Hirsch, B. Scholkopf, and H. P. A. Lensch, “Learning blind motion deblurring,” in _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , Oct 2017.
  * Mildenhall et al. [2018] B. Mildenhall, J. T. Barron, J. Chen, D. Sharlet, R. Ng, and R. Carroll, “Burst denoising with kernel prediction networks,” in _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2018.
  * Cho et al. [2021] W. Cho, S. Son, and D.-S. Kim, “Weighted multi-kernel prediction network for burst image super-resolution,” in _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , 2021, pp. 404–413.
  * Dudhane et al. [2022] A. Dudhane, S. W. Zamir, S. Khan, F. S. Khan, and M.-H. Yang, “Burst image restoration and enhancement,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2022, pp. 5749–5758.
  * Tico [2008] M. Tico, “Multi-frame image denoising and stabilization,” in _2008 16th European Signal Processing Conference_ , 2008, pp. 1–4.
  * Maggioni et al. [2012] M. Maggioni, G. Boracchi, A. Foi, and K. Egiazarian, “Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotemporal transforms,” _IEEE Transactions on Image Processing_ , vol. 21, no. 9, pp. 3952–3966, 2012.
  * Dabov et al. [2007b] K. Dabov, A. Foi, and K. Egiazarian, “Video denoising by sparse 3d transform-domain collaborative filtering,” in _2007 15th European Signal Processing Conference_ , 2007, pp. 145–149.
  * Bruhn et al. [2005] A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/kanade meets horn/schunck: Combining local and global optic flow methods,” _International journal of computer vision_ , vol. 61, no. 3, pp. 211–231, 2005.
  * Buades et al. [2016] A. Buades, J.-L. Lisani, and M. Miladinović, “Patch-based video denoising with optical flow estimation,” _IEEE Transactions on Image Processing_ , vol. 25, no. 6, pp. 2573–2586, 2016.
  * Dosovitskiy et al. [2015] A. Dosovitskiy, P. Fischer, E. Ilg, P. Hausser, C. Hazirbas, V. Golkov, P. Van Der Smagt, D. Cremers, and T. Brox, “Flownet: Learning optical flow with convolutional networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 2758–2766.
  * Ilg et al. [2017] E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox, “Flownet 2.0: Evolution of optical flow estimation with deep networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2462–2470.
  * Huang et al. [2022] Z. Huang, X. Shi, C. Zhang, Q. Wang, K. C. Cheung, H. Qin, J. Dai, and H. Li, “Flowformer: A transformer architecture for optical flow,” _arXiv preprint arXiv:2203.16194_ , 2022.
  * Butler et al. [2012] D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in _European Conf. on Computer Vision (ECCV)_ , ser. Part IV, LNCS 7577, A. Fitzgibbon et al. (Eds.), Ed. Springer-Verlag, Oct. 2012, pp. 611–625.
  * Paszke et al. [2019] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems 32_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds. Curran Associates, Inc., 2019, pp. 8024–8035. [Online]. Available: http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
  * Kingma and Ba [2014] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * ZHAO [2019] Y. ZHAO, “PyTorch implementation of Self Guided Network (ICCV),” 2019. [Online]. Available: https://github.com/zhaoyuzhi/Self-Guided-Network-for-Fast-Image-Denoising
  * Krasin et al. [2017] I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu-El-Haija, A. Kuznetsova, H. Rom, J. Uijlings, S. Popov, S. Kamali, M. Malloci, J. Pont-Tuset, A. Veit, S. Belongie, V. Gomes, A. Gupta, C. Sun, G. Chechik, D. Cai, Z. Feng, D. Narayanan, and K. Murphy, “Openimages: A public dataset for large-scale multi-label and multi-class image classification.” _Dataset available from https://storage.googleapis.com/openimages/web/index.html_ , 2017.
  * Healey and Kondepudy [1994] G. Healey and R. Kondepudy, “Radiometric ccd camera calibration and noise estimation,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 16, no. 3, pp. 267–276, 1994.
  * Xia et al. [2020] Z. Xia, F. Perazzi, M. Gharbi, K. Sunkavalli, and A. Chakrabarti, “Basis prediction networks for effective burst denoising with large kernels,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 11 841–11 850.


[a,b]Sebastian Hornung

# Understanding the background in dark matter searches by studying
antinucleosynthesis in the laboratory with ALICE

on behalf of the ALICE collaboration

###### Abstract

Antinuclei are considered to be one of the most promising probes in the
indirect search for dark matter (DM) annihilation in space. However, in light
of recent results on the production of light antinuclei in pp collisions at
the LHC, an abundant production of light (anti)nuclei is also expected from
Standard Model (SM) collisions of primary cosmic rays with the interstellar
medium. Hence further precise measurements are required to constrain the
production models of antinuclei in SM collisions to be sensitive to additional
DM annihilation events. The most recent results of the ALICE collaboration on
the production of antideuterons ($\overline{\mathrm{d}}$) and antihelium-3
(${}^{3}\overline{\text{He}}$) in proton-proton (pp) and proton-lead (p–Pb)
collisions are shown. These results provide valuable input for state-of-the-
art calculations of the production models currently used to estimate the SM
background to DM searches.

##  1 Antinuclei flux in space

Secondary antinuclei are expected to be produced via the decay or annihilation
of dark matter particles in space [1]. After the propagation through space
towards the earth, these particles could be observed as an excess of the light
antinuclei flux above the expectations from interactions of primary cosmic
rays (CR) and the interstellar medium (ISM) at low kinetic energies by
satellite-born or balloon experiments. As indicated in Figure 1, which shows
the predicted flux of ${}^{3}\overline{\text{He}}$ for different dark matter
models and decays as well as different background calculations, the exact
energy and strength of the excess related to dark matter depends on its
characteristics [1]. Thus, it is important to be able to precisely predict the
expected background from reactions of primary cosmic rays and the interstellar
medium, which both consist mostly of protons and ${}^{4}\text{He}$ [2, 3].

Figure 1: ${}^{3}\overline{\text{He}}$ flux predictions from different DM
models and astrophysical background calculations [4, 5, 6, 7, 8, 9, 2]. The
error bands illustrate uncertainties in the coalescence momentum (see text for
details), but also include propagation uncertainties. This figure is taken
from [1].

Unfortunately, the production of light (anti)nuclei in hadronic interactions
is not well understood and cannot be calculated from first principles. Thus,
two classes of phenomenological models are employed to describe this process,
the statistical hadronization and the coalescence models. In the Statistical
Hadronization Model (SHM) [10], the particles are produced from a medium in
statistical equilibrium and their relative abundances are fixed at the
chemical freeze-out, when the rate of inelastic scatterings between the
constituents of the medium produced in the collision becomes negligible. Even
though this model is very successful in describing the measured yield of light
(anti)nuclei and even (anti)hypertriton in lead-lead collisions at the LHC
using the grand canonical ensemble formulation [10], the description for
smaller system, like in pp and p–Pb collisions, requires the exact
conservation of charges, i.e. baryon number, strangeness number and electric
charge. Alternatively, the coalescence model is based on the assumption that
(anti)nuclei are produced from (anti)nucleons if they are close enough in
phase-space [11, 12]. The key parameter of these models is the coalescence
parameter $B_{\mathrm{A}}$, which is defined by

$\displaystyle
E_{\mathrm{A}}\frac{\mathrm{d}^{3}N_{\mathrm{A}}}{\mathrm{d}p_{\mathrm{A}}^{3}}=B_{\mathrm{A}}\left(E_{\mathrm{n}}\frac{\mathrm{d}^{3}N_{\mathrm{n}}}{\mathrm{d}p_{\mathrm{n}}^{3}}\right)^{A-Z}\left(E_{\mathrm{p}}\frac{\mathrm{d}^{3}N_{\mathrm{p}}}{\mathrm{d}p_{\mathrm{p}}^{3}}\right)^{Z}=B_{\mathrm{A}}{\left(E_{\mathrm{p}}\frac{\mathrm{d}^{3}N_{\mathrm{p}}}{\mathrm{d}p_{\mathrm{p}}^{3}}\right)^{A}}\Bigg{|}_{\vec{p}_{\mathrm{p}}=\vec{p}_{\mathrm{A}}/{A}}.$
(1)

It relates the invariant yield of nuclei with mass number $A$,
$E_{A}(\mathrm{d}^{3}N_{A}/\mathrm{d}p_{A}^{3})$, to the one of its
constituents. At midrapidity and LHC energies, i.e. center-of-mass energies
per nucleon-nucleon pair ($\sqrt{s_{\mathrm{NN}}}$) in the order of a few TeV,
the invariant yield of protons,
$E_{\mathrm{p}}(\mathrm{d}^{3}N_{\mathrm{p}}/\mathrm{d}p_{\mathrm{p}}^{3})$,
is expected to be identical to the one of neutrons, and it is therefore used
to describe both protons and neutrons. The momentum of the proton,
$\vec{p}_{\mathrm{p}}$, is given by the one of the nuclei, $\vec{p}_{A}$,
divided by their mass number. The coalescence parameter can also be related to
the production probability of the nucleus and can be calculated from the
overlap of the wave function of the nucleus with the phase space distribution
of its constituent nucleons via the Wigner formalism [13]. The estimate of the
background from the interactions of CRs and the ISM is usually based on the
coalescence approach employing either the analytical coalescence model [5, 9,
2] or a model based on the scaling of $B_{A}$ with the radius of the emission
source [8, 13]. In the former, all (anti)nucleons with momenta within a sphere
with a radius $p_{0}$, the coalescence momentum, form a bound state. In more
recent implementations of this model, the (anti)nucleons are produced
employing an hadronic event generator to take into account their possible
correlations [5, 2] and the coalescence process is taken into account by an
afterburner. The coalescence momentum cannot be calculated within these models
and has to be extracted from measurements. In [2] also the observed kinetic
energy dependence of $p_{0}$ has been taken into account. Both model
descriptions strongly rely on measurements of the production yields of light
antinuclei in pp and pA collisions, as for example performed by the ALICE
collaboration. The estimated antinuclei flux strongly depends on the precise
knowledge of the production rate together with a good understanding of the
propagation processes of the antinuclei through space. In this proceeding, we
focus on the application of the measurements of antideuterons and
${}^{3}\overline{\text{He}}$ in pp and p–Pb collisions at different collision
energies provided by the ALICE collaboration in the context of the predictions
of the expected top-of-the-atmosphere of antinuclei.

##  2 Production of light (anti)nuclei in ion collisions

The current estimates for top-of-the-atmosphere flux of secondary antinuclei
from the interaction of primary cosmic rays and the interstellar medium are
based on measurements of the production of light antinuclei in pA and pp
collisions including the published results from ALICE [14, 15]. Recently, the
ALICE collaboration has published new results for the transverse-momentum
spectra of (anti)deuterons in pp collisions at $\sqrt{s}=13$ TeV [16] and
(anti-)${}^{3}\text{He}$ in p–Pb collisions at $\sqrt{s_{\mathrm{NN}}}=5.02$
[17]. In addition, the preliminary results for (anti-)${}^{3}\text{He}$ in pp
collisions at $\sqrt{s}=13$ TeV and (anti)deuterons in p–Pb collisions at
$\sqrt{s_{\mathrm{NN}}}=8.16$ TeV are summarized in Figure 2.

Figure 2: $p_{\mathrm{T}}$ spectra of (anti-)${}^{3}\text{He}$ measured in pp
collisions at $\sqrt{s}=13$ TeV (left) and (anti)deuterons measured in p–Pb
collisions at $\sqrt{s_{\mathrm{NN}}}=8.16$ TeV (right) in different
multiplicity classes.

These preliminary measurements have not yet been used for the current
predictions in [2], but will help to reduce the differences between the
predictions and their respective uncertainty bands. These measurements can be
exploited to directly compare the corresponding predictions from the models
used to estimate the background in DM searches, as done in [2]. Moreover, the
corresponding coalescence parameter $B_{\mathrm{A}}$ can be either used to
extract the coalescence momentum $p_{0}$ or to compare with the models based
on the scaling with the size of the emission region [8].

##  3 Coalescence parameter

Figure 3: $B_{2}$ and $B_{3}$ measured as a function of the mean charged-
particle multiplicity density at midrapidity are compared to the coalescence
model expectations from [13] at a fixed value of $p_{\mathrm{T}}/A=0.75$
GeV/$c$ and $p_{\mathrm{T}}/A~{}=~{}0.73$ GeV/$c$, respectively. Two different
parameterizations for the radius of the emission region as a function of
$\langle\text{d}N_{\text{ch}}/\text{d}\eta_{\mathrm{lab}}\rangle_{|\eta_{\mathrm{lab}}|<0.5}$
are used. In the right panel, the expectations for the grand canonical (GC)
[10] and the canonical statistical hadronization model (CSM) [18] are also
indicated.

The evolution of $B_{A}$ with the size of the emission region can be studied
by measuring $B_{A}$ as a function of $p_{\mathrm{T}}/A$ for several collision
systems and energies [19, 20, 15, 17]. For a fixed value of
$p_{\mathrm{T}}/A=0.75$ GeV/$c$ and $p_{\mathrm{T}}/A=0.73$ GeV/$c$, $B_{2}$
and $B_{3}$ are shown as a function of the mean charged-particle multiplicity
density at midrapidity,
$\langle\text{d}N_{\text{ch}}/\text{d}\eta_{\mathrm{lab}}\rangle_{|\eta_{\mathrm{lab}}|<0.5}$,
in Figure 3. The mean charge-particle multiplicity density is proportional to
the volume of the emission region of the (anti)nuclei,
$R\propto\left(\langle\mathrm{d}N_{\mathrm{ch}}/d\eta_{\mathrm{lab}}\rangle_{|\eta_{\mathrm{lab}}|<0.5}\right)^{1/3}$.
Thus, these measurements directly prove the scaling of the coalescence
parameter with the size of the emission region as employed by one of the
coalescence models used to estimate the flux of secondary antinuclei at the
top of the atmosphere [8]. The measurements show a smooth transition from low
charged-particle multiplicity densities, which refer to a small emission
region, to large ones. This indicates that the production mechanism in small
collision systems evolves continuously to the one in large systems and also
that a single mechanism sensitive to the size of the emission region could be
possible. The experimental results are compared to calculations obtained from
the coalescence approach [13] for two different parameterizations for the size
of the emission region as a function of
$\langle\text{d}N_{\text{ch}}/\text{d}\eta_{\mathrm{lab}}\rangle_{|\eta_{\mathrm{lab}}|<0.5}$.
The theoretical calculations agree with the trend observed in data.

##  4 Summary and outlook

The ALICE collaboration has measured the production of light (anti)nuclei in
different collision systems and at different energies providing crucial input
information for the coalescence models used to estimate the expected
background from interactions of primary cosmic ray particle and the
interstellar medium. In particular, the most recent measurements of the
production of antideuterons and ${}^{3}\overline{\text{He}}$ can be used to
validate and constrain the model predictions. The theoretical descriptions
based on the scaling of the coalescence parameters $B_{2}$ and $B_{3}$ with
the size of the emission region are in agreement with the observed smooth
evolution of the coalescence parameters as a function of
$\langle\text{d}N_{\text{ch}}/\text{d}\eta_{\mathrm{lab}}\rangle_{|\eta_{\mathrm{lab}}|<0.5}$,
which is proportional to the volume of the emitting source.

In future data taking campaigns, the ALICE collaboration will be able to
provide more precise and more differential results on the production of
antideuterons and ${}^{3}\overline{\text{He}}$ as well as first measurements
of the production of ${}^{4}\overline{\text{He}}$ in pp and p–Pb collisions at
the LHC. These results will help to further constrain the predictions of the
expected background interactions of CR and the ISM which affects the searches
for an excess of the flux of antinuclei at the top of the atmosphere.

## References

  * [1] P. von Doetinchem et al., _Cosmic-ray antinuclei as messengers of new physics: status and outlook for the new decade_ , _JCAP_ 08 (2020) 035 [2002.04163].
  * [2] A. Shukla, A. Datta, P. von Doetinchem, D.-M. Gomez-Coral and C. Kanitz, _Large-scale Simulations of Antihelium Production in Cosmic-ray Interactions_ , _Phys. Rev. D_ 102 (2020) 063004 [2006.12707].
  * [3] D.-M. Gomez-Coral, A. Menchaca Rocha, V. Grabski, A. Datta, P. von Doetinchem and A. Shukla, _Deuteron and Antideuteron Production Simulation in Cosmic-Ray Interactions_ , _Phys. Rev. D_ 98 (2018) 023012 [1806.09303].
  * [4] A. Coogan and S. Profumo, _Origin of the tentative AMS antihelium events_ , _Phys. Rev. D_ 96 (2017) 083020 [1705.09664].
  * [5] M. Korsmeier, F. Donato and N. Fornengo, _Prospects to verify a possible dark matter hint in cosmic antiprotons with antideuterons and antihelium_ , _Phys. Rev. D_ 97 (2018) 103011 [1711.08465].
  * [6] Y.-C. Ding, N. Li, C.-C. Wei, Y.-L. Wu and Y.-F. Zhou, _Prospects of detecting dark matter through cosmic-ray antihelium with the antiproton constraints_ , _JCAP_ 06 (2019) 004 [1808.03612].
  * [7] M. Kachelriess, S. Ostapchenko and J. Tjemsland, _Revisiting cosmic ray antinuclei fluxes with a new coalescence model_ , _JCAP_ 08 (2020) 048 [2002.10481].
  * [8] K. Blum, K. C. Y. Ng, R. Sato and M. Takimoto, _Cosmic rays, antihelium, and an old navy spotlight_ , _Phys. Rev._ D96 (2017) 103021 [1704.05431].
  * [9] V. Poulin, P. Salati, I. Cholis, M. Kamionkowski and J. Silk, _Where do the AMS-02 antihelium events come from?_ , _Phys. Rev. D_ 99 (2019) 023016 [1808.08961].
  * [10] A. Andronic, P. Braun-Munzinger, K. Redlich and J. Stachel, _Decoding the phase structure of QCD via particle production at high energy_ , _Nature_ 561 (2018) 321 [1710.09425].
  * [11] J. I. Kapusta, _Mechanisms for deuteron production in relativistic nuclear collisions_ , _Phys. Rev._ C21 (1980) 1301.
  * [12] R. Scheibl and U. W. Heinz, _Coalescence and flow in ultrarelativistic heavy ion collisions_ , _Phys. Rev._ C59 (1999) 1585 [nucl-th/9809092].
  * [13] F. Bellini and A. P. Kalweit, _Testing coalescence and statistical-thermal production scenarios for (anti-)(hyper-)nuclei and exotic QCD objects at energies available at the CERN Large Hadron Collider_ , _Phys. Rev._ C99 (2019) 054905 [1807.05894].
  * [14] ALICE collaboration, _Production of deuterons, tritons, 3He nuclei and their antinuclei in pp collisions at $\mathbf{\sqrt{{\textit{s}}}}$ = 0.9, 2.76 and 7 TeV_, _Phys. Rev._ C97 (2018) 024615 [1709.08522].
  * [15] ALICE collaboration, _Production of light nuclei and anti-nuclei in pp and Pb–Pb collisions at energies available at the CERN Large Hadron Collider_ , _Phys. Rev._ C93 (2016) 024917 [1506.08951].
  * [16] ALICE collaboration, _(Anti-)Deuteron production in pp collisions at $\sqrt{s}=13$ TeV_, 2003.03184.
  * [17] ALICE collaboration, _Production of (anti-) 3He and (anti-)3H in p-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 5.02 TeV_, _Phys. Rev. C_ 101 (2020) 044906 [1910.14401].
  * [18] V. Vovchenko, B. Dönigus and H. Stoecker, _Multiplicity dependence of light nuclei production at LHC energies in the canonical statistical model_ , _Phys. Lett._ B785 (2018) 171 [1808.05245].
  * [19] ALICE collaboration, _Multiplicity dependence of (anti-)deuteron production in pp collisions at $\sqrt{s}$ = 7 TeV_, _Phys. Lett._ B794 (2019) 50 [1902.09290].
  * [20] ALICE collaboration, _Multiplicity dependence of light (anti-)nuclei production in p-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 5.02 TeV_, _Phys. Lett. B_ 800 (2020) 135043 [1906.03136].