# Wavelet-based Topological Loss for Low-Light Image Denoising

Alexandra Malyugina<EMAIL_ADDRESS>Nantheera Anantrasirichai
David Bull  Visual Information Laboratory, University of Bristol, 1 Cathedral
Square,Bristol, BS8 1UB, UK

###### Abstract

Despite extensive research conducted in the field of image denoising, many
algorithms still heavily depend on supervised learning and their effectiveness
primarily relies on the quality and diversity of training data. It is widely
assumed that digital image distortions are caused by spatially invariant
Additive White Gaussian Noise (AWGN). However, the analysis of real-world data
suggests that this assumption is invalid. Therefore, this paper tackles image
corruption by real noise, providing a framework to capture and utilise the
underlying structural information of an image along with the spatial
information conventionally used for deep learning tasks. We propose a novel
denoising loss function that incorporates topological invariants and is
informed by textural information extracted from the image wavelet domain. The
effectiveness of this proposed method was evaluated by training state-of-the-
art denoising models on the BVI-Lowlight dataset, which features a wide range
of real noise distortions. Adding a topological term to common loss functions
leads to a significant increase in the LPIPS (Learned Perceptual Image Patch
Similarity) metric, with the improvement reaching up to 25%. The results
indicate that the proposed loss function enables neural networks to learn
noise characteristics better. We demonstrate that they can consequently
extract the topological features of noise-free images, resulting in enhanced
contrast and preserved textural information.

###### keywords:

###### MSC:

68U10, 94A08, 57N65 image denoising, persistent homology, Topological Data
Analysis, loss function, wavelet transform

††journal: Computer Vision and Image Understanding

## Research Highlights

1. Proposed wavelet-based topological loss function for image denoising . 2. Tested on BVI-Lowlight dataset with real noise distortions. 3. Achieved significant improvements in LPIPS metric, up to 25%. 4. The loss fucntion reserves textural information while removing noise. 5. The novel loss is a promising approach for image enhancement techniques.

##  1 Introduction

Denoising is a common task when processing low-quality image data and forms an
important component in the processing pipeline for cases where acquisition is
performed under low light conditions. Applications include object detection
and object tracking in autonomous vehicles, biomedical image analysis, natural
history filmmaking and consumer video production. The primary objective of
image denoising is to eliminate the noise present while preserving the
original features. However, in practice denoising poses an inverse problem
with intrinsically ambiguous solutions and hence always results in a trade-off
between signal distortion and noise removal. Moreover, denoising methods
should not introduce new artefacts, for example over smoothing (blurring),
that can degrade the visual quality of the image.

Conventional denoising algorithms (pre machine learning), such as Non-Local
Means (NLM)Buades et al. (2005) and Block-Matching and 3D filtering (BM3D)
Dabov et al. (2009), rely on mathematical models and involve pre-defined
filters to remove noise from images. Latterly, state-of-the-art denoising
methods have exploited deep learning-based approaches Anantrasirichai and Bull
(2021a). Some methods have adopted a self-supervised learning framework to
address the issue of unavailable ground truth data, e.g. Krull et al. (2019),
Batson and Royer (2019),Ulyanov et al. (2018). However, the performance of
these methods is limited, particularly for low-light content where, for
example, edges are significantly distorted due to the presence of noise
Anantrasirichai and Bull (2021b). The best-performing methods hence rely on
supervised learning, where models are trained either by applying synthetic
noise to the clean images or by creating pseudo ground truth data from real
noise images Zhang et al. (2017) Anwar and Barnes (2019)Abdelhamed et al.
(2020).

Deep learning approaches are typically developed by designing a network
architecture and defining a loss function. In this paper, we focus on the
latter. The loss function plays a critical role in measuring the performance
of the neural network, and the goal of training is to minimise this loss.
Through the backpropagation algorithm, the loss function provides feedback to
the neural network by computing its gradient, which is then used to update the
network’s weights. Commonly used loss functions for image denoising include
$\ell_{1}$, $\ell_{2}$, Charbonnier, SSIM, Laplacian loss and a combination of
these Abdelhamed et al. (2020). Other losses have also been employed,
including perceptual loss, wavelet-based loss, gradient loss and total
variation loss, but no performance improvement has been reported with these
methods Anantrasirichai and Bull (2021b). Despite significant efforts to
improve the performance of denoising methods, their results still typically
exhibit distortion artifacts or lost signal information. Moreover, there are
no reports that analyse how the loss function influences noise removal for
different content classes. In this paper we introduce a novel loss function,
combining local low level features and topological features extracted from the
image. The denoising model is trained to be selective with respect to the
textured areas of the image, resulting in improvements in contrast and
performance on edges. Our proposed loss function is based on our previous work
in Malyugina et al. (2023); we further enhance performance by using wavelets
to categorise different content classes, such as homogeneous regions, edges,
structures, and textures. We then create a weight map to apply the appropriate
loss to each category. This is applied at the pixel level resulting in
enhanced sharpness and spatial contrast in the output image. Additionally, we
calculate topological invariants of the whole image instead of patches (as
reported in Malyugina et al. (2023)). We compare the performance of the new
loss function across popular denoising architectures, revealing that adding a
topological term to conventional loss functions helps to improve subjective
results by enhancing contrast while preserving texture. In conclusion, the
main contributions of our paper are

  * 1.

A novel denoising loss function that incorporates topological invariants and
leverages textural information from the image wavelet domain. The proposed
framework effectively integrates both the underlying structural information
and spatial characteristics of images.

  * 2.

Evaluation of the proposed loss function with multiple denoising models on the
BVI-Lowlight dataset, which is specifically designed for real noise
distortions in low light conditions.

The remainder of this paper is organised as follows. Problem statement and
existing solutions are presented in Section 2. Details of topological
invariants and the proposed loss function for image denoising are described in
Section 3 and Section 4, respectively. The performance of the method is
evaluated in Section 5, followed by the conclusions in Section 6.

##  2 Image Denoising Problem and Current Solutions

###  2.1 Image Denoising Problem

Given an image $\mathcal{I}^{N}$ corrupted by noise $N$, the goal is to obtain
a denoised image $\mathcal{I}^{O}$ that accurately represents the underlying
clean image $\mathcal{I}^{C}$. Mathematically, this can be expressed as:

$\mathcal{I}^{O}=f_{\bm{\theta}}(\mathcal{I}^{N},)$

where $f_{\bm{\theta}}$ represents the denoising function or model that takes
the noisy image as input and produces the denoised image as output. The noisy
image $\mathcal{I}^{N}$ in this case is the result of some function (e.g.
signal-dependent noise) of $\mathcal{I}^{C}$:

$\mathcal{I}^{N}={Noise}(\mathcal{I}^{C})$

The task of denoising can then be formulated as an optimisation problem, where
one searches for an optimal parameter set
$\bm{\theta}=\\{\theta_{i}\\}_{i\in\mathcal{I}}$ for a denoising model
$f_{\bm{\theta}}$ through minimising loss function $\mathcal{L}$:

$\hat{\bm{\theta}}=\arg\min_{\bm{\theta}}\mathcal{L}(f_{\bm{\theta}}(\mathcal{I}^{N}),\mathcal{I}^{C}).$

The denoising process aims to estimate the clean image $\mathcal{I}^{C}$ by
suppressing or removing the noise component from the noisy image
$\mathcal{I}^{N}$. This involves leveraging statistical properties of the
noise and exploiting the underlying structure and features present in the
image.

###  2.2 Current Image Denoising Methods

Denoising methods can be categorised based on the domain they operate on and
the type of approach employed.

In terms of the domain, denoising methods can be classified into spatial
domain, transform domain, and hybrid (spatio-domain) methods. Spatial domain
methods operate directly on the pixel values of an image. They include local
methods such as mean filters, order-statistic filters, and adaptive filters.
Non-local methods, like non-local means (NLM), take advantage of image self-
similarity for denoising. Techniques such as steering kernel regression (SKR)
Takeda et al. (2006) and Wiener filtering Wiener (1949) have been proposed to
improve the performance of spatial filters in structured areas containing
edges Tronicke and Böniger (2013).

Transform domain methods involve representing image patches using orthogonal
basic functions and adjusting coefficients to denoise the image. Wavelet-based
denoising methods have gained popularity due to their sparsity and
multiresolution properties. Thresholding techniques, such as VISUShrink Donoho
and Johnstone (1994) and BayesShrink Chipman et al. (1997), are commonly used
in wavelet domain denoising Shui (2009) Yu and Sapiro (2011) Foi et al.
(2007)Simoncelli and Adelson (1996) Mihcak et al. (1999).

Dual domain methods combine spatial and transform domain filtering. The BM3D
algorithm, proposed by Dabov et al. Dabov et al. (2007), utilizes
collaborative filtering in both the spatial and transform domains. It groups
similar image patches, applies transform-domain shrinkage, and performs
inverse transforms to obtain denoised images. Shape-adaptive versions of BM3D
further improve the preservation of image details Dabov et al. (2008)Dabov et
al. (2009).

In terms of the type of approach, denoising methods can be divided into
conventional and learning-based methods. Conventional methods typically employ
handcrafted algorithms and heuristics to remove noise from images. These
approaches include filters such as mean filters, median filters, order-
statistic filters and many of the above-mentioned techniques (NLM, BM3D and
thresholding).

Learning-based methods, such as sparse representation and dictionary learning,
have been applied to image denoising. Techniques such as K-SVD Aharon et al.
(2006) and locally learned dictionaries (K-LLD) Chatterjee and Milanfar (2009)
utilise sparse coding and clustering techniques to represent image patches.
The LSSC method proposed by Mairal et al. Mairal et al. (2009) combines local
and non-local sparsity within a neighborhood. Deep learning techniques,
specifically convolutional neural networks (CNNs), have shown remarkable
performance in image denoising. DnCNN, introduced by Zhang et al. Zhang et al.
(2017), is a deep neural network architecture based on residual learning that
has achieved excellent results. One of state-of-the-art image denoising
methods, RIDNET Anwar and Barnes (2019), utilise attention mechanisms with
short and long connects to learn both local and global features. More recent
learning-based denoiser combine transformer with UNet architectures, which
enhances feature extraction and hierarchical representation capabilities Fan
et al. (2022). The performance is comparable to RDUNet Gurrola-Ramos et al.
(2021), where residual dense network is employed through UNet architectures.

It is worth noting that there is a distinction between supervised and
unsupervised deep learning denoising. Supervised methods rely on ground truth
images for training, while unsupervised methods aim to denoise images without
ground truth information. Deep Image Prior Ulyanov et al. (2018) and
Noise2Noise Lehtinen et al. (2018) are examples of unsupervised deep learning
approaches, where networks are trained using only noisy input images.
Conventional techniques often suffer from insufficient adaptability due to
their reliance on fixed filters or predefined parameters, while learning-based
denoising methods face challenges related to data requirements, generalization
and handling new noise types. Both conventional and learning-based denoising
approaches may encounter difficulties in preserving fine image details while
reducing noise. However, learning-based algorithms have demonstrated better
capabilities in addressing this challenge compared to conventional techniques.

(a) Datacloud $\mathcal{X}$

(b) Persistent diagram $PD(\mathcal{X})$ for the datacloud $\mathcal{X}$.

Fig. 1: A persistent diagram $PD(\mathcal{X})$ of a data cloud $\mathcal{X}$
shows the birth (represented by the $x$ coordinate) and death (represented by
the $y$ coordinate) of topological features of different dimensions
(0-dimensional and 1-dimensional features, denoted as $H0$ and $H1$
respectively). Each point in the diagram corresponds to one of these features.

##  3 Topological Invariants of Image Data

This section describes the mathematical concepts needed to create a
topological loss function and to understand how can be exploited for image
denoising.

Topological Data Analysis (TDA) has emerged as a powerful tool for analyzing
complex data sets in various fields. It uses algebraic topology to study the
shape and structure of data, including clusters, holes, and voids, which is
particularly useful when dealing with data that may be high-dimensional or
noisy. The TDA technique we employ here is called ‘persistent homology’, since
it has been successfully applied to various problems, including genomics,
material science, analysis of financial networks and image processing Byrne et
al. (2023) Zomorodian and Carlsson (2005) Edelsbrunner et al. (2000)Epstein et
al. (2011) Malyugina et al. (2022b) Carlsson et al. (2008).

In particular, the concept of persistent homology in TDA has shown promising
results in topological image denoising Malyugina et al. (2023). It has been
shown that, by applying a topological based loss function that operates in
image patch space, noise removal can be enhanced while preserving essential
topological features of the image.

Persistent homology can be used to deal with images by providing a robust and
stable way to extract topological information from the image. By incorporating
this information in the backpropagation process, persistent homology can
assist in filtering out noise and extracting the underlying signal. The
proposed loss function ensures that persistent homology features represented
by persistent diagram of the denoised output of the neural network are close
to those of the clean image.

TDA was originally derived for continuous curves and manifolds, making it
unsuitable for discrete objects such as images. To identify and compute
topological invariants, which are properties of geometric objects that remain
constant under certain transformations, the concept of a simplicial complex is
introduced. A simplicial complex consists of simple shapes of various
dimensions like points, lines, triangles, and tetrahedrons. These shapes are
arranged by significance using a technique called filtration, which orders
them from simpler to more complex. The basic idea is to begin with the most
straightforward simplicial complex, which generally includes unconnected
points or vertices, and then systematically include additional simplices to
the complex in a controlled manner. This allows us to study the evolution of
the topology of the object as it becomes more complex.

######  Definition 3.1 (Simplicial Complexes).

Let $V$ as a finite nonempty set, the elements of which are called vertices. A
simplicial complex on $V$ is a collection $\mathcal{C}$ of nonempty subsets of
$V$ that satisfies the following conditions:

(i) $\forall v\in V$, the set $\\{v\\}$ lies in $\mathcal{C}$, and

(ii) $\forall\alpha\in\mathcal{C},$ and $\beta\subseteq\alpha$, where $\beta$
is also an element of $\mathcal{C}$. $\alpha$ and $\beta$ are called a simplex
and a face, respectively.

Conditions $(i)-(ii)$ mean that any face or subset of vertices of a simplex in
the collection is also in the collection and the intersection of any two
simplices in the collection is either empty or a face of both simplices.

The dimension of a simplex $\alpha\in\mathcal{C}$ is defined as
$dim(\alpha)=|\alpha|-1$, the dimension of $\mathcal{C}$ is the highest
dimension of constituent simplices.

In other words, a simplicial complex is a collection of simplices that fit
together like puzzle pieces, where the boundaries of each simplex are shared
with other simplices in the collection.

A subcomplex of a simplicial complex is a subset of its simplices that itself
forms a simplicial complex. In other words, a subcomplex is a simplicial
complex that is contained within another simplicial complex, or more formally,
$\mathcal{C}^{\prime}$ is called a subcomplex of $\mathcal{C}$, if
$\mathcal{C}^{\prime}\subseteq\mathcal{C}$ and $\mathcal{C}^{\prime}$ is a
simplicial complex itself. Intuitively, a subcomplex of a simplicial complex
is a smaller “piece” of the original complex that retains its simplicial
structure. For example, a triangle that is a face of a larger tetrahedron can
be considered a subcomplex of that tetrahedron.

Simplicial complex is a general concept in topology that provides a structured
way to study topological properties of spaces. One specific type of a
simplicial complex used in topological data analysis is the Vietoris-Rips
complex.

######  Definition 3.2 (Vietoris-Rips Complex).

Given a finite set of points $P$ in a metric space $(X,\rho)$ and a distance
threshold $\varepsilon>0$, the Vietoris-Rips complex $V_{\varepsilon}(P)$ is a
simplicial complex whose $k$-simplices correspond to subsets $S\subseteq P$
with diameter $\text{diam}(S)$ less than or equal to $\varepsilon$, i.e.,
$\text{diam}(S)=\max_{x,y\in S}rho(x,y)\leq\varepsilon$. Formally, a
$k$-simplex $S=\\{p_{0},p_{1},\ldots,p_{k}\\}$ belongs to $V_{\varepsilon}(P)$
if and only if $\rho(p_{i},p_{j})\leq\varepsilon$ for all $0\leq i,j\leq k$.

In other words, Vietoris-Rips complex is constructed from a set of data points
by connecting those points that are within a certain distance threshold of
each other. This type of complexes is useful for analyzing the shape and
clustering of multi dimensional data, revealing different levels of detail as
$\varepsilon$ changes.

There are other types of widely used complexes in topological analysis. One of
them, a cubical complex, specifically caters the data structure of digital
images Choe and Ramanna (2022). Cubical complexes provide a different approach
by utilising the structure composed of axis-parallel cubes that encapsulate
the geometry and relationships between cells of varying dimensions in a more
grid-like manner.

However, although cubical complexes are useful for digital image data Choe and
Ramanna (2022), they have inherent higher-dimensional nature and the larger
number of possible interactions between cells. This results in increased
computational overhead and slower processing times compared to the relatively
simpler pairwise distance calculations used in the Vietoris-Rips complex
construction. Computation times for RGB image patches are shown in Figure 2.
For example, for $512\times 512$ patch, the computation time for cubical
complex (the computation time is the same for 0- and 1- dimensions due to the
algorithm) is $\sim 5$ times higher than for 1-dim Vietoris-Rips and $\sim
170$ times higher than for 0-dim Vietoris-Rips complex (computed on Nvidia
2080 Ti). Due to this considerable limitation and critical importance of the
balance of computational speed and image patch size for deep-learning based
denoising, throughout this paper we will assume the use of Vietoris-Rips
complexes.

(a)

(b)

Fig. 2: (a) Computation times of Vietoris-Rips complexes and Cubical complexes
of dimensions 0 and 1. (b) Computation times of Vietoris-Rips complex of
dimension 0.

Simplices in a simplicial complex are arranged by significance using a
technique called filtration.

######  Definition 3.3 (Filtrations).

A (n-)filtration of a simplicial complex is a sequence of subcomplexes,
ordered by inclusion, such that each subcomplex is contained in the next. In
other words, it is a sequence of simplicial complexes:

$\mathcal{FC}_{0}\subset\mathcal{FC}_{1}\subset\mathcal{FC}_{2}\cdots\subset\mathcal{FC}_{n}=\mathcal{C}$
(1)

In TDA, while simplicial complexes are a convenient way to represent the
connectivity and topological relationships within data, a filtration is used
to provide a hierarchical representation of these relationship, revealing the
evolution and persistence of topological features across different scales.

The purpose of filtrations is to capture the occurrence (“birth”) and
disappearance (“death”) of topological like homology classes Edelsbrunner et
al. (2000), that can emerge and vanish as the parameter $\varepsilon$ of the
filtration $\mathcal{C}_{\varepsilon}$ changes (for Vietoris-Rips complexes).

To visualise the changes in the topology of a simplicial complex over a range
of filtration scales, a persistent diagram is created. The diagram displays
each homology class Edelsbrunner et al. (2000) as a point, where the
horizontal axis and the vertical axis represent the “birth” and “death” times,
respectively.

######  Definition 3.4 (Persistent diagrams).

A persistent diagram of a filtration $\mathcal{F}$ on a $n$-dimensional
complex $\mathcal{C}$ is a collection of mappings
$PD=\\{PD_{k}\\}_{k=1}^{m<=n}$, each of which map every $i^{th}$
$k$-dimensional topological feature to a pair
$(b_{i},d_{i})\in\mathbb{R}^{2}\cup{\infty}$:

$PD_{k}:(\mathcal{C},\mathcal{F})\to\\{b_{i},d_{i}\\}_{i\in I_{k}},$ (2)

where $b_{i}$ represents the appearance (“birth”) of the feature when
$\varepsilon=b_{i}$ and $d_{i}$ represents its disappearance (“death”) when
$\varepsilon=d_{i}$ (see Fig. 1). When $k=0$, each point in the persistent
diagram represent a connected component and when $k=1$, a point corresponds to
a 1-dimensional holes in filtered subcomplexes of $\mathcal{C}$.

For image data, we calculate persistent diagrams on the filtrations of
Vietoris-Rips complexes over image intensity scale, tracking the occurence and
disappearance of topological features at different intensity values. For our
task, we consider $k=0$ to represent the number of connected components and
$k=1$ to represent the number of 1-dimensional holes in filtered subcomplexes
of $\mathcal{C}$. As this paper focuses on specific aspects of the topic and
cannot accommodate a comprehensive discussion of concepts such as homology
classes and groups, we refer the readers to Edelsbrunner et al. (2000) for
more theoretical foundations and a deeper theoretical understanding of
persistent diagrams.

##  4 Wavelet-based Topological Loss Function

###  4.1 Topological Loss Function

To construct a topological loss for a pair of ground truth and output images
$(\mathcal{I}^{C},\mathcal{I}^{O})$, we first calculate persistence diagrams
$PD(\mathcal{I}^{C})$ and $PD(\mathcal{I}^{O})$ using intensity-based
filtrations of Vietoris-Rips complexes calculated in the spatial dimension of
the images. The topological distance between two diagrams is then calculated
using total persistence as a dissimilarity measure.

######  Definition 4.1 (Total Persistence).

$p$-total persistence of a persistence diagram $PD$ is defined as a total sum
of the lifespans of each point in a diagram:

$TPers(PD)=\sum_{x\in PD}{(d(x)-b(x))^{p}},$ (3)

where $d(x)$ and $b(x)$ are “death” and “birth” values of $x\in PD$. For our
experiments we use $p=1$.

Total persistence can be interpreted as the overall importance of topological
features in a simplicial complex over a range of filtration scales.

Finally, we can use total persistence as the dissimilarity measure for the
persistense diagrams space. We introduce the topological component of our
future loss function, that we define as shown in Eq. 4, where the absolute
value is used instead of squaring to ensure robustness to outliers:

$\mathcal{L}_{top}(\mathcal{I}^{O},\mathcal{I}^{C})=\\\
|TPers(PD(\mathcal{I}^{O}))-TPers(PD(\mathcal{I}^{C}))|.$ (4)

A topological loss function based on persistent homology in the space of
contrast patches sampled spatially from an image was shown to improve image
denoising task by Malyugina et al. in Malyugina et al. (2023). Unlike
Malyugina et al. (2023), in this work we obtain topological information
directly from the image spatial domain, leading to increased information of
both local and global features. We then combine it with a spatial loss to
ensure that local spatial information is retained. We used torch-topological
111https://github.com/aidos-lab/pytorch-topological for calculation of
complexes, filtrations and persistence diagrams.

###  4.2 Texture Mask

In our experiments with $\mathcal{L}_{top}$ defined in Equation 4, we observed
that the topological loss component works better in textured areas or on
edges, while sometimes creating undesired artifacts in plain untextured areas
(see Fig. 5). Therefore we modified our approach to train and apply
topological loss using a texture mask based on wavelet decompositions.
Wavelets have been proven to be successful in capturing textual information in
images through various applications, including texture classification
Anantrasirichai et al. (2015a) and image denoising with texture preservation
Hill et al. (2015). The patterns and variations in the wavelet coefficients
can provide information about the spatial frequency content and orientation of
the texture.

The Discrete Wavelet Transform (DWT) decomposes a signal into a set of
mutually orthogonal basis wavelet functions. To retrieve textured areas from
the ground truth image $\mathcal{I}^{C}$, we first apply 1-level decomposition
with Haar base functions Schmidt (1907) and take only low frequency bands,
$LL$. This aims to ignore the areas with fine textural detail and noise
Anantrasirichai et al. (2015b), producing a smoother mask. Subsequently, we
apply another DWT to $LL$ and retrieve absolute values from the high-pass
subbands $HH,HV,HD$ of $LL$, where $HH$, $HV$ and $HD$ denote high-pass
subbands in horizontal, vertical and diagonal directions, respectively.
Finally, we take mean values of these three level 2 DWT subbands, thus
resulting in a texture mask (see Fig. 3):

$Mask(\mathcal{I}^{C})=\text{mean}\\{d_{H}\\}|_{H\in\\{HH,HV,HD\\}},$

where

$d_{H}=|\ DWT_{LL}\circ DWT_{H}\circ\uparrow_{4}(\mathcal{I}^{C})|,$

and $\uparrow_{4}$ denotes upscaling with the factor of $4$ and $\circ$ is a
function superposition.

(a)

(b)

(c)

Fig. 3: (a) Ground truth image ${I}^{C}$; (b) wavelet bands of 2-level
decomposition $DWT(\mathcal{I}^{C})$; (c) resulting wavelet mask
$Mask(\mathcal{I}^{C})$ built by averaging $2^{\text{nd}}$ order high-pass
bands (green in (b)).

Finally, to apply topological loss only to textured areas, we apply the
texture mask to the image $\mathcal{I}^{C}$, producing $Mask(\mathcal{I}^{C})$
and combine it with pixelwise basic loss $\mathcal{L}_{base}$, where
$\mathcal{L}_{base}=\ell_{1}$ or $\ell_{2}$:

$\displaystyle\mathcal{L}_{wvcomb}(\mathcal{I}^{O},\mathcal{I}^{C})=\mathcal{L}_{top}(\mathcal{I}^{O},\mathcal{I}^{C})\odot
Mask(\mathcal{I}^{C})+$
$\displaystyle\alpha\mathcal{L}_{base}(\mathcal{I}^{O},\mathcal{I}^{C})\odot(\mathbb{I}-Mask(\mathcal{I}^{C}))$

where $\odot$ is element-wise product and $\alpha$ is a scalar hyperparameter.
The respective method is shown in Algorithm 1.

Input : Noisy image $I^{N}$, Groundtruth image $I^{C}$, Trainable model
$f_{\bm{\theta}}$

Output : Optimised parameter set $\hat{\bm{\theta}}$

Step 1: Wavelet Decomposition;

Perform wavelet decomposition on $I^{C}$ to obtain $LL$ band wavelet
coefficients; $\mathcal{I}^{C}_{LL}=DWT_{LL}(\mathcal{I}^{C})$

Step 2: Texture Mask Calculation;

Calculate the texture mask;
$Mask(\mathcal{I}^{C})=DWT_{H}(\mathcal{I}^{C}_{LL})$

Step 3: Acquire model output $\mathcal{I}^{O}$;

Apply model $f_{\bm{\theta}}$ to noisy image:
$\mathcal{I}^{O}=f_{\bm{\theta}}(\mathcal{I}^{N})$

Step 4: Persistence Diagram Calculation; Calculate the persistence diagrams
$PD(I^{C})$ and $PD(I^{O})$ of $I^{C}$ and $I^{O}$, respectively;

Step 5: Topological Loss Term Calculation;

Calculate the total persistence values $TPers(I^{C})$ and $TPers(I^{O})$ for
$PD(I^{C})$ and $PD(I^{O})$, respectively;

Calculate the topological loss component $\mathcal{L}_{top}$:
$\mathcal{L}_{top}(\mathcal{I}^{O},\mathcal{I}^{C})=|TPers(PD(\mathcal{I}^{O}))-TPers(PD(\mathcal{I}^{C}))|;$

Step 6: Base Loss Term Calculation;

Calculate the base loss component $\mathcal{L}_{base}$ as the $\ell_{p}$ loss
between $I^{C}$ and $I^{O}$;
$\mathcal{L}_{base}=||(\mathcal{I}^{O},\mathcal{I}^{C})||_{p},\quad p=1\text{
or }2$

Step 7: Combined Loss Calculation;

Calculate the pixelwise mask-weighted topological loss to enforce the
topological guidance in textured areas:
$\mathcal{L}_{top}^{masked}=\mathcal{L}_{top}\odot Mask(\mathcal{I}^{C});$ and
in remaining “plain” areas tune it down:
$\mathcal{L}_{base}^{masked}=\mathcal{L}_{base}\odot(1-Mask(\mathcal{I}^{C}));$

Calculate the combined wavelet loss $\mathcal{L}{wvcomb}$ as the
($\alpha$-weighted) sum of the mask-weighted losses with gain $\alpha$ ;

$\mathcal{L}_{wvcomb}(\mathcal{I}^{O},\mathcal{I}^{C})=\mathcal{L}_{base}^{masked}+\alpha\mathcal{L}_{top}^{masked}$
Step 8: Find the parameters of the network, minimising combined loss;

$\hat{\bm{\theta}}=\arg\min_{\bm{\theta}}\mathcal{L}_{wvcomb}(f_{\bm{\theta}}(\mathcal{I}^{N}),\mathcal{I}^{C})$
return $\hat{\bm{\theta}}$;

Algorithm 1 Wavelet Topological Image Denoising

Figure 4 shows the diagram of the proposed wavelet-based topological loss
function. The restored image output of the denoising network is transformed
into topological features, which are subsequently compared with those of the
groundtruth, $\mathcal{L}_{top}$. The final loss function is a combination of
$\mathcal{L}_{top}$ and $\mathcal{L}_{base}$ with the weighting determined by
the textural information extracted from the wavelet domain.

Fig. 4: Wavelet topological loss calculation for a pair of images: output
image ${I}^{O}$ and groundtruth ${I}^{C}$. First, the texture mask of
${I}^{C}$ is calculated based on wavelet decomposition of ground truth image
$\mathcal{I}^{C}$. The topological loss component $\mathcal{L}_{top}$ is
calculated as the absolute difference of total persistence values of diagrams
$PD(\mathcal{I}_{C})$ and $PD(\mathcal{I}_{O})$. We also calculate
$\mathcal{L}_{base}$ ($l_{1}$ loss) to retain image spatial information. The
resulting wavelet combined loss $\mathcal{L}_{wvcomb}$ is calculated as
pixelwise mask-weighted base loss $M({I}^{C})\times\mathcal{L}_{top}$ and
$\mathcal{L}_{base}$.

##  5 Experiments and discussion

###  5.1 Dataset

We trained and tested the models on the BVI-Lowlight dataset Malyugina et al.
(2022a, 2023), which was collected to overcome the limitations of existing
denoising datasets in terms of variations in noise levels and diverse content
with varying textures. This dataset consists of 31,800 14-bit images captured
from 20 scenes using two cameras, Nikon D7000 and Sony A7SII, with ISO
settings ranging from 100 to 25600 and 100 to 409600, respectively. We ensured
consistent lighting conditions by using non-flickering LED lights. Pseudo
ground truth images were generated by applying postprocessing techniques,
including the removal of oversaturated pixels, intensity alignment, and image
registration. Further details can be found in Malyugina et al. (2023).

###  5.2 Training

In our experiments, we employed both traditional and state-of-the-art
architectures for our baselines: i) A residual-based denoising convolutional
neural network (DnCNN) introduced in Zhang et al. (2017) by Zhang et al.,
which is a widely-used method for benchmarking deep image denoisers. ii) UNet,
an encoder-decoder style fully convolutional network Ronneberger et al.
(2015), which has been extensively used as a foundational framework for image
denoising. iii) RIDNet, a single-stage blind real image denoising network that
incorporates a feature attention mechanism Anwar and Barnes (2019).

We compared the proposed loss with other denoising losses, including
conventional losses $\ell_{1}$, $\ell_{2}$, and VGG loss ($\alpha$ is set to
$0.99942857$ as in Johnson et al. (2016)). In our experiments, we set
topological loss term gain $\alpha$ to $0.004$ as it demonstrated best
performance on test sets. We also tested several combinations of losses and
models.

All models presented in this paper were trained using 11K patches from the
BVI-Lowlight dataset. The patches were sized 256$\times$256, and we trained
the models for $40$ epochs. The initial learning rate was set to $0.0001$, and
the batch size was $16$, which was limited by the memory capacity of our
computing system.

###  5.3 Results

We evaluated the performance of our method by training state-of-the-art
denoising models, using different loss functions and their combintaions. The
results based on objective metrics are presented in Table 1 and examples for
subjective assessment are shown in Fig. 5 and Fig.6. These results demonstrate
improved performance in terms of objective metrics that correlate more
strongly with subjective quality assessment Zhang et al. (2018).

Fig. 5: Using wavelet mask with $\mathcal{L}_{top}$. Top row: Sample patches
from dataset ground truth. Second row: Sample patches from noisy images with
ISO varying from 160000 to 400000. Third row: Sample patches of the outputs
from RIDNET trained with $\ell_{1}$ only. Fourth row: Sample patches of the
outputs from DnCNN trained with topology loss (unmasked) combined with
$\ell_{1}$. Bottom row: Sample patches of the outputs from DnCNN trained with
topology loss (masked) combined with $\ell_{1}$. PSNR (yellow), LPIPS (cyan)
and SSIM (orange) values are calculated per corresponding pairs. Note the
preserved edges, enhanced contrast and improvement in LPIPS metric values when
using masked topological loss compared to $\ell_{1}$ loss only. Using wavelet
texture mask also prevents the network from producing undesirable artifacts.
Fig. 6: Results. Top row: Sample patches from dataset ground truth. Second
row: Sample patches from noisy images with ISO varying from 160000 to 400000.
Third row: Sample patches of the outputs from DnCNN trained with $\ell_{1}$
only. Fourth row: Sample patches of the outputs from DnCNN trained with VGG
loss combined with $\ell_{1}$. Fifth row: Sample patches of the outputs from
DnCNN trained with topology loss (masked) combined with $\ell_{1}$. Bottom
row: Sample patches of the outputs from DnCNN trained with topology loss
(masked) combined with VGG and $\ell_{1}$. PSNR (yellow), LPIPS (cyan) and
SSIM (orange) values are calculated per corresponding pairs.

The highest and most consistent gain throughout all the architectures is
achieved using the LPIPS metric: this shows improvements compared to
conventional loss functions, as well as when combined with VGG. We have
observed that perceptually, adding VGG loss in some cases helps to attenuate
artifacts that can appear in plain areas when using masked topological loss
only. Although with the RIDNET architecture, PSNR is lower for the models
trained with topological loss compared to ${\ell_{1}}$ or ${\ell_{2}}$, LPIPS
values are still considerably higher. It is also worth noting that adding VGG
loss term to a minimization criterion does not always improve the performance,
comparing to models trained with basic loss only (see values for DnCNN with
${\ell_{2}}$). Figure 5 shows that using a wavelet mask together with a
topological term for the loss helps to reduce the occurrence of artifacts in
homogeneous areas while preserving the texture and enhancing the contrast. For
example, the denoised coin image using RIDNET with our proposed loss function
exhibits sharper edges and better details compared to those denoised using
$\ell_{2}$ alone or without a mask. Additionally, RIDNET with the unmasked
loss function fails to completely remove color noise.

Figure 6 shows the denoised results using DnCNN. The results obtained by
training it with a topological loss are shown with wavelet-based masks.
Comparing the results in the 5th and 6th rows to those in the 3rd and 4th
rows, our proposed masked topological loss produces sharper results with
better fine details. Specifically, the fonts in the third columns become more
readable, such as ‘R61’. Without a topological loss, the VGG loss appears to
remove the fine texture present in the top-right area of the images in the
fourth column, which is the white area of the lama’s doll fur. More subjective
results are available in supplemental material, see Fig. 1.

Overall, the evaluation of the results based on the BVI-Lowlight dataset,
featuring real-world noise levels, has demonstrated the effectiveness of the
proposed loss function in enabling denoising models to better learn both noise
characteristics and the underlying signal. This, in turn, leads to enhanced
contrast and preserved textural information in the denoised images.

In the presence of noise, the topological properties of an image can become
distorted or disrupted. By incorporating topological invariants into the loss
function, the algorithm can effectively capture and preserve the original
structural features of the image while removing the noise. This approach helps
to maintain the integrity of textured areas, and enhance the overall visual
quality of the denoised image, as well as provide a valuable framework for
improving the performance of existing image denoising models and combating
common problems such as oversmoothing.

Table 1: Metrics values for the outputs of a corresponding model (DnCNN, RIDNET or UNet) trained with (i) ${\ell_{i}}$ only (ii) combination of ${\ell_{i}}$ with VGG loss (iii) combination of ${\ell_{i}}$ with persistence-based loss $\mathcal{L}_{top}$ (iv) combination of ${\ell_{i}}$ with persistence-based loss $\mathcal{L}_{top}$ and VGG loss $\mathcal{L}_{vgg}$. | LPIPS | PSNR | SSIM
---|---|---|---
Noisy | 0.430 | 29.19 | 0.632
DnCNN+l1 | 0.172 | 35.26 | 0.918
DnCNN+l1+vgg | 0.166 | 35.33 | 0.917
DnCNN+l1+topo | 0.129 | 35.60 | 0.923
DnCNN+l1+topo+vgg | 0.134 | 35.66 | 0.921
DnCNN+l2 | 0.154 | 35.54 | 0.923
DnCNN+l2+vgg | 0.186 | 34.83 | 0.914
DnCNN+l2+topo | 0.131 | 35.42 | 0.918
DnCNN+l2+topo+vgg | 0.124 | 35.55 | 0.919
RIDNET+l1 | 0.119 | 36.97 | 0.934
RIDNET+l1+vgg | 0.115 | 37.06 | 0.936
RIDNET+l1+topo | 0.096 | 37.24 | 0.938
RIDNET+l1+topo+vgg | 0.108 | 37.23 | 0.937
RIDNET+l2 | 0.098 | 37.30 | 0.937
RIDNET+l2+vgg | 0.098 | 37.23 | 0.939
RIDNET+l2+topo | 0.094 | 36.65 | 0.928
RIDNET+l2+topo+vgg | 0.089 | 36.91 | 0.933
UNet+l1 | 0.180 | 35.36 | 0.916
UNet+l1+vgg | 0.179 | 35.34 | 0.915
UNet+l1+topo | 0.136 | 35.68 | 0.923
UNet+l1+topo+vgg | 0.139 | 35.65 | 0.924
UNet+l2 | 0.171 | 35.64 | 0.919
UNet+l2+vgg | 0.165 | 35.75 | 0.922
UNet+l2+topo | 0.121 | 35.82 | 0.919
UNet+l2+topo+vgg | 0.116 | 35.88 | 0.923

##  6 Conclusion and Future Work

In this paper, we introduce a novel method for image denoising that
incorporates a wavelet-based topological loss function and utilizes textural
information from the image wavelet domain. Our proposed loss function
effectively captures noise characteristics while preserving image texture,
resulting in denoised images with enhanced visual quality and preserved
perceptual fidelity of the original content. The experimental results on real
noise confirm the efficacy of our method, showing superior perceptual results
and improved performance in objective metrics that are highly correlated with
subjective quality assessment.

For future research, the generalisability of the wavelet-based topological
loss function should be further investigated across diverse datasets and noise
types. Additionally, investigating the integration of this approach with other
denoising methods may yield further improvements.

## Acknowledgments

This work was supported by the UKRI MyWorld Strength in Places Programme
(SIPF00006/1) and Bristol+Bath Creative R+D under AHRC grant (AH/S002936/1).

## References

  * Abdelhamed et al. (2020) Abdelhamed, A., Afifi, M., Timofte, R., Brown, M.S., 2020.  NTIRE 2020 Challenge on Real Image Denoising: Dataset, Methods and Results, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
  * Aharon et al. (2006) Aharon, M., Elad, M., Bruckstein, A., et al., 2006.  K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation.  IEEE Transactions on signal processing 54, 4311.
  * Anantrasirichai and Bull (2021a) Anantrasirichai, N., Bull, D., 2021a.  Artificial intelligence in the creative industries: a review.  Art. Int. Rev. .
  * Anantrasirichai and Bull (2021b) Anantrasirichai, N., Bull, D., 2021b.  Contextual colorization and denoising for low-light ultra high resolution sequences, in: ICIP proc., pp. 1614–1618.
  * Anantrasirichai et al. (2015a) Anantrasirichai, N., Burn, J., Bull, D., 2015a.  Terrain classification from body-mounted cameras during human locomotion.  IEEE Transactions on Cybernetics 45, 2249–2260.  doi:10.1109/TCYB.2014.2368353.
  * Anantrasirichai et al. (2015b) Anantrasirichai, N., Burn, J., Bull, D.R., 2015b.  Robust texture features based on undecimated dual-tree complex wavelets and local magnitude binary patterns, in: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3957–3961.  doi:10.1109/ICIP.2015.7351548.
  * Anwar and Barnes (2019) Anwar, S., Barnes, N., 2019.  Real image denoising with feature attention.  IEEE International Conference on Computer Vision (ICCV-Oral) .
  * Batson and Royer (2019) Batson, J., Royer, L., 2019.  Noise2self: Blind denoising by self-supervision, in: ICML proc., PMLR. pp. 524–533.
  * Buades et al. (2005) Buades, A., Coll, B., Morel, J.M., 2005.  A non-local algorithm for image denoising, in: CVPR proc., IEEE. pp. 60–65.
  * Byrne et al. (2023) Byrne, N., Clough, J.R., Valverde, I., Montana, G., King, A.P., 2023.  A persistent homology-based topological loss for CNN-based multiclass segmentation of CMR.  IEEE Transactions on Medical Imaging 42, 3–14.  URL: https://doi.org/10.1109%2Ftmi.2022.3203309, doi:10.1109/tmi.2022.3203309.
  * Carlsson et al. (2008) Carlsson, G., Ishkhanov, T., Silva, V.D., Zomorodian, A., 2008.  On the local behavior of spaces of natural images.  Int. j. of comp. vision 76, 1–12.
  * Chatterjee and Milanfar (2009) Chatterjee, P., Milanfar, P., 2009.  Clustering-based denoising with locally learned dictionaries.  IEEE Transactions on Image Processing 18, 1438–1451.
  * Chipman et al. (1997) Chipman, H.A., Kolaczyk, E.D., McCulloch, R.E., 1997.  Adaptive bayesian wavelet shrinkage.  Journal of the Amer. Stat. Assoc. 92, 1413–1421.
  * Choe and Ramanna (2022) Choe, S., Ramanna, S., 2022.  Cubical homology-based machine learning: An application in image classification.  Axioms 11, 112.
  * Dabov et al. (2007) Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K., 2007.  Image denoising by sparse 3-d transform-domain collaborative filtering.  IEEE Transactions on image processing 16, 2080–2095.
  * Dabov et al. (2008) Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K., 2008.  A nonlocal and shape-adaptive transform-domain collaborative filtering.  Proc. Int. Workshop on Local and Non-Local Approx. in Image Process., LNLA .
  * Dabov et al. (2009) Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K., 2009.  BM3D image denoising with shape-adaptive principal component analysis, in: SPARS’09.
  * Donoho and Johnstone (1994) Donoho, D.L., Johnstone, J.M., 1994.  Ideal spatial adaptation by wavelet shrinkage.  biometrika 81, 425–455.
  * Edelsbrunner et al. (2000) Edelsbrunner, H., Letscher, D., Zomorodian, A., 2000.  Topological persistence and simplification, in: Proc. ann. symp. on found. of comp. sc., IEEE. pp. 454–463.
  * Epstein et al. (2011) Epstein, C., Carlsson, G., Edelsbrunner, H., 2011.  Topological data analysis.  Inverse Problems 27, 120201.
  * Fan et al. (2022) Fan, C.M., Liu, T.J., Liu, K.H., 2022.  Sunet: Swin transformer unet for image denoising, in: 2022 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2333–2337.  doi:10.1109/ISCAS48785.2022.9937486.
  * Foi et al. (2007) Foi, A., Katkovnik, V., Egiazarian, K., 2007.  Pointwise shape-adaptive dct for high-quality denoising and deblocking of grayscale and color images.  IEEE Trans. on Image Proc. 16, 1395–1411.
  * Gurrola-Ramos et al. (2021) Gurrola-Ramos, J., Dalmau, O., Alarcón, T.E., 2021.  A residual dense u-net neural network for image denoising.  IEEE Access 9, 31742–31754.  doi:10.1109/ACCESS.2021.3061062.
  * Hill et al. (2015) Hill, P., Anantrasirichai, N., Achim, A., Al-Mualla, M., Bull, D., 2015.  Undecimated dual-tree complex wavelet transforms.  Signal Processing: Image Communication 35, 61–70.  URL: https://www.sciencedirect.com/science/article/pii/S0923596515000715, doi:https://doi.org/10.1016/j.image.2015.04.010.
  * Johnson et al. (2016) Johnson, J., Alahi, A., Fei Fei, L., 2016.  Perceptual losses for real-time style transfer and super-resolution.  CoRR abs/1603.08155.  URL: http://arxiv.org/abs/1603.08155, arXiv:1603.08155.
  * Krull et al. (2019) Krull, A., Buchholz, T.O., Jug, F., 2019.  Noise2void-learning denoising from single noisy images, in: CVPR proc., pp. 2129--2137.
  * Lehtinen et al. (2018) Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., Aila, T., 2018.  Noise2noise: Learning image restoration without clean data.  arXiv preprint arXiv:1803.04189 .
  * Mairal et al. (2009) Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A., 2009.  Non-local sparse models for image restoration, in: CVPR proc., IEEE. pp. 2272--2279.
  * Malyugina et al. (2022a) Malyugina, A., Anantrasirichai, N., Bull, D., 2022a.  Bvi-lowlight.  URL: https://dx.doi.org/10.21227/zp7a-0683, doi:10.21227/zp7a-0683.
  * Malyugina et al. (2022b) Malyugina, A., Anantrasirichai, N., Bull, D., 2022b.  A topological loss function: Image denoising on a low-light dataset.  URL: https://arxiv.org/abs/2208.04573, doi:10.48550/ARXIV.2208.04573.
  * Malyugina et al. (2023) Malyugina, A., Anantrasirichai, N., Bull, D., 2023.  A topological loss function for image denoising on a new BVI-lowlight dataset.  Signal Processing , 109081URL: https://www.sciencedirect.com/science/article/pii/S016516842300155X, doi:https://doi.org/10.1016/j.sigpro.2023.109081.
  * Mihcak et al. (1999) Mihcak, M.K., Kozintsev, I., Ramchandran, K., Moulin, P., 1999.  Low-complexity image denoising based on statistical modeling of wavelet coefficients.  IEEE Signal Processing Letters 6, 300--303.
  * Ronneberger et al. (2015) Ronneberger, O., Fischer, P., Brox, T., 2015.  U-net: Convolutional networks for biomedical image segmentation, in: MICCAI, Springer. pp. 234--241.
  * Schmidt (1907) Schmidt, E., 1907.  Zur theorie der linearen und nichtlinearen integralgleichungen.  Mathematische Annalen 63, 433--476.
  * Shui (2009) Shui, P.L., 2009.  Image denoising using 2-d separable oversampled dft modulated filter banks.  IET Image processing 3, 163--173.
  * Simoncelli and Adelson (1996) Simoncelli, E.P., Adelson, E.H., 1996.  Noise removal via bayesian wavelet coring, in: Proc. 3rd IEEE Intern. Conf. on Image Proc.
  * Takeda et al. (2006) Takeda, H., Farsiu, S., Milanfar, P., et al., 2006.  Kernel regression for image processing and reconstruction .
  * Tronicke and Böniger (2013) Tronicke, J., Böniger, U., 2013.  Steering kernel regression: An adaptive denoising tool to process gpr data, in: Advanced Ground Penetrating Radar (IWAGPR), 2013 7th International Workshop on, IEEE. pp. 1--4.
  * Ulyanov et al. (2018) Ulyanov, D., Vedaldi, A., Lempitsky, V., 2018.  Deep image prior, in: CVPR proc., pp. 9446--9454.
  * Wiener (1949) Wiener, N., 1949.  Extrapolation, interpolation and smoothing of stationary time series with engineering applications .
  * Yu and Sapiro (2011) Yu, G., Sapiro, G., 2011.  DCT image denoising: a simple and effective image denoising algorithm.  Image Processing On Line 1, 292--296.
  * Zhang et al. (2017) Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L., 2017.  Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising.  IEEE Transactions on Image Processing 26, 3142--3155.
  * Zhang et al. (2018) Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O., 2018.  The unreasonable effectiveness of deep features as a perceptual metric, in: CVPR.
  * Zomorodian and Carlsson (2005) Zomorodian, A., Carlsson, G., 2005.  Computing persistent homology.  Disc. & Comp. Geometry 33, 249--274.