# Gap-closing Matters: Perceptual Quality Evaluation and Optimization of Low-
Light Image Enhancement

Baoliang Chen*, Lingyu Zhu*, Hanwei Zhu, Wenhan Yang, Member, IEEE, Linqi
Song, Senior Member, IEEE, and Shiqi Wang, Senior Member, IEEE *The authors
contributed equally to this work. Baoliang Chen, Lingyu Zhu, Hanwei Zhu, and
Linqi Song are with the Department of Computer Science, City University of
Hong Kong, Hong Kong, China. (e-mail: {blchen6-c, lingyzhu-c,
<EMAIL_ADDRESS>linqi.song@cityu.edu.hk).
Shiqi Wang is with the Department of Computer Science, City University of Hong
Kong, Hong Kong, China, and also with the Shenzhen Research Institute, City
University of Hong Kong, Shenzhen, China (e-mail: shiqwang@cityu.edu.hk).
Wenhan Yang is with Peng Cheng Laboratory, Shenzhen, China. (e-mail:
<EMAIL_ADDRESS>
Corresponding author: Shiqi Wang.

###### Abstract

There is a growing consensus in the research community that the optimization
of low-light image enhancement approaches should be guided by the visual
quality perceived by end users. Despite the substantial efforts invested in
the design of low-light enhancement algorithms, there has been comparatively
limited focus on assessing subjective and objective quality systematically. To
mitigate this gap and provide a clear path towards optimizing low-light image
enhancement for better visual quality, we propose a gap-closing framework. In
particular, our gap-closing framework starts with the creation of a large-
scale dataset for Subjective QUality Assessment of REconstructed LOw-Light
Images (SQUARE-LOL). This database serves as the foundation for studying the
quality of enhanced images and conducting a comprehensive subjective user
study. Subsequently, we propose an objective quality assessment measure that
plays a critical role in bridging the gap between visual quality and
enhancement. Finally, we demonstrate that our proposed objective quality
measure can be incorporated into the process of optimizing the learning of the
enhancement model toward perceptual optimality. We validate the effectiveness
of our proposed framework through both the accuracy of quality prediction and
the perceptual quality of image enhancement. Our database and code will be
made publicly available to facilitate further research in this area.

###### Index Terms:

Low-light image enhancement, blind image quality assessment, and optimization.

##  I Introduction

The prevalence of image acquisition devices has led to an increased demand for
low-light image enhancement techniques that can improve the quality of
photographs taken under inadequate lighting conditions. As a result of
unbalanced lighting, low-light enhancement algorithms aim to mitigate the
effects on visibility, color distortion, and excessive noise caused by
unbalanced lighting. Early efforts focused on reconstructing images taken in
low-light conditions to resemble those taken under normal illumination,
including approaches based on Histogram Equalization (HE) [1, 2, 3, 4] and
Retinex theory [5, 6, 7, 8]. However, simply brightening the image without
incorporating additional image priors can result in amplified noise. Recently,
the deep learning-based methods [9, 10, 11] have achieved significant
breakthroughs. These approaches utilize extensive databases containing pairs
of low and high-quality images to train enhancement mappings in a fully-
supervised fashion.

Optimizing fully-supervised low-light enhancement methods can be challenging
due to the lack of guidance from perceptually-calibrated quality models.
Although it has been widely accepted that incorporating advanced quality
measures in optimization might lead to more visually pleasant results [12,
13], proven quality measures that behave consistently with the human
perception are still largely lacking. Consequently, the quality model may not
consistently correlate with the perceptual quality of the enhanced images,
leading to unanticipated distortions such as biased color and residual noise.
To address this issue, researchers have attempted to learn low-light
enhancement models without paired supervision [14, 15]. This approach involves
training the model using only low or normal-light images without requiring
paired data. To enable this, Generative Adversarial Network (GAN) [16] is used
to facilitate style transfer from low-light images to natural lighting
conditions. Despite the potential of these methods to utilize the scene
statistics of natural images, it remains uncertain whether the learned
statistics accurately reflect human perception in the absence of explicit
guidance. Moreover, without the paired supervision, it is very challenging to
recover the fine details and remove the sensor noise due to the weak
supervision.

TABLE I: Comparisons of the IQA databases. Database |  $\\#$ of Ref. Images/Patches | Quality Degradation |  $\\#$ of Distortion Types |  $\\#$ of Images Patches | Testing Methodology
---|---|---|---|---|---
LIVE [17] | 29 | Synthetic distortions | 5 | 797 | DMOS
CSIQ [18] | 30 | Synthetic distortions | 29 | 866 | DMOS
TID2013 [19] | 25 | Synthetic distortions | 25 | 3,000 | MOS
BAPPS [12] | 187.7k | Synthetic distortions + CNN alg. outputs | 425 | 375.4k | 2 Alternative Choices
PieAPP [20] | 200 | Synthetic distortions + CNN alg. outputs | 75 | 20,280 | Probability of Preference
EHNQ [21] | 100 | alg. outputs (traditional) | 15 | 1,500 | MOS
Ours | 290 |  Low-light enhancement alg. outputs (traditional and CNN) | 10 | 2,900 | 2 Alternative Choices

Therefore, despite the demonstrated success, there are still some fundamental
issues in low light enhancement regarding the algorithm evaluation and
optimization for human vision. Specifically, the comparison, evaluation, and
optimization of the perceptual quality of enhanced images become crucial.
Though Chen et al. [22] have attempted to optimize the enhancement method by
an objective quality model with a loop game manner, it suffers from unreliable
enhancement and difficulty in optimization due to the unreliable pseudo
labels. In this paper, we address this issue by bridging the gap between
quality assessment and enhancement sequentially. In particular, we contribute
a novel dedicated database with reliable subjective ratings of diverse images
and various low-light image enhancement models, highlighting the potential
challenges faced by competing quality assessment methods in evaluating the
quality of enhanced images. Subsequently, we take a further step to develop a
learning-based quality assessment measure in an effort to assess the
perceptual visual quality of the enhanced images from a variety of enhancement
models automatically. Finally, we optimize the proven quality measure that
achieves a high correlation with human perception on the baseline low-light
enhancement models. Comprehensive experimental results indicate that the
proposed framework effectively bridges the gap, resulting in a significant
improvement in quality assessment accuracy and ensuring superior quality of
the enhanced images. The primary contributions of this paper can be summarized
as follows,

  * •

We establish a large-scale database named SQUARE-LOL (Subjective Quality
Assessment of Reconstructed Low-Light Image), which consists of diverse image
contents and different enhancement methods. The database contains 2,900 images
and pairwise subjective testing labels for the 13,050 pairs. It provides a
solid foundation and a reliable way for gauging and optimizing the visual
quality of enhanced images acquired under low-light conditions.

  * •

We develop a Blind Image Quality Assessment (BIQA) model, specifically
designed for low-light image enhancement applications. The proposed
Illumination Aware and Content Adaptive (IACA) quality assessment model is
grounded on the exploitation of deep feature representations that govern the
visual quality, delivering state-of-the-art performance on the proposed
database.

  * •

We propose an optimization approach for image enhancement tasks based on the
IACA model, which bridges the gap between quality evaluation and optimization
in a principled way. By embedding the proposed BIQA measure, the proposed
scheme outperforms the vanilla version in terms of visual quality, further
demonstrating the feasibility and effectiveness of this framework.

##  II Related Works

###  II-A Image Quality Assessment Databases

To evaluate IQA methods, databases with human perception scores need to be
constructed. Some widely used IQA databases include LIVE [17], TID2013 [19],
CSIQ [18], and Waterloo Exploration Database [23]. These databases have been
created by simulating multiple distortions to varying degrees. However,
synthetic distorted images cannot accurately reflect the realistic distortions
that occur during image acquisition, processing, or storage. Besides, there
has been a growing interest in quality evaluation for images in the wild, and
several typical image databases have been created. These include LIVE
Challenge [24], KonIQ-10k [25], and SPAQ [26]. Additionally, the proliferation
of AI-driven optimization algorithms, such as super-resolution, deblurring,
denoising, and compression, has introduced complex artifacts and opened new
avenues for image quality assessment. Correspondingly, related databases have
been constructed to investigate perceptual similarities [12, 20]. Notably,
Perceptual Image-Error Assessment through Pairwise Preference (PieAPP) [20]
collects user judgments in a pairwise manner for perceptual preferences, thus
providing more reliable scores.

In addition to the above-mentioned databases, other databases have been
created to assess the quality of images under specific conditions. For
example, in [27], the quality assessment of enhanced images from hazy,
underwater, and low-light scenarios was conducted, introducing different types
of corruption. The enhanced night-time image quality dataset [21] contains a
relatively smaller number of enhanced images. Table I presents a detailed
comparison of databases that rely on synthetic and Convolutional Neural
Networks (CNN) to generate images.

###  II-B Blind Image Quality Assessment Models

In order to evaluate the degradation of naturalness in images, BIQA methods
rely on natural scene statistics (NSS). A number of NSS features, such as
local spatial normalized luminance coefficients, have been extensively
investigated by researchers such as Mittal et al. [28]. The free energy theory
has also been explored for image quality assessment [29, 30]. To extract more
discriminative features, researchers have utilized techniques like
constructing pseudo-reference images [31] and singular value decomposition
(SVD) [32]. Gu et al. [33] applied statistical naturalness, information
entropy, and structural preservation to estimate the quality of tone-mapping
images. In recent years, deep learning-driven BIQA techniques have exhibited
better performance when compared to conventional approaches. Zhang et al. [34]
utilized the deep bilinear model to build a robust BIQA, achieving promising
results for quality assessment of synthetic and realistic distortions. Talebi
et al. developed the Neural IMage Assessment (NIMA) model [35], which predicts
the distribution of quality ratings on the AVA database [36] . Su et al. [37]
aggregated the proposed local and global feature representation to learn a
content-aware module, and an adaptive multi-scale hyper-network was
subsequently proposed to predict image quality score. Zhu et al. [38] proposed
a novel deep meta-learning-based IQA model that can be easily adapted to
unknown distortions by leveraging meta-knowledge transferred by humans. Bosse
et al. [39] presented an end-to-end method that can simultaneously handle
full-reference and no-reference IQA tasks.

Figure 1: Sampled enhanced low-light images in SQUARE-LOL database. The
corresponding reference image of each scene is also provided in the first
column. The images can be zoomed in to a higher magnification for a closer
examination.

###  II-C Low-light Image Enhancement Methods

Low-light image enhancement has been extensively studied to improve the
visibility of hidden information. Some pioneering works were developed based
on physical assumptions and priors, such as Histogram Equalization (HE) [1, 2,
3, 4] and Retinex-based methods [7, 40, 41]. HE is a simple and effective
method that stretches the valid luminance range of given low-light images.
Differential histogram equalization (DHE) [42] was a variational method that
improves visual quality. However, HE lacks consideration of spatial details,
resulting in undesirable illumination and noise. To overcome these
limitations, Retinex theory [43] was introduced, which decomposes images into
an illumination layer and reflectance layer. The typical Retinex-based methods
include the single-scale Retinex model (SSR) [40], Multiscale Retinex (MSR)
[7], and variational methods [lee2013adaptive]. Fu et al. [44] fused the
adjusted illumination and reflectance components to generate enhanced images,
while Guo et al. [6] adopted an initial illumination map by applying
structure-aware prior and refining it based on an Augmented Lagrangian
Multiplier (ALM) algorithm. Although traditional methods can improve image
brightness to some extent, they still have limitations, especially when
dealing with images with complex backgrounds and large noise. In recent years,
deep learning-based enhancement methods have gained increasing attention. For
example, Low-light Net (LLNet) [9] attempted to achieve contrast enhancement
and denoising simultaneously. The deep recursive band network (DRBN) proposed
by Yang et al. [45] combined fidelity learning and adversarial learning. Hao
et al. [46] proposed a novel semi-decoupled Retinex-based enhancement method.
The previous works [47, 48] introduced Near-Infrared enlightened (NIRE) images
as guidance to purify low-light images. Additionally, obtaining paired data
consisting of low-light and normal-light images is often challenging. To
overcome this limitation, EnlightenGAN, as proposed by Jiang et al. [14], was
designed based on the principles of adversarial learning to achieve optimal
perceptual quality, and Guo et al. [49] investigated the enhancement curve and
proposed zero-reference optimization for image enhancement.

##  III Database Construction

To better understand the perceptual characteristics of enhanced images and
gain insights into the quality assessment of low-light image enhancement
methods, we have created a dedicated database named SQUARE-LOL. This database
consists of 2,900 enhanced images, each with corresponding subjective quality
ratings. The number of source low-level images is 290, and each image is
enhanced by ten classical or state-of-the-art low-level image enhancement
methods, including traditional methods and deep learning-based methods. This
large-scale database allows us to reveal that a well-designed IQA model and
readily-deployable optimization framework can be developed to improve the
performance of image enhancement ultimately.

###  III-A Data Preparation

To construct our database, we utilize the publicly available LOw-Light
database (LOL) [50], which contains 689 low-light and normal-light image pairs
for training and 100 low-light and normal-light image pairs for testing.
According to diverse image content, we selected 247 and 43 low-light images
from the LOL training and testing sets, respectively. We apply ten classical
and state-of-the-art low-light image enhancement methods to enhance the
selected low-light images. The traditional methods include the Camera Response
Model (CRM) [51], Joint Enhancement and Denoising Method (JED) [52],
Multiscale Retinex (MR) [7], Simultaneous Reflectance and Illumination
Estimation (SRIE) [5], Multiple Fusion (MF) [44]. The deep learning-based
enhancement algorithms tend to introduce different distortions; as such, we
add Deep Retinex Decomposition (DRD) [50], ZeroDCE [49], EnlightenGAN [14],
Self-supervised [53], and Deep Recursive Band Network (DRBN) [45] to enhance
the selected low-light images. The resulting images contain a variety of
quality degradations. As illustrated in Fig. 1, these approaches can introduce
diverse quality impairments during the enhancement procedure, such as noise
contamination, blurring, over-exposure, under-exposure, contrast reduction,
color shift, and detail loss. Furthermore, it is worth noting that noise can
be amplified in the image, resulting in distinct artifacts that are different
from synthetic distortions. These artifacts can be introduced intentionally or
unintentionally and may include mixed distortion patterns that can have a
notable influence on the overall quality of the image. It is important to
consider these types of artifacts when evaluating the performance of different
image enhancement methods.

###  III-B Subjective Testing

The primary objective of this work is to conduct a detailed analysis of the
perceptual visual quality of enhanced results and the behavior of low-light
image enhancement algorithms. To achieve this goal, we have created a
dedicated database that focuses exclusively on pairwise preference. In other
words, each trial in the database presents two enhanced images to the subject,
who must choose the one with a more pleasing visual quality. This method is
commonly known as the Two-Alternative Forced Choice (2AFC) and is widely used
in psychophysical studies due to its high discrimination power, which leads to
more trustworthy subjective quality score.  In detail, each subject attends a
tutorial with detailed instructions before the subjective experiment. The
subjects are required to identify the image with better quality within
2$\sim$5 seconds. We use LCD monitors (resolution: $1920\times 1080$ pixels)
to perform subjective experiments. These experiments are calibrated according
to recommendations in [54]. The subjective experiments are conducted with
random display pairs, and each pair is generated with an identical low-light
image but different enhancement models. As such, the total number of pairs is
$C_{10}^{2}\times 290=13,050.$ We follow [55] and adopt the sanity check to
ensure the accuracy of the comparison after each session. In total, $N$
subjects are invited in the subjective testing, and herein $N$=30. Eventually,
for each enhanced image, we are able to collect $9\times 30=270$ votes from
the 30 subjects. In Fig. 2 and Fig. 3, we show two scenes sampled from our
SQUARE-LOL database, and the corresponding subjective testing results are
shown in Table II. From Table II, we can observe that the image enhanced by
EnlightenGAN is assigned 197 votes and acquire the highest opinion score
($197/270=0.7296$) among the ten enhancement algorithms, revealing the
superior perceptual quality.

(a) Input

(b) JED, OS: 0.1778

(c) Self-supervised ,OS: 0.3667

(d) DRD, OS: 0.3704

(e) DRBN, OS: 0.4519

(f) SRIE, OS: 0.4630

(g) MR, OS: 0.4704

(h) MF, OS: 0.5741

(i) Zero, OS: 0.6333

(j) EnlightenGAN, OS: 0.6852

(k) CRM, OS: 0.8074

(l) Reference

Figure 2:  Illustration of the enhancement results with different methods in
the SQUARE-LOL database. The corresponding reference image and low-light image
are presented in sub-figure (a) and (l), respectively. The opinion score (OS)
is provided below each sub-image.

(a) Input

(b) MR, OS: 0.1704

(c) DRD, OS: 0.2630

(d) JED, OS: 0.3593

(e) Self-supervised, OS: 0.3667

(f) DRBN, OS: 0.4926

(g) SRIE, OS: 0.5444

(h) CRM, OS: 0.6074

(i) Zero, OS: 0.6148

(j) EnlightenGAN, OS: 0.7185

(k) MF, OS: 0.8000

(l) Reference

Figure 3:  Illustration of the enhancement results with different methods in the SQUARE-LOL database. The corresponding reference image and low-light image are presented in sub-figure (a) and (l), respectively. The opinion score (OS) is provided below each sub-image. TABLE II:  Pairwise comparison results provided by 30 subjects. The reported values correspond to the number of times that the method in the row was preferred over the method in the column, indicating its relative superiority. WT denotes Winning Times, and OS denotes Opinion Score. | CRM | EnlightenGAN | JED | MF | MR | DRD | Self-supervised | DRBN | SRIE | ZeroDCE | WT | Total | OS
---|---|---|---|---|---|---|---|---|---|---|---|---|---
CRM | - | 14 | 19 | 18 | 29 | 30 | 19 | 15 | 22 | 19 | 185 | 270 | 0.6852
EnlightenGAN | 16 | - | 22 | 22 | 30 | 30 | 29 | 15 | 19 | 14 | 197 | 270 | 0.7296
JED | 11 | 8 | - | 9 | 18 | 23 | 15 | 9 | 14 | 11 | 118 | 270 | 0.4370
MF | 12 | 8 | 21 | - | 30 | 30 | 22 | 14 | 16 | 13 | 166 | 270 | 0.6148
MR | 1 | 0 | 12 | 0 | - | 30 | 11 | 11 | 5 | 0 | 70 | 270 | 0.2593
DRD | 0 | 0 | 7 | 0 | 0 | - | 2 | 9 | 2 | 2 | 22 | 270 | 0.0815
Self-supervised | 11 | 1 | 15 | 8 | 19 | 28 | - | 15 | 7 | 3 | 107 | 270 | 0.3963
DRBN | 15 | 15 | 21 | 16 | 19 | 21 | 15 | - | 2 | 11 | 135 | 270 | 0.5000
SRIE | 8 | 11 | 16 | 14 | 25 | 28 | 23 | 28 | - | 14 | 167 | 270 | 0.6185
ZeroDCE | 11 | 16 | 19 | 17 | 30 | 28 | 27 | 19 | 16 | - | 183 | 270 | 0.6778

###  III-C Data Analyses

To quantify the image quality, we assign “1” to the particular image when it
is preferred in the pairwise comparison for each trail. Given one image
$I_{i,j}$, where $i$ represents the image index and $j$ denotes the
enhancement model, in total, there are $T=N\times(M-1)$ pairwise comparisons
where $N$ and $M$ are the numbers of subjects and models, respectively.
Assuming that the image is in favor of for $V_{i,j}$ times, the ground-truth
opinion score that quantifies the quality with the particular enhancement
model is given by,

$\sigma_{i,j}=\frac{V_{i,j}}{T}.$ (1)

The distribution of $\sigma_{i,j}$ is presented in Fig. 4, indicating that our
database covers a broad spectrum of quality levels. However, it is worth
noting that there are very few enhanced images within the range of [0.8, 1.0].
This can be attributed to the lack of a universal algorithm with high
generalization capability for images with diverse content. The proposed
database, with its subjective ratings, offers valuable insights into the
development of more effective enhancement systems.

Figure 4: The distribution of opinion scores of our SQUARE-LOL database.
Figure 5:  Illustration of the proposed comprehensive scheme to bridge the gap
between quality assessment and enhancement for low-light images. This scheme
can be summarized as follows: a) The construction of the valuable SQUARE-LOL
database, which provides enhanced low-light images generated using different
enhancement (traditional and deep-learning) methods with corresponding quality
labels; b) The development of a deep learning-based IQA measure (IACA)
dedicated to enhanced images by bridging the gap from enhancement to
assessment; c) The optimization of enhancement models with the guidance of the
IACA model, filling in the gap from assessment to enhancement.

##  IV Methodology

As depicted in Fig. 5, this operation chain is bridged by the proposed IQA
model, which plays an essential role in ensuring that the final enhanced
images meet the essential requirement of perceptual quality. The approach
involves learning to enhance low-light images using a full operation chain
that transfers knowledge from subjective quality assessment to the enhancement
process. To accomplish this, we develop the IACA (Image Assessment with
Convolutional Attention) model, which is trained using the SQUARE-LOL
database. Using this IQA model, we then aim to learn how to enhance low-light
images, ultimately creating visually more pleasing results.

###  IV-A From Enhancement to Assessment

By leveraging the established SQUARE-LOL database, we propose a deep learning-
based IQA model (IACA) for low-light enhancement images. We show the
architecture of our deep network in Fig. 5. In our model, we adopt ResNeXt
[56] as the backbone for feature extraction. Given an image $I$, it is passed
through the CNN (ResNeXt - $32\times 8d$ [56]) for multi-scale feature
extraction. In particular, the ResNeXt - $32\times 8d$ is pre-trained on
ImageNet [57], and the final fully-connected layer is discarded. However, the
illumination information, which plays a critical role in the low-light
enhancement, has not been specifically considered in the pre-trained ResNeXt.
As such, the illumination modulation strategy is incorporated into our model.
Moreover, to further enhance the generalization capability, a content
adaptation module based on channel attention is employed for feature
aggregation.

####  IV-A1 Illumination Modulation

There are several reasons that account for the ignorance of illumination when
the vanilla ResNeXt model is adopted for feature extraction. First, the
ResNeXt is pre-trained for object classification, expecting to be robust and
invariant even under different illumination conditions. Second, the data used
for training in ImageNet are usually images with normal light. In view of
this, we first estimate the luminance map of input images with the Max-RGB
strategy [58], such that the highest pixel values in each channel (R, G, B) of
the input image can be extracted, which are further denoted as $I_{max}$. For
each scale, we adopt max pooling layer to pool $I_{max}$ into the luminance
map with a spatial size identical to the corresponding feature map. Following
the pooled luminance map, three convolutional layers are adopted to extract
the illumination-aware features, which are merged with the feature maps from
the original images. Following this vein, the luminance information can be
actively involved in the model learning process.

####  IV-A2 Content Adaptation

To encode the generalization capability into feature representation, we
leverage statistical pooling moments and aggregate the features in the fourth
layer $Conv4$ and the fifth layer $Conv5$ with global average pooling (mean
pooling) and standard deviation pooling (std pooling) layers. In Fig. 5, we
use the notation $f_{s1}^{mean}$, $f_{s1}^{std}$, $f_{s2}^{mean}$, and
$f_{s2}^{std}$ to represent the aggregated features from layer $Conv4$ and
$Conv5$. Specifically, $f_{s1}^{mean}$ and $f_{s1}^{std}$ denote the mean and
standard deviation of the features from $Conv4$, while $f_{s2}^{mean}$ and
$f_{s2}^{std}$ denote the mean and standard deviation of the features from
$Conv5$. Simply concatenating the pooled features $f_{s1}^{mean}$ and
$f_{s2}^{mean}$ for quality regression may not be practical. This is because
these features are highly correlated with semantic information [59, 60], which
can cause overfitting to specific scenes in the training set. Instead of
discarding $f_{s1}^{mean}$ and $f_{s2}^{mean}$, we treat them as semantically
meaningful features working as an integral part of the attention-based multi-
scale feature extraction. Specifically, we adopt four Fully-Connected (FC)
layers with the following units: 256, 64, 256, 1024, and 256, 64, 256, 2048
for $f_{s1}^{mean}$ and $f_{s2}^{mean}$, respectively. For the first three FC
layers, each FC is followed by a Rectified Linear Unit (ReLU) activation layer
[61].  For the last layer, the activation layer is Sigmoid, by which the
attention map can be estimated. Subsequently, the attention weighted
$f_{s1}^{std}$ and $f_{s2}^{std}$ are further encoded by three FC layers, then
we concatenate the encoded features and use the FC layer for quality
regression.

TABLE III: Comparisons of quality prediction performance on SQUARE-LOL. The values corresponding to the best and second-best performance are highlighted in boldface and underlined, respectively. Models | Conventional Methods | Deep-learning Based Methods | IACA (Ours)
---|---|---|---
NIQE | BRISQUE | CORNIA | HOSA | Bilinear | WaDIQaM-NR | Normal-In-Normal | Meta-IQA
SROCC | 0.276 | 0.658 | 0.625 | 0.685 | 0.792 | 0.838 | 0.850 | 0.654 | 0.875
PLCC | 0.286 | 0.649 | 0.631 | 0.694 | 0.783 | 0.841 | 0.844 | 0.668 | 0.878

###  IV-B From Assessment to Enhancement

The underlying philosophy of many low-light image enhancement technologies is
to recover images using the Maximum A Posteriori (MAP) framework. This
approach involves simultaneously maintaining fidelity (fidelity term) and the
alignment of scene statistics (prior term) [62]. By incorporating both
fidelity and prior terms, the MAP framework can effectively enhance low-light
images while preserving important scene information.  In our optimization
framework, we follow this vein and adopt the widely used full-reference IQA
model structural similarity (SSIM) index [63] for signal-level comparisons in
an effort to push the image to be closer to the ground truth in supervised
learning. More importantly, regarding the prior term, instead of adopting
hand-crafted image priors such as dark channel [64], Total Variation (TV) [65]
or Markov random field (MRF) [66], the valid IACA model in the perceptual
sense is incorporated. Thus, we can define the loss function for learning the
model in the following way,

$\mathcal{L}=\phi^{f}\left(\mathbf{X}^{r},\mathcal{N}(\mathbf{X};\mathbf{W})\right)+\lambda\phi^{q}\left(\mathcal{N}(\mathbf{X};\mathbf{W})\right),$
(2)

where the enhancement network and its parameters are denoted as $\mathcal{N}$
and $\mathbf{W}$, respectively. And the notation $\mathbf{X}$ and
$\mathbf{X}^{r}$ is used to refer to the input low-light image and its
corresponding reference image, respectively The fidelity loss term
$\phi^{f}(\cdot,\cdot)$ is based on SSIM, and the quality loss term
$\phi^{q}(\cdot)$ is given by,

$\phi^{q}(\mathcal{N}(\mathbf{X};\mathbf{W}))=\left|Q^{max}-IACA(\mathcal{N}(\mathbf{X};\mathbf{W}))\right|,$
(3)

where $Q^{max}$ denotes the maximum quality score the IACA model can achieve
on the training set. The hyper-parameter $\lambda$ in Eqn. (2) is used to
adjust the relative importance of the IACA model in the loss function.

To accurately represent the intrinsic performance of model optimization, we
employ the SRCNN [67] framework and the DRBN [45] architecture, which is
considered state-of-the-art in low-light enhancement, for model learning.
Specifically, we substitute the original loss functions utilized in these two
models with the proposed quality-aware loss function (see Eqn. (2)). For fair
comparisons, we fix the network architecture of the two models, including
kernel size, layer numbers, and weight initialization. Moreover, the baseline
is generated by the removal of the term guided by IACA in the loss function.

(a) SRCNN Ori.

(b) SRCNN Opt.

(c) DRBN Ori.

(d) DRBN Opt.

Figure 6: Visual quality comparisons of enhanced images by original baselines
(SRCNN Ori. and DRBN Ori.) and our optimized models (SRCNN Opt. and DRBN
Opt.).

##  V Experiments

###  V-A Experimental Settings

During the IACA model learning, we divide our database into a training set
(2,470 images) and a testing set (430 images) according to the content, and
this is consistent with the LOL database [50] to ensure no content
overlapping. To increase the diversity of the training data, we randomly crop
patches with dimensions of $224\times 224$ from the original images. During
optimization, we use a batch size of 32 and apply the Adam optimizer with a
learning rate of 1e-4. Training is terminated after 100 epochs. In the testing
phase, the images are tested at the original size without cropping. For the
faster convergence of our model, the “Norm-In-Norm” [68] loss is adopted for
regression. The loss function is performed not only on the final concatenated
features but also on the features in the two sub-scales, and their weights are
set to be identical. We use the Spearman Rank Order Correlation Coefficient
(SROCC) and the Pearson Linear Correlation Coefficient (PLCC) to evaluate the
IACA model.

To train the enhancement model, we adopt the default data splitting strategy
from the LOL database [50], which involves creating distinct training and
testing sets. Notably, this approach ensures that the testing sets and
training sets used in this paper do not overlap, either in terms of IACA
prediction or enhancement optimization.

As part of the training process, we perform data augmentation by randomly
cropping the images to dimensions of $256\times 256$, while the original sizes
are used during testing. A total of 200 epochs are allowed during training,
with the initial learning rate set to 1e-4. After the 200 epochs, the learning
rate is decreased by 0.5. Furthermore, we set the hyper-parameter $\lambda$ in
Eqn. (2) to be 5e-3 in SRCNN and 1e-3 in DRBN, respectively.

###  V-B Experimental Results

####  V-B1 Quality Prediction Performance Evaluation

This subsection presents a comprehensive comparison of our proposed IACA model
with state-of-the-art BIQA methods, including hand-crafted feature-based (NIQE
[69], BRISQUE [28], CORNIA [70], HOSA [71]) and deep learning-based (Bilinear
[34], WaDIQaM-NR [39], Meta-IQA [38] and Normal-In-Normal [68]) models. For
the hand-crafted feature-based methods, we directly evaluate the models on our
SQUARE-LOL database with open-source codes. Regarding the deep learning-based
methods, we follow the parameter initialization of those models and retrain
them on our database using the same training set as the proposed IACA model.

Table III clearly demonstrates that the performance of conventional BIQA
methods is significantly below the desired level. Due to the intrinsic
differences between the enhanced low-light images and the normal-light images,
the readily deployed IQA models may not suffice in the low-light enhancement
scenarios. It is also worth mentioning that “Norm-In-Norm” shares the same
backbone with ours with only the mean pooling utilized for feature
aggregation, causing the model to be less effective. Compared with the above
deep learning-based methods, we introduce the illumination modulation in the
feature extraction as well as the attention strategy, leading to a more
generalized model. The superior performance validates the effectiveness of the
proposed IACA method.

TABLE IV: Ablation study of each component in the IACA model. MS-Feat | Illumination Modulation | Content Adaptation | Std Pooling | Mean Pooling | SROCC | PLCC
---|---|---|---|---|---|---
| ✓ | ✓ |  |  | 0.864 | 0.865
✓ |  | ✓ |  |  | 0.866 | 0.871
✓ | ✓ |  | ✓ |  | 0.859 | 0.866
✓ | ✓ |  |  | ✓ | 0.861 | 0.866
✓ | ✓ | ✓ |  |  | 0.875 | 0.878
Figure 7: The predicted quality score difference between the optimization
results and the original results. For each image pair, a positive score
difference means the quality of the optimization result is better than the
original one. The SRCNN and DRBN architectures are utilized for optimization,
respectively. Figure 8: The subjective testing results for each image in the
testing set are reported. The percentage in which the subject is preferred is
provided. Left: SRCNN; Right: DRBN. Figure 9: The preference of the proposed
method and original method for each individual subject on the 100 testing
images of LOL [50]. For each image, the two enhanced images by these two
methods form a pair for comparison. Left: The SRCNN architecture is adopted
for optimization. Right: The DRBN architecture is adopted for optimization.

####  V-B2 Ablation Study of the IACA model

In order to analyze the functionalities of the various modules in our proposed
IACA model, we perform the ablation study in this subsection. The influence of
each design choice is shown in Table IV, and the best result in our experiment
is achieved with multi-scale feature extraction, illumination modulation as
well as content adaptation performed. More specifically, we first ablate the
multi-scale feature learning and use the last stage (Conv5) of ResNeXt for
training. The results reveal that features in different scales (denoted as MS-
Feat in Table IV) are more capable of handling such more complicated
distortions. Subsequently, we ablate the illumination modulation from our
original model, and significant performance degradation (SROCC from 0.875 to
0.866) can be observed. This phenomenon further demonstrates that illumination
information plays a critical role in quality perception. Moreover, we also
analyze different spatial pooling strategies by maintaining the MS-Feat and
illumination modulation. As shown in Table IV, neither the individual std-
pooling nor mean-pooling can achieve the best results. By contrast, with our
proposed content adaptation module, the content adaptive quality features can
be finally acquired, further providing evidence of the effectiveness of our
adaptation component.

####  V-B3 Validations on Enhancement Optimization

In Fig. 6, we visually compare the enhanced results of the original baseline
as well as the optimized models. Our optimization framework delivers visually
better results, while the images enhanced by the original SRCNN are darker
with lower contrast. Moreover, the DRBN tends to over-smooth images compared
with our method. In Fig. 7, we adopt our IACA model for the quality score
prediction of the optimization results and the original results, and the score
difference of each image pair is presented. From the figure, we can observe
that 94% score difference is positive for SRCNN and 86% score difference is
positive for DRBN models, verifying that most of the optimization results
achieve higher quality scores than the original ones. This phenomenon
demonstrates the effectiveness of the proposed full-chain optimization guided
by the perceptual quality of different network architectures.

Herein, our target is not to achieve the best enhancement performance. The
intended message that we are trying to convey is that optimization guided by
perceptual characteristics can successfully improve the image quality
generated by these widely adopted models. Furthermore, we conduct subjective
evaluations by comparing the images generated from original and optimized
models. We adopt a similar strategy as in the creation of SQUARE-LOL. During
each trial of the experiment, the 2AFC strategy was used to compare the
quality of the images generated by the original baseline and the proposed
models. Specifically, the subject was presented with a pair of images and
asked to select the one with better quality. In total, 12 subjects
participated in this experiment. The percentage that the proposed optimization
scheme is in favor of against the original model for each image is shown in
Fig. 8 and the overall percentages are 70 % for SRCNN and 62 % for DRBN
models, revealing the subjective quality has been improved by a large margin
with both SRCNN and DRBN architectures, demonstrating the promising capability
of model optimization from a perceptual perspective. In Fig. 9, we also report
the preference between the optimized images and the originally enhanced images
(without optimization) of each subject. We can conclude that more than 90 %
(SRCNN) and 83 % (DRBN) subjects have more votes for our optimized images when
compared with the original ones, further providing evidence that more visually
pleasant enhancement results can be obtained by our optimization scheme.

##  VI Conclusions

In this paper, we propose a principled framework for evaluating the subjective
quality of low-light images, designing quality assessment algorithms, and
optimizing low-light image enhancement methods to improve perceptual quality.
Our experimental findings demonstrate that the proposed objective quality
model IACA agrees well with the subjective opinions and reveals the potential
in perceptually optimizing the enhancement models in a data-driven fashion.
While the proposed gap-closing framework represents one of the initial efforts
in this area of research, further refinement can be achieved by establishing
the sophisticated closed-loop via quality assessment of the enhanced images
from the optimized models. Additionally, exploring the impact of varying image
content on quality perception poses novel challenges for IQA and low-light
enhancement research, thereby providing impetus for innovative future
explorations.

## References

  * [1] M. Abdullah-Al-Wadud, M. H. Kabir, M. A. A. Dewan, and O. Chae, “A dynamic histogram equalization for image contrast enhancement,” _IEEE Transactions on Consumer Electronics_ , vol. 53, no. 2, pp. 593–600, 2007.
  * [2] D. Coltuc, P. Bolon, and J.-M. Chassery, “Exact histogram specification,” _IEEE Transactions on Image Processing_ , vol. 15, no. 5, pp. 1143–1152, 2006.
  * [3] J. A. Stark, “Adaptive image contrast enhancement using generalizations of histogram equalization,” _IEEE Transactions on Image Processing_ , vol. 9, no. 5, pp. 889–896, 2000.
  * [4] T. Arici, S. Dikbas, and Y. Altunbasak, “A histogram modification framework and its application for image contrast enhancement,” _IEEE Transactions on Image Processing_ , vol. 18, no. 9, pp. 1921–1935, 2009.
  * [5] X. Fu, D. Zeng, Y. Huang, X.-P. Zhang, and X. Ding, “A weighted variational model for simultaneous reflectance and illumination estimation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 2782–2790.
  * [6] X. Guo, Y. Li, and H. Ling, “LIME: Low-light image enhancement via illumination map estimation,” _IEEE Transactions on Image Processing_ , vol. 26, no. 2, pp. 982–993, 2016.
  * [7] D. J. Jobson, Z.-u. Rahman, and G. A. Woodell, “A multiscale retinex for bridging the gap between color images and the human observation of scenes,” _IEEE Transactions on Image processing_ , vol. 6, no. 7, pp. 965–976, 1997.
  * [8] S. Wang, J. Zheng, H.-M. Hu, and B. Li, “Naturalness preserved enhancement algorithm for non-uniform illumination images,” _IEEE Transactions on Image Processing_ , vol. 22, no. 9, pp. 3538–3548, 2013.
  * [9] K. G. Lore, A. Akintayo, and S. Sarkar, “Llnet: A deep autoencoder approach to natural low-light image enhancement,” _Pattern Recognition_ , vol. 61, pp. 650–662, 2017.
  * [10] W. Ren, S. Liu, L. Ma, Q. Xu, X. Xu, X. Cao, J. Du, and M.-H. Yang, “Low-light image enhancement via a deep hybrid network,” _IEEE Transactions on Image Processing_ , vol. 28, no. 9, pp. 4364–4375, 2019.
  * [11] R. Wang, Q. Zhang, C.-W. Fu, X. Shen, W.-S. Zheng, and J. Jia, “Underexposed photo enhancement using deep illumination estimation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 6849–6857.
  * [12] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 586–595.
  * [13] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in _European Conference on Computer Vision_. Springer, 2016, pp. 694–711.
  * [14] Y. Jiang, X. Gong, D. Liu, Y. Cheng, C. Fang, X. Shen, J. Yang, P. Zhou, and Z. Wang, “Enlightengan: Deep light enhancement without paired supervision,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 2340–2349, 2021.
  * [15] Z. Ni, W. Yang, S. Wang, L. Ma, and S. Kwong, “Unpaired image enhancement with quality-attention generative adversarial network,” in _Proceedings of the 28th ACM International Conference on Multimedia_ , 2020, pp. 1697–1705.
  * [16] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” _arXiv preprint arXiv:1406.2661_ , 2014.
  * [17] H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” _IEEE Transactions on Image Processing_ , vol. 15, no. 11, pp. 3440–3451, 2006.
  * [18] E. C. Larson and D. M. Chandler, “Most apparent distortion: full-reference image quality assessment and the role of strategy,” _Journal of Electronic Imaging_ , vol. 19, no. 1, p. 011006, 2010.
  * [19] N. Ponomarenko, L. Jin, O. Ieremeiev, V. Lukin, K. Egiazarian, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti _et al._ , “Image database TID2013: Peculiarities, results and perspectives,” _Signal processing: Image communication_ , vol. 30, pp. 57–77, 2015.
  * [20] E. Prashnani, H. Cai, Y. Mostofi, and P. Sen, “Pieapp: Perceptual image-error assessment through pairwise preference,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 1808–1817.
  * [21] Y. Yang, T. Xiang, S. Guo, X. Lv, H. Liu, and X. Liao, “Ehnq: Subjective and objective quality evaluation of enhanced night-time images,” _IEEE Transactions on Circuits and Systems for Video Technology_ , 2023.
  * [22] B. Chen, L. Zhu, H. Zhu, W. Yang, F. Lu, and S. Wang, “The loop game: Quality assessment and optimization for low-light image enhancement,” _arXiv preprint arXiv:2202.09738_ , 2022.
  * [23] K. Ma, Z. Duanmu, Q. Wu, Z. Wang, H. Yong, H. Li, and L. Zhang, “Waterloo exploration database: New challenges for image quality assessment models,” _IEEE Transactions on Image Processing_ , vol. 26, no. 2, pp. 1004–1016, 2016.
  * [24] D. Ghadiyaram and A. C. Bovik, “Massive online crowdsourced study of subjective and objective picture quality,” _IEEE Transactions on Image Processing_ , vol. 25, no. 1, pp. 372–387, 2015.
  * [25] V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, “Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 4041–4056, 2020.
  * [26] Y. Fang, H. Zhu, Y. Zeng, K. Ma, and Z. Wang, “Perceptual quality assessment of smartphone photography,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 3677–3686.
  * [27] Z. Chen, T. Jiang, and Y. Tian, “Quality assessment for comparing image enhancement algorithms,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2014, pp. 3003–3010.
  * [28] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” _IEEE Transactions on Image Processing_ , vol. 21, no. 12, pp. 4695–4708, 2012.
  * [29] K. Gu, G. Zhai, X. Yang, and W. Zhang, “Using free energy principle for blind image quality assessment,” _IEEE Transactions on Multimedia_ , vol. 17, no. 1, pp. 50–63, 2014.
  * [30] Y. Liu, G. Zhai, K. Gu, X. Liu, D. Zhao, and W. Gao, “Reduced-reference image quality assessment in free-energy principle and sparse representation,” _IEEE Transactions on Multimedia_ , vol. 20, no. 2, pp. 379–391, 2017.
  * [31] X. Min, K. Gu, G. Zhai, J. Liu, X. Yang, and C. W. Chen, “Blind quality assessment based on pseudo-reference image,” _IEEE Transactions on Multimedia_ , vol. 20, no. 8, pp. 2049–2062, 2017.
  * [32] B. Sadou, A. Lahoulou, T. Bouden, A. R. Avila, T. H. Falk, Z. Akhtar _et al._ , “Blind image quality assessment using singular value decomposition based dominant eigenvectors for feature selection,” in _Proceedings of the 5th International Conference on Signal and Image Processing (SIPRO’19), Toronto, ON, Canada_ , 2019, pp. 13–14.
  * [33] K. Gu, S. Wang, G. Zhai, S. Ma, X. Yang, W. Lin, W. Zhang, and W. Gao, “Blind quality assessment of tone-mapped images via analysis of information, naturalness, and structure,” _IEEE Transactions on Multimedia_ , vol. 18, no. 3, pp. 432–443, 2016.
  * [34] W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang, _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 30, no. 1, pp. 36–47, 2018.
  * [35] H. Talebi and P. Milanfar, “Nima: Neural image assessment,” _IEEE Transactions on Image Processing_ , vol. 27, no. 8, pp. 3998–4011, 2018.
  * [36] N. Murray, L. Marchesotti, and F. Perronnin, “Ava: A large-scale database for aesthetic visual analysis,” in _2012 IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 2012, pp. 2408–2415.
  * [37] S. Su, Q. Yan, Y. Zhu, C. Zhang, X. Ge, J. Sun, and Y. Zhang, “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 3667–3676.
  * [38] H. Zhu, L. Li, J. Wu, W. Dong, and G. Shi, “Metaiqa: deep meta-learning for no-reference image quality assessment,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 14 143–14 152.
  * [39] S. Bosse, D. Maniry, K.-R. Müller, T. Wiegand, and W. Samek, “Deep neural networks for no-reference and full-reference image quality assessment,” _IEEE Transactions on Image Processing_ , vol. 27, no. 1, pp. 206–219, 2017.
  * [40] D. J. Jobson, Z.-u. Rahman, and G. A. Woodell, “Properties and performance of a center/surround retinex,” _IEEE transactions on image processing_ , vol. 6, no. 3, pp. 451–462, 1997.
  * [41] E. H. Land, “The retinex theory of color vision,” _Scientific american_ , vol. 237, no. 6, pp. 108–129, 1977.
  * [42] K. Nakai, Y. Hoshi, and A. Taguchi, “Color image contrast enhacement method based on differential intensity/saturation gray-levels histograms,” in _2013 International Symposium on Intelligent Signal Processing and Communication Systems_. IEEE, 2013, pp. 445–449.
  * [43] E. H. Land, “The retinex,” _American Scientist_ , vol. 52, no. 2, pp. 247–264, 1964.
  * [44] X. Fu, D. Zeng, Y. Huang, Y. Liao, X. Ding, and J. Paisley, “A fusion-based enhancing method for weakly illuminated images,” _Signal Processing_ , vol. 129, pp. 82–96, 2016.
  * [45] W. Yang, S. Wang, Y. Fang, Y. Wang, and J. Liu, “From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 3063–3072.
  * [46] S. Hao, X. Han, Y. Guo, X. Xu, and M. Wang, “Low-light image enhancement with semi-decoupled decomposition,” _IEEE transactions on multimedia_ , vol. 22, no. 12, pp. 3025–3038, 2020.
  * [47] Z. Li, H.-M. Hu, W. Zhang, S. Pu, and B. Li, “Spectrum characteristics preserved visible and near-infrared image fusion algorithm,” _IEEE Transactions on Multimedia_ , vol. 23, pp. 306–319, 2020.
  * [48] R. Wan, B. Shi, W. Yang, B. Wen, L.-Y. Duan, and A. C. Kot, “Purifying low-light images via near-infrared enlightened image,” _IEEE Transactions on Multimedia_ , 2022.
  * [49] C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong, and R. Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 1780–1789.
  * [50] C. Wei, W. Wang, W. Yang, and J. Liu, “Deep retinex decomposition for low-light enhancement,” _arXiv preprint arXiv:1808.04560_ , 2018.
  * [51] Z. Ying, G. Li, Y. Ren, R. Wang, and W. Wang, “A new low-light image enhancement algorithm using camera response model,” in _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , 2017, pp. 3015–3022.
  * [52] X. Ren, M. Li, W.-H. Cheng, and J. Liu, “Joint enhancement and denoising method via sequential decomposition,” in _Proceedings of the IEEE International Symposium on Circuits and Systems_ , 2018, pp. 1–5.
  * [53] Y. Zhang, X. Di, B. Zhang, and C. Wang, “Self-supervised image enhancement network: Training with low light images only,” _arXiv e-prints_ , pp. arXiv–2002, 2020.
  * [54] R. I.-R. BT, “Methodology for the subjective assessment of the quality of television pictures,” _International Telecommunication Union_ , 2002.
  * [55] W.-S. Lai, J.-B. Huang, Z. Hu, N. Ahuja, and M.-H. Yang, “A comparative study for single image blind deblurring,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 1701–1709.
  * [56] S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 1492–1500.
  * [57] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.
  * [58] M. A. Hussain and A. S. Akbari, “Max-RGB based colour constancy using the sub-blocks of the image,” in _Proceedings of the IEEE International Conference on Developments in eSystems Engineering_ , 2016, pp. 289–294.
  * [59] W. Wan, J. Chen, T. Li, Y. Huang, J. Tian, C. Yu, and Y. Xue, “Information entropy based feature pooling for convolutional neural networks,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 3405–3414.
  * [60] B. Chen, L. Zhu, G. Li, F. Lu, H. Fan, and S. Wang, “Learning generalized spatial-temporal deep feature representation for no-reference video quality assessment,” _IEEE Transactions on Circuits and Systems for Video Technology_ , 2021.
  * [61] X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in _Proceedings of the International Conference on Artificial Intelligence and Statistics_ , 2011, pp. 315–323.
  * [62] M. R. Banham and A. K. Katsaggelos, “Digital image restoration,” _IEEE Signal Processing Magazine_ , vol. 14, no. 2, pp. 24–41, 1997.
  * [63] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE Transactions on Image Processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * [64] K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” _IEEE transactions on Pattern Analysis and Machine Intelligence_ , vol. 33, no. 12, pp. 2341–2353, 2010.
  * [65] B. Chen, C. Jung, and Z. Zhang, “Variational fusion of time-of-flight and stereo data for depth estimation using edge-selective joint filtering,” _IEEE Transactions on Multimedia_ , vol. 20, no. 11, pp. 2882–2890, 2018.
  * [66] S. Z. Li, _Markov random field modeling in image analysis_. Springer Science & Business Media, 2009.
  * [67] C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 38, no. 2, pp. 295–307, 2015.
  * [68] D. Li, T. Jiang, and M. Jiang, “Norm-in-norm loss with faster convergence and better performance for image quality assessment,” in _Proceedings of the ACM International Conference on Multimedia_ , 2020, pp. 789–797.
  * [69] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,” _IEEE Signal Processing Letters_ , vol. 20, no. 3, pp. 209–212, 2012.
  * [70] P. Ye, J. Kumar, L. Kang, and D. Doermann, “Unsupervised feature learning framework for no-reference image quality assessment,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 2012, pp. 1098–1105.
  * [71] J. Xu, P. Ye, Q. Li, H. Du, Y. Liu, and D. Doermann, “Blind image quality assessment based on high order statistics aggregation,” _IEEE Transactions on Image Processing_ , vol. 25, no. 9, pp. 4444–4457, 2016.

|  Baoliang Chen received his B.S. degree in Electronic Information Science
and Technology from Hefei University of Technology, Hefei, China, in 2015, his
M.S. degree in Intelligent Information Processing from Xidian University,
Xian, China, in 2018, and his Ph.D. degree in computer science from the City
University of Hong Kong, Hong Kong, in 2022. He is currently a postdoctoral
researcher with the Department of Computer Science, City University of Hong
Kong. His research interests include image/video quality assessment and
transfer learning.
---|---
|  Lingyu Zhu received the B.S. degree from the Wuhan University of Technology
in 2018 and the master’s degree from Hong Kong University of Science and
Technology in 2019. He is currently pursuing a Ph.D. degree at the City
University of Hong Kong. His research interests include image/video quality
assessment, image/ video processing, and deep learning.
---|---
|  Hanwei Zhu received the B.E and M.S. degrees from the Jiangxi University of
Finance and Economics, Nanchang, China, in 2017 and 2020, respectively. He is
currently pursuing a Ph.D. degree in the Department of Computer Science, City
University of Hong Kong. His research interest includes perceptual image
processing and computational photography.
---|---
|  Shiqi Wang (Senior Member, IEEE) received the B.S. degree in computer
science from the Harbin Institute of Technology in 2008 and the Ph.D. degree
in computer application technology from Peking University in 2014. From 2014
to 2016, he was a Post-Doctoral Fellow with the Department of Electrical and
Computer Engineering, University of Waterloo, Waterloo, ON, Canada. From 2016
to 2017, he was a Research Fellow with the Rapid-Rich Object Search
Laboratory, Nanyang Technological University, Singapore. He is currently an
Associate Professor with the Department of Computer Science, City University
of Hong Kong. He has proposed more than 50 technical proposals to ISO/MPEG,
ITU-T, and AVS standards, and authored or coauthored more than 200 refereed
journal articles/conference papers. His research interests include video
compression, image/video quality assessment, and image/video search and
analysis. He received the Best Paper Award from IEEE VCIP 2019, ICME 2019,
IEEE Multimedia 2018, and PCM 2017. His coauthored article received the Best
Student Paper Award in the IEEE ICIP 2018. He was a recipient of the 2021 IEEE
Multimedia Rising Star Award in ICME 2021. He serves as an Associate Editor
for IEEE Transactions on Circuits and Systems for Video Technology.
---|---


# Precision tests of the nonlinear mode coupling of anisotropic flow
via high-energy collisions of isobars

Jiangyong Jia<EMAIL_ADDRESS>Department of Chemistry, Stony
Brook University, Stony Brook, NY 11794, USA  Physics Department, Brookhaven
National Laboratory, Upton, NY 11976, USA  Giuliano Giacalone
<EMAIL_ADDRESS>Institut für Theoretische Physik, Universität
Heidelberg, Philosophenweg 16, 69120 Heidelberg, Germany  Chunjian Zhang
Department of Chemistry, Stony Brook University, Stony Brook, NY 11794, USA

###### Abstract

Valuable information on the dynamics of expanding fluids can be inferred from
the response of such systems to perturbations in their initial geometry. We
apply this technique in high-energy 96Ru+96Ru and 96Zr+96Zr collisions to
scrutinize the expansion dynamics of the quark-gluon plasma, where the initial
geometry perturbations are sourced by the differences in deformations and
radial profiles between 96Ru and 96Zr, and the collective response is captured
by the change in anisotropic flow $V_{n}$ between the two collision systems.
Using a transport model, we analyze how the nonlinear coupling between lower-
order flow harmonics $V_{2}$ and $V_{3}$ to the higher-order flow harmonics
$V_{4}$ and $V_{5}$, expected to scale as $V_{4\mathrm{NL}}=\chi_{4}V_{2}^{2}$
and $V_{5\mathrm{NL}}=\chi_{5}V_{2}V_{3}$, gets modified as one moves from
96Ru+96Ru to 96Zr+96Zr systems. We find that these scaling relations are valid
to a high precision: variations of order 20% in $V_{4\mathrm{NL}}$ and
$V_{5\mathrm{NL}}$ due to differences in quadrupole deformation, octupole
deformation and nuclear skin modify $\chi_{4}$ and $\chi_{5}$ by about 1–2%.
Percent-level deviations are however larger than the expected experimental
uncertainties, and could be measured. Therefore, collisions of isobars with
different nuclear structure are an unique tool to isolate subtle nonlinear
effects in the expansion of the quark gluon plasma that would be otherwise
impossible to access in a single collision system.

###### pacs:

25.75.Gz, 25.75.Ld, 25.75.-1

## Introduction.

The space-time evolution of the quark-gluon plasma (QGP) produced in high-
energy nuclear collisions is driven by pressure-gradient forces that convert
spatial deformations in the initial geometry into momentum-space deformations
that are captured by azimuthal correlations among final-state hadrons
Ollitrault (1992); Alver and Roland (2010); Teaney and Yan (2011). These
correlations emerge in the Fourier spectrum of the azimuthal particle
distribution, $p(\phi)\propto\sum_{n=-\infty}^{\infty}V_{n}e^{in\phi}$ Heinz
and Snellings (2013). The dissipative effects in the expansion of QGP quickly
damp the $V_{n}$ spectrum as one goes higher in $n$ Teaney and Yan (2012). The
dominant coefficients, reflecting genuine deformations of the QGP geometry,
are elliptic flow, $V_{2}$, and triangular flow, $V_{3}$. Damped higher
harmonics are strongly affected by their couplings to $V_{2}$ and $V_{3}$.
Quadrangular flow, $V_{4}$, for instance, receives a large contribution from
its coupling to $V_{2}$, which scales as $V_{2}^{2}$.

Over the past decade, many theoretical Gardim  _et al._ (2012); Teaney and Yan
(2012, 2014); Gardim  _et al._ (2015); Yan and Ollitrault (2015); Qian  _et
al._ (2016); Giacalone  _et al._ (2016); Qian and Heinz (2016); Giacalone  _et
al._ (2017); Qian  _et al._ (2017); Giacalone  _et al._ (2018); Magdy (2022a);
Zhao  _et al._ (2022) and experimental Aad  _et al._ (2014, 2015); Acharya
_et al._ (2017, 2020); Adam  _et al._ (2020) investigations have clarified the
mechanism by which $V_{2}$ and $V_{3}$ source harmonics of higher order. The
resulting picture is that $V_{n,n>3}$ contains a contribution reflecting a
genuine $n^{\mathrm{th}}$-order deformation, denoted as $V_{n\mathrm{L}}$,
plus a contribution from couplings to $V_{2}$ and $V_{3}$, denoted as
$V_{n\mathrm{NL}}$. For $V_{4}$ and $V_{5}$, the decompositions are Yan and
Ollitrault (2015),

$\displaystyle V_{4}=V_{4\mathrm{L}}+\chi_{4}\left(V_{2}\right)^{2}\;,\hskip
20.0ptV_{5}=V_{5\mathrm{L}}+\chi_{5}V_{2}V_{3}\;,$ (1)

from which we define the nonlinear parts as
$V_{4\mathrm{NL}}=\chi_{4}\left(V_{2}\right)^{2}$,
$V_{5\mathrm{NL}}=\chi_{5}V_{2}V_{3}$. The coefficient $\chi_{4}$, for
example, determines the coupling strength between elliptic and quadrangular
flow. Measurements of these couplings are of great interest as they are
dynamically generated during the QGP expansion. They probe transport and
hadronization properties of the QGP, whose characterization is one of the main
goals of high-energy heavy-ion collision experiments.

This paper establishes a new method to probe the nonlinear coupling of flow
harmonics in the QGP expansion. We look at two collision systems at identical
multiplicities but with small differences in their initial geometries. We
study how the QGP responds to such differences in the dynamically-generated
couplings between $V_{2}$, $V_{3}$ and higher-order harmonics $V_{4}$,
$V_{5}$. We argue that the cleanest method to achieve this is to exploit
isobars. These are nuclei with identical mass number but different
deformations and radial profiles, providing the desired differences in
collision geometries. We focus on 96Ru+96Ru and 96Zr+96Zr collisions, for
which high-precision experimental data are available from the Relativistic
Heavy Ion Collider (RHIC) Abdallah  _et al._ (2022). Using a transport model,
we predict $\chi_{n}$ in these two systems, and systematically address the
impact of nuclear structure on such observables. The $\chi_{n}$ show very
small differences between 96Ru+96Ru and 96Zr+96Zr, much smaller than the
calculated differences for corresponding nonlinear terms $V_{n\mathrm{NL}}$.
These small differences are ascribed to subleading nonlinear couplings, so far
undetected in heavy-ion collisions, which are within the reach of the existing
isobar data. We conclude that the different structure of two isobars can be
exploited as a precision tool to access subtle nonlinear phenomena in the QGP
expansion.

## Model, observables, goal.

We study 96Ru+96Ru and 96Zr+96Zr collisions within the Glauber Monte Carlo
model Miller  _et al._ (2007). The colliding ions are treated as collections
of nucleons that are randomly distributed in each event according to a Woods-
Saxon density,

$\displaystyle\rho(r,\theta,\phi)$
$\displaystyle\propto\frac{1}{1+e^{[r-R_{0}\left(1+\beta_{2}Y_{2}^{0}(\theta,\phi)+\beta_{3}Y_{3}^{0}(\theta,\phi)\right)]/a_{0}}},$
(2)

with four nuclear structure parameters: nuclear diffusivity $a_{0}$, half-
width radius $R_{0}$, quadrupole deformation $\beta_{2}$, and octupole
deformation $\beta_{3}$. Each collision has a number of nucleons that
participate in the interaction, which give rise to the QGP. The evolution of
QGP is modeled via the multi-phase transport code (AMPT) Lin  _et al._ (2005),
version v2.26t5 in string-melting mode, which generates the hadrons in the
final state. Observables are calculated using hadrons with
$0.2<p_{\mathrm{T}}<2$ GeV for events sorted in intervals of
$N_{\mathrm{ch}}$: the charged hadron multiplicity for $p_{\mathrm{T}}>0.1$
GeV and $|\eta|<0.5$, similar to the experiment Abdallah  _et al._ (2022). The
observables of interest involve products of flow vectors
$V_{n}=v_{n}e^{in\Psi_{n}}$ averaged over events with the same
$N_{\mathrm{ch}}$, where $\Psi_{n}$ is the orientation of the harmonic and
$v_{n}\equiv|V_{n}|$ is its amplitude. Such averages are computed within the
framework of multi-particle correlations Bilandzic  _et al._ (2011, 2014). In
particular, we use a sub-event method Jia  _et al._ (2017) by correlating
particles in the pseudorapidity window $0<\eta<2$ with those having
$-2<\eta<0$ to reduce the impact of non-collective (non-flow) correlations.

Our focus is on ratios of observables taken between 96Ru+96Ru and 96Zr+96Zr
collisions. We study the dynamical response of the QGP to small differences in
the initial geometry induced by the nuclear structure. The $V_{2}$ and $V_{3}$
emerge as a response to the initial spatial deformations (or eccentricities)
$\mathcal{E}_{2}$ and $\mathcal{E}_{3}$ Teaney and Yan (2011), respectively.
The response coefficient $K_{n}=V_{n}/\mathcal{E}_{n}$ is a probe of the
medium properties, and has been investigated in model studies of isobar
collisions Zhang and Jia (2022); Jia and Zhang (2021); Nijs and van der Schee
(2021). Here, we look instead at the coupling between harmonics represented by
$\chi_{n}$. The advantage of such quantities is that they do not directly
depend on eccentricities (Eq. (3)), and can be extracted directly from
experimental $V_{n}$ data.

To this end, we continue from the decompositions in Eq. (1). Defining
$V_{n\mathrm{L}}$ as the vector that is uncorrelated with $V_{n\mathrm{NL}}$
in the considered event class Yan and Ollitrault (2015), $\left\langle
V_{4\mathrm{L}}(V_{2}^{*})^{2}\right\rangle=\left\langle
V_{5\mathrm{L}}V_{2}^{*}V_{3}^{*}\right\rangle=0$, one obtains an unique
expression for the coupling coefficients,

$\displaystyle\chi_{4}=\frac{\langle V_{4}(V_{2}^{2})^{*}\rangle}{\langle
v_{2}^{4}\rangle},\hskip 30.0pt\chi_{5}=\frac{\langle
V_{5}(V_{2}V_{3})^{*}\rangle}{\langle v_{2}^{2}v_{3}^{2}\rangle}.$ (3)

It is clear both $\chi_{4}$ and $\chi_{5}$ are expressed in terms of
quantities that are measurable. By construction, the $\chi_{n}$ encode as well
the effect of subleading couplings that go beyond the $V_{2}^{2}$ or
$V_{2}V_{3}$ terms. If the impact of such subleading couplings become more
visible as we vary the initial geometry, then $\chi_{n}$ would change
accordingly. The goal of this work is to expose and make a precision test of
this feature using collisions of isobars, which permits us to claim evidence
of subleading couplings in $V_{4}$ and $V_{5}$, driven by small differences in
the initial geometry between 96Ru+96Ru and 96Zr+96Zr collisions.

The same could in principle be achieved by comparing other species at the same
multiplicities, for instance, 238U+238U versus 197Au+197Au or 208Pb+208Pb
versus 129Xe+129Xe, collisions that were already taken at RHIC and the Large
Hadron Collider (LHC), respectively. However, as the expected contribution
from subleading modes to $\chi_{n}$ is small, it might be beyond the genuine
measurement systematics. In contrast, the isobar running mode guarantees that
ratios of observables are devoid of measurement systematics Adam  _et al._
(2021), as well as system dependence of final state effects Zhang  _et al._
(2022), which makes the extraction of novel nonlinear effects possible.

Figure 1:  Isobar ratios of ${\left\langle
V_{4}\left(V_{2}^{*}\right)^{2}\right\rangle}$, $\left\langle
v_{2}^{4}\right\rangle$, $\left\langle v_{2}^{2}\right\rangle^{2}$ (top), and
${\left\langle V_{5}V_{2}^{*}V_{3}^{*}\right\rangle}$, $\left\langle
v_{2}^{2}v_{3}^{2}\right\rangle$, $\left\langle
v_{2}^{2}\right\rangle\left\langle v_{3}^{2}\right\rangle$ (bottom). We
consider the impact of $\beta_{2}$, $\beta_{3}$, $a_{0}$, $R_{0}$, and all of
them included, moving from left to right in the figure. The locations for 1%
and 5% most central collisions are indicated by vertical dashed lines.

## Nonlinear coupling coefficients.

To make progress, we perform the exercise recently proposed in Ref. Zhao  _et
al._ (2022); Magdy (2022b). We denote by $\mathcal{R}_{\mathcal{O}}$ the ratio
of observable $\mathcal{O}$ taken between 96Ru+96Ru and 96Zr+96Zr collisions
at fixed $N_{\mathrm{ch}}$,

$\displaystyle\mathcal{R}_{\mathcal{O}}(N_{\mathrm{ch}})=\frac{\mathcal{O}_{\mathrm{Ru+Ru}}(N_{\mathrm{ch}})}{\mathcal{O}_{\mathrm{Zr+Zr}}(N_{\mathrm{ch}})}.$
(4)

We shall refer to such operation as the isobar ratio. Taking the isobar ratio
of the plane correlators $\langle V_{4}(V_{2}^{2})^{*}\rangle$ and $\langle
V_{5}V_{2}^{*}V_{3}^{*}\rangle$ yields,

$\displaystyle\mathcal{R}_{\left\langle
V_{4}\left(V_{2}^{*}\right)^{2}\right\rangle}=\mathcal{R}_{\chi_{4}}\mathcal{R}_{\left\langle
v_{2}^{4}\right\rangle},\;\;\;\mathcal{R}_{\left\langle
V_{5}V_{2}^{*}V_{3}^{*}\right\rangle}=\mathcal{R}_{\chi_{5}}\mathcal{R}_{\left\langle
v_{2}^{2}v_{2}^{3}\right\rangle}.$ (5)

The ratio at the same $N_{\mathrm{ch}}$ ensures a nearly perfect cancellation
of final state effects for flow observables due to e.g. viscosity and
hadronization Zhang  _et al._ (2022). One also in general expects that
$\chi_{n}$ are nearly independent of initial-state features, i.e.,
$\mathcal{R}_{\chi_{n}}\approx 1$ Yan and Ollitrault (2015). Therefore, the
ratio of the plane correlators should be given by the ratio of $\langle
v_{2}^{4}\rangle$ or $\langle v_{2}^{2}v_{3}^{2}\rangle$, which in turn are
directly influenced by the nuclear structure parameters. To see this
explicitly, we simulate generic high-statistics isobar collisions for several
choices of Woods-Saxon densities listed in Tab. 1. These permit us to isolate
the impact of individual features of the nuclear profiles. For instance,
Case1/Case2 isolates the impact of $\beta_{2}$, while the combined effect of
all four parameters is reflected by Case1/Case5.

Figure 1 shows the $N_{\rm ch}$ dependence of the ratios in Eq. (5), including
the impact of the four Woods-Saxon parameters one at a time. The top row shows
results for $V_{4}$. Remarkably, we see that the ratio of plane correlators
$\mathcal{R}_{\langle V_{4}(V_{2}^{2})^{*}\rangle}$ follows very precisely
$\mathcal{R}_{\left\langle v_{2}^{4}\right\rangle}$, irrespective of the
source of modifications in the nuclear structure induced to $v_{2}$. Within
the precision of this plot, $\mathcal{R}_{\chi_{n}}=1$ is confirmed. This
result reproduces nicely the preliminary measurement of this observable by the
STAR collaboration Chunjian Zhang, , Observation of nuclear deformation in
isobar collisions() (STAR Collabration). It is interesting to note that the
$\mathcal{R}_{\langle v_{2}^{2}\rangle^{2}}$ (open diamonds) also follows very
closely the $\mathcal{R}_{\left\langle v_{2}^{4}\right\rangle}$, with a
visible difference appearing only after a change in surface diffuseness,
$a_{0}$. This behavior can be understood from the following identity,

$\displaystyle\mathcal{R}_{\left\langle
v_{2}^{2}\right\rangle^{2}}-\mathcal{R}_{\left\langle
v_{2}^{4}\right\rangle}=x(\mathcal{R}_{v_{2}\\{4\\}^{4}}-\mathcal{R}_{\left\langle
v_{2}^{4}\right\rangle}),\hskip
5.0ptx\equiv\\!v_{2}\\{4\\}^{4}/\left(2\left\langle
v_{2}^{2}\right\rangle^{2}\right),$

where we have introduced the fourth-order cumulant of the fluctuations of
$v_{2}$: $v_{2}\\{4\\}^{4}=2\langle v_{2}^{2}\rangle^{2}-\langle
v_{2}^{4}\rangle$. In the top-middle panel of Fig. 1, a quick estimate gives
$x\sim 0.1$ and $\mathcal{R}_{v_{2}\\{4\\}^{4}}\sim 1.35$ in mid-central
collisions, which explains quantitatively why $\mathcal{R}_{\left\langle
v_{2}^{2}\right\rangle^{2}}$ is about 2% larger than
$\mathcal{R}_{\left\langle v_{2}^{4}\right\rangle}$.

The lower panels of Fig. 1 display similar comparisons for $V_{5}$. Our
conclusions are unchanged. In this case, we predict in addition perfect
agreement between the ratio of $\langle v_{2}^{2}v_{3}^{2}\rangle$ and that of
$\langle v_{2}^{2}\rangle\langle v_{3}^{2}\rangle$ for all Woods-Saxon
parameters. This is expected from the rather weak correlation between $v_{2}$
and $v_{3}$ in heavy ion collisions Lacey  _et al._ (2016); Huo  _et al._
(2014); Aad  _et al._ (2015); Adam  _et al._ (2016).

|  $R_{0}$ (fm) |  $a_{0}$ (fm) | $\beta_{2}$ | $\beta_{3}$
---|---|---|---|---
Case1 96Ru | 5.09 | 0.46 | 0.162 | 0
Case2 | 5.09 | 0.46 | 0.06 | 0
Case3 | 5.09 | 0.46 | 0.06 | 0.20
Case4 | 5.09 | 0.52 | 0.06 | 0.20
Case5 96Zr | 5.02 | 0.52 | 0.06 | 0.20
Ratios |  $\frac{\mathrm{\textstyle\small Case1}}{\mathrm{\textstyle\small Case2}}$ , $\frac{\mathrm{\textstyle\small Case2}}{\mathrm{\textstyle\small Case3}}$ , $\frac{\mathrm{\textstyle\small Case3}}{\mathrm{\textstyle\small Case4}}$ , $\frac{\mathrm{\textstyle\small Case4}}{\mathrm{\textstyle\small Case5}}$ , $\frac{\mathrm{\textstyle\small Case1}}{\mathrm{\textstyle\small Case5}}$
Table 1:  Woods-Saxon parameters for 96Ru (Case1), 96Zr (Case5). The three
other cases allow us to study the impact of nuclear structure on observables
step by step. About 170 million AMPT events are generated for each case, about
10% of the available real isobar data for one collision system. The choice of
parameters is motivated by previous comparisons between AMPT results and
experimental data Jia and Zhang (2021).

Having exposed the nonlinear couplings, we zoom in these curves and examine
the actual values of $\mathcal{R}_{\chi_{n}}$. Our predictions are given in
Fig. 2. We do observe small systematic deviations of up to 1–2% level induced
by nuclear structure effects. Concerning $\chi_{4}$, the most significant
deviations are associated with the nuclear deformation parameters $\beta_{2}$
and $\beta_{3}$. Larger $\beta_{2}$ in 96Ru leads to a slight decrease of
$\chi_{4}$ from peripheral to central collisions, while the larger $\beta_{3}$
of 96Zr leads to the opposite effect. Similar effects are observed for
$\chi_{5}$. Both ratios are, on the other hand, more weakly-dependent on the
radial profile parameters, $a_{0}$ and $R_{0}$.

The insets of Fig. 2 show our predictions for $\chi_{4}$ and $\chi_{5}$ as a
function of $N_{\mathrm{ch}}$. They exhibit a weak centrality dependence, with
only a decrease in the 0–5% centrality range, consistent with recent STAR
measurement Adam  _et al._ (2020) in 197Au+197Au collisions. We stress that
the precision reached in our analysis is in fact worse than what could be
achieved based on the datasets available to the STAR collaboration. Therefore,
percent level deviations could be easily detected with the isobar data.

This leads us to our main conclusion. The fact that
$\mathcal{R}_{\chi_{n}}\neq 1$ implies the presence of additional nonlinear
contributions to $V_{4}$ and $V_{5}$, driven by the nuclear structure effects.
For instance, an important nonlinear contribution to $V_{4}$ beyond
$V_{2}^{2}$ should be $V_{3}V_{1}$. It makes sense, then, that $\chi_{4}$ is
affected by the large $\beta_{3}$ in 96Zr, as both $V_{1}$ and $V_{3}$ have a
leading dependence on $\beta_{3}$ Jia (2022). However, since both $V_{1}$ and
$V_{3}$ do not have a leading dependence on $\beta_{2}$, the dependence of
$\mathcal{R}_{\chi_{4}}$ on $\beta_{2}$ suggests another nonlinear
contribution, potentially in the form of $V_{3}^{2}V_{2}^{*}$ 111In principle,
the nonlinear mode $V_{1}^{2}V_{2}$ is also allowed. However, the
$p_{\mathrm{T}}$-integrated $V_{1}$ is known to be very small Aad  _et al._
(2012). Thus, due to presence of $V_{1}^{2}$, this mode should be strongly
suppressed.. A detailed study of such geometry-induced subleading couplings is
beyond the scope of this paper. However, this analysis can be readily
performed if our prediction is confirmed by the experimental measurements. We
emphasize once more that, without resorting to isobar ratios, it would be
impossible to achieve the precision required to detect such effects. This is a
remarkable consequence of the isobar collision campaign, with profound
implications for future precision QGP studies.

Figure 2:  Isobar ratios of nonlinear response coefficients $\chi_{4}$ (top
panel) and $\chi_{5}$ (bottom panel) as a function of $N_{\rm ch}$. The impact
of various Woods-Saxon parameters is exhibited separately, as well as combined
together. The insets show predictions for $\chi_{4}$ and $\chi_{5}$ averaged
over all cases.

## Subleading nonlinear modes in $v_{4L}$.

In the framework of nonlinear coupling coefficients, the definition of the
nonlinear term $V_{n\mathrm{NL}}$ (and therefore $\chi_{n}$) and the linear
term $V_{n\mathrm{L}}$ are intertwined. This implies that, if $\chi_{n}$
coefficients are impacted by subleading nonlinear modes, such effect may also
occur for $v_{n\mathrm{L}}$ and show up in the isobar ratio of
$v_{n\mathrm{L}}$. We focus here on $n=4$.

The linear component can be isolated by combing Eq. (1) with a two-particle
measurement of $V_{4}$, $v_{4}\\{2\\}^{2}\equiv\left\langle
v_{4}^{2}\right\rangle$ Aad  _et al._ (2014, 2015),

$\displaystyle
v_{4\mathrm{L}}^{2}=v_{4}\\{2\\}^{2}-v_{4,\mathrm{NL}}^{2},\hskip
20.0ptv_{\mathrm{4NL}}^{2}\equiv\chi_{4}^{2}\left\langle
v_{2}^{4}\right\rangle.$ (6)

The isobar ratios of the three quantities in the above equation are related
via a simple idendity,

$\displaystyle\mathcal{R}_{v_{4}\\{2\\}^{2}}=\mathcal{R}_{v_{4\mathrm{L}}^{2}}+(\mathcal{R}_{v_{4\mathrm{NL}}^{2}}-\mathcal{R}_{v_{4\mathrm{L}}^{2}})r,\;\;\;r\equiv\\!\frac{v_{4\mathrm{NL}}^{2}}{v_{4}\\{2\\}^{2}}$
$\displaystyle\mathcal{R}_{v_{4}\\{2\\}}\approx\mathcal{R}_{v_{4\mathrm{L}}}+(\mathcal{R}_{v_{4\mathrm{NL}}}-\mathcal{R}_{v_{4\mathrm{L}}})r$
(7)

The second line of the above equation is valid when all ratios are close to
unity. Figure 3 shows our predictions for the isobar ratio of $v_{4}$, as well
as how the impact of nuclear structure manifests in the linear,
$v_{\mathrm{4L}}$, and nonlinear mode, $v_{\mathrm{4NL}}$, extracted using Eq.
(6). The left panel displays the ratio $\mathcal{R}_{v_{4}\\{2\\}}$, which
reveals small but rather complex dependencies on the nuclear structure
parameters. The right panel shows the impact of such parameters on the
nonlinear component, which we already observed from $\mathcal{R}_{\left\langle
v_{2}^{4}\right\rangle}$ in the top panels of Fig. 1 (however, note that
$\mathcal{R}_{v_{4\mathrm{L}}}\approx\sqrt{\mathcal{R}_{\left\langle
v_{2}^{4}\right\rangle}}$). The fraction of nonlinear contribution, shown in
the inset, approaches zero in central collisions, so that $v_{\mathrm{4NL}}$
impacts $\mathcal{R}_{v_{4}\\{2\\}}$ mostly in the non-central region.

The middle panel of Fig. 3 shows the ratio of linear modes,
$\mathcal{R}_{v_{\mathrm{4L}}}$. We observe something important. The
coefficient $\chi_{4}$ in Fig. 2 shows a sensitivity to $\beta_{2}$ and
$\beta_{3}$. The same does not occur for $v_{\mathrm{4L}}$, which can be
understood from the fact that fourth-order eccentricity $\mathcal{E}_{4}$ does
not depend on $\beta_{2}$ and $\beta_{3}$ Jia (2022). On the other hand,
$v_{\mathrm{4L}}$ is more strongly impacted by the radial profile parameters,
$a_{0}$ and $R_{0}$. These results imply that the effect of subleading modes
to $v_{4\mathrm{L}}$ is complementary to that observed for $v_{4\mathrm{NL}}$.
Both observables probe such phenomena, but respond to initial-state
deformations in different ways. We stress, once more, that all these features
can be accessed experimentally via high statistics isobar data.

Figure 3:  The ratios of quadrangular flow from two-particle method,
$v_{4}\\{2\\}$ (left), and the ratios of estimated linear component,
$v_{\mathrm{4L}}$ (middle), and nonlinear component, $v_{\mathrm{4NL}}$
(right), as a function of $N_{\mathrm{ch}}$ in isobar collisions. Nuclear
structure differences follow Table 1. The inset shows the fraction of
$v_{4}^{2}$ carried by the nonlinear mode.

## Conclusion & Prospects.

Detailed transport simulations predict that the precision reached in the
isobar ratios of higher-order flow harmonic coefficients permits one to
scrutinize the effect of subleading couplings to $V_{4}$ and potentially
$V_{5}$. If this prediction is confirmed in experiments, it will establish
isobar collisions as an unique tool to study subleading nonlinear effects in
the QGP expansion, and exposed by the different structures of the two isobars.
The largest subleading modes to $V_{4}$ should come from $V_{1}V_{3}$ and
$V_{3}^{2}V_{2}^{*}$, namely,

$V_{4}=V_{4\mathrm{L}}+\chi_{4,22}V_{2}^{2}+\chi_{4,31}V_{1}V_{3}+\chi_{4,332}V_{3}^{2}V_{2}^{*}.$
(8)

A self-consistent framework to define both $V_{4\mathrm{L}}$ and the coupling
coefficients in the presence of multiple nonlinear components has been derived
in Ref. Giacalone  _et al._ (2018), and could be readily applied to isobar
data analysis. So far, subleading couplings have only been discussed in the
analysis of $V_{6}$, $V_{7}$, $V_{8}$ in 208Pb+208Pb collisions at LHC
energies. The rich nonlinear structure of, e.g., $V_{6}$ is, however,
difficult to fully clarify via isobar ratios in 96Ru+96Ru and 96Zr+96Zr
collisions at RHIC, as multiplicities are not high enough. If larger isobar
pairs are collided in future, potentially at higher collision energies at the
LHC, such possibility will be realized. It will permit us to access
experimentally the full intricacies of the nonlinear hydrodynamic response,
thus opening a new promising opportunity for data-driven determinations of the
QGP transport properties.

## Acknowledgments.

This research of J.J and C.Z is supported by DOE DE-FG02-87ER40331. The
research of G.G. is funded by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under Germanys Excellence Strategy EXC2181/1-390900948
(the Heidelberg STRUCTURES Excellence Cluster), within the Collaborative
Research Center SFB1225 (ISOQUANT, Project-ID 273811115). We acknowledge
Somadutta Bhatta for useful comments.

## References

  * Ollitrault (1992) Jean-Yves Ollitrault, “Anisotropy as a signature of transverse collective flow,” Phys. Rev. D 46, 229–245 (1992).
  * Alver and Roland (2010) B. Alver and G. Roland, “Collision geometry fluctuations and triangular flow in heavy-ion collisions,” Phys. Rev. C 81, 054905 (2010), [Erratum: Phys.Rev.C 82, 039903 (2010)], arXiv:1003.0194 [nucl-th] .
  * Teaney and Yan (2011) Derek Teaney and Li Yan, “Triangularity and Dipole Asymmetry in Heavy Ion Collisions,” Phys. Rev. C 83, 064904 (2011), arXiv:1010.1876 [nucl-th] .
  * Heinz and Snellings (2013) Ulrich Heinz and Raimond Snellings, “Collective flow and viscosity in relativistic heavy-ion collisions,” Ann. Rev. Nucl. Part. Sci. 63, 123–151 (2013), arXiv:1301.2826 [nucl-th] .
  * Teaney and Yan (2012) Derek Teaney and Li Yan, “Non linearities in the harmonic spectrum of heavy ion collisions with ideal and viscous hydrodynamics,” Phys. Rev. C 86, 044908 (2012), arXiv:1206.1905 [nucl-th] .
  * Gardim  _et al._ (2012) Fernando G. Gardim, Frederique Grassi, Matthew Luzum, and Jean-Yves Ollitrault, “Mapping the hydrodynamic response to the initial geometry in heavy-ion collisions,” Phys. Rev. C85, 024908 (2012), arXiv:1111.6538 [nucl-th] .
  * Teaney and Yan (2014) D. Teaney and L. Yan, “Event-plane correlations and hydrodynamic simulations of heavy ion collisions,” Phys. Rev. C 90, 024902 (2014), arXiv:1312.3689 [nucl-th] .
  * Gardim  _et al._ (2015) Fernando G. Gardim, Jacquelyn Noronha-Hostler, Matthew Luzum, and Frédérique Grassi, “Effects of viscosity on the mapping of initial to final state in heavy ion collisions,” Phys. Rev. C 91, 034902 (2015), arXiv:1411.2574 [nucl-th] .
  * Yan and Ollitrault (2015) Li Yan and Jean-Yves Ollitrault, “$\nu_{4},\nu_{5},\nu_{6},\nu_{7}$: nonlinear hydrodynamic response versus LHC data,” Phys. Lett. B 744, 82–87 (2015), arXiv:1502.02502 [nucl-th] .
  * Qian  _et al._ (2016) Jing Qian, Ulrich W. Heinz, and Jia Liu, “Mode-coupling effects in anisotropic flow in heavy-ion collisions,” Phys. Rev. C 93, 064901 (2016), arXiv:1602.02813 [nucl-th] .
  * Giacalone  _et al._ (2016) Giuliano Giacalone, Li Yan, Jacquelyn Noronha-Hostler, and Jean-Yves Ollitrault, “Symmetric cumulants and event-plane correlations in Pb + Pb collisions,” Phys. Rev. C 94, 014906 (2016), arXiv:1605.08303 [nucl-th] .
  * Qian and Heinz (2016) Jing Qian and Ulrich Heinz, “Hydrodynamic flow amplitude correlations in event-by-event fluctuating heavy-ion collisions,” Phys. Rev. C 94, 024910 (2016), arXiv:1607.01732 [nucl-th] .
  * Giacalone  _et al._ (2017) Giuliano Giacalone, Li Yan, Jacquelyn Noronha-Hostler, and Jean-Yves Ollitrault, “The fluctuations of quadrangular flow,” J. Phys. Conf. Ser. 779, 012064 (2017), arXiv:1608.06022 [nucl-th] .
  * Qian  _et al._ (2017) Jing Qian, Ulrich Heinz, Ronghua He, and Lei Huo, “Differential flow correlations in relativistic heavy-ion collisions,” Phys. Rev. C 95, 054908 (2017), arXiv:1703.04077 [nucl-th] .
  * Giacalone  _et al._ (2018) Giuliano Giacalone, Li Yan, and Jean-Yves Ollitrault, “Nonlinear coupling of flow harmonics: Hexagonal flow and beyond,” Phys. Rev. C 97, 054905 (2018), arXiv:1803.00253 [nucl-th] .
  * Magdy (2022a) Niseem Magdy, “Investigations of the linear and non-linear flow harmonics using the a multi-phase transport model,” J. Phys. G 49, 015105 (2022a), arXiv:2106.09484 [nucl-th] .
  * Zhao  _et al._ (2022) Shujun Zhao, Hao-jie Xu, Yu-Xin Liu, and Huichao Song, “Probing the nuclear deformation with three-particle asymmetric cumulant in RHIC isobar runs,” (2022), arXiv:2204.02387 [nucl-th] .
  * Aad  _et al._ (2014) Georges Aad _et al._ (ATLAS), “Measurement of event-plane correlations in $\sqrt{s_{NN}}=2.76$ TeV lead-lead collisions with the ATLAS detector,” Phys. Rev. C 90, 024905 (2014), arXiv:1403.0489 [hep-ex] .
  * Aad  _et al._ (2015) Georges Aad _et al._ (ATLAS), “Measurement of the correlation between flow harmonics of different order in lead-lead collisions at $\sqrt{s_{NN}}$=2.76 TeV with the ATLAS detector,” Phys. Rev. C 92, 034903 (2015), arXiv:1504.01289 [hep-ex] .
  * Acharya  _et al._ (2017) Shreyasi Acharya _et al._ (ALICE), “Linear and non-linear flow modes in Pb-Pb collisions at $\sqrt{s_{\rm NN}}=$ 2.76 TeV,” Phys. Lett. B 773, 68–80 (2017), arXiv:1705.04377 [nucl-ex] .
  * Acharya  _et al._ (2020) Shreyasi Acharya _et al._ (ALICE), “Higher harmonic non-linear flow modes of charged hadrons in Pb-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 5.02 TeV,” JHEP 05, 085 (2020), arXiv:2002.00633 [nucl-ex] .
  * Adam  _et al._ (2020) J. Adam _et al._ (STAR), “Investigation of the linear and mode-coupled flow harmonics in Au+Au collisions at $\sqrt{S_{NN}}$ = 200 GeV,” Phys. Lett. B 809, 135728 (2020), arXiv:2006.13537 [nucl-ex] .
  * Abdallah  _et al._ (2022) Mohamed Abdallah _et al._ (STAR), “Search for the chiral magnetic effect with isobar collisions at $\sqrt{s_{NN}}$=200 GeV by the STAR Collaboration at the BNL Relativistic Heavy Ion Collider,” Phys. Rev. C 105, 014901 (2022), arXiv:2109.00131 [nucl-ex] .
  * Miller  _et al._ (2007) Michael L. Miller, Klaus Reygers, Stephen J. Sanders, and Peter Steinberg, “Glauber modeling in high energy nuclear collisions,” Ann. Rev. Nucl. Part. Sci. 57, 205–243 (2007), arXiv:nucl-ex/0701025 .
  * Lin  _et al._ (2005) Zi-Wei Lin, Che Ming Ko, Bao-An Li, Bin Zhang, and Subrata Pal, “A Multi-phase transport model for relativistic heavy ion collisions,” Phys. Rev. C72, 064901 (2005), arXiv:nucl-th/0411110 [nucl-th] .
  * Bilandzic  _et al._ (2011) Ante Bilandzic, Raimond Snellings, and Sergei Voloshin, “Flow analysis with cumulants: Direct calculations,” Phys. Rev. C 83, 044913 (2011), arXiv:1010.0233 [nucl-ex] .
  * Bilandzic  _et al._ (2014) Ante Bilandzic, Christian Holm Christensen, Kristjan Gulbrandsen, Alexander Hansen, and You Zhou, “Generic framework for anisotropic flow analyses with multiparticle azimuthal correlations,” Phys. Rev. C 89, 064904 (2014), arXiv:1312.3572 [nucl-ex] .
  * Jia  _et al._ (2017) Jiangyong Jia, Mingliang Zhou, and Adam Trzupek, “Revealing long-range multiparticle collectivity in small collision systems via subevent cumulants,” Phys. Rev. C 96, 034906 (2017), arXiv:1701.03830 [nucl-th] .
  * Zhang and Jia (2022) Chunjian Zhang and Jiangyong Jia, “Evidence of Quadrupole and Octupole Deformations in Zr96+Zr96 and Ru96+Ru96 Collisions at Ultrarelativistic Energies,” Phys. Rev. Lett. 128, 022301 (2022), arXiv:2109.01631 [nucl-th] .
  * Jia and Zhang (2021) Jiangyong Jia and Chun-Jian Zhang, “Scaling approach to nuclear structure in high-energy heavy-ion collisions,” (2021), arXiv:2111.15559 [nucl-th] .
  * Nijs and van der Schee (2021) Govert Nijs and Wilke van der Schee, “Inferring nuclear structure from heavy isobar collisions using Trajectum,” (2021), arXiv:2112.13771 [nucl-th] .
  * Adam  _et al._ (2021) J. Adam _et al._ (STAR), “Methods for a blind analysis of isobar data collected by the STAR collaboration,” Nucl. Sci. Tech. 32, 48 (2021), arXiv:1911.00596 [nucl-ex] .
  * Zhang  _et al._ (2022) Chunjian Zhang, Somadutta Bhatta, and Jiangyong Jia, “Ratios of collective flow observables in high-energy isobar collisions are insensitive to final state interactions,” (2022), arXiv:2206.01943 [nucl-th] .
  * Magdy (2022b) Niseem Magdy, “Impact of nuclear deformation on collective flow observables in relativistic U+U collisions,” (2022b), arXiv:2206.05332 [nucl-th] .
  * Chunjian Zhang, , Observation of nuclear deformation in isobar collisions() (STAR Collabration) Chunjian Zhang, (STAR Collabration), Observation of nuclear deformation in isobar collisions, “https://indico.cern.ch/event/895086/contributions/4749420/,” .
  * Lacey  _et al._ (2016) Roy A. Lacey, D. Reynolds, A. Taranenko, N. N. Ajitanand, J. M. Alexander, Fu-Hu Liu, Yi Gu, and A. Mwai, “Acoustic scaling of anisotropic flow in shape-engineered events: implications for extraction of the specific shear viscosity of the quark gluon plasma,” J. Phys. G 43, 10LT01 (2016), arXiv:1311.1728 [nucl-ex] .
  * Huo  _et al._ (2014) Peng Huo, Jiangyong Jia, and Soumya Mohapatra, “Elucidating the event-by-event flow fluctuations in heavy-ion collisions via the event shape selection technique,” Phys. Rev. C 90, 024910 (2014), arXiv:1311.7091 [nucl-ex] .
  * Adam  _et al._ (2016) Jaroslav Adam _et al._ (ALICE), “Correlated event-by-event fluctuations of flow harmonics in Pb-Pb collisions at $\sqrt{s_{{}_{\rm NN}}}=2.76$ TeV,” Phys. Rev. Lett. 117, 182301 (2016), arXiv:1604.07663 [nucl-ex] .
  * Jia (2022) Jiangyong Jia, “Shape of atomic nuclei in heavy ion collisions,” Phys. Rev. C 105, 014905 (2022), arXiv:2106.08768 [nucl-th] .
  * Note (1) In principle, the nonlinear mode $V_{1}^{2}V_{2}$ is also allowed. However, the $p_{\mathrm{T}}$-integrated $V_{1}$ is known to be very small Aad  _et al._ (2012). Thus, due to presence of $V_{1}^{2}$, this mode should be strongly suppressed.
  * Aad  _et al._ (2012) Georges Aad _et al._ (ATLAS), “Measurement of the azimuthal anisotropy for charged particle production in $\sqrt{s_{\mathrm{NN}}}=2.76$ TeV lead-lead collisions with the ATLAS detector,” Phys. Rev. C86, 014907 (2012), arXiv:1203.3087 [hep-ex] .