# QPT V2: Masked Image Modeling Advances Visual Scoring

Qizhi Xie  Tsinghua University<EMAIL_ADDRESS>,  Kun
Yuan${}^{~{}\textrm{{\char 0\relax}}}$ Kuaishou Technology
<EMAIL_ADDRESS>,  Yunpeng Qu  Tsinghua University
<EMAIL_ADDRESS>,  Mingda Wu  Kuaishou Technology
<EMAIL_ADDRESS>,  Ming Sun  Kuaishou Technology<EMAIL_ADDRESS>,  Chao Zhou  Kuaishou Technology<EMAIL_ADDRESS>and  Jihong
Zhu${}^{~{}\textrm{{\char 0\relax}}}$ Tsinghua University
<EMAIL_ADDRESS>
(2024)

###### Abstract.

Quality assessment and aesthetics assessment aim to evaluate the perceived
quality and aesthetics of visual content. Current learning-based methods
suffer greatly from the scarcity of labeled data and usually perform sub-
optimally in terms of generalization. Although masked image modeling (MIM) has
achieved noteworthy advancements across various high-level tasks (e.g.,
classification, detection etc. ). In this work, we take on a novel perspective
to investigate its capabilities in terms of quality- and aesthetics-awareness.
To this end, we propose Quality- and aesthetics-aware PreTraining (QPT V2),
the first pretraining framework based on MIM that offers a unified solution to
quality and aesthetics assessment. To perceive the high-level semantics and
fine-grained details, pretraining data is curated. To comprehensively
encompass quality- and aesthetics-related factors, degradation is introduced.
To capture multi-scale quality and aesthetic information, model structure is
modified. Extensive experimental results on 11 downstream benchmarks clearly
show the superior performance of QPT V2 in comparison with current state-of-
the-art approaches and other pretraining paradigms. Code and models will be
released at https://github.com/KeiChiTse/QPT-V2.

visual scoring, quality and aesthetics assessment, self-supervised learning,
masked image modeling

††copyright: acmlicensed††journalyear: 2024††doi: XXXXXXX.XXXXXXX††isbn:
978-1-4503-XXXX-X/18/06††ccs: Computing methodologies††ccs: Computing
methodologies Artificial intelligence††ccs: Computing methodologies Computer
vision††ccs: Computing methodologies Computer vision tasks††ccs: Computing
methodologies Scene understanding††footnotetext: ${}^{\textrm{{\char
0\relax}}}$ Corresponding authors. This work is done when Qizhi Xie and
Yunpeng Qu are research interns at Kuaishou Technology.

##  1\. Introduction

Figure 1. QPT V2: a new MIM-based pretraining paradigm for visual scoring. For
pretraining, dataset $\mathcal{D}_{I}$ provides HR & HFC images, augmented by
quality- and aesthetics-aware degradation $\mathcal{A}(\cdot)$. A multi-scale
autoencoder $\mathcal{G}(\cdot)$ outputs the reconstructed images. Through
finetuning of the encoder, it can solve visual scoring tasks like IQA, VQA,
and IAA.

The aims of Image Quality Assessment (IQA), Visual Quality Assessment (VQA),
and Image Aesthetics Assessment (IAA) are to appraise the quality and
aesthetics of visual content, serving as critical components across a
multitude of vision applications including video enhancement, transcoding, and
transmission (Rippel et al., 2019; Zhang et al., 2019; Qu et al., 2024;
Isogawa et al., 2019). While being studied separately for a considerable
period, these tasks present strong resemblance in various aspects. All these
tasks share the same core objective, that is, to mimic the Human Visual System
(HVS), so as to generate accurate scores aligned with human perception (He et
al., 2014; Wang et al., 2023). Moreover, the proliferation of User-Generated
Content (UGC) (Tu et al., 2021a; Sun et al., 2022) and AI-Generated Content
(AIGC) (Yuan et al., 2023a) has become a trend in recent years, which greatly
contributed to the exponential growth of image and video data (Barnett et al.,
2018). The complexity and interrelation of quality-related and aesthetics-
related factors in emerging content are unprecedented, and analyzing single
factors alone is insufficient to achieve a comprehensive perception of visual
content aligned with human perception. In response to the aforementioned
resemblance and trend, we refer to IQA, VQA, and IAA jointly as Visual Scoring
(VS) for analysis.

Facilitated by the advancements of deep neural networks (Krizhevsky et al.,
2012; He et al., 2016; Vaswani et al., 2017), learning-based methods (Su et
al., 2020; Li et al., 2022; He et al., 2022b) have surpassed traditional
methods (Mittal et al., 2013; Xu et al., 2016; Tu et al., 2021b) based on
handcrafted features on multiple VS benchmarks (Wang et al., 2019; Ciancio et
al., 2011; Fang et al., 2020). They acquire features with strong
expressiveness via regressing from the Mean Opinion Scores (MOS). However, one
of the primary obstacles in solving VS lies in the limited size of labeled
datasets (Li et al., 2022, 2021a; Zhao et al., 2023; Madhusudana et al., 2022;
Sun et al., 2023). Due to the high cost associated with collecting MOS through
extensively annotated subjective studies, the scale of VS datasets is often
only a fraction, ranging from one-tenth to even one-hundredth, of other high-
level visual task datasets (e.g., classification). At all events, the paucity
of labeled data restricts the capabilities of deep learning methods.

To tackle this problem, some previous efforts increased data size by
patch/frame-level augmentation (Ke et al., 2021; Li et al., 2021a; Bosse et
al., 2018; Kang et al., 2014) or mixed-database training (Korhonen, 2019).
However, the quality and aesthetics scores of local patches often differ from
those of the entire content, and subjective differences are observed across
datasets, thus hindering the achievement of promising results. On the other
hand, a different research line (Wu et al., 2022; Tu et al., 2022; Wang et
al., 2023; Ke et al., 2023; Yang et al., 2022) exploits knowledge valuable for
VS from datasets and model weights of other domains, by tapping into the power
of pretrained vision or vision-language (VL) models (Zhang et al., 2020; Wu et
al., 2022; Yang et al., 2022; Ying et al., 2020; Tu et al., 2022; Radford et
al., 2021; Yu et al., 2022). These works attempt to extract knowledge that is
more quality- or aesthetic-aware from large-scale datasets by carefully
designing pretraining objectives(Madhusudana et al., 2022; Li et al., 2022),
and are then finetuned on downstream VS tasks. The pretraining objectives of
existing works are mainly based on contrastive learning (van den Oord et al.,
2018; Schroff et al., 2015), which can be viewed as a global self-supervised
learning (SSL) approach, as it groups similar samples closer and diverse
samples far from each other (He et al., 2020; Zhao et al., 2023; Madhusudana
et al., 2022). However, this ”sample-level” discernment is insufficient for
capturing local distortions and visual attributes (Park et al., 2023).
Therefore, exploring more effective pixel-level discrimination may be
beneficial for incorporating pretrained priors into downstream VS tasks.

Masked Image Modeling (MIM) (He et al., 2022a), which learns representation by
pixel-level reconstruction of the masked regions in the input, has
demonstrated its impressive ability of semantics- and texture-aware perception
in visual tasks (Bao et al., 2022; Peng et al., 2022). In this paper, we
conduct a detailed exploration of MIM, in which we observe MIM can learn both
sample-level and pixel-level information of the visual content, showing the
potential to serve as a general pretraining recipe to VS tasks. As shown in
Fig. 1, we propose QPT V2, the first pretraining framework based on MIM that
offers a unified solution for VS tasks. To enhance the acquisition of prior
knowledge by MIM for VS tasks, we propose further improvements and
optimizations of the vanilla MIM from the perspectives of data, degradation,
and model. Regarding the realm of data, we curate a dataset with high
resolution (HR) and high foreground coverage (HFC), thereby aiding the pretext
task of MIM. Regarding the realm of degradation, we propose an optimal
strategy for applying degradations to the reconstruction target, exploring the
type and composition of degradations to acquire prior knowledge of practical
scenarios. Regarding the realm of model, we use a drop-in strategy to learn
multi-scale representations by adaptively fusing features of different layers.

Our main contributions can be summed as follows:

  * •

To the best of our knowledge, we are the first to validate the capability of
MIM in adeptly unifying downstream visual scoring tasks. We decompose MIM into
three crucial components: data, degradation, and model, and individually
investigate their respective influences.

  * •

We propose QPT V2, which stands as the pioneering MIM-based pretraining
framework, offering a unified solution for VS tasks. To enhance the
acquisition of prior knowledge through MIM, we make targeted improvements in
the aspects of data, degradation, and model.

  * •

QPT V2 achieves state-of-the-art (SOTA) results on 11 benchmarks in IQA, VQA,
and IAA, surpassing other pretraining paradigms as well. Extensive ablation
studies prove the validity of each enhancement of MIM.

##  2\. Related Work

###  2.1. Visual Scoring

Visual scoring necessitates precise scoring of visual content in terms of
quality (e.g., IQA, VQA) and aesthetics (e.g., IAA). In this work, we focus on
Non-reference QA (e.g., NR-IQA and NR-VQA), since the availability of pristine
data is too hard in the real world. At the early stage, handcrafted features
based on natural scene statistics (NSS) dominate the realm of VS (Simoncelli
and Olshausen, 2001; Mittal et al., 2013, 2012). Later, data-driven methods
enhanced the performance significantly with the rise of deep learning (Kang et
al., 2014; Li et al., 2019; Esfandarani and Milanfar, 2018; Wang et al., 2023;
Wu et al., 2022; Yuan et al., 2024; Lu et al., 2024; Yuan et al., 2023b).
Nonetheless, they rely heavily on label-intensive supervision. Previous works
attempt to solve this problem by data augmentation (Ke et al., 2021; Bosse et
al., 2018; You, 2021; Kang et al., 2014), mixed-database training (Li et al.,
2021a; Sun et al., 2023), rank-based learning (Liu et al., 2017; Li et al.,
2021b) and general knowledge transfer (Wang et al., 2023; Zhang et al., 2023b;
Wu et al., 2022; Esfandarani and Milanfar, 2018).

Several researches focus on extracting quality or aesthetics information by
large-scale pretraining. Among them, CONTRIQUE (Madhusudana et al., 2022)
learns distortion-related information on images with synthetic and realistic
distortions based on contrastive learning. Similarly, Re-IQA (Saha et al.,
2023) re-engineers the MoCo-v2 (Chen et al., 2020a) framework and applies
intricate data augmentations to learn quality-aware features. Moreover, QPT
(Zhao et al., 2023) introduces a diverse array of degradations and composites
to mimic real-world distortions, which greatly expands the pretraining data
volume. Instead, we devise a pretraining framework based on MIM to learn
quality- and aesthetics-related representations.

###  2.2. Masked Image Modeling

Masked modeling learns representation by reconstructing a masked portion of
the input. Driven by the success of BERT (Devlin et al., 2019) in NLP, MIM has
become a representative SSL method in computer vision (Bao et al., 2022; He et
al., 2022a; Peng et al., 2022; Wang et al., 2022; Zhang et al., 2023a). As a
pioneer work, BEiT (Bao et al., 2022) proposes to reconstruct the features of
DALL-E (Ramesh et al., 2021). MAE (He et al., 2022a) directly reconstructs raw
pixels of the masked areas, which greatly simplifies the whole pretraining
pipeline. Some studies prove that pixel-based MIM is biased towards
reconstructing low-level details, thus hindering the performance on high-level
tasks (Liu et al., 2023a; Park et al., 2023; Bao et al., 2022). As a result,
the following works introduce a more complicated reconstruction target rather
than using raw pixels (Wei et al., 2022a; Fang et al., 2023; Hou et al., 2022;
Wei et al., 2022b). While previous MIM studies mainly focus on high-level
tasks, in this paper, we make the first attempt to adapt MIM to visual
scoring.

##  3\. Methodology

Figure 2.  Semantics- and distortion-awareness of the pixel-based MIM. (a) MIM
has the ability to understand the semantics; (b) Pixel-based MIM can
reconstruct the distortions applied to original images, the left column and
the right column are high and low-frequency intervals, respectively.  Figure
3. Illustration of the gap in FC between SA-1B and DIV2K a) qualitatively and
b) quantitatively after cropping. Figure 4. Overview of our proposed QPT V2.
QPT V2 incorporates three improvements based on pixel-based MIM tailored for
VS. To curate HR & HFC training data, we examine the resolution and foreground
coverage of various datasets and samples. To determine quality- and
aesthetics-aware degradation, we explore the degradation type and composition.
To perceive distortion and aesthetics information in multi-scale fashion, we
design a pretrain-only feature fusion module in a hierarchical encoder.

We first revisit MIM concisely in Sec.3.1, and then describe the motivation of
QPT V2 in Sec.3.2. Last, the key designs incorporated in QPT V2 are
elucidated.

###  3.1. A Revisit of Masked Image Modeling

There are three major steps in MIM: (1) split the image into visible and
masked patches, (2) reconstruct the masked patches and (3) calculate the
reconstruction loss.

Given the original image $\mathbf{I}\in\mathbb{R}^{H\times W\times 3}$, where
$H$, $W$ are the height and width of the image. First, specific degradations
$\mathcal{A}(\cdot)$ (e.g., resizing) are applied to the image, generating
non-overlapping visible patches $\mathbf{I}_{v}$ and masked patches
$\mathbf{I}_{m}$ with masking $\mathcal{M}$:

(1)
$\begin{split}&\mathbf{I}_{v}=(1-\mathcal{M})\odot\mathcal{A}(\mathbf{I})\\\
&\mathbf{I}_{m}=\mathcal{M}\odot\mathcal{A}(\mathbf{I})\\\ \end{split}$

Second, only the visible patches $\mathbf{I}_{v}$ are fed into the autoencoder
$\mathcal{G}(\cdot)$ to reconstruct the masked patches $\hat{\mathbf{I}}_{m}$
as:

(2)
$\begin{split}&\hat{\mathbf{I}}_{m}=\mathcal{G}(\mathbf{I}_{v},\mathbf{e}_{[\mathcal{M}]})\\\
\end{split}$

The autoencoder $\mathcal{G}(\cdot)$ consists of an encoder $\mathcal{F}_{e}$
and a decoder $\mathcal{F}_{d}$, both are stacked Transformer blocks. Here, a
shared learnable mask token $\mathbf{e}_{[\mathcal{M}]}$ functions as the
placeholder of masked patches, which are combined with the encoder’s output
and fed into the decoder. Last, an MSE loss $\mathcal{L}(\cdot)$ is computed
at masked positions for self-supervision as
$\mathcal{L}=\|\mathbf{I}_{m}-\hat{\mathbf{I}}_{m}\|_{2}^{2}$.

###  3.2. Motivation

To accurately score the quality and aesthetics of visual content, a broad
range of VS-related factors necessitate examination, namely high-level
attributes (e.g., semantics, composition etc. ) and low-level distortions
(e.g., blur, noise etc. ). By analysing the insightful features of MIM, we
believe the pretrained models have the potential to be both quality-aware and
aesthetics-aware, described next.

First, it has been proved that MIM has the ability to comprehend the high-
level semantics of the image (Bao et al., 2022; He et al., 2022a). During
pretraining, the large masking ratio forces the model to reconstruct the
masked area provided with a few visible patches. Depicted by Fig. 2 (a), the
pretrained model gives semantically plausible reconstruction even when 90% of
the pixels are masked. Second, MIM is proved to be biased towards low-level
details when reconstructing (Liu et al., 2023b, a) the raw pixels. Due to the
perfect reconstruction of pixel values, the model focuses on intricate details
(e.g., texture with repeated patterns) besides understanding the content,
allowing for a better perception of low-level distortion. To better illustrate
the distortion-awareness of MIM, we separately apply blurring and noise to the
same image. Reconstruction results in Fig. 2 (b) show that the pretrained
model can perceive distortions in low- and high-frequency intervals,
respectively.

Despite MIM has the potential to encompass VS-related factors comprehensively,
unleashing its power on downstream VS tasks still presents a non-trivial
endeavor. We dissect the MIM framework and identify three components that
contribute to this gap:

  * •

Data. ImageNet (Deng et al., 2009) has become the de-facto pretraining data in
MIM studies. The images generally exhibit low resolution and lack intricate
details. Over the years, supporting evidence from psychophysical studies has
indicated the richness of details (e.g., spatial complexity) in visual content
directly impacts human eye’s perception of quality and aesthetics (Egiazarian
et al., 2006; He et al., 2014; Gupta et al., 2011; Wang et al., 2008). Thus,
pretraining on data lacking details might not be sufficient for model to
exploit fine-grained quality and aesthetics information. In all, curating
pretraining data tailored for VS is of utmost importance.

  * •

Degradation. MIM achieves excellent results in high-level tasks with simple
degradations (e.g., random cropping) (He et al., 2022a; Xie et al., 2022).
Thus, previous works pay less attention to the degradation design. Yet, simple
degradations can only encompass VS-related factors presented in common
scenarios (e.g., content editing), overlooking other factors introduced by
various visual applications, such as compression, transmission, and
unprofessional shooting. Therefore, degradations that cover extensive VS-
related factors need to be considered.

  * •

Model. HVS assesses quality and aesthetics in a multi-scale fashion (Ke et
al., 2021). Additionally, numerous previous works have proved the benefits of
utilizing multi-scale features in other vision tasks (Lin et al., 2017; Hosu
et al., 2019a). As a result, to mimic HVS and capture both fine-grained and
coarse-grained VS-related factors, improving the model structure is
exceedingly crucial.

###  3.3. Data

Demonstrated in Fig. 4, pretraining data of QPT V2 is curated from two
criteria: high resolution (HR) and high foreground coverage (HFC). As argued
above, by reconstructing the rich textures and local structures within the HR
images, models are prone to perceive a broad range of quality and aesthetics
information during pretraining. In addition, FC is defined as the proportion
of foreground region of the entire image. Since the foreground region encodes
way more semantics and texture than the background, pretraining on HFC images
ensures the model’s sensitivity to both high-level and low-level visual
attributes.

Based on the two criteria, multiple datasets with various resolutions and FC
are investigated. We resort to SA-1B (Kirillov et al., 2023) as the
pretraining data source for the following reasons. First, SA-1B has an average
resolution of approximately 1600$\times$2100, which is significantly higher
than that of ImageNet (469$\times$387). Second, although widely used datasets
(e.g., DIV2K (Agustsson and Timofte, 2017), UnsplashFull (Lugmayr et al.,
2021) etc. ) in super-resolution (SR) task possess higher resolution
(1972$\times$1435 and 4300$\times$3938 on average), SA-1B exhibits a
significantly higher FC. To maintain the resolution of HR images while
adapting to the small input size of the model (e.g., 224$\times$224),
degradations $\mathcal{A}(\cdot)$ in Equ. 1 typically include random cropping,
which further widens the gap between SA-1B and SR datasets in terms of FC.
Fig. 3 highlights this difference. Note that all binary masks are generated by
(Reis et al., 2023). Third, SA-1B provides a straightforward criterion, namely
the number of objects per image, which allows us to further filter the dataset
to get images with higher FC. Eventually, the pretraining dataset consists of
1.28 million HR images filtered from SA-1B, with each image containing 50 or
more objects. The effectiveness of HR & HFC data on downstream VS tasks is
validated in Sec.4.3.

###  3.4. Degradation

Figure 5. Illustration of the studied degradations, each transforms data
stochastically. Figure 6. Illustration of our comprehensive degradation
selection. We only consider frequency-based degradations that result in
different frequency distributions.

To comprehensively cover the VS-related factors, degradation type and
composition are studied. Fig. 5 visualizes all the degradations studied in
this work. First, to account for VS-related factors introduced by geometric
transformation, resizing is considered. Second, to cover the factors
introduced by frequency shift, blurring, sharpening, and gaussian noise are
studied. Last, to incorporate the factors introduced by color changing, color
jittering and color space transformation (CST) are considered. Following
(Madhusudana et al., 2022), we employ four color spaces including RGB, LAB,
HSV and grayscale. Fig. 6 gives the completeness of our degradations. In terms
of degradation composition, two strategies are adopted. First, we compose
degradations sequentially. Second, inspired by recent progress in SR (Zhao et
al., 2023; Wang et al., 2021), an advanced composition including shuffling,
skipping and high-order operations is used to simulate complex degradations.
Random cropping is applied after all the degradations.

Benefiting from the comprehensiveness of our degradation selection, we
discover that CST stands out as the most quality- and aesthetics-aware
degradation. Previous NSS-based VS studies have demonstrated that the VS-
related information exists in various color spaces (Ghadiyaram and Bovik,
2016b; Tu et al., 2021c) and subsequent studies further proposed that the
information of different color spaces are complementary to each other
(Madhusudana et al., 2022). Therefore, we speculate that applying CST to the
reconstruction target exposes a richer set of quality and aesthetics factors
to the model, improving the data diversity during pretraining. Different from
previous pretraining objectives based on contrastive learning, we further
reveal the fact that QPT V2 does not benefit from the sequential or advanced
composition of degradations. More details of both findings can be found in
Sec.4.3.

###  3.5. Model

To perceive the quality and aesthetics information at different scales,
encoder architecture and multi-scale feature fusion are considered. Regarding
the selection of the encoder architecture, the common choices are ViT
(Dosovitskiy et al., 2021) and hierarchical backbones (e.g., Swin (Liu et al.,
2021) and HiViT (Zhang et al., 2023a)). Compared to ViT, hierarchical
backbones are better at learning multi-scale features by leveraging image-
related inductive biases. Thus, a representative hierarchical backbone HiViT
is selected as the encoder.

There are three stages of different scales in HiViT. Upon that, we devise a
fusion module to incorporate the multi-scale features output by different
stages. The fusion process is described next. The hierarchical encoder
$\mathcal{F}_{e}$ outputs features at multiple stages during pretraining,
shown in Fig. 4. These features are denoted by $X=\\{x_{i}\\}_{1\leq i\leq
N}$, where $N$ represents the number of stage. First, $x_{i}$ is processed by
a projection layer $\mathcal{P}_{i}(\cdot)$, which aligns the feature space
between outputs of different stages, as:

(3) $\begin{split}\overline{X}=\\{\mathcal{P}_{i}(x_{i})\\}_{1\leq i\leq N}\\\
\end{split}$

Second, the projected features of all stages $\overline{X}$, are integrated by
a fusion layer $\widetilde{F}(\cdot)$ as:

(4) $\begin{split}Y=\widetilde{F}(\overline{X})\\\ \end{split}$

$Y$ will be fed into the decoder $\mathcal{F}_{d}$ for pixel reconstruction.
Note that the fusion process is only introduced during pretraining, without
affecting the finetuning stage. More details of the architecture selection and
feature fusion are in Sec.4.3.

##  4\. Experiments

In this section, experimental setups are first introduced in Sec.4.1. By
comparing to existing SOTA methods in Sec.4.2, QPT V2 is evaluated on 11
benchmarks from all three VS tasks. Last, an in-depth ablation over QPT V2 is
provided in Sec.4.3.

###  4.1. Evaluation Setups

#### Criteria

SRCC (Spearman rank correlation coef.) and PLCC (Pearson linear correlation
coef.) are adopted as evaluation criteria for all three tasks, both ranging in
[0, 1]. A larger SRCC indicates a better ranking between samples, and a larger
PLCC shows a more accurate score prediction.

#### Benchmarks

11 benchmarks are selected from IQA, VQA, and IAA to comprehensively evaluate
the visual scoring ability of QPT V2. For IQA, three synthetically degraded
datasets (TID2013 (Ponomarenko et al., 2013), LIVE (Sheikh et al., 2006),
KADID (Lin et al., 2019)) and three datasets with real-world distortions
(KonIQ10K (Hosu et al., 2020), CLIVE (Ghadiyaram and Bovik, 2016a), FLIVE
(Ying et al., 2020)) are included. For VQA, we choose three public NR-VQA
datasets, including LIVE-VQC (Sinno and Bovik, 2019), KoNViD-1k (Hosu et al.,
2017), and LSVQ (Ying et al., 2021). For IAA, AVA (Murray et al., 2012) is
selected for evaluation.

The key designs of QPT V2 are ablated on FLIVE, LIVE-VQC, and AVA. For all the
datasets without official splitting, we randomly split them into 80% for
training and 20% for testing. The finetuning/evaluation procedure is conducted
on 10 different splittings to avoid randomness, and the average SRCC and PLCC
are reported.

#### Pretraining details

Table 1. Performance of existing SOTA methods and the proposed QPT V2 on three
synthetic and three real-world IQA datasets. ”-” means missing corresponding
results in the original paper. The best and second-best results are bolded and
underlined. Method Synthetic Real-world LIVE TID2013 KADID FLIVE CLIVE
KonIQ10K SRCC PLCC SRCC PLCC SRCC PLCC SRCC PLCC SRCC PLCC SRCC PLCC NIQE
(Mittal et al., 2013) 0.907 0.901 0.315 0.393 0.374 0.428 0.211 0.288 0.454
0.468 0.526 0.475 BRISQUE (Mittal et al., 2012) 0.939 0.935 0.604 0.694 0.528
0.567 0.288 0.373 0.601 0.621 0.715 0.702 ILNIQE (Zhang et al., 2015) 0.902
0.906 0.521 0.648 0.503 0.496 0.219 0.256 0.453 0.511 0.503 0.496 CORNIA (Ye
et al., 2012) 0.947 0.950 0.678 0.768 0.516 0.558 - - - - - - HOSA (Xu et al.,
2016) 0.946 0.950 0.735 0.815 0.618 0.653 - - - - - - DB-CNN (Zhang et al.,
2020) 0.968 0.971 0.816 0.865 0.851 0.856 0.554 0.652 0.844 0.862 0.878 0.887
HyperIQA (Su et al., 2020) 0.962 0.966 0.840 0.858 0.852 0.845 0.535 0.623
0.855 0.871 0.908 0.921 CONRTIQUE (Madhusudana et al., 2022) 0.960 0.961 0.843
0.857 0.934 0.937 0.580 0.641 0.854 0.890 0.896 0.901 Re-IQA (Saha et al.,
2023) 0.970 0.971 0.804 0.861 0.872 0.885 0.645 0.733 0.840 0.854 0.914 0.923
MUSIQ (Ke et al., 2021) - - - - - 0.566 0.661 - - 0.916 0.928 TReS (Golestaneh
et al., 2022) 0.969 0.968 0.863 0.883 0.859 0.858 0.554 0.625 0.846 0.877
0.915 0.928 QPT (Zhao et al., 2023) - - - - - - 0.610 0.677 0.895 0.914 0.927
0.941 QPT V2 0.972 0.973 0.874 0.885 0.897 0.896 0.645 0.684 0.897 0.902 0.913
0.930
---
Table 2. Performance of existing SOTA methods and the proposed QPT V2 on four
in-the-wild VQA datasets. ”-” means missing corresponding results in the
original paper. The best and second-best results are bolded and underlined.
Method Intra-dataset Cross-dataset LSVQtest LSVQ1080p LIVE-VQC KoNViD-1k SRCC
PLCC SRCC PLCC SRCC PLCC SRCC PLCC BRISQUE (Mittal et al., 2012) 0.579 0.576
0.497 0.531 0.524 0.536 0.646 0.647 TLVQM (Korhonen, 2019) 0.772 0.774 0.589
0.616 0.670 0.691 0.732 0.724 VIDEVAL (Tu et al., 2021b) 0.794 0.783 0.545
0.554 0.630 0.640 0.751 0.741 VSFA (Li et al., 2019) 0.801 0.796 0.675 0.704
0.734 0.772 0.784 0.794 BVQA (Li et al., 2022) 0.852 0.854 0.772 0.788 0.816
0.824 0.839 0.830 SimpleVQA (Sun et al., 2022) 0.867 0.861 0.764 0.803 - -
0.860 - PVQwo/patch (Ying et al., 2021) 0.814 0.816 0.686 0.708 0.781 0.781
0.747 0.796 PVQw/patch (Ying et al., 2021) 0.827 0.828 0.711 0.739 0.770 0.807
0.791 0.795 FastVQA (Wu et al., 2022) 0.876 0.877 0.779 0.814 0.823 0.844
0.859 0.855 Q-Align (Wu et al., 2023) 0.883 0.882 0.797 0.830 - - 0.865 0.877
QPT V2 0.886 0.889 0.785 0.822 0.827 0.853 0.866 0.865

All the experiments are conducted on 4 NVIDIA V100 GPUs. The pretraining data,
degradation and model are specified in Sec.3.3, Sec.3.4, and Sec.3.5,
respectively. We randomly mask 75% of the pixels following (He et al., 2022a)
and the input image size is 224$\times$224\. Hyperparameters are inherited
from (He et al., 2022a).

#### Finetuning strategy

For IQA, we implement the regression head with a simple MLP (e.g., two linear
layers with a GeLU activation in between). Following (Sun et al., 2023), we
resize the shorter edge of images to 340 while keeping the aspect ratio, then
randomly crop sub-images with size 224$\times$224\. AdamW is adopted for
optimization, with a weight decay of 0.01. The initial learning rate is 2e-5
and decayed by cosine annealing without warmup. Pretrained models are
finetuned for 200 epochs, and the checkpoint of the last epoch is selected for
evaluation. When testing, we take the four corners and the center crops and
average their predictions for the final score.

For VQA, we follow the settings in (Wu et al., 2022) for finetuning. The
pretraining weight is inflated to adapt video input, as done in (Tong et al.,
2022). For hyperparameters, AdamW is used with a weight decay of 0.05 and
batch size of 16. The initial learning rate is set to 1e-3 and decayed with a
cosine annealing strategy. The pretrained models are finetuned for 30 epochs
on LSVQtrain followed by evaluating on LSVQtest, LSVQ1080p and two other
smaller datasets, LIVE-VQC and KoNViD-1k. We uniformly sample four 32-frame
clips from an input video, and average the predicted scores as the final
results.

For IAA, pretrained models are finetuned on AVAtrain for 60 epochs and then
evaluate on AVAtest, images are resized to 224$\times$224 for evaluation. The
regression head and hyperparameters are kept consistent with IQA.

###  4.2. Comparison with state-of-the-arts

#### IQA

We compare QPT V2 with 5 traditional methods and 7 deep learning-based
methods. Results in Tab. 1 show that QPT V2 achieves superior or comparable
performances to current SOTA methods. Previous deep learning-based methods
have achieved outstanding performances on three synthetic datasets. Therefore,
further improvements on these datasets can be challenging to attain. Still,
QPT V2 improves the results on LIVE and TID2013 (e.g., +1.1% of SRCC on
TID2013). Moreover, our method also reaches leading SRCC on FLIVE and CLIVE
(+0.4% of SRCC on FLIVE), showcasing its ability to perceive real-world
distortions effectively. Besides, Tab. 1 includes methods that also harness
the power of pretraining by designing contrastive pretext tasks (e.g.,
CONRTIQUE, Re-IQA, and QPT). For example, Re-IQA respectively learns a
content-aware encoder on ImageNet-1K and a distortion-aware encoder on 758K
distorted images. In comparison, QPT V2 consumes less pretraining data and
achieves better performance.

#### VQA

We compare QPT V2 to three traditional methods and six deep learning-based
methods. Results given in Tab. 2 provide the following conclusions. First, QPT
V2 exceeds all the traditional methods that rely on hand-crafted features by a
large margin, and beats most data-driven methods on four VQA datasets. Second,
under the intra-dataset setting, QPT V2 pushes the SRCC by 0.3% and PLCC by
0.7% on LSVQtest, exhibiting accurate quality assessment. Third, under the
cross-dataset setting, we surpass the current SOTAs as well (e.g., with 0.4%
and 0.9% gains in SRCC and PLCC on LIVE-VQC), presenting impressive
generalization capability.

#### IAA

Table 3. IAA performance on the AVA dataset. The best and second-best results
are bolded and underlined.

Method | AVA_test
---|---
SRCC | PLCC
NIMA (Esfandarani and Milanfar, 2018) | 0.612 | 0.636
MLSP (Hosu et al., 2019b) | 0.756 | 0.757
AFDC (Chen et al., 2020b) | 0.649 | 0.671
MUSIQ (Ke et al., 2021) | 0.726 | 0.738
MaxViT (Tu et al., 2022) | 0.708 | 0.745
CLIP-IQA+ (Wang et al., 2023) | 0.619 | 0.586
Aesthetic Predictor (LAION, 2023) | 0.721 | 0.723
TANet (He et al., 2022b) | 0.758 | 0.765
GAT×3-GATP (Ghosal and Smolic, 2022) | 0.762 | 0.764
LIQE (Zhang et al., 2023b) | 0.776 | 0.763
VILA (Ke et al., 2023) | 0.774 | 0.774
Q-Align (Wu et al., 2023) | 0.822 | 0.817
QPT V2 (60% finetuning data) | 0.766 | 0.780
QPT V2 | 0.865 | 0.875

We select 11 deep learning-based methods for comparison. Tab. 3 indicates that
our method significantly surpasses previous SOTA results on AVA dataset,
reaching 0.865 (+4.3%) of SRCC and 0.875 (+5.8%) of PLCC. It is worth noting
that Q-Align (Wu et al., 2023) leverages the power of large multi-modality
models (LMMs). In comparison, our work introduces a new pretraining paradigm,
and exhibits lower computation and smaller model size. The advantages become
more evident when comparing to methods without utilizing LMMs (e.g., +8.9% of
SRCC and +10.1% of PLCC). The finetuning data amount is further reduced to
investigate the power of QPT V2. The results show that QPT V2 achieves parity
with some previous SOTA methods (e.g., LIQE, VILA) using only 60% finetuning
data, realizing a more data-efficient transfer. Both LIQE and VILA solve IAA
by using auxiliary knowledge in text description. In comparison, QPT V2
achieves SOTA results without the assistance of text modality.

#### QPT V2 vs. other pretext tasks

Table 4. Comparisons of finetuning evaluation using different pretext tasks on
CLIVE and LIVE-VQC, and AVA.

Pretext task | CLIVE | LIVE-VQC | AVA
---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
QPT V2 | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875
QPT (Zhao et al., 2023) | 0.610 | 0.677 | - | - | - | -
MoCo (He et al., 2020) | 0.578 | 0.629 | 0.819 | 0.828 | 0.707 | 0.712
Supervised | 0.556 | 0.604 | 0.810 | 0.825 | 0.704 | 0.690
w/o | 0.451 | 0.475 | 0.696 | 0.731 | 0.545 | 0.552

We compare QPT V2 with four pretext tasks, including QPT, MoCo, ImageNet-1K
supervised training and train-from-scratch, on three VS benchmarks. Note that
both supervised training and train-from-scratch use the same encoder as QPT
V2, which is HiViT. MoCo and QPT are based on semantics-aware and quality-
aware contrastive learning, respectively. The results in Tab. 4 verify the
superiority of QPT V2. In addition, QPT V2 also achieves better performances
than the supervised training and the one without pretrained weights in all
three VS tasks.

###  4.3. Ablation Studies

#### Effectiveness of HR & HFC data

Table 5. Ablation on resolution and foreground coverage of pretraining data.
IN1K and UF denote ImageNet-1K and UnsplashFull for simplicity.

Source | HR | HFC | FLIVE | LIVE-VQC | AVA
---|---|---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
IN1K | ✗ | ✔ | 0.617 | 0.653 | 0.812 | 0.825 | 0.759 | 0.780
UF | ✔ | ✗ | 0.602 | 0.631 | 0.799 | 0.828 | 0.778 | 0.801
SA-1B | ✔ | ✔ | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875

We demonstrate the effectiveness of HR & HFC data in QPT V2 by comparing to
models pretrained on data with different resolutions and FC. The following
conclusions can be drawn from Tab. 5. First, pretraining on data with both HR
and HFC leads to the best downstream performances. Second, high resolution
matters. When the foreground coverage is generally high, pretraining on HR
images yields noticeably superior performance on all three representative VS
datasets. e.g., +2.9% on FLIVE, +2.2% on LIVE-VQC and +10.1% on the AVA
dataset. Last, HFC is essential. When the resolution is relatively high, HFC
data always prevail. e.g., +4.8% on FLIVE, +2.7% on LIVE-VQC , and +8.1% on
the AVA dataset.

#### Effectiveness of quality- and aesthetics-aware degradation

Table 6. Ablation on single degradation type, each transforms data
stochastically.

Deg. | FLIVE | LIVE-VQC | AVA
---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
None | 0.616 | 0.664 | 0.813 | 0.836 | 0.832 | 0.820
Resizing | 0.593 | 0.621 | 0.797 | 0.815 | 0.774 | 0.752
Blurring | 0.628 | 0.664 | 0.813 | 0.833 | 0.827 | 0.831
Sharpening | 0.617 | 0.650 | 0.803 | 0.820 | 0.801 | 0.786
Noise | 0.602 | 0.614 | 0.793 | 0.810 | 0.773 | 0.747
CST | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875
Color jittering | 0.623 | 0.649 | 0.809 | 0.826 | 0.788 | 0.792

Table 7. Ablation on different forms of degradation composition. CST and B
denote color space transformation and blurring for simplicity.

Comp. | Deg. | FLIVE | LIVE-VQC | AVA
---|---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
None | CST | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875
B | 0.628 | 0.664 | 0.813 | 0.833 | 0.827 | 0.831
Sequential |  B$\rightarrow$CST | 0.645 | 0.671 | 0.806 | 0.824 | 0.821 | 0.840
CST$\rightarrow$B | 0.637 | 0.674 | 0.815 | 0.838 | 0.855 | 0.874
Advanced | All | 0.603 | 0.652 | 0.797 | 0.811 | 0.820 | 0.839

Tab. 6 displays the downstream performances after applying six different
degradations to the reconstructed target in QPT V2. Results obtained without
employing any form of degradation serve as the baseline. First, the CST
degradation incorporated in QPT V2 performs the best, demonstrating its
quality- and aesthetics-awareness. Second, blurring brings a slight
improvement in IAA (+0.5% of SRCC and +1.1% of PLCC). Recent MIM studies find
that removing the high-frequency components of pixels helps the model to focus
on semantics, benefiting downstream high-level tasks (Liu et al., 2023a).
Thus, we attribute the gains to the fact that IAA places greater emphasis on
high-level visual attributes compared to IQA and VQA (Wu et al., 2023). Last,
the geometry-based resizing, frequency-based sharpening and noise, and color
jittering impair the downstream performances on all three benchmarks. This
suggests that, altering the spatial layout of the reconstruction target,
enriching its high-frequency details, corrupting its frequency spectrum, or
perturbing its color are all detrimental to the model’s quality- and
aesthetics-awareness.

To study the effect of degradation composition, two top-performing
degradations in Tab. 6, namely, CST and blurring are arranged sequentially.
Also, all six degradations are included in an advanced composition protocol
discussed in Sec.3.4. Tab. 7 indicates two findings. First, CST and blurring
cannot synergize when arranged sequentially, leading to slightly inferior
results. Second, QPT V2 does not benefit from a complicated degradation space.
The above findings are inconsistent with those in contrastive learning, we
attribute them to the distinction between two pretraining paradigms.

#### Effectiveness of multi-scale model

Tab. 8 validates the effectiveness of the encoder architecture selection. With
similar model capacity, hierarchical backbones outperform plain ViT in all
three VS tasks. For example, Swin-T reaches 0.868 of SRCC on AVA dataset, 4.2%
higher than ViT-S. Since two hierarchical models exhibit similar downstream
performances, we opt for HiViT-T as the encoder in QPT V2 for better training
efficiency. Though increasing the model capacity might potentially yield
better results, we did not use larger models out of tradeoffs, which can be
done in future work.

To validate the effectiveness of the multi-scale feature fusion strategy
proposed in Sec.3.5, we fuse features at different stages, and the downstream
results are given in Tab. 9. By default, the output of the last stage (stage
3) is always fed to the decoder. Tab. 9 indicates that multi-scale feature
fusion always provides benefits for VS tasks. Particularly, fusing features
from the shallow stage (stage 1) yields the most significant gains on three
downstream datasets. Due to the inclusion of more low-level details in shallow
layer features, we believe that fusing these features assists the model in
better perceiving low-level VS-related factors. Additionally, the
implementation choices of the projection layer $\mathcal{P}_{i}(\cdot)$ and
the fusion layer $\widetilde{F}(\cdot)$ specified in Sec.3.5 are discussed in
Tab. 10. Following conclusions can be drawn: First, a simple linear layer is
sufficient to project representation into the same feature space. A more
complex MLP (e.g., Linear-GeLU-Linear structure) cannot bring improvement
while introducing non-negligible computational overhead. We think the non-
linearity may increase the optimization difficulty as for pretraining. Second,
weighted average pooling is better suited for integrating projected features
compared to the simple summation.

Table 8. Ablation on the selection of the encoder architecture. MS denotes
multi-scale for simplicity.

Model | MS | Param | FLIVE | LIVE-VQC | AVA
---|---|---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
ViT-S | ✗ | 22M | 0.614 | 0.651 | 0.809 | 0.835 | 0.822 | 0.832
Swin-T | ✔ | 28M | 0.647 | 0.671 | 0.828 | 0.850 | 0.868 | 0.863
HiViT-T | ✔ | 19M | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875

Table 9. Ablation on the location for feature fusion.

Stage | FLIVE | LIVE-VQC | AVA
---|---|---|---
1 | 2 | SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
✗ | ✗ | 0.643 | 0.650 | 0.814 | 0.837 | 0.848 | 0.858
✔ | ✗ | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875
✗ | ✔ | 0.643 | 0.671 | 0.819 | 0.838 | 0.842 | 0.863
✔ | ✔ | 0.654 | 0.672 | 0.818 | 0.838 | 0.854 | 0.869

Table 10. Ablation on different feature fusion implementations. Linear and MLP
represent the projection layer, while Pool and Sum represent the fusion
strategies.

Linear | MLP | Pool | Sum | FLIVE | LIVE-VQC | AVA
---|---|---|---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
✔ |  | ✔ |  | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875
✔ |  |  | ✔ | 0.638 | 0.667 | 0.804 | 0.821 | 0.820 | 0.830
| ✔ | ✔ |  | 0.656 | 0.682 | 0.820 | 0.848 | 0.858 | 0.864

#### Impact of pretraining masking ratio

A clear conclusion can be drawn from Tab. 11: Masking ratio needs to be within
a reasonable range (i.e., 60% or 75%). An excessively high ratio (i.e., 90%)
will result in a significant loss of quality and aesthetics information, while
a low ratio (i.e., 30%) overly simplifies the pixel reconstruction task,
preventing the model from perceiving the quality and aesthetics information
within the image. As a result, we simply adhere to (He et al., 2022a) for
better performances and fair comparison.

Table 11. Ablation on masking ratio during pretraining.

ratio | FLIVE | LIVE-VQC | AVA
---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
75% | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875
30% | 0.619 | 0.652 | 0.788 | 0.808 | 0.721 | 0.733
60% | 0.645 | 0.679 | 0.823 | 0.854 | 0.857 | 0.870
90% | 0.604 | 0.644 | 0.781 | 0.798 | 0.747 | 0.751

#### Impact of pretraining data amount

We study the impact of data amount on QPT V2 by using 20%, 50% and 100%
percentages of the pretraining data. Given by Tab. 12, the performances on
three downstream datasets continue to improve as the data amount increases.
Surprisingly, we find that even when using only 50% of the data, QPT V2 still
achieves comparable performances to SOTA methods in IAA (e.g., 0.770 of SRCC,
0.774 of PLCC on AVA dataset), demonstrating its strong aesthetics perception
capabilities. Furthermore, using diverse data sources to scale up HR & HFC
data remains an open question and could be explored in future work.

Table 12. Impact of data amount for the pretext task of QPT V2, using
different percentages of the pretraining dataset.

Percentage | FLIVE | LIVE-VQC | AVA
---|---|---|---
SRCC | PLCC | SRCC | PLCC | SRCC | PLCC
20% | 0.529 | 0.533 | 0.772 | 0.796 | 0.586 | 0.602
50% | 0.610 | 0.647 | 0.805 | 0.829 | 0.770 | 0.774
100% | 0.645 | 0.684 | 0.827 | 0.853 | 0.865 | 0.875

##  5\. Conclusion

We propose QPT V2, a novel MIM-based pretraining paradigm crafted for visual
scoring tasks, aiming at alleviating the obstacle of insufficient annotated
data. To enhance the quality- and aesthetics-awareness of the pretraining
objective, we provide a meticulous analysis of the vanilla MIM framework and
make targeted improvements on three key components: pretraining data,
degradation, and model structure. After pretraining, QPT V2 achieves SOTA
results on 11 downstream benchmarks in IQA, VQA, and IAA, demonstrating
impressive capability and generalization ability. In all, we hope this work
will inspire the community to reflect and explore the possibility of different
pretraining paradigms for VS.

## References

  * (1)
  * Agustsson and Timofte (2017) Eirikur Agustsson and Radu Timofte. 2017.  NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In _CVPR Workshops_. IEEE Computer Society, 1122–1131.
  * Bao et al. (2022) Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. 2022.  BEiT: BERT Pre-Training of Image Transformers. In _ICLR_. OpenReview.net.
  * Barnett et al. (2018) Thomas Barnett, Shruti Jain, Usha Andra, and Taru Khurana. 2018.  Cisco visual networking index (vni) complete forecast update, 2017–2022.  _Americas/EMEAR Cisco Knowledge Network (CKN) Presentation_ (2018), 1–30.
  * Bosse et al. (2018) Sebastian Bosse, Dominique Maniry, Klaus-Robert Müller, Thomas Wiegand, and Wojciech Samek. 2018.  Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment.  _IEEE Trans. Image Process._ 27, 1 (2018), 206–219.
  * Chen et al. (2020b) Qiuyu Chen, Wei Zhang, Ning Zhou, Peng Lei, Yi Xu, Yu Zheng, and Jianping Fan. 2020b.  Adaptive Fractional Dilated Convolution Network for Image Aesthetics Assessment. In _CVPR_. Computer Vision Foundation / IEEE, 14102–14111.
  * Chen et al. (2020a) Xinlei Chen, Haoqi Fan, Ross B. Girshick, and Kaiming He. 2020a.  Improved Baselines with Momentum Contrastive Learning.  _CoRR_ abs/2003.04297 (2020).
  * Ciancio et al. (2011) Alexandre G. Ciancio, André Luiz N. Targino da Costa, Eduardo A. B. da Silva, Amir Said, Ramin Samadani, and Pere Obrador. 2011.  No-Reference Blur Assessment of Digital Pictures Based on Multifeature Classifiers.  _IEEE Trans. Image Process._ 20, 1 (2011), 64–75.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009.  ImageNet: A large-scale hierarchical image database. In _CVPR_. IEEE Computer Society, 248–255.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _NAACL-HLT (1)_. Association for Computational Linguistics, 4171–4186.
  * Dosovitskiy et al. (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In _ICLR_. OpenReview.net.
  * Egiazarian et al. (2006) Karen Egiazarian, Jaakko Astola, Nikolay Ponomarenko, Vladimir Lukin, Federica Battisti, and Marco Carli. 2006.  New full-reference quality metrics based on HVS. In _Proceedings of the second international workshop on video processing and quality metrics_ , Vol. 4. 4.
  * Esfandarani and Milanfar (2018) Hossein Talebi Esfandarani and Peyman Milanfar. 2018.  NIMA: Neural Image Assessment.  _IEEE Trans. Image Process._ 27, 8 (2018), 3998–4011.
  * Fang et al. (2023) Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023.  EVA: Exploring the Limits of Masked Visual Representation Learning at Scale. In _CVPR_. IEEE, 19358–19369.
  * Fang et al. (2020) Yuming Fang, Hanwei Zhu, Yan Zeng, Kede Ma, and Zhou Wang. 2020.  Perceptual Quality Assessment of Smartphone Photography. In _CVPR_. Computer Vision Foundation / IEEE, 3674–3683.
  * Ghadiyaram and Bovik (2016a) Deepti Ghadiyaram and Alan C. Bovik. 2016a.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality.  _IEEE Trans. Image Process._ 25, 1 (2016), 372–387.
  * Ghadiyaram and Bovik (2016b) Deepti Ghadiyaram and Alan C. Bovik. 2016b.  Perceptual Quality Prediction on Authentically Distorted Images Using a Bag of Features Approach.  _CoRR_ abs/1609.04757 (2016).
  * Ghosal and Smolic (2022) Koustav Ghosal and Aljosa Smolic. 2022.  Image Aesthetics Assessment Using Graph Attention Network. In _ICPR_. IEEE, 3160–3167.
  * Golestaneh et al. (2022) S. Alireza Golestaneh, Saba Dadsetan, and Kris M. Kitani. 2022.  No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency. In _WACV_. IEEE, 3989–3999.
  * Gupta et al. (2011) Prateek Gupta, Priyanka Srivastava, Satyam Bhardwaj, and Vikrant Bhateja. 2011.  A modified PSNR metric based on HVS for quality assessment of color images. In _2011 International Conference on Communication and Industrial Application_. IEEE, 1–4.
  * He et al. (2022a) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. 2022a.  Masked Autoencoders Are Scalable Vision Learners. In _CVPR_. IEEE, 15979–15988.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020.  Momentum Contrast for Unsupervised Visual Representation Learning. In _CVPR_. Computer Vision Foundation / IEEE, 9726–9735.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep Residual Learning for Image Recognition. In _CVPR_. IEEE Computer Society, 770–778.
  * He et al. (2014) Lihuo He, Fei Gao, Weilong Hou, and Lei Hao. 2014.  Objective image quality assessment: a survey.  _Int. J. Comput. Math._ 91, 11 (2014), 2374–2388.
  * He et al. (2022b) Shuai He, Yongchang Zhang, Rui Xie, Dongxiang Jiang, and Anlong Ming. 2022b.  Rethinking Image Aesthetics Assessment: Models, Datasets and Benchmarks. In _IJCAI_. ijcai.org, 942–948.
  * Hosu et al. (2019a) Vlad Hosu, Bastian Goldlücke, and Dietmar Saupe. 2019a.  Effective Aesthetics Prediction With Multi-Level Spatially Pooled Features. In _CVPR_. Computer Vision Foundation / IEEE, 9375–9383.
  * Hosu et al. (2019b) Vlad Hosu, Bastian Goldlücke, and Dietmar Saupe. 2019b.  Effective Aesthetics Prediction With Multi-Level Spatially Pooled Features. In _CVPR_. Computer Vision Foundation / IEEE, 9375–9383.
  * Hosu et al. (2017) Vlad Hosu, Franz Hahn, Mohsen Jenadeleh, Hanhe Lin, Hui Men, Tamás Szirányi, Shujun Li, and Dietmar Saupe. 2017.  The Konstanz natural video database (KoNViD-1k). In _QoMEX_. IEEE, 1–6.
  * Hosu et al. (2020) Vlad Hosu, Hanhe Lin, Tamás Szirányi, and Dietmar Saupe. 2020.  KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment.  _IEEE Trans. Image Process._ 29 (2020), 4041–4056.
  * Hou et al. (2022) Zejiang Hou, Fei Sun, Yen-Kuang Chen, Yuan Xie, and Sun-Yuan Kung. 2022.  MILAN: Masked Image Pretraining on Language Assisted Representation.  _CoRR_ abs/2208.06049 (2022).
  * Isogawa et al. (2019) Mariko Isogawa, Dan Mikami, Kosuke Takahashi, Daisuke Iwai, Kosuke Sato, and Hideaki Kimata. 2019.  Which is the Better Inpainted Image?Training Data Generation Without Any Manual Operations.  _Int. J. Comput. Vis._ 127, 11-12 (2019), 1751–1766.
  * Kang et al. (2014) Le Kang, Peng Ye, Yi Li, and David S. Doermann. 2014.  Convolutional Neural Networks for No-Reference Image Quality Assessment. In _CVPR_. IEEE Computer Society, 1733–1740.
  * Ke et al. (2021) Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. 2021.  MUSIQ: Multi-scale Image Quality Transformer. In _ICCV_. IEEE, 5128–5137.
  * Ke et al. (2023) Junjie Ke, Keren Ye, Jiahui Yu, Yonghui Wu, Peyman Milanfar, and Feng Yang. 2023.  VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining. In _CVPR_. IEEE, 10041–10051.
  * Kirillov et al. (2023) Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloé Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross B. Girshick. 2023.  Segment Anything. In _ICCV_. IEEE, 3992–4003.
  * Korhonen (2019) Jari Korhonen. 2019.  Two-Level Approach for No-Reference Consumer Video Quality Assessment.  _IEEE Trans. Image Process._ 28, 12 (2019), 5923–5938.
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012.  ImageNet Classification with Deep Convolutional Neural Networks. In _NIPS_. 1106–1114.
  * LAION (2023) LAION. 2023.  _aesthetic-predictor_.  https://github.com/LAION-AI/aesthetic-predictor
  * Li et al. (2022) Bowen Li, Weixia Zhang, Meng Tian, Guangtao Zhai, and Xianpei Wang. 2022.  Blindly Assess Quality of In-the-Wild Videos via Quality-Aware Pre-Training and Motion Perception.  _IEEE Trans. Circuits Syst. Video Technol._ 32, 9 (2022), 5944–5958.
  * Li et al. (2019) Dingquan Li, Tingting Jiang, and Ming Jiang. 2019.  Quality Assessment of In-the-Wild Videos. In _ACM Multimedia_. ACM, 2351–2359.
  * Li et al. (2021a) Dingquan Li, Tingting Jiang, and Ming Jiang. 2021a.  Unified Quality Assessment of in-the-Wild Videos with Mixed Datasets Training.  _Int. J. Comput. Vis._ 129, 4 (2021), 1238–1257.
  * Li et al. (2021b) Dingquan Li, Tingting Jiang, Ming Jiang, Vajira Lasantha Thambawita, and Haoliang Wang. 2021b.  Reproducibility Companion Paper: Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment. In _ACM Multimedia_. ACM, 3615–3618.
  * Lin et al. (2019) Hanhe Lin, Vlad Hosu, and Dietmar Saupe. 2019.  KADID-10k: A Large-scale Artificially Distorted IQA Database. In _QoMEX_. IEEE, 1–3.
  * Lin et al. (2017) Tsung-Yi Lin, Piotr Dollár, Ross B. Girshick, Kaiming He, Bharath Hariharan, and Serge J. Belongie. 2017.  Feature Pyramid Networks for Object Detection. In _CVPR_. IEEE Computer Society, 936–944.
  * Liu et al. (2017) Xialei Liu, Joost van de Weijer, and Andrew D. Bagdanov. 2017.  RankIQA: Learning from Rankings for No-Reference Image Quality Assessment. In _ICCV_. IEEE Computer Society, 1040–1049.
  * Liu et al. (2023a) Yuan Liu, Songyang Zhang, Jiacheng Chen, Kai Chen, and Dahua Lin. 2023a.  PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling.  _CoRR_ abs/2303.02416 (2023).
  * Liu et al. (2023b) Yuan Liu, Songyang Zhang, Jiacheng Chen, Zhaohui Yu, Kai Chen, and Dahua Lin. 2023b.  Improving Pixel-based MIM by Reducing Wasted Modeling Capability. In _ICCV_. IEEE, 5338–5349.
  * Liu et al. (2021) Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In _ICCV_. IEEE, 9992–10002.
  * Lu et al. (2024) Yiting Lu, Xin Li, Yajing Pei, Kun Yuan, Qizhi Xie, Yunpeng Qu, Ming Sun, Chao Zhou, and Zhibo Chen. 2024.  Kvq: Kwai video quality assessment for short-form videos. In _CVPR_. 25963–25973.
  * Lugmayr et al. (2021) Andreas Lugmayr, Martin Danelljan, and Radu Timofte. 2021.  NTIRE 2021 Learning the Super-Resolution Space Challenge. In _CVPR Workshops_. Computer Vision Foundation / IEEE, 596–612.
  * Madhusudana et al. (2022) Pavan C. Madhusudana, Neil Birkbeck, Yilin Wang, Balu Adsumilli, and Alan C. Bovik. 2022.  Image Quality Assessment Using Contrastive Learning.  _IEEE Trans. Image Process._ 31 (2022), 4149–4161.
  * Mittal et al. (2012) Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012.  No-Reference Image Quality Assessment in the Spatial Domain.  _IEEE Trans. Image Process._ 21, 12 (2012), 4695–4708.
  * Mittal et al. (2013) Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. 2013.  Making a ”Completely Blind” Image Quality Analyzer.  _IEEE Signal Process. Lett._ 20, 3 (2013), 209–212.
  * Murray et al. (2012) Naila Murray, Luca Marchesotti, and Florent Perronnin. 2012.  AVA: A large-scale database for aesthetic visual analysis. In _CVPR_. IEEE Computer Society, 2408–2415.
  * Park et al. (2023) Namuk Park, Wonjae Kim, Byeongho Heo, Taekyung Kim, and Sangdoo Yun. 2023.  What Do Self-Supervised Vision Transformers Learn?. In _ICLR_. OpenReview.net.
  * Peng et al. (2022) Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei. 2022.  BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers.  _CoRR_ abs/2208.06366 (2022).
  * Ponomarenko et al. (2013) Nikolay N. Ponomarenko, Oleg Ieremeiev, Vladimir V. Lukin, Karen O. Egiazarian, Lina Jin, Jaakko Astola, Benoît Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, and C.-C. Jay Kuo. 2013.  Color image database TID2013: Peculiarities and preliminary results. In _EUVIP_. IEEE, 106–111.
  * Qu et al. (2024) Yunpeng Qu, Kun Yuan, Kai Zhao, Qizhi Xie, Jinhua Hao, Ming Sun, and Chao Zhou. 2024.  XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution.  _CoRR_ abs/2403.05049 (2024).
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021.  Learning Transferable Visual Models From Natural Language Supervision. In _ICML_ _(Proceedings of Machine Learning Research, Vol. 139)_. PMLR, 8748–8763.
  * Ramesh et al. (2021) Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021.  Zero-Shot Text-to-Image Generation. In _ICML_ _(Proceedings of Machine Learning Research, Vol. 139)_. PMLR, 8821–8831.
  * Reis et al. (2023) Dillon Reis, Jordan Kupec, Jacqueline Hong, and Ahmad Daoudi. 2023.  Real-time flying object detection with YOLOv8.  _arXiv preprint arXiv:2305.09972_ (2023).
  * Rippel et al. (2019) Oren Rippel, Sanjay Nair, Carissa Lew, Steve Branson, Alexander G. Anderson, and Lubomir D. Bourdev. 2019.  Learned Video Compression. In _ICCV_. IEEE, 3453–3462.
  * Saha et al. (2023) Avinab Saha, Sandeep Mishra, and Alan C. Bovik. 2023.  Re-IQA: Unsupervised Learning for Image Quality Assessment in the Wild. In _CVPR_. IEEE, 5846–5855.
  * Schroff et al. (2015) Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015.  FaceNet: A unified embedding for face recognition and clustering. In _CVPR_. IEEE Computer Society, 815–823.
  * Sheikh et al. (2006) Hamid R. Sheikh, Muhammad F. Sabir, and Alan C. Bovik. 2006.  A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms.  _IEEE Trans. Image Process._ 15, 11 (2006), 3440–3451.
  * Simoncelli and Olshausen (2001) Eero P Simoncelli and Bruno A Olshausen. 2001.  Natural image statistics and neural representation.  _Annual review of neuroscience_ 24, 1 (2001), 1193–1216.
  * Sinno and Bovik (2019) Zeina Sinno and Alan Conrad Bovik. 2019.  Large-Scale Study of Perceptual Video Quality.  _IEEE Trans. Image Process._ 28, 2 (2019), 612–627.
  * Su et al. (2020) Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. 2020.  Blindly Assess Image Quality in the Wild Guided by a Self-Adaptive Hyper Network. In _CVPR_. Computer Vision Foundation / IEEE, 3664–3673.
  * Sun et al. (2022) Wei Sun, Xiongkuo Min, Wei Lu, and Guangtao Zhai. 2022.  A Deep Learning based No-reference Quality Assessment Model for UGC Videos. In _ACM Multimedia_. ACM, 856–865.
  * Sun et al. (2023) Wei Sun, Xiongkuo Min, Danyang Tu, Siwei Ma, and Guangtao Zhai. 2023.  Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Training.  _IEEE J. Sel. Top. Signal Process._ 17, 6 (2023), 1178–1192.
  * Tong et al. (2022) Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. 2022.  VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training. In _NeurIPS_.
  * Tu et al. (2022) Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan C. Bovik, and Yinxiao Li. 2022.  MaxViT: Multi-axis Vision Transformer. In _ECCV (24)_ _(Lecture Notes in Computer Science, Vol. 13684)_. Springer, 459–479.
  * Tu et al. (2021a) Zhengzhong Tu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, and Alan C. Bovik. 2021a.  UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content.  _IEEE Trans. Image Process._ 30 (2021), 4449–4464.
  * Tu et al. (2021b) Zhengzhong Tu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, and Alan C. Bovik. 2021b.  UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content.  _IEEE Trans. Image Process._ 30 (2021), 4449–4464.
  * Tu et al. (2021c) Zhengzhong Tu, Xiangxu Yu, Yilin Wang, Neil Birkbeck, Balu Adsumilli, and Alan C. Bovik. 2021c.  RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content.  _CoRR_ abs/2101.10955 (2021).
  * van den Oord et al. (2018) Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018.  Representation Learning with Contrastive Predictive Coding.  _CoRR_ abs/1807.03748 (2018).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is All you Need. In _NIPS_. 5998–6008.
  * Wang et al. (2008) Bo Wang, Zhibing Wang, Yupeng Liao, and Xinggang Lin. 2008.  HVS-based structural similarity for image quality assessment. In _2008 9th International Conference on Signal Processing_. IEEE, 1194–1197.
  * Wang et al. (2023) Jianyi Wang, Kelvin C. K. Chan, and Chen Change Loy. 2023.  Exploring CLIP for Assessing the Look and Feel of Images. In _AAAI_. AAAI Press, 2555–2563.
  * Wang et al. (2022) Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, and Furu Wei. 2022.  Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks.  _CoRR_ abs/2208.10442 (2022).
  * Wang et al. (2021) Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. 2021.  Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data. In _ICCVW_. IEEE, 1905–1914.
  * Wang et al. (2019) Yilin Wang, Sasi Inguva, and Balu Adsumilli. 2019.  YouTube UGC Dataset for Video Compression Research. In _MMSP_. IEEE, 1–5.
  * Wei et al. (2022a) Chen Wei, Haoqi Fan, Saining Xie, Chao-Yuan Wu, Alan L. Yuille, and Christoph Feichtenhofer. 2022a.  Masked Feature Prediction for Self-Supervised Visual Pre-Training. In _CVPR_. IEEE, 14648–14658.
  * Wei et al. (2022b) Longhui Wei, Lingxi Xie, Wengang Zhou, Houqiang Li, and Qi Tian. 2022b.  MVP: Multimodality-Guided Visual Pre-training. In _ECCV (30)_ _(Lecture Notes in Computer Science, Vol. 13690)_. Springer, 337–353.
  * Wu et al. (2022) Haoning Wu, Chaofeng Chen, Jingwen Hou, Liang Liao, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. 2022.  FAST-VQA: Efficient End-to-End Video Quality Assessment with Fragment Sampling. In _ECCV (6)_ _(Lecture Notes in Computer Science, Vol. 13666)_. Springer, 538–554.
  * Wu et al. (2023) Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guangtao Zhai, and Weisi Lin. 2023.  Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels.  _CoRR_ abs/2312.17090 (2023).
  * Xie et al. (2022) Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. 2022.  SimMIM: a Simple Framework for Masked Image Modeling. In _CVPR_. IEEE, 9643–9653.
  * Xu et al. (2016) Jingtao Xu, Peng Ye, Qiaohong Li, Haiqing Du, Yong Liu, and David S. Doermann. 2016.  Blind Image Quality Assessment Based on High Order Statistics Aggregation.  _IEEE Trans. Image Process._ 25, 9 (2016), 4444–4457.
  * Yang et al. (2022) Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. 2022.  MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment. In _CVPR Workshops_. IEEE, 1190–1199.
  * Ye et al. (2012) Peng Ye, Jayant Kumar, Le Kang, and David S. Doermann. 2012.  Unsupervised feature learning framework for no-reference image quality assessment. In _CVPR_. IEEE Computer Society, 1098–1105.
  * Ying et al. (2021) Zhenqiang Ying, Maniratnam Mandal, Deepti Ghadiyaram, and Alan C. Bovik. 2021.  Patch-VQ: ’Patching Up’ the Video Quality Problem. In _CVPR_. Computer Vision Foundation / IEEE, 14019–14029.
  * Ying et al. (2020) Zhenqiang Ying, Haoran Niu, Praful Gupta, Dhruv Mahajan, Deepti Ghadiyaram, and Alan C. Bovik. 2020.  From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality. In _CVPR_. Computer Vision Foundation / IEEE, 3572–3582.
  * You (2021) Junyong You. 2021.  Long Short-term Convolutional Transformer for No-Reference Video Quality Assessment. In _ACM Multimedia_. ACM, 2112–2120.
  * Yu et al. (2022) Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. 2022.  CoCa: Contrastive Captioners are Image-Text Foundation Models.  _Trans. Mach. Learn. Res._ 2022 (2022).
  * Yuan et al. (2023a) Jiquan Yuan, Xinyan Cao, Changjin Li, Fanyi Yang, Jinlong Lin, and Xixin Cao. 2023a.  PKU-I2IQA: An Image-to-Image Quality Assessment Database for AI Generated Images.  _CoRR_ abs/2311.15556 (2023).
  * Yuan et al. (2023b) Kun Yuan, Zishang Kong, Chuanchuan Zheng, Ming Sun, and Xing Wen. 2023b.  Capturing Co-existing Distortions in User-Generated Content for No-reference Video Quality Assessment. In _Proceedings of the 31st ACM International Conference on Multimedia_. 1098–1107.
  * Yuan et al. (2024) Kun Yuan, Hongbo Liu, Mading Li, Muyi Sun, Ming Sun, Jiachao Gong, Jinhua Hao, Chao Zhou, and Yansong Tang. 2024.  PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild. In _CVPR_. 2835–2845.
  * Zhang et al. (2015) Lin Zhang, Lei Zhang, and Alan C. Bovik. 2015.  A Feature-Enriched Completely Blind Image Quality Evaluator.  _IEEE Trans. Image Process._ 24, 8 (2015), 2579–2591.
  * Zhang et al. (2019) Wenlong Zhang, Yihao Liu, Chao Dong, and Yu Qiao. 2019.  RankSRGAN: Generative Adversarial Networks With Ranker for Image Super-Resolution. In _ICCV_. IEEE, 3096–3105.
  * Zhang et al. (2020) Weixia Zhang, Kede Ma, Jia Yan, Dexiang Deng, and Zhou Wang. 2020.  Blind Image Quality Assessment Using a Deep Bilinear Convolutional Neural Network.  _IEEE Trans. Circuits Syst. Video Technol._ 30, 1 (2020), 36–47.
  * Zhang et al. (2023b) Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. 2023b.  Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective. In _CVPR_. IEEE, 14071–14081.
  * Zhang et al. (2023a) Xiaosong Zhang, Yunjie Tian, Lingxi Xie, Wei Huang, Qi Dai, Qixiang Ye, and Qi Tian. 2023a.  HiViT: A Simpler and More Efficient Design of Hierarchical Vision Transformer. In _ICLR_. OpenReview.net.
  * Zhao et al. (2023) Kai Zhao, Kun Yuan, Ming Sun, Mading Li, and Xing Wen. 2023.  Quality-aware Pre-trained Models for Blind Image Quality Assessment.  _CoRR_ abs/2303.00521 (2023).