# Reduced-Reference Quality Assessment of Point Clouds via Content-Oriented
Saliency Projection

Wei Zhou, Guanghui Yue, Ruizeng Zhang, Yipeng Qin, and Hantao Liu This work
was supported in part by NSFC under Grant 62001302 and Guangdong Basic and
Applied Basic Research Foundation 2019A1515111205.W. Zhou is with the
Department of Electrical and Computer Engineering, University of Waterloo,
Waterloo, ON N2L 3G1, Canada (e-mail: wei.zhou@uwaterloo.ca).G. Yue is with
the School of Biomedical Engineering, Shenzhen University, Shenzhen 518060,
China (e-mail: yueguanghui@szu.edu.cn).R. Zhang is with Beijing Institute of
Technology, Beijing 100081, China (e-mail: hireason@163.com).Y. Qin and H. Liu
are with the School of Computer Science and Informatics, Cardiff University,
Cardiff, CF24 4AX, United Kingdom (e-mail<EMAIL_ADDRESS>liuh35@cardiff.ac.uk).

###### Abstract

Many dense 3D point clouds have been exploited to represent visual objects
instead of traditional images or videos. To evaluate the perceptual quality of
various point clouds, in this letter, we propose a novel and efficient
Reduced-Reference quality metric for point clouds, which is based on Content-
oriented sAliency Projection (RR-CAP). Specifically, we make the first attempt
to simplify reference and distorted point clouds into projected saliency maps
with a downsampling operation. Through this process, we tackle the issue of
transmitting large-volume original point clouds to user-ends for quality
assessment. Then, motivated by the characteristics of the human visual system
(HVS), the objective quality scores of distorted point clouds are produced by
combining content-oriented similarity and statistical correlation
measurements. Finally, extensive experiments are conducted on SJTU-PCQA and
WPC databases. The experimental results demonstrate that our proposed
algorithm outperforms existing reduced-reference and no-reference quality
metrics, and significantly reduces the performance gap between state-of-the-
art full-reference quality assessment methods. In addition, we show the
performance variation of each proposed technical component by ablation tests.

###### Index Terms:

Point clouds, reduced-reference quality metric, visual content, saliency
projection, human visual system.

##  I Introduction

Massive visual data are emerging in our daily lives. Especially in recent
years, due to the rapid development of 3D capture and rendering technologies,
point cloud has become one of the most popular immersive media formats. It
leads to lots of real-world applications, such as automatic diving, mixed
reality, remote sensing, and so on [1, 2]. Usually, a dense point cloud is a
3D model and has a set of scattered points in space, which is employed to
represent an object. Each point owns geometry coordinates (i.e., $x$, $y$,
$z$) and photometric attributes (e.g., RGB color information). Similar to
conventional images/videos, point clouds have also undergone a variety of
artifacts during the multimedia signal processing chain, from acquisition,
compression, to transmission, reconstruction, and display [3, 4, 5]. In other
words, these procedures will inevitably cause quality degradation at user-
ends. Therefore, the quality evaluation of point clouds is crucial to monitor
and guarantee satisfactory user quality-of-experience.

Since humans are the ultimate receivers of point clouds, subjective quality
assessment is the most reliable method. By carrying out relevant subjective
tests, several point cloud quality databases have been established [6, 7, 8,
9]. However, such subjective tests are often time-consuming, expensive, and
labor-intensive. Thus, how to design effectively objective quality assessment
methods based on the human visual system (HVS) is a challenging yet promising
research direction.

Generally, for the objective quality metrics of point clouds, there exist
three categories, including full-reference (FR), no-reference (NR), and
reduced-reference (RR). When pristine information is entirely accessible, FR
methods are proposed by comparing the difference or similarity between the
distorted and the corresponding original reference point clouds. The earliest
FR metrics are used for the standardization body of MPEG point cloud
compression, which consist of point-to-point [10], point-to-plane [11], point-
to-mesh [12], and plane-to-plane [13], etc. These methods compute the distance
deviation between reference and distorted point clouds. Afterward, Yang et al.
[14] proposed the graph similarity index (GraphSIM) method based on graph
signal processing, and also extended it to a multiscale variant [15]. Inspired
by the structural similarity index (SSIM) [16], Alexiou et al. [17] exploited
geometry, normal vectors, curvature values, and color information to form
PointSSIM. Moreover, in [18], Meynet et al. used local curvature statistics to
construct PC-MSDM and further proposed the point cloud quality metric (PCQM)
[19] according to the optimally weighted linear combination of curvature and
color features. Lu et al. [20] compared the 3D edge similarity to quantify
point cloud quality. Except for direct comparisons on 3D point cloud models,
another kind of FR methods is to project point clouds into multiple 2D images
from different views. Then, the mainstream 2D quality evaluation algorithms
can be utilized, such as PSNR and SSIM.

In real scenarios, the full information of original reference point clouds may
not be available. Several NR methods are developed to predict the visual
quality from distorted point clouds, both from hand-crafted and learning-based
aspects. For example, Zhang et al. [21] proposed NR-3DQA for evaluating point
cloud quality on the basis of 3D natural scene statistics (NSS). In [22], a
point cloud quality assessment network (PQA-Net) was designed by multi-task
learning. In addition, by view projection, 2D quality metrics in NR manner can
be applied, e.g., the representative NSS methods called blind/referenceless
image spatial quality evaluator (BRISQUE) [23] and natural image quality
evaluator (NIQE) [24]. Suppose when part of reference information can be
obtained, RR metrics provide an intermediate solution. Nevertheless, very few
RR quality assessment methods for point clouds have been proposed in the
literature. Liu et al. [25] exploited quantization parameters to estimate the
perceptual quality of V-PCC compressed point clouds. To the best of our
knowledge, there is only one general-purpose RR metric named PCMRR [26] which
relies on a small set of statistical features from the original point clouds
regarding geometry, normal vector, and color. The characteristics of the HVS
are lacking in the design philosophy of the PCMRR method.

To fill the blank of the above-mentioned problem, we propose the RR metric
based on Content-oriented sAliency Projection (RR-CAP). The main contributions
of this letter are summarized as follows:

  1. 1.

Motivated by the HVS properties, we propose the first image-based RR point
cloud quality assessment method via saliency projection.

  2. 2.

The content-oriented similarity and statistical correlation measurements are
developed in the proposed framework.

  3. 3.

Our proposed quality metric can relieve the transmission burden for the large
amount of point data, while still show competitive performance when tested on
subject-rated databases.

The rest of this letter is organized as follows. Section II first introduces
the proposed RR-CAP method in detail. In Section III, we then show the
experimental results and analyses on publicly available point cloud quality
databases. Finally, the conclusion is given in Section IV.

##  II Proposed RR-CAP Metric

The overall framework of our proposed RR-CAP is shown in Fig. 1. To reduce the
large-scale point cloud data from reference side, we first extract downsampled
saliency maps after view projection. The content-oriented similarity and
statistical correlation are then combined to estimate the perceptual quality
of test point cloud.

Figure 1: Overall framework of the proposed RR-CAP, where
$\normalsize{\scriptsize{D}⃝}\normalsize$ and
$\normalsize{\scriptsize{P}⃝}\normalsize$ are the downsampling and pooling
operations, respectively.

###  II-A Saliency Extraction in Projected Planes

When subjects browse the targeted point clouds, they would synthesize the
quality sensation from all views, resulting in the final experience.
Therefore, we project both original and test 3D models into multiple 2D image
planes. Here, we adopt the six perpendicular projections [6] which can
uniformly cover most of the viewed content.

Because introducing artifacts may change the saliency behaviors and the HVS is
more sensitive to distortions in salient areas, visual saliency plays a vital
role in quality evaluation [27, 28]. After view projection, we extract
saliency maps by image signature [29]. Specifically, for each projected image
$I$, we first downsample it to a coarser counterpart by:

$\tilde{I}(i,j)=A*I(si,sj),$ (1)

where $s$ represents the downsampling scale. $i=1,2,\ldots,\frac{I}{s}$ and
$j=1,2,\ldots,\frac{J}{s}$ are indexes for the row and column of downsampled
image, respectively. $A$ is a low-passing filter and $*$ denotes the
convolution operation.

Then, the image signature is defined as the sign function of discrete cosine
transform (DCT) coefficients [30] for downsampled projection, which can be
computed as follows:

$\hat{I}=\operatorname{sign}(\operatorname{DCT}(\tilde{I})).$ (2)

With the image signature in the transformed format, we can convert it back to
the spatial domain by inverse DCT (IDCT) and calculate the saliency map as:

$m=\operatorname{IDCT}(\hat{I})\circ\operatorname{IDCT}(\hat{I}),$ (3)

where $\circ$ indicates the Hadamard product. As can be seen from Fig. 1,
instead of using visual saliency as weighting maps in most existing works, the
downsampled saliency maps serve as the reduced-reference information in our
proposed framework, which can alleviate the large transmission data of
original point clouds.

Figure 2: An example to illustrate the relationship between visual content and
quality degradation, where P1 and P2 are two projection views.

###  II-B Content-Oriented Similarity Measurement

Intuitively, we can measure image structural similarity between original and
test projected saliency maps. Given a distorted saliency map $m_{d}$ and the
corresponding reference saliency map $m_{r}$, we compute the structural
similarity between $m_{d}$ and $m_{r}$ as follows:

$S=\frac{\left(2\mu_{r}\mu_{d}+C_{1}\right)\left(2\sigma_{rd}+C_{2}\right)}{\left(\mu_{r}^{2}+\mu_{d}^{2}+C_{1}\right)\left(\sigma_{r}^{2}+\sigma_{d}^{2}+C_{2}\right)},$
(4)

where $\mu_{r}$, $\mu_{d}$, $\sigma_{r}^{2}$, $\sigma_{d}^{2}$, and
$\sigma_{rd}$ represent the corresponding local mean, variance, and covariance
of original reference and distorted saliency maps. $C_{1}$ and $C_{2}$ are
stabilizing constants.

Along with pooling the similarities for all views, we employ a content-
oriented weighting strategy, which shows more consistent with the HVS
perception. Considering that content is usually quantified by spatial
information by using the Sobel filter [31], here the spatial variation is
taken as the weight and calculated by:

$w=\left|\operatorname{std}\left[\operatorname{Sobel}\left(I_{d}\right)\right]-\operatorname{std}\left[\operatorname{Sobel}\left(I_{r}\right)\right]\right|,$
(5)

where $\rm{std[\cdot]}$ denotes the standard deviation that operates over the
pixels in the image space. $I_{r}$ and $I_{d}$ are the reference and distorted
projected images, respectively. Let $n$ be the number of projection views. The
content-oriented similarity measurement is obtained as follows:

$S_{w}=\frac{1}{n}\sum S^{w}.$ (6)

To show how the proposed content-oriented weighting strategy works, we provide
two pairs of projection views with small and large distortion degrees, as
illustrated in Fig. 2. We can observe that the visual contents from various
views generally have different spatial variations. Additionally, the
projection view with large distortion causes more spatial variation.

###  II-C Statistical Correlation Measurement

Apart from the content-oriented similarity measurement, statistical
information from the saliency maps is also important for the perceptual
quality of point clouds. Therefore, for each test point cloud, we formulate
the statistical correlation measurement by:

$H_{c}=\frac{1}{n}\sum\frac{\mathrm{E}\left[h_{r}h_{d}\right]-\mathrm{E}\left[h_{r}\right]\mathrm{E}\left[h_{d}\right]}{\sqrt{\mathrm{E}\left[h_{r}^{2}\right]-\left(\mathrm{E}\left[h_{r}\right]\right)^{2}}\sqrt{\mathrm{E}\left[h_{d}{}^{2}\right]-\left(\mathrm{E}\left[h_{d}\right]\right)^{2}}},$
(7)

where $\mathrm{E}[\cdot]$ represents the expectation operator. $h_{r}$ and
$h_{d}$ are the statistical histograms of original reference and distorted
saliency maps, respectively.

Through the aforementioned procedures, we have two quality measurements, i.e.,
$S_{w}$ and $H_{c}$, for content-oriented similarity and statistical
correlation, respectively. Finally, the objective quality scores of distorted
point clouds can be calculated as:

$Q=S_{w}\cdot H_{c}.$ (8)

##  III Results and Analyses

In this section, we compare our proposed RR-CAP with state-of-the-art quality
assessment methods on public subject-rated quality databases for point clouds.
Besides, the ablation study is also conducted to further validate the
performance of each proposed technical component.

###  III-A Experimental Protocols

We carry out extensive experiments on the SJTU-PCQA [6] and WPC [7] point
cloud quality databases. Both databases provide mean opinion score (MOS) as
ground-truth subjective quality for each distorted point cloud sample.

  * •

SJTU-PCQA database: is composed of 9 reference point cloud samples. Several
distortion types including geometry Gaussian noise, color noise, downscaling,
octree-based compression, and their mixture are applied to generate 378
distorted point clouds. Each distortion type involves six levels.

  * •

WPC database: has 20 original point clouds with four distortion types,
resulting in 740 distorted point clouds. To be specific, there exist 60
downsampling distorted point clouds, 180 distorted point clouds with Gaussian
noise, 320 G-PCC (T) compressed point clouds, and 180 V-PCC compressed point
clouds.

Four widely used evaluation criteria are adopted for performance comparisons,
including Spearman rank-order correlations coefficient (SROCC), Kendall rank-
order correlation coefficient (KROCC), Pearson linear correlation coefficient
(PLCC), and root mean square error (RMSE). Among them, SROCC and PLCC/RMSE are
employed to measure prediction monotonicity and accuracy, respectively. The
KROCC is used for the ordinal association between two measured quantities. An
excellent quality metric should obtain the performance values of SROCC, KROCC,
and PLCC near 1, and the value of RMSE closes to 0. Note that before computing
the PLCC and RMSE for different quality metrics, a nonlinear fitting process
[32] is executed to map the predicted quality scores onto the common scale
space of subjective scores.

TABLE I: Performance comparisons of objective quality metrics on SJTU-PCQA and
WPC databases. The best performance values for FR, NR, and RR are in bold.

Ref | Type | Method | SJTU-PCQA Database | WPC Database
---|---|---|---|---
SROCC | KROCC | PLCC | RMSE | SROCC | KROCC | PLCC | RMSE
FR | Model-based | GraphSIM [14] | 0.8483 | 0.6448 | 0.8449 | 1.5721 | 0.5831 | 0.4194 | 0.6163 | 17.1939
PointSSIM [17] | 0.6867 | 0.4964 | 0.7136 | 1.7001 | 0.4542 | 0.3278 | 0.4667 | 20.2733
PCQM [19] | 0.8544 | 0.6586 | 0.8653 | 1.2162 | 0.7434 | 0.5601 | 0.7499 | 15.1639
Image-based | PSNR | 0.2422 | 0.1077 | 0.2317 | 2.3124 | 0.4235 | 0.3080 | 0.4872 | 15.8133
SSIM [16] | 0.2987 | 0.1919 | 0.3476 | 2.1770 | 0.3878 | 0.3234 | 0.4944 | 15.7749
NR | Model-based | NR-3DQA [21] | 0.7144 | 0.5174 | 0.7382 | 1.7686 | 0.6479 | 0.4417 | 0.6514 | 16.5716
Image-based | BRISQUE [23] | 0.2051 | 0.1121 | 0.2241 | 2.2428 | 0.3781 | 0.2444 | 0.4176 | 22.5414
NIQE [24] | 0.2214 | 0.1512 | 0.3764 | 2.2671 | 0.3887 | 0.2551 | 0.3957 | 22.5502
RR | Model-based | PCMRR [26] | 0.4816 | 0.3362 | 0.6191 | 1.9342 | 0.3097 | 0.2082 | 0.3433 | 21.5302
Image-based | Proposed RR-CAP | 0.7577 | 0.5508 | 0.7691 | 1.5512 | 0.7162 | 0.5260 | 0.7307 | 15.6485

###  III-B Performance Comparisons

To verify the performance of our proposed RR-CAP method, we compare it with
existing objective quality evaluation approaches including FR, NR, and RR
metrics. The results are shown in TABLE I. In general, the compared metrics
can be categorized into two types: model-based and image-based methods. The
model-based metrics directly operate from 3D models, which consist of GraphSIM
[14], PointSSIM [17], PCQM [19], NR-3DQA [21], and PCMRR [26]. The image-based
metrics project the 3D models into 2D image planes, and then perform quality
assessment on the corresponding projections. The compared image-based metrics
involve PNSR, SSIM [16], BRISQUE [23], and NIQE [24]. Note that PCMRR is the
only RR method and belongs to model-based metrics. That is, there has been no
investigation on image-based RR metrics specifically designed for point clouds
yet. Therefore, we try to fill this gap and propose an image-based RR point
cloud metric based on the characteristics of the HVS.

From the quantitative table, we make the following analyses. First,
traditional image-based metrics show fewer effects in estimating perceptual
quality since they do not take the properties of point clouds into account.
Second, our proposed RR-CAP outperforms state-of-the-art NR and RR metrics
including both model-based and image-based ones. This demonstrates the
superiority of the proposed method although it is an image-based metric. Note
that our proposed metric significantly performs better than PCMRR. This is
mainly due to the consideration of the HVS perception and viewing process in
our framework. Third, we find that the proposed RR metric can greatly reduce
the performance gap compared to the FR model-based quality assessment methods.

###  III-C Ablation Tests

It is interesting to test the performance variation of each proposed technical
component and parameter of our method. In Fig. 3 and Fig. 4, we show the tests
regarding the content-oriented weighting strategy, statistical histogram
feature, and downsampling scales.

From the figures, we can see that our method improves gradually by adding the
weighting strategy and statistical information. This validates the
effectiveness of the proposed technical components. In addition, larger
downsampling scales can save more transmitted resources. To obtain the best
performance and also relieve transmission burden, we choose the downsampling
scale equaling to $16$ in the proposed framework.

Figure 3: Performance variation regarding each technical component, where “-”
indicates removing the corresponding item. W and H are the proposed weighting
strategy and statistical histogram feature, respectively. (a) Run on SJTU-PCQA
database; (b) Run on WPC database.

Figure 4: Performance change with respect to downsampling scales. (a) Run on
SJTU-PCQA database; (b) Run on WPC database.

##  IV Conclusion

In this letter, we propose a new general-purpose RR metric for objective
quality assessment of point clouds. Inspired by the characteristics of the
HVS, our proposed method is based on saliency projection, followed by content-
oriented similarity and statistical correlation measurements. Experiments show
that the proposed RR-CAP can achieve high consistency with the subjective
ratings, compared to state-of-the-art quality metrics. An implementation of
the metric is available at: https://github.com/weizhou-geek/RR-CAP.

## References

  * [1] Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3D point clouds: A survey,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 43, no. 12, pp. 4338–4364, 2020.
  * [2] S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Cohen, M. Krivokuća, S. Lasserre, Z. Li _et al._ , “Emerging MPEG standards for point cloud compression,” _IEEE Journal on Emerging and Selected Topics in Circuits and Systems_ , vol. 9, no. 1, pp. 133–148, 2018.
  * [3] S. Sun, T. Yu, J. Xu, W. Zhou, and Z. Chen, “GraphIQA: Learning distortion graph representations for blind image quality assessment,” _IEEE Transactions on Multimedia_ , 2022.
  * [4] S. Chikkerur, V. Sundaram, M. Reisslein, and L. J. Karam, “Objective video quality assessment methods: A classification, review, and performance comparison,” _IEEE Transactions on Broadcasting_ , vol. 57, no. 2, pp. 165–182, 2011.
  * [5] W. Zhou, Q. Yang, Q. Jiang, G. Zhai, and W. Lin, “Blind quality assessment of 3D dense point clouds with structure guided resampling,” _arXiv preprint arXiv:2208.14603_ , 2022.
  * [6] Q. Yang, H. Chen, Z. Ma, Y. Xu, R. Tang, and J. Sun, “Predicting the perceptual quality of point cloud: A 3D-to-2D projection-based exploration,” _IEEE Transactions on Multimedia_ , vol. 23, pp. 3877–3891, 2020.
  * [7] Q. Liu, H. Su, Z. Duanmu, W. Liu, and Z. Wang, “Perceptual quality assessment of colored 3D point clouds,” _IEEE Transactions on Visualization and Computer Graphics_ , 2022.
  * [8] E. Alexiou, I. Viola, T. M. Borges, T. A. Fonseca, R. L. De Queiroz, and T. Ebrahimi, “A comprehensive study of the rate-distortion performance in MPEG point cloud compression,” _APSIPA Transactions on Signal and Information Processing_ , vol. 8, 2019.
  * [9] A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “Point cloud rendering after coding: Impacts on subjective and objective quality,” _IEEE Transactions on Multimedia_ , vol. 23, pp. 4049–4064, 2020.
  * [10] R. Mekuria, Z. Li, C. Tulvan, and P. Chou, “Evaluation criteria for point cloud compression,” _ISO/IEC MPEG_ , no. 16332, 2016.
  * [11] D. Tian, H. Ochimizu, C. Feng, R. Cohen, and A. Vetro, “Geometric distortion metrics for point cloud compression,” in _IEEE International Conference on Image Processing_ , 2017, pp. 3460–3464.
  * [12] P. Cignoni, C. Rocchini, and R. Scopigno, “Metro: Measuring error on simplified surfaces,” in _Computer Graphics Forum_ , vol. 17, no. 2. Wiley Online Library, 1998, pp. 167–174.
  * [13] E. Alexiou and T. Ebrahimi, “Point cloud quality assessment metric based on angular similarity,” in _IEEE International Conference on Multimedia and Expo_ , 2018, pp. 1–6.
  * [14] Q. Yang, Z. Ma, Y. Xu, Z. Li, and J. Sun, “Inferring point cloud quality via graph similarity,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2020.
  * [15] Y. Zhang, Q. Yang, and Y. Xu, “MS-GraphSIM: Inferring point cloud quality via multiscale graph similarity,” in _Proceedings of the 29th ACM International Conference on Multimedia_ , 2021, pp. 1230–1238.
  * [16] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE Transactions on Image Processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * [17] E. Alexiou and T. Ebrahimi, “Towards a point cloud structural similarity metric,” in _IEEE International Conference on Multimedia & Expo Workshops_, 2020, pp. 1–6.
  * [18] G. Meynet, J. Digne, and G. Lavoué, “PC-MSDM: A quality metric for 3D point clouds,” in _IEEE International Conference on Quality of Multimedia Experience_ , 2019, pp. 1–3.
  * [19] G. Meynet, Y. Nehmé, J. Digne, and G. Lavoué, “PCQM: A full-reference quality metric for colored 3D point clouds,” in _IEEE International Conference on Quality of Multimedia Experience_ , 2020, pp. 1–6.
  * [20] Z. Lu, H. Huang, H. Zeng, J. Hou, and K.-K. Ma, “Point cloud quality assessment via 3D edge similarity measurement,” _IEEE Signal Processing Letters_ , vol. 29, pp. 1804–1808, 2022.
  * [21] Z. Zhang, W. Sun, X. Min, T. Wang, W. Lu, and G. Zhai, “No-reference quality assessment for 3D colored point cloud and mesh models,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 32, no. 11, pp. 7618–7631, 2022.
  * [22] Q. Liu, H. Yuan, H. Su, H. Liu, Y. Wang, H. Yang, and J. Hou, “PQA-Net: Deep no reference point cloud quality assessment via multi-view projection,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 31, no. 12, pp. 4645–4660, 2021.
  * [23] A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” _IEEE Transactions on Image Processing_ , vol. 21, no. 12, pp. 4695–4708, 2012.
  * [24] A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blin” image quality analyzer,” _IEEE Signal Processing Letters_ , vol. 20, no. 3, pp. 209–212, 2012.
  * [25] Q. Liu, H. Yuan, R. Hamzaoui, H. Su, J. Hou, and H. Yang, “Reduced reference perceptual quality model with application to rate control for video-based point cloud compression,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 6623–6636, 2021.
  * [26] I. Viola and P. Cesar, “A reduced reference metric for visual quality evaluation of point cloud contents,” _IEEE Signal Processing Letters_ , vol. 27, pp. 1660–1664, 2020.
  * [27] W. Lin, S. Lee _et al._ , “Visual saliency and quality evaluation for 3D point clouds and meshes: An overview,” _APSIPA Transactions on Signal and Information Processing_ , vol. 11, no. 1, 2022.
  * [28] X. Min, K. Gu, G. Zhai, M. Hu, and X. Yang, “Saliency-induced reduced-reference quality index for natural scene and screen content images,” _Signal Processing_ , vol. 145, pp. 127–136, 2018.
  * [29] X. Hou, J. Harel, and C. Koch, “Image signature: Highlighting sparse salient regions,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 34, no. 1, pp. 194–201, 2011.
  * [30] M. A. Saad, A. C. Bovik, and C. Charrier, “Blind image quality assessment: A natural scene statistics approach in the DCT domain,” _IEEE Transactions on Image Processing_ , vol. 21, no. 8, pp. 3339–3352, 2012.
  * [31] H. Yu and S. Winkler, “Image complexity and spatial information,” in _IEEE International Workshop on Quality of Multimedia Experience_ , 2013, pp. 12–17.
  * [32] A. M. Rohaly, P. J. Corriveau, J. M. Libert, A. A. Webster, V. Baroncini, J. Beerends, J.-L. Blin, L. Contin, T. Hamada, D. Harrison _et al._ , “Video quality experts group: Current results and future directions,” in _Visual Communications and Image Processing_ , vol. 4067. SPIE, 2000, pp. 742–753.