11institutetext:  Shenzhen Institute for Advanced Study,
University of Electronic Science and Technology of China
11email<EMAIL_ADDRESS>
# Beyond Viewpoint: Robust 3D Object Recognition under Arbitrary Views through
Joint Multi-Part Representation

Linlong Fan  0000-0002-6350-7875 Ye Huang Corresponding author
0000-0001-5668-5529 Yanqi Ge  0009-0000-0086-4958 Wen Li  0000-0002-5559-8594
Lixin Duan  0000-0002-0723-4016

###### Abstract

Existing view-based methods excel at recognizing 3D objects from predefined
viewpoints, but their exploration of recognition under arbitrary views is
limited. This is a challenging and realistic setting because each object has
different viewpoint positions and quantities, and their poses are not aligned.
However, most view-based methods, which aggregate multiple view features to
obtain a global feature representation, hard to address 3D object recognition
under arbitrary views. Due to the unaligned inputs from arbitrary views, it is
challenging to robustly aggregate features, leading to performance
degradation. In this paper, we introduce a novel Part-aware Network (PANet),
which is a part-based representation, to address these issues. This part-based
representation aims to localize and understand different parts of 3D objects,
such as airplane wings and tails. It has properties such as viewpoint
invariance and rotation robustness, which give it an advantage in addressing
the 3D object recognition problem under arbitrary views. Our results on
benchmark datasets clearly demonstrate that our proposed method outperforms
existing view-based aggregation baselines for the task of 3D object
recognition under arbitrary views, even surpassing most fixed viewpoint
methods.

###### Keywords:

3D object recognition weakly-supervised learning

##  1 Introduction

Figure 1: In practical scenarios, 3D objects are observed from arbitrary
views(right). The definition of arbitrary views is that each object is
unaligned, and the viewpoint positions and quantities vary for each object.
Previous works have focused on 3D object recognition in aligned views(left),
where the pose of each object is aligned, and the positions and quantities of
viewpoints are predefined. Rotated views(middle) is an extension that
introduces random rotation for each 3D object while keeping the viewpoint
positions and quantities unchanged. Our work focuses on arbitrary views.
Figure 2: Comparison between multi-view aggregation representation (a) and
part-based representation (b). View aggregation methods integrate features
from different views and use the aggregated features to represent 3D objects.
However, since the views are not aligned, the aggregated features lack
robustness, making it difficult to handle 3D recognition on arbitrary views.
Additionally, view aggregation methods suffer from information loss, thus
unable to fully utilize the features from each view. We newly propose a part-
based representation that focuses on multi-view part awareness and combines
multiple parts to robustly represent 3D objects.

Learning 3D with multi-view supervision has received widespread attention. It
covers various research domains such as NeRFs [40, 15], BEV detection [30,
62], 3D generation [33], 3D reasoning [21, 9], etc., among which 3D object
recognition is considered one of the most fundamental problems. Thanks to
mature and transferable 2D vision models [11, 47] and large-scale image
datasets [10], rendering 3D objects into multiple 2D images and applying 2D
networks to indirectly recognize 3D objects has become the state-of-the-art
method [57]. Furthermore, compared with direct methods (_e.g_. , point-based
[1, 61], voxel-based [38]), view-based methods are closer to human perception
of 3D objects in the real world.

Previous view-based methods [50, 5, 56, 49, 6] focus on addressing 3D object
recognition tasks under fixed viewpoints, while recognition under arbitrary
views still lacks extensive research. As shown in Fig. 1, arbitrary views
represent a more challenging setting where each object may have different
viewpoint positions and quantities, and the poses of each object are not
aligned.

However, most view-based methods [56, 49, 13, 6, 63, 50, 26, 5, 18, 65, 64,
41, 55, 60, 57], which aggregate multiple view features to obtain a global
feature representation, hard to effectively address 3D object recognition
under arbitrary views. This is because the misaligned nature of arbitrary
views complicates the representation and aggregation of view features, which
leads to performance degradation. For example, when applied to arbitrary
views, the performance of CVR [55] drops by 6.31% and 14.23% in ModelNet [59]
and ScanObjectNN [53] datasets. Even some multi-view aggregation methods [56,
26] cannot be applied to arbitrary views because they require fixed viewpoint
position information. Furthermore, view aggregation methods [49, 13] typically
jointly optimize feature extraction and aggregation modules to maximize
performance, but this may lead the extraction module to overly focus on
certain views while ignoring information from others, resulting in information
loss.

To address these challenges, we propose a novel part-based representation, as
shown in Fig. 2. This idea is inspired by cognitive science and biology [20],
which aims to understand different part features or specific regions in a
multi-view sequence to better represent 3D objects. For example, a person can
be composed of multiple parts with unique attributes, such as the head, arms,
body, legs, and so on. By locating and integrating the features of these
parts, we can more accurately represent a person. Compared with the multi-view
aggregation features, part features offer several advantages. Firstly, an
individual view contains abundant part information. Even if only a few views
are utilized, we can still extract stable part features, thereby adapting to
changes in the number of viewpoints in arbitrary view settings. Secondly, the
part features are more stable than the multi-view aggregation features [37],
which can better adapt to changes in object rotation and viewpoint positions.
Therefore, the part-based representation method is robust and can better
handle the challenge of 3D object recognition under arbitrary views. We also
provide comprehensive experimental results in Sec. 4 to demonstrate the
effectiveness of part-based representation. When switching to arbitrary views,
we do not observe significant performance degradation in our method, and it
even outperforms most fixed viewpoint methods.

Specifically, this paper proposes a novel part-aware network (PANet). Firstly,
it employs weakly supervised learning to locate and recognize parts in each
view, obtaining local view part features. Then, local view parts are input
into the adaptive part refinement (APR) module. This module adaptively
perceives the correlation of local view parts and further integrates these
view parts to obtain global parts, which aims to adapt to the viewpoint
changes of parts across different views. Finally, we combine these global
parts to represent a 3D object. It is worth noting that we also introduce a
regularization loss in the part localization module to ensure that part
features can better focus on the object.

The primary contributions of this work can be summarized as follows:

$-$ Unlike existing view aggregation methods, we propose a novel part-based
representation. We consider a 3D object as a collection of part features
rather than an abstract multi-view aggregated feature. This part-based
representation is not limited by the position and quantity of viewpoints or
affected by object poses, making it more flexibly applicable to various
viewpoint scenarios, especially in arbitrary views.

$-$ Further, we newly develop a Part-Aware Network (PANet), for extracting
parts in multiple views. The network employs a weakly supervised method to
localize parts on each view, enabling it to obtain local view parts. Then, we
further refines these view parts through the proposed Adaptive Part Refinement
(APR) module to obtain global parts. These global parts can better adapt to
differences in parts across viewpoints. Finally, integrating these global
parts forms a comprehensive description of a 3D object.

$-$ We conduct experiments on various datasets, including ScanObjectNN,
ModelNet, and RGBD. Our method outperforms existing methods in both fixed
views and arbitrary views. In particular, the performance in arbitrary views
exceeds most existing fixed viewpoint methods based on fixed views, achieving
robust recognition of 3D objects.

##  2 Relative Work

3D object Recognition methods:  3D object recognition using deep neural
networks can be broadly divided into three main categories: point-based
methods [42, 43, 17], voxel-based methods [38, 28, 46], and view-based
methods. View-based methods render 3D objects into multiple views and utilize
multiple views to jointly represent 3D objects. Benefiting from mature and
transferable 2D vision models and large-scale image datasets, they can better
handle 3D object recognition tasks.

MVCNN [48] is a pioneering multi-view network for 3D object classification. It
utilizes a view pooling strategy to integrate features from different views.
Yang _et al_. [63] proposes a relational network to effectively connect
corresponding regions from different viewpoints, thus enhancing the
information of each view. 3D2Seq2views [18] method treats multiple views as a
sequence and employs a convolutional neural network with novel hierarchical
attention aggregation to aggregate sequential views. DRCNN [50] utilizes the
dynamic routing algorithm of capsule networks to construct a new Dynamic
Routing Layer (DRL), which integrates features from each view. View-GCN [56]
employs multiple views as nodes in a view graph, leveraging a graph
convolutional network (GCN) architecture to hierarchically aggregate
information and learn robust 3D shape descriptors. These methods assume that
each object has a predefined set of viewpoints and are trained and tested on
these viewpoints. This assumption may limit model performance when dealing
with 3D object recognition in arbitrary views. As far as we know, there are
few studies [34, 55, 31] that have made significant progress beyond fixed
viewpoint settings. The MVTN [34] implements viewpoint regression for 3D
object recognition based on the latest research progress of differentiable
rendering. The motivation of CVR [55] is to restore the intrinsic alignment of
arbitrary views, transform it into aligned representations and then perform
aggregation. In our work, we newly propose part-based representation. Part
features exhibit viewpoint invariance and rotation robustness, making them
more suitable for handling changes in viewpoint positions and quantities in
arbitrary views. Moreover, the key idea of part-based representation is not to
rely on multi-view aggregation, thus avoiding information loss.

Discriminative Part Localization:  The perception and localization of object
part-level features are usually challenging. Many existing approaches [35, 25,
66, 67] suggest using bounding boxes and additional annotations to accurately
locate discriminative regions of objects. However, this method often requires
expert knowledge and extensive annotation efforts, rendering it impractical
for real-world applications. Fortunately, more and more research [69, 14, 68,
23, 45] has shown that important object regions can also be localized using
weakly supervised methods. These methods rely solely on image-level
supervision to identify part features. Early works usually locate class-
specific discriminative regions via CAM [69]. Fu  _et al_. proposes Recursive
Attention CNN (RACNN) [14] to iteratively focus on predicting the locations of
attention regions and extract discriminative features. Zheng  _et al_.
proposes Multi-Attention CNN (MACNN) [68], which proposes channel grouping
loss to generate multiple parts by clustering. WSDAN [23] proposes weakly
supervised attention learning to generate the distribution of parts. Meng  _et
al_. [39] introduces an end-to-end part discovery model (PDM) that aims to
uncover robust and diverse object parts within a unified framework. Our PANet
focuses on perceiving multiple discriminative regions in each view, extracting
sufficient local view parts, and proposing the Adaptive Part Refinement (APR)
module to further refine them to obtain global parts. These global parts can
adapt to changes in viewpoint for parts across different views, making them
more suitable for 3D objects.

##  3 Proposed Method

Figure 3: Overview of our proposed Part-Aware Network (PANet). The network
takes multi-view images as input, encodes them by CNN, obtains view features
I, and uses the cross-view association (CVA) module to enhance the view
features. Firstly, employ weakly-supervised methods to perceive part regions
and generate view part sequences T. Then, given some learnable part tokens P,
the sequences T and P are concatenated and input to the adaptive part
refinement (APR) module. The module aims to refine the sequence T into a more
compact representation, resulting in the global part features. Finally,
multiple global parts $\underline{\textbf{P}}$ together serve as a
representation of a 3D object.

In this work, we newly propose a Part-Aware Network (PANet) that aims to solve
the challenge of 3D object recognition (especially under arbitrary views). The
overall architecture of the proposed PANet is shown in Fig. 3, consisting of
three modules: weakly supervised part localization, adaptive part refinement,
and joint part representation. We first introduce a weakly supervised method
to perform part localization and feature extraction on each view (Sec. 3.1).
Secondly, to adapt to the variations of part features across views, we propose
the Adaptive Part Refinement (APR) module, which refines redundant part
features from multiple views into more compact representations (Sec. 3.2).
Finally, we integrate these refined features to represent a 3D object (Sec.
3.3).

###  3.1 Weakly-Supervised Part Localization

It is worth noting that we do not rely on supervised positional annotations
[67], such as bounding boxes or keypoints, for part localization. Instead, we
employ a more cost-efficient weakly-supervised learning approach [23, 68] that
utilizes image-level annotations to identify discriminative regions, which
requires no expertise and no extensive annotation effort.

We generate a multi-view sequence
$\mathcal{X}=\\{\textbf{X}_{1},\textbf{X}_{2},\ldots,\textbf{X}_{v}\\}$ by
rendering a 3D object from different viewpoints. Here, $v$ denotes the number
of views, $\textbf{X}_{i}$ represents the $i$-th view, where
$i\in\left[1,v\right]$. Then, each view is encoded by an image classification
network _e.g_. , ResNet [19] to obtain view features
$\textbf{I}={\\{\textbf{I}_{1},\textbf{I}_{2},\ldots,\textbf{I}_{v}\\}}$,
where $\textbf{I}_{i}\in\mathbb{R}^{{\rm H}\times{\rm W}\times{\rm
C}}~{},i\in\left[1,v\right]$111H, W, and C represent the height, width, and
number of channels of the feature layer, respectively is corresponding to the
feature map of the $i$-th view $\textbf{X}_{i}$. To align the features from
multiple views into the same feature space, we share the weights among each of
the convolutional neural network.

Enhance Multi-view Features:  However, the associations between view features
I are not explicitly modeled. Therefore, we develop a cross-view association
(CVA) module to obtain enhanced view features $\textbf{F}_{i}$, enabling each
view to incorporate information from other views.

For each view feature $\textbf{I}_{i}$, the cross-view association (CVA)
module calculates the matching score between it and the overall view feature
I. Here, the dot product function is selected as the matching function. Then,
$\textbf{I}_{i}$ is enhanced based on the correlation scores with other views.
The enhanced view feature $\textbf{F}_{i}\in\mathbb{R}^{{\rm H}\times{\rm
W}\times{\rm C}}$ can be expressed in detail as follows:

$\textbf{F}_{i}={softmax\left({\omega(\textbf{I}_{i})^{T}\cdot\omega(\textbf{I})}\right)\textbf{I}},$
(1)

where $\omega(\textbf{I}_{i})\in\mathbb{R}^{{\rm C}}$ represents the use of
average pooling operations.

Part Sampler:  After passing through the cross-view association module, the
enhanced view features F comprehensively incorporate information from other
views. Next, we sample parts from the enhanced view features F.

Inspired by [45, 23], we implement the attention model $\psi$ to learn the
spatial distribution of each part within each view. For view $i$, the
distribution of view parts is represented by the attention map
$\textbf{A}_{i}\in\mathbb{R}^{{\rm H}\times{\rm W}\times{\rm M}}$, which can
be computed by:

$\textbf{A}_{i}=\psi(\textbf{F}_{i})=\left\\{\textbf{A}_{i,1},\textbf{A}_{i,2},\ldots,\textbf{A}_{i,\rm
M}\right\\},$ (2)

where $\textbf{A}_{i,j}\in~{}\mathbb{R}^{{\rm H}\times{\rm
W}},i\in\left[1,v\right],j\in\left[1,\rm M\right]$ represents the $j$-th part
region or visual pattern of the object’s $i$-th views, such as a car wheel or
an airplane wing. $\rm M$ denotes the number of attention maps, each view
extracts the same number of regions of interest.

Next, we employ the attention map $\textbf{A}_{i,j}$ to softly weight the
enhanced feature maps $\textbf{F}_{i}$, followed by the global average pooling
operation $\omega$ to obtain the parts on view $i$ (See Eq. 3).

$\boldsymbol{t}_{i,j}=\omega\left({\textbf{F}_{i}\otimes\textbf{A}_{i,j}}\right),$
(3)

where $\boldsymbol{t}_{i,j}\in\mathbb{R}^{{\rm C}}$, $\otimes$ denotes
element-wise multiplication between two tensors.

Finally, we concatenate all the view part features together, resulting in a 3D
multi-view part sequence $\textbf{T}\in~{}\mathbb{R}^{\rm N\times C}$ (see Eq.
4).

$\displaystyle\textbf{T}=[\boldsymbol{t}_{0,0},\boldsymbol{t}_{0,1},\ldots,\boldsymbol{t}_{0,{\rm
M}},\boldsymbol{t}_{1,0},\boldsymbol{t}_{1,1},\ldots,\boldsymbol{t}_{1,{\rm
M}},...,\boldsymbol{t}_{v,0},\boldsymbol{t}_{v,1},\ldots,\boldsymbol{t}_{v,{\rm
M}}],$ (4)

where $\rm N$ represents the total number of parts $\left(v\times{\rm
M}\right)$ in the multi-view.

###  3.2 Adaptive Part Refinement

In Sec. 3.1, we have obtained a multi-view part sequence T. However, there are
differences in parts across different views, where each element $t_{ij}$ in T
can only represent local view features. It is challenging to generalize the
overall features of a 3D object relying solely on local view parts. Therefore,
we newly propose an Adaptive Part Refinement (APR) module, which aims to
explicitly model the correlations between parts across views, thus enhancing
the robustness of part features to viewpoint variations. This module
adaptively integrates part information from multiple views and refines it to
obtain global part features that are more suitable for 3D objects.

Formally, the APR (Adaptive Part Refinement) module contains a transformer
block. Given some global part tokens [7, 11, 4] $\textbf{P}\in\mathbb{R}^{\rm
L\times C}$, where $\rm L$ represents the number of parts. We concatenate P
and T as input to the APR module. After passing through the APR module, local
parts $\boldsymbol{t}_{i,j}$ can learn cross-view correlations and obtain
global parts $\textbf{\text@underline{P}}\in~{}\mathbb{R}^{\rm L\times C}$
suitable for 3D objects. We also observe the diversity of global part features
P (See Fig. 4).

###  3.3 Joint Multi-Part Representation

Relying solely on individual parts may not be sufficient to represent a 3D
object. To overcome this limitation, we jointly describe the 3D object using
multiple parts. We make predictions for each part and then average them to
obtain the prediction for the 3D object. Each 3D object can be represented as
Eq. 5:

$\textbf{\text@underline{P}}=\left[\underline{\boldsymbol{p}}_{1},\underline{\boldsymbol{p}}_{2},\ldots,\underline{\boldsymbol{p}}_{\rm
L}\right],~{}~{}~{}\rm
where~{}\textbf{\text@underline{P}}\in~{}\mathbb{R}^{\rm L\times C}$ (5)

The proposed PANet is optimized through supervised learning, including
instance classification loss $\mathcal{L}_{ce}$ and part-aware loss
$\mathcal{L}_{awe}$. The loss function for multi-view $\mathcal{X}$ is defined
as follows:

$\begin{split}&p^{i}=softmax\left(\underline{\boldsymbol{p}}_{i}\boldsymbol{W}_{p}+\boldsymbol{b}_{p}\right),~{}~{}~{}{\rm
where}~{}\underline{\boldsymbol{p}}_{i}\in\mathbb{R}^{\rm 1\times
C},~{}i\in\left[1,\rm L\right]\\\
&q^{j}=softmax\Big{(}\boldsymbol{t}_{j}\boldsymbol{W}_{q}+\boldsymbol{b}_{q}\Big{)},~{}~{}~{}{\rm
where}~{}\boldsymbol{t}_{j}\in\mathbb{R}^{\rm 1\times(M\times
C)},~{}j\in\left[1,v\right],\\\
&\mathcal{L}=~{}\mathcal{L}_{ce}+\mathcal{L}_{awe}=-\sum_{k=1}^{K}{y_{k}}{log\hat{p}_{k}}+\gamma\left(-\frac{1}{v}\sum_{i=1}^{v}\sum_{k=1}^{K}{y_{k}}{logq^{j}_{k}}\right),\end{split}$
(6)

where $\boldsymbol{W}_{p}$, $\boldsymbol{W}_{q}$, $\boldsymbol{b}_{p}$, and
$\boldsymbol{b}_{q}$ are learnable parameters. $p^{i}$ represents the
predicted probability of the $i$-th global part, $\hat{p}_{k}$ represents the
average of the predicted probabilities for $\rm L$ global parts, and $q^{j}$
represents the predicted probability for the $j$-th view part. $y_{k}$ is the
ground truth label of class $k$. $\mathcal{L}_{ce}$ represents the cross-
entropy loss of the final parts. $\mathcal{L}_{awe}$ represents the
classification loss of $\rm M$ part features in each view, which is used to
constrain the generation of parts. Furthermore, we apply techniques such as
flipping and random erasure to augment the image for more robust part results.

##  4 Experiments

###  4.1 Datasets and Experimental Setup

We conduct an extensive evaluation of PANet on several benchmark datasets,
namely ScanObjectNN [53], ModelNet [59], and RGBD [29]. In order to facilitate
fair comparisons, we strictly adhere to the methodologies proposed in prior
works [48, 26, 55] for preparing the training and testing data on the
different datasets. Specifically, we adopt the training scheme introduced by
MVCNN [48] and adopt the same viewpoint generation and rendering method as
RotationNet [26] to handle the multi-view images from the ModelNet and
ScanObjectNN datasets. Experiment on rotated datasets also follows CVR [55]
setting. For arbitrary views, the number of views for each 3D object ranges
randomly from 10 to 20. Throughout the training process, we utilize the AdamW
[36] optimizer with a learning rate of 1e-5. We select ResNet50 as our
backbone network and utilize label smoothing regularization [51] to enhance
the generalization capability of our model. $\gamma$ is the weighting factor
for the part-aware loss as in Eq. 6, which we set $\gamma=1$ for the
experiments. Please refer to the Appendix for implementation details of our
method.

###  4.2 Comparison with State-of-the-Art Method

Table 1: Classification on ScanObjectNN. ‘Aligned’ means that every 3D object is in the same pose. ‘Rotated’ means that each 3D object is misaligned. ‘-’ indicates that the specific result is not reported. Method | Type | Aligned | Rotated
---|---|---|---
Per Class Acc. | Per Ins. Acc. | Per Class Acc. | Per Ins. Acc.
PointNet++ [43] | Point | 82.10% | 84.30% | - | -
DGCNN [54] | 84.00% | 86.20% | - | -
MVCNN [48] | View | 85.71% | 87.82% | 78.21% | 80.62%
GVCNN [13] | (fixed) | 86.64% | 88.68% | 82.86% | 83.70%
RotationNet [26] | 84.88% | 86.90% | 74.68% | 76.16%
View-GCN [56] | 88.67% | 90.39% | 81.99% | 83.50%
CVR [55] | 88.39% | 90.74% | 84.70% | 85.59%
View-GCN++ [57] | 89.10% | 91.30% | 85.80% | 86.80%
Ours | 93.21% | 93.99% | 86.15% | 87.82%
CVR [55] | (arbitrary) | - | - | 68.07% | 71.36%
Ours | 91.77% | 93.48% | 84.08% | 86.45%

Table 2: Classification on Aligned ModelNet40. ‘-’ indicates that the specific result is not reported, ‘fixed’ denotes sampling from a fixed viewpoint. ‘free’ means that the arbitrary number of viewpoints. | Aligned ModelNet40 |
---|---|---
Method | Type | Class Acc. | Ins. Acc.
3DShapeNets [58] | Voxel | 77.3% | -
VoxNet [38] | 83.0% | -
VRNEnsemble [3] | 91.4% | 93.8%
PointNet++ [43] | Point | - | 91.9%
Kd-Networks [27] | 88.5% | 91.8%
RS-CNN [22] | - | 93.6%
KP-Convs [52] | - | 92.9%
MVTN [34] | 92.0% | 93.8%
MVCNN-new [48] |  | 92.40% | 95.00%
GVCNN [13] |  | 90.70% | 93.10%
HEAR [5] |  | 95.20% | 96.70%
RotationNet [26] |  | - | 97.37%
View-GCN [56] |  | 96.50% | 97.60%
MVT [6] |  | - | 97.50%
HMVCM [32] |  | - | 94.57%
MVD [24] | View | 92.61% | 94.41%
DAN [41] |  | - | 93.50%
DRCNN [50] |  | 94.86% | 96.84%
View-GCN++ [57] |  | 96.50% | 97.60%
CVR [55] |  | 95.77% | 97.16%
HGNN+ [16] |  | - | 96.92%
Ours |  | 96.60% | 97.73%
Ours | (free) | 96.37% | 97.53%

Table 3:  The accuracy of 3D object classification on Rotated ModelNet40. | Rotated ModelNet40 |
---|---|---
Method | #View | Class Acc. | Ins. Acc.
MVCNN [48] | fixed | 87.95% | 88.17%
GVCNN [13] | 89.69% | 88.10%
EMV [12] | 88.80% | 90.00%
RotationNet [26] | 84.74% | 85.29%
View-GCN [56] | 85.90% | 88.25%
CVR [55] | 91.12% | 92.22%
View-GCN++ [57] | 90.70% | 92.80%
Ours | 90.35% | 92.87%
CVR [55] | arbitrary | 84.01% | 86.91%
Ours | 90.69% | 92.23%

Table 4: The accuracy of object classification on RGBD (in%).

Method |  $\\#$View | Ins. Acc.
---|---|---
MDSICNN [2] |  $\geq$ 120 | 89.6%
CFK[8] |  $\geq$ 120 | 86.8%
MMDCNN [44] |  $\geq$ 120 | 86.8%
RotationNet [26] | 12 | 89.3%
View-GCN [56] | 12 | 94.3%
View-GCN++ [57] | 12 | 92.5%
Ours | 12 | 96.5%
MVCNN [48] | 4-12 | 89.0%
GVCNN [13] | 89.8%
CVR [55] | 91.8%
Ours | 95.9%

The ScanObjectNN Datasets. As a recently introduced real-world 3D object
classification dataset, ScanObjectNN [53] provides scanned indoor scene data.
The dataset comprises approximately 15,000 objects, spanning 15 categories,
and includes 2,902 unique object instances. Compared with CAD handcrafted
datasets, ScanObjectNN offers more realistic challenges as it reflects the
noise, complexity, and incompleteness present in real-world environments,
incorporating the appearance and geometric shapes of actual objects.
Consequently, it is better for assessing the robustness of the model. We train
PANet under different viewpoint settings and compare with point-based methods
[43, 54] and view-based methods [48, 13, 26, 56, 55, 57], as shown in Tab. 1.
View-based methods primarily rely on view aggregation, and it is difficult to
robustly aggregate features from structurally misaligned views, especially for
arbitrary views. When switching to arbitrary views, the performance of these
methods drops sharply. Our PANet is observed to significantly outperform
previous approaches in both class accuracy and instance accuracy, whether in
aligned views, rotated views, or arbitrary views. In aligned views, compared
with the current state-of-the-art methods _e.g_. , View-GCN++ [57] and CVR
[55], PANet improves instance accuracy and class accuracy by 2.69% and 4.11%.
PANet also achieves significant improvements of 1.02% and 0.35% in rotated
views. Surprisingly, even under arbitrary views, our performance surpasses
that of most fixed viewpoint methods, demonstrating that PANet can robustly
recognize 3D objects unconstrained by the scene. Please refer to the Appendix
for more detailed comparisons.

The ModelNet Datasets. ModelNet [59] is a dataset of hand-drawn point clouds
generated from CAD models. It comprises 12,311 3D shapes, covering 40
categories, with 9,483 models allocated for training and 2,468 for testing.
The experimental results on the ModelNet dataset are presented in Tab. 4.2. We
first compare with point-based and voxel-based methods, including PointNet++
[43], Kd-Networks [27], RS-CNN [22], KP-Convs [52], MVTN [34], 3DShapeNets
[58], VoxNet [38], and VRNEnsemble [3]. Our PANet outperforms them by more
than 3.93% and 4.6% in instance accuracy and class accuracy and is also 3.73%
and 4.37% higher in free viewpoints. We also compared with view-based
approaches, including MVCNN-new [48], GVCNN [13], HEAR [5], RotationNet [26],
View-GCN [56], MVT [6], HMVCM [32], MVD [24], DAN [41], DRCNN [50], EMV [12],
View-GCN++ [57], CVR [55], and HGNN+ [16]. PANet consistently outperforms
these methods in fixed views. Its performance at free viewpoints surpasses
that of most existing approaches. We next report the robustness of PANet in
rotated views, as shown in Tab. 4.2. Under the rotated views, PANet also
achieves the highest score in instance accuracy.

The RGBD Datasets. The RGBD [29] dataset comprises 51 categories of domestic
objects captured in the real world, encompassing RGBD views from multiple
viewpoints. To ensure a fair comparison, we adopt the identical experimental
setup as RotationNet [26] and View-GCN [56]. Specifically, we uniformly sample
12 RGB images from the same circle for each 3D object as input and report the
average results using ten-fold cross-validation. We keep the same settings as
CVR [55] in arbitrary views, sampling images from 4 to 12 viewpoints as input
to the network. The experimental results are summarized in Tab. 4. The
performance of PANet in arbitrary views is impressive. Compared with work in
arbitrary views _e.g_. , CVR [55], PANet significantly improves by 4.1%.
Additionally, compared with the work in fixed views, such as view-GCN [56],
PANet has also improved by 1.6%. This further validates the superiority of our
method.

Figure 4: Visualization of correlations between part features
$\underline{\textbf{P}}$. The left figure illustrates the correlation between
$\underline{\textbf{P}}$ after applying $\mathcal{L}_{awe}$.

Table 5: PANet component ablations. The first line serves as the baseline.
$\mathcal{L}_{awe}$ refers to the weakly supervised part-aware loss function.
CVA stands for Cross-View Association module.

CVA | $\mathcal{L}_{awe}$ | Class Acc. | Ins. Acc.
---|---|---|---
|  | 83.76% | 84.56%
✓ |  | 84.99% | 85.25%
| ✓ | 85.45% | 86.96%

###  4.3 Model Analysis

All experiments are conducted on the ScanObjectNN dataset in rotated views.
Please refer to the Appendix for detailed experimental analyses.

PANet component ablations We evaluate the role of the cross-view association
module and $\mathcal{L}_{awe}$, as shown in Tab. 4. The CVA module plays a
positive role in enhancing the performance of PANet, improving instance
accuracy and class accuracy by 0.69% and 1.23%. The results show that the CVA
module enables each view to more effectively integrate information from other
views, further enhancing the representation capability of the view features.
Our method also improves by 2.40% and 1.76% in two metrics by introducing the
$\mathcal{L}_{awe}$ constraint. In addition, we also conduct correlation
analysis between part features $\underline{\textbf{P}}$, as shown in Fig. 4.
We observe that without applying $\mathcal{L}_{awe}$ constraints, the
correlation between parts exhibits a flat similarity matrix, indicating that
the parts are unvaried and cannot be distinguished in the feature space. In
contrast, the part features become more diverse with $\mathcal{L}_{awe}$
constraint. Therefore, we conclude that $\mathcal{L}_{awe}$ can supervise the
perception of parts to obtain diverse part features.

Table 6: Explore the impact of different viewpoint sampling on performance. Dataset | Method | Aligned | Rotated
---|---|---|---
Class Acc. | Ins. Acc. | Class Acc. | Ins. Acc.
ModelNet40 | random sampling | 96.37% | 97.53% | 90.69% | 92.23%
furthest point sampling | 95.44% | 97.41% | 89.73% | 92.26%
ScanObjectNN | random sampling | 91.77% | 93.48% | 84.08% | 86.45%
furthest point sampling | 92.97% | 93.48% | 84.80% | 85.93%

Influence of Different Viewpoint Sampling We introduce two different sampling
methods: random sampling and furthest point sampling (FPS). Random sampling
refers to the process of randomly selecting 10 to 20 views from each object.
This method is closer to real-world scenarios, and we showcased PANet’s
performance in Tab. 1, 4.2, 4.2, 4 based on this sampling method. FPS selects
10 to 20 views based on the furthest point sampling principle. It provides a
uniform sampling method that covers every orientation of the 3D objects. As
shown in Tab. 6, we observed that the performance of random sampling and FPS
across different datasets is nearly similar. This result shows that PANet is
not sensitive to the choice of sampling methods. The model demonstrates
robustness in adapting to changes in viewpoint caused by different sampling
methods while maintaining stable performance.

Table 7: Explore the impact of number of views and backbones on performance.

Backbone | Views
---|---
1 views | 5 views | 10 views | 15 views | 20 views
resnet18 | 45.63% | 70.84% | 75.66% | 79.90% | 82.85%
resnet50 | 67.07% | 79.59% | 82.50% | 85.93% | 86.62%

Influence of the Number of the Views We further evaluate PANet by considering
different backbone networks and numbers of views, as shown in Tab. 7. In
previous studies, such as View-GCN [56], because view aggregation leads to
feature loss. Choosing a stronger backbone network may not necessarily lead to
performance improvement in certain dataset. However, our observations reveal
that the choice of backbone significantly affects PANet performance. This is
because PANet directly integrates multiple parts to characterize 3D objects.
Therefore, a stronger backbone network is more advantageous for perceiving
part features. Furthermore, we notice a positive correlation between the
number of views and the performance of PANet. It is worth noting that previous
studies on multi-view aggregation methods, such as MHBN [65], MVLADN [64], and
GVCNN [13], have indicated that increasing the number of views does not
necessarily lead to better representation of 3D objects. PANet combines
multiple parts for representation, and the APR module can also filter out some
redundant parts. This means that PANet is not affected by redundant views. Our
experimental results also verify this idea.

Table 8: Exploring the effect of the number of attention maps, ${\rm M}$ denotes the number of attention maps per view. #Attention Maps | Class Acc. | Ins. Acc.
---|---|---
${\rm M=16}$ | 84.57% | 86.28%
${\rm M=32}$ | 85.36% | 87.14%
${\rm M=64}$ | 86.50% | 87.31%
${\rm M=128}$ | 86.38% | 87.14%

Table 9: Exploring the effect of the number of part tokens, $\rm L$ indicates the number of parts. #Number of Parts | Class Acc. | Ins. Acc.
---|---|---
${\rm L=1}$ | 85.50% | 86.28%
${\rm L=8}$ | 86.20% | 87.14%
${\rm L=16}$ | 86.50% | 87.31%
${\rm L=32}$ | 85.63% | 86.31%

Influence of the Number of the Attention Maps and Parts We show the impact of
the number of attention maps $\rm M$ and the number of part tokens $\rm L$, as
shown in Tab. 9 and Tab. 9. The number of attention maps $\rm M$ determines
the discriminative region in each view, and the part tokens $\rm L$ determine
the number of global parts. More $\rm M$ and $\rm L$ usually contribute to
better performance. MACNN [68] and WSDAN [23] report similar conclusions. The
experiments in Tab. 9 show that as the value of $\rm M$ increases, the
classification accuracy gradually improves. When $\rm M=64$, the performance
tends to stabilize. The results in Tab. 9 show that the performance improves
as the number of parts increases and reaches a peak at $\rm L=16$. Compared
with a single part, multiple parts provide richer information and enhance the
generalization of the model.

###  4.4 Visualization

Figure 5: Comparing the semantic differences between view-based aggregation
methods and PANet in feature extraction.

We randomly select some 3D models from a test set and compare the semantic
differences between the features extracted by MVCNN and PANet, as shown in
Fig. 5. MVCNN [48] is a classic feature aggregation method. We notice that the
features extracted by MVCNN are not entirely focused on the object. This is
because the uneven weight distribution leads to view aggregation methods
ignoring certain view information. It is worth noting that PANet does not
discard any views but instead considers the features of each view, leveraging
the valuable information within each view. We can observe that different views
emphasize various part locations, with each view capturing multiple
discriminative regions (_e.g_. , the airplane nose, body, and wings). This
observation further validates the diversity of captured parts. Furthermore, we
observe that the view aggregation method does not explicitly model the unique
attributes of 3D objects, leading to feature ambiguity. In contrast, part-
based representation reveals the inherent properties of objects, enhancing the
interpretability of features and making recognition easier. Please refer to
the Appendix for more visualization results.

##  5 Conclusion

In this paper, we propose a novel part-aware network (PANet), which is a part-
based representation. Compared with view aggregation methods, this part-based
representation has properties such as viewpoint invariance and rotation
robustness, which can more effectively address the 3D object recognition
problem under arbitrary views. Moreover, we proposed the part-aware loss to
ensure that part features can better focus on the object, thereby fully
utilizing the information from each view. We evaluate the performance of PANet
on several 3D datasets, including ScanObjectNN, ModelNet, and RGBD. The
experimental results show that PANet has made significant progress, especially
in arbitrary views. We also conduct ablation and visualization experiments to
study the impact of different components in our method. It is worth noting
that in these experiments, we found that our method is not affected by
redundant views (Sec. 4.3), and the part features can better reveal the
inherent properties of objects (Sec. 4.4). In conclusion, the experimental
results prove the effectiveness of our method.

Acknowledgement This research is supported by the Shenzhen Science and
Technology Plan Project No. KJZD20230923113800002 and Shenzhen LongHua
Fundamental Research Program No. 10162A20230325B73A546.

## References

  * [1] Afham, M., Dissanayake, I., Dissanayake, D., Dharmasiri, A., Thilakarathna, K., Rodrigo, R.: Crosspoint: Self-supervised cross-modal contrastive learning for 3d point cloud understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9902–9912 (2022)
  * [2] Asif, U., Bennamoun, M., Sohel, F.A.: A multi-modal, discriminative and spatially invariant cnn for rgb-d object labeling. IEEE transactions on pattern analysis and machine intelligence 40(9), 2051–2065 (2017)
  * [3] Brock, A., Lim, T., Ritchie, J.M., Weston, N.: Generative and discriminative voxel modeling with convolutional neural networks. arXiv preprint arXiv:1608.04236 (2016)
  * [4] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16. pp. 213–229. Springer (2020)
  * [5] Chen, J., Qin, J., Shen, Y., Liu, L., Zhu, F., Shao, L.: Learning attentive and hierarchical representations for 3d shape recognition. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16. pp. 105–122. Springer (2020)
  * [6] Chen, S., Yu, T., Li, P.: Mvt: Multi-view vision transformer for 3d object recognition. arXiv preprint arXiv:2110.13083 (2021)
  * [7] Cheng, B., Schwing, A., Kirillov, A.: Per-pixel classification is not all you need for semantic segmentation. Advances in Neural Information Processing Systems 34, 17864–17875 (2021)
  * [8] Cheng, Y., Cai, R., Zhao, X., Huang, K.: Convolutional fisher kernels for rgb-d object recognition. In: 2015 International Conference on 3D Vision. pp. 135–143. IEEE (2015)
  * [9] Delitzas, A., Parelli, M., Hars, N., Vlassis, G., Anagnostidis, S., Bachmann, G., Hofmann, T.: Multi-clip: Contrastive vision-language pre-training for question answering tasks in 3d scenes. arXiv preprint arXiv:2306.02329 (2023)
  * [10] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)
  * [11] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
  * [12] Esteves, C., Xu, Y., Allen-Blanchette, C., Daniilidis, K.: Equivariant multi-view networks. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1568–1577 (2019)
  * [13] Feng, Y., Zhang, Z., Zhao, X., Ji, R., Gao, Y.: Gvcnn: Group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 264–272 (2018)
  * [14] Fu, J., Zheng, H., Mei, T.: Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4438–4446 (2017)
  * [15] Gao, K., Gao, Y., He, H., Lu, D., Xu, L., Li, J.: Nerf: Neural radiance field in 3d vision, a comprehensive review. arXiv preprint arXiv:2210.00379 (2022)
  * [16] Gao, Y., Feng, Y., Ji, S., Ji, R.: Hgnn+: General hypergraph neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(3), 3181–3199 (2022)
  * [17] Guo, Y., Wang, H., Hu, Q., Liu, H., Liu, L., Bennamoun, M.: Deep learning for 3d point clouds: A survey. IEEE transactions on pattern analysis and machine intelligence 43(12), 4338–4364 (2020)
  * [18] Han, Z., Lu, H., Liu, Z., Vong, C.M., Liu, Y.S., Zwicker, M., Han, J., Chen, C.P.: 3d2seqviews: Aggregating sequential views for 3d global feature learning by cnn with hierarchical attention aggregation. IEEE Transactions on Image Processing 28(8), 3986–3999 (2019)
  * [19] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [20] Herreras, E.B.: Cognitive neuroscience; the biology of the mind. Cuadernos de Neuropsicología/Panamerican Journal of Neuropsychology 4(1), 87–90 (2010)
  * [21] Hong, Y., Lin, C., Du, Y., Chen, Z., Tenenbaum, J.B., Gan, C.: 3d concept learning and reasoning from multi-view images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9202–9212 (2023)
  * [22] Hu, L., Qin, M., Zhang, F., Du, Z., Liu, R.: Rscnn: a cnn-based method to enhance low-light remote-sensing images. Remote Sensing 13(1), 62 (2020)
  * [23] Hu, T., Qi, H., Huang, Q., Lu, Y.: See better before looking closer: Weakly supervised data augmentation network for fine-grained visual classification. arXiv preprint arXiv:1901.09891 (2019)
  * [24] Huang, J., Yan, W., Li, T., Liu, S., Li, G.: Learning the global descriptor for 3-d object recognition based on multiple views decomposition. IEEE Transactions on Multimedia 24, 188–201 (2022). https://doi.org/10.1109/TMM.2020.3047762
  * [25] Huang, S., Xu, Z., Tao, D., Zhang, Y.: Part-stacked cnn for fine-grained visual categorization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1173–1182 (2016)
  * [26] Kanezaki, A., Matsushita, Y., Nishida, Y.: Rotationnet for joint object categorization and unsupervised pose estimation from multi-view images. IEEE transactions on pattern analysis and machine intelligence 43(1), 269–283 (2019)
  * [27] Klokov, R., Lempitsky, V.: Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. In: Proceedings of the IEEE international conference on computer vision. pp. 863–872 (2017)
  * [28] Kumawat, S., Raman, S.: Lp-3dcnn: Unveiling local phase in 3d convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4903–4912 (2019)
  * [29] Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view rgb-d object dataset. In: 2011 IEEE international conference on robotics and automation. pp. 1817–1824. IEEE (2011)
  * [30] Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Qiao, Y., Dai, J.: Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers. In: European conference on computer vision. pp. 1–18. Springer (2022)
  * [31] Lin, Y., Gou, Y., Liu, X., Bai, J., Lv, J., Peng, X.: Dual contrastive prediction for incomplete multi-view representation learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45(4), 4447–4461 (2022)
  * [32] Liu, A.A., Zhou, H., Nie, W., Liu, Z., Liu, W., Xie, H., Mao, Z., Li, X., Song, D.: Hierarchical multi-view context modelling for 3d object classification and retrieval. Information Sciences 547, 984–995 (2021)
  * [33] Liu, R., Wu, R., Van Hoorick, B., Tokmakov, P., Zakharov, S., Vondrick, C.: Zero-1-to-3: Zero-shot one image to 3d object. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 9298–9309 (2023)
  * [34] Liu, S., Nguyen, V., Rehg, I., Tu, Z.: Recognizing objects from any view with object and viewer-centered representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11784–11793 (2020)
  * [35] Liu, X., Han, Z., Liu, Y.S., Zwicker, M.: Fine-grained 3d shape classification with hierarchical part-view attention. IEEE Transactions on Image Processing 30, 1744–1758 (2021)
  * [36] Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
  * [37] Lowe, D.: Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision. vol. 2, pp. 1150–1157 vol.2 (1999). https://doi.org/10.1109/ICCV.1999.790410
  * [38] Maturana, D., Scherer, S.: Voxnet: A 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS). pp. 922–928. IEEE (2015)
  * [39] Meng, M., Zhang, T., Yang, W., Zhao, J., Zhang, Y., Wu, F.: Diverse complementary part mining for weakly supervised object localization. IEEE Transactions on Image Processing 31, 1774–1788 (2022). https://doi.org/10.1109/TIP.2022.3145238
  * [40] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65(1), 99–106 (2021)
  * [41] Nie, W., Zhao, Y., Song, D., Gao, Y.: Dan: Deep-attention network for 3d shape recognition. IEEE Transactions on Image Processing 30, 4371–4383 (2021)
  * [42] Qi, C.R., Su, H., Mo, K., Guibas, L.J.: Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 652–660 (2017)
  * [43] Qi, C.R., Yi, L., Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems 30 (2017)
  * [44] Rahman, M.M., Tan, Y., Xue, J., Lu, K.: Rgb-d object recognition with multimodal deep convolutional neural networks. In: 2017 IEEE International Conference on Multimedia and Expo (ICME). pp. 991–996. IEEE (2017)
  * [45] Rao, Y., Chen, G., Lu, J., Zhou, J.: Counterfactual attention learning for fine-grained visual categorization and re-identification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1025–1034 (2021)
  * [46] Riegler, G., Osman Ulusoy, A., Geiger, A.: Octnet: Learning deep 3d representations at high resolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3577–3586 (2017)
  * [47] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)
  * [48] Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.: Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision. pp. 945–953 (2015)
  * [49] Su, J.C., Gadelha, M., Wang, R., Maji, S.: A deeper look at 3d shape classifiers. In: Proceedings of the European Conference on Computer Vision (ECCV) Workshops. pp. 0–0 (2018)
  * [50] Sun, K., Zhang, J., Liu, J., Yu, R., Song, Z.: Drcnn: Dynamic routing convolutional neural network for multi-view 3d object recognition. IEEE Transactions on Image Processing 30, 868–877 (2020)
  * [51] Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2818–2826 (2016)
  * [52] Thomas, H., Qi, C.R., Deschaud, J.E., Marcotegui, B., Goulette, F., Guibas, L.J.: Kpconv: Flexible and deformable convolution for point clouds. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 6411–6420 (2019)
  * [53] Uy, M.A., Pham, Q.H., Hua, B.S., Nguyen, T., Yeung, S.K.: Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 1588–1597 (2019)
  * [54] Wang, Y., Sun, Y., Liu, Z., Sarma, S.E., Bronstein, M.M., Solomon, J.M.: Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (tog) 38(5), 1–12 (2019)
  * [55] Wei, X., Gong, Y., Wang, F., Sun, X., Sun, J.: Learning canonical view representation for 3d shape recognition with arbitrary views. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 407–416 (2021)
  * [56] Wei, X., Yu, R., Sun, J.: View-gcn: View-based graph convolutional network for 3d shape analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1850–1859 (2020)
  * [57] Wei, X., Yu, R., Sun, J.: Learning view-based graph convolutional network for multi-view 3d shape analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022)
  * [58] Wu, S., Khosla, Y., Zhang, T.: D shapenets: A deep representation for volumetric shape modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. pp. 7–12 (2015)
  * [59] Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3d shapenets: A deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1912–1920 (2015)
  * [60] Xu, Y., Zheng, C., Xu, R., Quan, Y., Ling, H.: Multi-view 3d shape recognition via correspondence-aware deep learning. IEEE Transactions on Image Processing 30, 5299–5312 (2021)
  * [61] Xue, L., Gao, M., Xing, C., Martín-Martín, R., Wu, J., Xiong, C., Xu, R., Niebles, J.C., Savarese, S.: Ulip: Learning unified representation of language, image and point cloud for 3d understanding. arXiv preprint arXiv:2212.05171 (2022)
  * [62] Yang, C., Chen, Y., Tian, H., Tao, C., Zhu, X., Zhang, Z., Huang, G., Li, H., Qiao, Y., Lu, L., et al.: Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17830–17839 (2023)
  * [63] Yang, Z., Wang, L.: Learning relationships for multi-view 3d object recognition. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 7505–7514 (2019)
  * [64] Yu, T., Meng, J., Yang, M., Yuan, J.: 3d object representation learning: A set-to-set matching perspective. IEEE Transactions on Image Processing 30, 2168–2179 (2021). https://doi.org/10.1109/TIP.2021.3049968
  * [65] Yu, T., Meng, J., Yuan, J.: Multi-view harmonized bilinear network for 3d object recognition. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 186–194 (2018). https://doi.org/10.1109/CVPR.2018.00027
  * [66] Zhang, H., Xu, T., Elhoseiny, M., Huang, X., Zhang, S., Elgammal, A., Metaxas, D.: Spda-cnn: Unifying semantic part detection and abstraction for fine-grained recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1143–1152 (2016)
  * [67] Zhang, N., Donahue, J., Girshick, R., Darrell, T.: Part-based r-cnns for fine-grained category detection. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13. pp. 834–849. Springer (2014)
  * [68] Zheng, H., Fu, J., Mei, T., Luo, J.: Learning multi-attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE international conference on computer vision. pp. 5209–5217 (2017)
  * [69] Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2921–2929 (2016)


Nonlocality of quantum matter corrections
and cosmological constant running

E.V. Gorbara ***E-mail address<EMAIL_ADDRESS>and Ilya L. Shapirob
†††E-mail address<EMAIL_ADDRESS>

${a)}$ Physics Faculty, Taras Shevchenko National University of Kyiv, 64/13,
Volodymyrska Street, 01601 Kyiv, Ukraine

${b)}$ Departamento de Física, ICE, Universidade Federal de Juiz de Fora,

Juiz de Fora, 36036-900, Minas Gerais, Brazil

###### Abstract

Semiclassical contributions to the gravitational action include various terms
with zero, two, and four derivatives of the metric as well as nonlocal form
factors for these terms. Contributions to some of these terms could be
confused with others on a specific metric background or for a particular gauge
fixing. We present a critical analysis of the recent works where the tensor
structure and the number of derivatives in the action of gravity were not
properly taken into account. Taking these relevant aspects into account, we
show that although some contributions owing to the quantum fluctuations of
massive or massless scalar as well as fermion and vector fields may be
attributed to the “running” of the cosmological constant, in reality they
correspond to the fourth derivative terms of the action.

Keywords: Effective action, non-locality, quantum corrections, cosmological
constant

##  1 Introduction

Quantum matter corrections to the effective gravitational action is one of the
main points of interest in semiclassical gravity staring from the seminal
pioneering works [1, 2]. A characteristic feature of the contributions of
massive quantum field is the presence of more complicated nonlocal form
factors in the vacuum effective action [3, 4, 5, 6]111 See also subsequent
works [9, 8] for the extension to the total derivative form factor of the
Einstein term. that boil down to the logarithmic expressions in the UV (at
large energies) [7, 3]. The references [3, 4, 5, 6, 8] employ the heat-kernel
technique providing one of the most direct and unambiguous methods to derive
the mentioned form factors. An alternative equivalent approach is to use
Feynman diagrams on the flat Minkowski spacetime background $\eta_{\mu\nu}$
[5, 9]. Decomposing the metric as follows

$\displaystyle g_{\mu\nu}\,=\,\eta_{\mu\nu}+h_{\mu\nu},$ (1)

one has to calculate the self-energy diagrams (or the polarization operator
[2]) for the graviton propagator $G_{\mu\nu\alpha\beta}=-i\langle
h_{\mu\nu}h_{\alpha\beta}\rangle$ shown in Fig. 1 and then determine the
semiclassical corrections to the gravitational action.

The role of mass for the problem under consideration is crucial. At low energy
(in the IR), in the momentum representation, one can make an expansion in the
ratio $k^{2}/m^{2}$ (where $k^{2}=k^{\mu}k_{\mu}$ is the square of Euclidean
momentum) and arrive at the gravitational analog of the Appelquist and
Carazzone decoupling theorem [10]. This decoupling has been found in [5], but
only for the $C^{2}$ and $R^{2}$ terms, where $C^{\mu\nu\alpha\beta}$ is the
Weyl tensor. In a recent work [8], one can see the decoupling taking place for
the Einstein-Hilbert term, but not for the cosmological constant term.

One of the interesting outputs of calculations [5] is that they do not provide
a nonlocal form factor for the cosmological constant term. The reason is that
the hypothetical nonlocal form favor $k_{\Lambda}(\operatorname{\square})$
should act on a constant term, giving zero. However, it was suggested in [5]
and in a slightly different framework in [11] that an effective running of the
cosmological term $\rho_{\Lambda}$ would be possible if we meet, in the
quantum corrections to the gravitational action, the logarithm form factors in
the terms quadratic in the Ricci tensor or the curvature scalar,

$\displaystyle\int
d^{4}x\,\sqrt{-g}\,R_{\mu\nu}\,\frac{m^{4}}{\operatorname{\square}^{2}}\,R^{\mu\nu}\,,\qquad\int
d^{4}x\,\sqrt{-g}\,R\,\frac{m^{4}}{\operatorname{\square}^{2}}\,R\,,$ (2)

or in a similar term with the Riemann tensor, or in higher order terms such as
$\int F(\operatorname{\square}^{-1}R)$, etc. To see the reason why terms (2)
are nonlocal analogs of the cosmological constant term

$\displaystyle-\int
d^{4}x\,\sqrt{-g}\rho_{\Lambda}\,,\qquad\rho_{\Lambda}=\frac{\Lambda}{8\pi G}$
(3)

(called the CC term, in what follows), one has to consider a local (similar to
the cosmological evolution) scaling of the metric

$\displaystyle g_{\mu\nu}\,\,\,\longrightarrow\,\,\,a^{2}(\eta)g_{\mu\nu},$
(4)

where $\eta$ is the conformal time. For $a=const$, terms (2) scale exactly
like the CC term. Euristically, the d´Alembert operator in denominator is
traded for two derivatives in $R_{\mu\nu}$ or $R$. For a time-dependent
$a(\eta)$, these nonlocal terms provide typically a mild deviation from the CC
term and this explains the phenomenological success of the corresponding
cosmological models [12] (see also further references therein). In general,
cosmological models with slowly varying CC term attract a lot of attention and
for this reason, it would be very interesting to derive the nonlocal terms (2)
explicitly. However, the attempts to do so, failed so far. In particular, the
nonlocal terms that emerge in the vacuum action from a spontaneous symmetry
breaking (SSB) [13] (see also more detailed consideration, including highly
non-trivial derivation of the energy-momentum tensor of vacuum, in [14])
produced a negative result. Although there are many nonlocal terms in the SSB-
induced action, the ones of type (2) cancel out.

> Figure 1: Two diagrams of the one-loop correction to $\langle
> h_{\mu\nu}h_{\alpha\beta}\rangle$ due to a quantum matter field represented
> by thick lines. Diagram (a) gives a contribution to the nonlocal form
> factor, while diagram (b) contributes only to the local expression, i.e.,
> divergences. Both plots are needed to provide a correspondence between
> logarithmic divergences and the logarithmic UV limit for the form factors
> [5].

In this situation, it looks very promising that similar terms with extra
logarithm factor

$\displaystyle\mathcal{L}\,=\,\frac{m^{4}}{40\pi^{2}}\bigg{\\{}\Big{(}\frac{1}{\operatorname{\square}}R_{\lambda\sigma}\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}+m^{2}}{m^{2}}\Big{)}\,\Big{(}\frac{1}{\operatorname{\square}}R_{\lambda\sigma}\Big{)}\,-\,\frac{1}{8}\Big{(}\frac{1}{\operatorname{\square}}R\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}+m^{2}}{m^{2}}\Big{)}\Big{(}\frac{1}{\operatorname{\square}}R\Big{)}\bigg{\\}}$
(5)

were found in a recent work [15]. According to [15], these expressions were
obtained from the very same diagrams in Fig. 1 calculated in [5, 9] and this
fact requires, in view of the above mentioned, an explanation.

An alternative description of quantum matter effects is the renormalization
group running. The physical beta functions can be derived using the momentum-
subtraction scheme of renormalization. In the IR, these beta functions
demonstrate the quadratic decoupling, however, only for the coefficients of
the $C^{2}$, $R^{2}$ terms [5], and the $R$-term [8].

It is worth noting that in the case of the minimal subtraction (MS)
renormalization scheme, there is a well-defined beta function for
$\rho_{\Lambda}$ [16, 17]. An interesting paper [18] entitled “Cosmology is
not a renormalization group flow” compares the MS-scheme beta function for
$\rho_{\Lambda}$ with the explicit calculation of quantum contributions to the
vacuum energy on de Sitter space and finds different results. However, this
comparison was made for the quantum effects of a massless scalar field. On the
other hand, the beta function for $\rho_{\Lambda}$ vanishes in the MS or any
other scheme of renormalization for all known types of massless fields. Thus,
one has to explain the discrepancy between the vanishing beta function and the
nonzero result on a de Sitter background.

It turns out that the explanation of the two results concerning the quantum
contributions to the vacuum energy density $\rho_{\Lambda}$ [15] and [18], is
quite similar, regardless that they were obtained by completely different
methods. In both cases, subtleties in the interpretation of the calculations
are connected with the tensor structure of the gravitational action. We
believe that explaining these results in more detail will be instructive and
present the corresponding explanations in the rest of this paper.

The work is organized as follows. Section 2 briefly reviews the expression for
the nonlocal effective action derived in [5]. In Sec. 3, we discuss quantum
matter corrections in de Sitter space and in Sec. 4 provide an in detail
analysis of the results of the calculation of Feynman diagrams in [15].
Finally, we draw our conclusions in Sec. 5.

##  2 Form factors for the gravitational action

Let us start by introducing some standard notions. The general expression for
the action of semiclassical gravity providing renormalizable semiclassical
theory, has the form

$\displaystyle S_{vacuum}\,=\,S_{EH}\,+\,S_{HD}$ (6)

and consists of the Einstein-Hilbert term

$\displaystyle S_{EH}\,=\,-\,\frac{1}{16\pi G}\,\int
d^{4}x\,\sqrt{-g}\,(R+2\Lambda)$ (7)

and the higher derivative term

$\displaystyle S_{HD}\,=\,\int
d^{4}x\,\sqrt{-g}\,\left\\{\,a_{1}C^{2}+a_{2}R^{2}+a_{3}E_{4}+a_{4}\operatorname{\square}R\,\right\\}\,,$
(8)

which includes the square of the Weyl tensor
$C^{2}=R_{\mu\nu\alpha\beta}^{2}-2R_{\alpha\beta}^{2}+1/3\,R^{2}$ and the
integrand of the Gauss-Bonnet topological invariant
$E_{4}=R_{\mu\nu\alpha\beta}R^{\mu\nu\alpha\beta}-4\,R_{\alpha\beta}R^{\alpha\beta}+R^{2}$.
Here $\,a_{1}$,…, $a_{4}$, $\,G\,$, and $\,\Lambda\,$ are the parameters of
the semiclassical gravitational action. As in [5] and [15], we consider
quantum matter corrections due to a free massive scalar field

$\displaystyle S_{m}\,\,=\,\,\frac{1}{2}\int
d^{4}x\,\sqrt{-g}\,\left\\{g^{\mu\nu}\partial_{\mu}\varphi\partial_{\nu}\varphi-m^{2}\varphi^{2}+\xi
R\varphi^{2}\right\\}$ (9)

with the nonminimal parameter of interaction $\xi$.

The one-loop contribution $\bar{\Gamma}_{vac}$ to the gravitational action was
obtained in [5] by using the diagram approach with the expansion in the
curvature tensor and its covariant derivatives up to the second order in
curvature (and verified employing the heat kernel method). The result of these
calculations has the form

$\displaystyle\bar{\Gamma}_{vac}\,=\,\frac{1}{2(4\pi)^{2}}\,\int
d^{4}x\,\sqrt{-g}\,\bigg{\\{}\,\frac{m^{4}}{2}\,\Big{(}\frac{1}{\epsilon_{\mu}}\,+\,\frac{3}{2}\Big{)}\,+\,\tilde{\xi}m^{2}R\,\Big{[}\frac{1}{\epsilon_{\mu}}\,+\,1\Big{]}$
$\displaystyle\qquad\,\,+\,\frac{1}{2}\,C_{\mu\nu\alpha\beta}\,\Big{[}\frac{1}{60\epsilon_{\mu}}\,+\,k_{W}(a)\,\Big{]}C^{\mu\nu\alpha\beta}\,+\,R\,\Big{[}\,\frac{1}{2\epsilon_{\mu}}\,\tilde{\xi}^{2}\,+\,k_{R}(a)\,\Big{]}\,R\,\bigg{\\}}\,,\qquad$
(10)

with nonlocal form factors

$\displaystyle
k_{W}(a)\,=\,\frac{8A}{15\,a^{4}}\,+\,\frac{2}{45\,a^{2}}\,+\,\frac{1}{150}\,,$
$\displaystyle
k_{R}(a)\,=\,\Big{(}A+\frac{1}{18}\Big{)}\tilde{\xi}^{2}+\Big{(}\frac{2A}{3a^{2}}-\frac{A}{6}\Big{)}\tilde{\xi}+\frac{A}{9a^{4}}-\frac{A}{18a^{2}}+\frac{A}{144}+\frac{1}{108\,a^{2}}-\frac{7}{2160}\,.\qquad$
(11)

In these formulas, we used the notation

$\displaystyle\frac{1}{\epsilon_{\mu}}\,=\,\frac{1}{2-w}+\,\mbox{ln}\,\Big{(}\frac{4\pi\mu^{2}}{m^{2}}\Big{)}\,,\qquad\tilde{\xi}=\xi-\frac{1}{6},\qquad
a^{2}=\frac{4\operatorname{\square}}{\operatorname{\square}-4m^{2}}\,,$
$\displaystyle
A=-\frac{1}{2}\int_{0}^{1}d\alpha\,\mbox{ln}\,\Big{[}1+\alpha(1-\alpha)u\Big{]}=1-\frac{1}{a}\,\mbox{ln}\,\frac{1+a/2}{1-a/2}\,.$
(12)

Two things are worth mentioning here.

i) The one-loop contribution (10) is complete in the
$\mathcal{O}(R^{2}_{...})$ approximation.

ii) In the massless case, there is no $m^{4}$-type counterterm and the
$MS$-scheme beta function for $\rho_{\Lambda}$ vanishes. Let us note, in
passing, that in the massless case there are no corrections to the CC term
also at higher loops. In both massive and massless cases, there is no nonlocal
form factor for the CC term. Only the local counterterm $\propto m^{4}$ is
present in the massive case.

Results i) and ii) are in a direct conflict with the interpretations of [18]
and [15], where the quantum correction to the CC cosmological constant
equation of state $\omega=-1$ in the massless theory and the $m^{4}$-type
nonlocal one-loop correction to the CC were found, respectively. We clarify an
apparent contradiction of the obtained results in the sections below and show
that all results are consistent taking into account that the loop
contributions to the vacuum action correspond not only to the
$\rho_{\Lambda}$-term but, also, to other relevant terms in the classical
action (6). However, only terms (8) have logarithmic divergences and,
therefore, only these terms gain logarithm
($\operatorname{\square}$-dependent) form factors from the loop corrections.

##  3 IR dependence on $a$ in the massless case

The logic of [18] is based on the comparison of the renormalization group
equation for $\rho_{\Lambda}$ in [19], [20], and other papers on the
renormalization group running of the CC and the quantum corrections to the
vacuum equation of state obtained in [21]. The last calculation is based on
the well-defined stochastic quantization of massive scalar field on a de
Sitter background [22]. This technique enables one to obtain higher-loop
perturbative and even nonperturbative results.222Compared to the recent
attempt to apply the stochastic formalism on an arbitrary metric background
[23], using the de Sitter metric provides significant advantages. As we will
show below, this also means the need for a special care in the interpretation
of results. The massless limit in this method is smooth and one should expect
a good fit with other methods of calculations.

The first-order logarithm corrections for the energy density and the pressure
of vacuum, quoted in [18], are given by

$\displaystyle\delta\rho_{\Lambda}\,\sim\,-\frac{7}{2}\,H^{4}\,\,\mbox{ln}\,(a),\qquad\delta\big{(}\rho_{\Lambda}+p_{\Lambda}\big{)}\,\sim\,-\frac{4}{3}\,H^{4}\,\,\mbox{ln}\,(a).$
(13)

Since the one-loop contribution to the effective action (10) has nor CC-type
divergences, neither nonlocal corrections to the CC term in the massless case,
how one can resolve the contradiction between (13) and (10) in the limit $m\to
0$?

Let us recall that the calculations of [21] were done on a de Sitter
background, where the terms in action (6) boil down to the expressions

$\displaystyle E_{4}=\frac{1}{6}\,R^{2},\qquad C^{2}=0,\quad\mbox{and}\quad
R=-12H^{2}.$ (14)

Taking this into account and looking at Eq. (13), we readily see that the
logarithmic “running” should not be attributed to the CC term
$\rho_{\Lambda}$, but rather to a linear combination of $E_{4}=24H^{4}$ and
$R^{2}=144H^{4}$. Indeed, when making the comparison of the running of the
parameters in action (10) with the output of de Sitter calculations, the
unique reasonable attribution is to the running of $a_{2}$ and $a_{3}$ and not
of the CC term. The reason is (as we mentioned above) that the constant
$\rho_{\Lambda}$ does not run in massless models and, also, it is not
$\mathcal{O}(H^{4})$, while the $E_{4}$ and $R^{2}$-terms are exactly of order
$H^{4}$.

One can note that, in the conformal ($\xi=1/6$) massless case, the corrections
to $\rho$ and $p$ can be easily recovered from the trace anomaly [24, 25] and
the conservation equation, as described in [26] and, more recently, in [27]
(see further references therein). Unfortunately, our attempt to recover the
coefficients in (13) from the known one-loop beta functions did not work well,
probably because of the second loop “contamination”.

Another relevant observation is that, in the higher derivative model [19],
there are both massless and massive modes, but it is not clear how to separate
them in the IR and arrive at the meaningful comparison with the IR evolution
described in [21] in the massless case.

Let us comment on the possible use of results (13) concerning the running in
the $S_{HD}$ sector of the vacuum action (6). The renormalization group
running of the higher derivative terms can be considered in the form of
anomaly-induced action (see, e.g., [28, 17] for the introduction and further
references). This is a local version of the renormalization group running
which, in difference to a global scaling (as correctly noted in [18]), can be
consistently applied to cosmology. However, this action is well-defined only
for a classically conformal theory and mainly at the one-loop level. It is
clear that relations (13) and their higher-order generalizations [22, 21] can
be used beyond this framework, e.g., in the non-conformal models and at higher
loops. A consistent use of these important results is possible only on the
basis of their correct interpretation and we hope that the present
communication will contribute in this direction.

##  4 Nonlocal $m^{4}$-type form factors

Let us return to the remarkable result of [15] given by Eq.(5) suggesting an
intriguing nonlocal partner for the CC term in the gravitational action. This
seems to be a very good progress compared to expression (10), where one cannot
find any form factor for the $\rho_{\Lambda}$ term. Therefore, first of all,
we should check whether (10) is compatible with (5) derived from the same
diagrams in Fig. 1.

The diagram calculation in [5] is based on the decomposition of the
polarization operator in different tensor structures and using expansion (1)
to the second order in $h_{\mu\nu}$,

$\displaystyle\int d^{4}x\sqrt{-g}\,=\,\int
d^{4}x\,h^{\mu\nu}\,\Big{(}\,\frac{1}{8}\,\eta_{\mu\nu}\,\eta_{\alpha\beta}-\,\frac{1}{4}\,\delta_{\mu\nu\,,\,\alpha\beta}\,\Big{)}\,h^{\alpha\beta}\,+...\,,$
(15) $\displaystyle\int d^{4}x\sqrt{-g}R\,=\,\int
d^{4}x\,\,\,h^{\mu\nu}\,\Big{[}\,\frac{1}{4}\,\big{(}\delta_{\mu\nu\,,\,\alpha\beta}-\eta_{\mu\nu}\,\eta_{\alpha\beta}\big{)}\operatorname{\square}-\,\frac{1}{2}\,\eta_{\mu\alpha}\partial_{\nu}\partial_{\beta}$
$\displaystyle\qquad\,\,\qquad\,\,\qquad\,\,+\,\frac{1}{4}\,(\eta_{\mu\nu}\partial_{\alpha}\partial_{\beta}+\eta_{\alpha\beta}\partial_{\mu}\partial_{\nu})\,\Big{]}\,h^{\alpha\beta}\,+...\,,$
$\displaystyle\int d^{4}x\sqrt{-g}R^{2}=\int
d^{4}x\,\,h^{\mu\nu}\Big{[}\,\partial_{\mu}\partial_{\nu}\partial_{\alpha}\partial_{\beta}+\eta_{\mu\nu}\eta_{\alpha\beta}\operatorname{\square}^{2}$
$\displaystyle\qquad\,\,\qquad\,\,\qquad\,\,-\,(\eta_{\mu\nu}\partial_{\alpha}\partial_{\beta}+\eta_{\alpha\beta}\partial_{\mu}\partial_{\nu})\operatorname{\square}\Big{]}h^{\alpha\beta}+..\,,$
$\displaystyle\int d^{4}x\sqrt{-g}\,W\,=\,\int
d^{4}x\,h^{\mu\nu}\,\Big{[}\frac{1}{4}\,\Big{(}\delta_{\mu\nu\,,\,\alpha\beta}\,-\,\frac{1}{3}\,\eta_{\mu\nu}\,\eta_{\alpha\beta}\Big{)}\operatorname{\square}^{2}-\frac{1}{2}\,\eta_{\mu\alpha}\,\partial_{\beta}\partial_{\nu}\operatorname{\square}$
$\displaystyle\qquad\,\,\qquad\,\,\qquad\,\,+\,\frac{1}{12}\,(\eta_{\mu\nu}\partial_{\alpha}\partial_{\beta}+\eta_{\alpha\beta}\partial_{\mu}\partial_{\nu})\,\operatorname{\square}+\frac{1}{6}\,\partial_{\mu}\partial_{\nu}\partial_{\alpha}\partial_{\beta}\Big{]}\,h^{\alpha\beta}\,+...\,,\mbox{\qquad}$
(16)

where dots stand for the lower and higher order (in $h_{\mu\nu}$) terms. Here
we used the notation $2W=C^{2}-E_{4}$ which simplifies the expansion without
the loss of generality. It is easy to see that there are five distinct tensor
structures. In momentum space, we have [17]

$\displaystyle{\hat{T}}_{1}=\delta_{\mu\nu,\alpha\beta}=\frac{1}{2}\big{(}\eta_{\mu\alpha}\eta_{\nu\beta}+\eta_{\mu\beta}\eta_{\nu\alpha}\big{)},\qquad{\hat{T}}_{2}=\eta_{\mu\nu}\eta_{\alpha\beta},\qquad{\hat{T}}_{3}=\frac{1}{k^{2}}\big{(}\eta_{\mu\nu}k_{\alpha}k_{\beta}+\eta_{\alpha\beta}k_{\mu}k_{\nu}\big{)},$
$\displaystyle{\hat{T}}_{4}\,=\,\frac{1}{4k^{2}}\big{(}\eta_{\mu\alpha}k_{\beta}k_{\nu}+\eta_{\nu\alpha}k_{\beta}k_{\mu}+\eta_{\mu\beta}k_{\alpha}k_{\nu}+\eta_{\nu\beta}k_{\alpha}k_{\mu}\big{)}\,,\quad{\hat{T}}_{5}\,=\,\frac{1}{k^{4}}\,k_{\alpha}k_{\beta}k_{\mu}k_{\nu}.\mbox{\qquad}$
(17)

After decomposing the polarization operator in these tensor structures, one
should calculate the loop integrals and arrive at the expressions equivalent
to (10).

However, the calculations were performed in [15] using the gauge fixing
condition $\partial_{\mu}h^{\mu}_{\,\,\nu}=\frac{1}{2}\partial_{\nu}h$ for the
external metric perturbation (1). In this case, the structures
${\hat{T}}_{3,4,5}$ in (17) disappear and one cannot distinguish different
terms in expansion (16). The unique remaining criterion is that the term $W$
(or, equivalently, $C^{2}$) contributes to the propagator of the traceless
mode $\bar{h}_{\mu\nu}=h_{\mu\nu}-\frac{1}{4}hg_{\mu\nu}$ rather than the
propagation of the trace mode $h=h^{\mu}_{\,\,\mu}$, while the $R^{2}$-term
contributes to the propagator of the trace mode only. This enables one to
separate the two terms in (5) but still one may confuse the $C^{2}$ and
$R^{2}$ form factors with the ones for the CC and Einstein-Hilbert terms.

Thus, the problem is to extract (5) from the full expression (10) under the
gauge fixing condition
$\partial_{\mu}h^{\mu}_{\,\,\nu}=\frac{1}{2}\partial_{\nu}h$, where we meet
only the two tensor structures ${\hat{T}}_{1}$ and ${\hat{T}}_{2}$. Let us
present the corresponding procedure in detail.

For both nonlocal form factors $k_{W}(a)$ and $k_{R}(a)$, logarithm terms are
present only in the UV, thus, we have to consider the regime $k^{2}\gg m^{2}$.
In this case, by using the relations

$\displaystyle\frac{4}{a^{2}}=1+\frac{m^{2}}{k^{2}}\,,\qquad\,\mbox{ln}\,\frac{1+a/2}{1-a/2}\,\,\longrightarrow\,\,\,\mbox{ln}\,\Big{(}\frac{k^{2}}{m^{2}}\Big{)},$
(18)

we obtain

$\displaystyle
k_{W}\,=\,-\,\Big{[}\frac{1}{60}+\frac{1}{6}\,\frac{m^{2}}{k^{2}}+\frac{1}{2}\,\frac{m^{4}}{k^{4}}\Big{]}\,\,\mbox{ln}\,\Big{(}\frac{k^{2}}{m^{2}}\Big{)}\,+\,\dots\,\,,$
$\displaystyle
k_{R}\,=\,-\,\bigg{[}\frac{\tilde{\xi}^{2}}{2}+\Big{(}\frac{\tilde{\xi}}{3}+\tilde{\xi}^{2}\Big{)}\,\frac{m^{2}}{k^{2}}\,+\,\Big{(}\frac{1}{18}+\frac{2\tilde{\xi}}{3}-\tilde{\xi}^{2}\Big{)}\,\frac{m^{4}}{k^{4}}\bigg{]}\,\,\mbox{ln}\,\Big{(}\frac{k^{2}}{m^{2}}\Big{)}\,+\,\dots\,\,,$
(19)

where dots indicate $\mathcal{O}(m^{6}/k^{6})$ and non-logarithm terms. One
can extract the UV limit of the one-loop contributions (10), present it in the
coordinate representation, and use the Ricci basis, i.e., replace
$(1/2)C^{2}\to W=R_{\lambda\sigma}^{2}-(1/3)R^{2}$. This procedure gives us
the following expression for the effective Lagrangian:

$\displaystyle\mathcal{L}^{(UV,2)}_{scal}\,=\,\frac{1}{(4\pi)^{2}}\bigg{\\{}\,\frac{m^{4}}{4}\Big{(}\frac{1}{\epsilon_{\mu}}+\frac{3}{2}\Big{)}\,+\,\frac{\tilde{\xi}m^{2}}{2}\,\Big{(}\frac{1}{\epsilon_{\mu}}+1\Big{)}R$
$\displaystyle\qquad\qquad+\,\,\,R_{\lambda\sigma}\,\bigg{[}\frac{1}{60\epsilon_{\mu}}\,-\,\frac{1}{2}\Big{(}\frac{1}{60}+\frac{1}{6}\,\frac{m^{2}}{\operatorname{\square}}+\frac{1}{2}\,\frac{m^{4}}{\operatorname{\square}^{2}}\,+\,\dots\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\bigg{]}R^{\lambda\sigma}$
$\displaystyle\qquad\qquad+\,\,\,\frac{1}{2}\,R\bigg{[}\frac{1}{\epsilon_{\mu}}\,\Big{(}\frac{\tilde{\xi}^{2}}{2}-\frac{1}{180}\Big{)}\,-\,\frac{1}{2}\Big{(}\frac{\tilde{\xi}^{2}}{2}+\frac{(3\tilde{\xi}^{2}+\tilde{\xi})}{6}\frac{m^{2}}{\operatorname{\square}}$
$\displaystyle\qquad\qquad+\,\,\,\frac{(1+12\tilde{\xi}-18\tilde{\xi}^{2})}{18}\frac{m^{4}}{\operatorname{\square}^{2}}\,+\,\dots\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\bigg{]}\,R\,\bigg{\\}}\,.$
(20)

Here the superscript (UV,2) serves to remember that the nonlocal part of this
formula is the UV piece, up to the second order in $m^{2}/k^{2}$, of the full
expression (10).

The first observation about expression (20) is that leading
$\mathcal{O}(m^{0}/k^{0})$ logarithm terms in the coefficients correspond to
the divergences of the full expression (10), as it has to be. The sub-leading
logarithmic terms have $\mathcal{O}(m^{2}/k^{2})$ and
$\mathcal{O}(m^{4}/k^{4})$. The last kind of terms is of special interest to
us in order to reproduce (2). The non-local terms in (20) come from the
fourth-derivative form factors in the general expression (10) and are not
connected with the renormalization of the CC term.

The reduction of the nonlocal $\mathcal{O}(m^{4}/k^{4})$-terms from expression
(20) to formula (5), which we reproduced from Ref. [15], requires one more
operation. In the arguments of the logarithms in (19) and (20), one has to
perform an ad hoc change, replacing $k^{2}\to k^{2}+m^{2}$ and
$\operatorname{\square}\to\operatorname{\square}+m^{2}$, respectively. In
principle, this is a legitimate operation because the whole expression (20) is
an expansion valid only in the UV regime $m^{2}\ll k^{2}$ and making such a
replacement does not modify the coefficient of the logarithm terms.333 Let us
mention that this also provides a qualitative and apparent similarity with the
complete expression (10) even in the IR decoupling regime, as it was explained
in [15].  After the described operation, the second order,
$\mathcal{O}(m^{4}/k^{4})$, terms in the brackets of (20) produce the desired
structures
$\big{(}m^{4}/\operatorname{\square}^{2}\big{)}\,\mbox{ln}\,\big{(}1+\operatorname{\square}/m^{2}\big{)}$
exactly like in (5).

Finally, to complete the comparison with formula (5), let us consider the
minimal theory with $\xi=0$ or, equivalently, with $\tilde{\xi}=-1/6$. In this
way, preserving only the $\mathcal{O}(m^{4}/k^{4})$ logarithmic terms, we
arrive at an analog of (5), extracted from the full form factor (10),

$\displaystyle\mathcal{L}^{(UV,2,nl)}_{scal}\,=\,-\,\frac{m^{4}}{4(4\pi)^{2}}\bigg{\\{}\Big{(}\frac{1}{\operatorname{\square}}R_{\lambda\sigma}\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\,\Big{(}\frac{1}{\operatorname{\square}}R^{\lambda\sigma}\Big{)}\,-\,\frac{1}{2}\Big{(}\frac{1}{\operatorname{\square}}R\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\Big{(}\frac{1}{\operatorname{\square}}R\Big{)}\bigg{\\}}.$
(21)

Here and in other similar formulas presented below, superscript (UV,2,nl)
means that this is the expression for the second order in $m^{2}/k^{2}$
nonlocal terms, extracted from the UV limit of the fourth derivative terms in
the complete formula (10).

The apparent difference between the two expressions (up to the replacement
$\operatorname{\square}\to\operatorname{\square}+m^{2}$ discussed above) (5)
and (21) is only in the value of coefficients, which we cannot explain.
However, the main point is that contributions (21) do not correspond to the
genuine Einstein-Hilbert $R$-term or to the CC term. Clearly, they are some
pieces of the form factors $k_{W}$ and $k_{R}$ of the fourth-derivative terms.
This conclusion is proved by our analysis of the tensor structure of the
polarization operator. Let us stress again that this analysis is possible only
without imposing gauge condition on the external field $h_{\mu\nu}$.

It is instructive to provide results similar to (21) in the cases of massive
fermion and vector fields. The expressions similar to (20) can be easily
derived from the form factors given in [6], but we do not include these
technical details here. For the fermion field, we find

$\displaystyle\mathcal{L}^{(UV,2,nl)}_{ferm}\,=\,\frac{m^{4}}{2(4\pi)^{2}}\bigg{\\{}\Big{(}\frac{1}{\operatorname{\square}}R_{\lambda\sigma}\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\,\Big{(}\frac{1}{\operatorname{\square}}R^{\lambda\sigma}\Big{)}\bigg{\\}}.$
(22)

It is interesting that, compared to the scalar field, in the
$R_{\lambda\sigma}^{2}$ term there is the change of sign typical for the
divergences of the $m^{4}$-type (however, the coefficient is $-2$ instead of
$-4$). On the other hand, the $R^{2}$ term cancels in this case. Thus, the
similarity with the $m^{4}$-type divergences in (10) is a pure coincidence.

Finally, for the massive vector (Proca) model, we have

$\displaystyle\mathcal{L}^{(UV,2,nl)}_{Proca}\,=\,\frac{3m^{4}}{4(4\pi)^{2}}\bigg{\\{}\Big{(}\frac{1}{\operatorname{\square}}R_{\lambda\sigma}\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\,\Big{(}\frac{1}{\operatorname{\square}}R^{\lambda\sigma}\Big{)}\,-\,\frac{1}{2}\Big{(}\frac{1}{\operatorname{\square}}R\Big{)}\,\mbox{ln}\,\Big{(}\frac{\operatorname{\square}}{m^{2}}\Big{)}\Big{(}\frac{1}{\operatorname{\square}}R\Big{)}\bigg{\\}}.$
(23)

Up to the overall factor, this expression coincides with that for massive
scalar field (21). We leave to the interested reader to check that this
coincidence concerns only the $\mathcal{O}(m^{4})$ nonlocal term and does not
take place in the zero- and second order terms of the expansion analogous to
(20).

Thus, physical interpretation of the $\mathcal{O}(m^{2}/k^{2})$ and
$\mathcal{O}(m^{4}/k^{4})$ terms is subtle. In the UV regime (where $m^{2}$ in
arguments of logarithms becomes irrelevant), these terms really have a global
scaling property identical to that of the CC term. Even if the tensor
structure of these terms forces us attributing them to the $C^{2}$ and $R^{2}$
sectors of the action, the scaling properties are those of the lower-
derivative terms. In this respect, one may use these expansions as an
alternative to the anomaly-induced actions for the quantum massive fields at
high energy (see, e.g., [28] and further references therein).

It is known that the anomaly-induced action modified for nonzero masses of
quantum fields has interesting cosmological applications [28, 29]. It would be
certainly very interesting (albeit certainly more difficult) to explore the
effects of masses by using the complete nonlocal action (10). It is worth
noting that in the recent work [30] one can find a useful formalism for the
massless logarithm form factors. One may expect that this interesting method
can be adapted for a theory with nonzero masses. In this case, terms (2) and
(22) will appear in the action as the second order corrections, while the
first order terms (which we derived but not analysed here) are the leading
corrections.

##  5 Conclusions

We analysed non-locality of quantum corrections due to free massive scalar,
fermion, vector fields to the effective gravitational action. Although the
genuine cosmological term does not admit a non-local form factor, the form
factors in the $C^{2}$ and $R^{2}$ terms produce contributions (21) which at
energies larger than the corresponding particle mass look like an effective
cosmological term. However, there is an important difference, as these non-
local analogs of the cosmological constant are, in fact, the second-order
terms in the $m^{2}/k^{2}$ expansions of the fourth derivative terms in the
UV. Certainly, it would be very interesting to determine the impact of these
and other similar low-mass terms on the evolution of the early Universe. It is
worth adding that the role of $f(1/\operatorname{\square}R)$ terms in nonlocal
cosmology was analysed in [31, 32].

Similarly, calculations performed in a de Sitter background should be
correctly interpreted as those which describe the running of the fourth
derivative (Gauss-Bonnet and $R^{2}$) terms, rather than the scale-dependence
of the cosmological term. It is interesting that these terms do not produce
higher derivative ghosts on a flat background and play very important role in
the Starobinsky inflationary model [33]. Thus, the de Sitter - based relations
such as (13) may be a useful alternative to the anomaly-induced effective
action, especially for the phenomenologically interesting large values of the
nonminimal parameter $\xi$.

## Acknowledgments

I.Sh. is partially supported by Conselho Nacional de Desenvolvimento
Científico e Tecnológico - CNPq (Brazil) under the grant 303635/2018-5 and by
Fundação de Amparo à Pesquisa de Minas Gerais - FAPEMIG under the project
PPM-00604-18.

## References

  * [1] R. Utiyama and B.S. DeWitt, Renormalization of a classical gravitational field interacting with quantized matter fields, J. Math. Phys. 3 (1962) 608.
  * [2] Ya.B. Zeldovich and A.A. Starobinsky, Particle production and vacuum polarization in an anisotropic gravitational field, Sov. Phys. JETP 34 (1972) 1159.
  * [3] A.O. Barvinsky and G.A. Vilkovisky, Covariant perturbation theory. 2: Second order in the curvature. General algorithms, Nucl. Phys. B 333 (1990) 471.
  * [4] I.G. Avramidi, Covariant methods for the calculation of the effective action in quantum field theory and investigation of higher-derivative quantum gravity, (PhD thesis, Moscow University, 1986). hep-th/9510140; Covariant methods of study on the nonlocal structure of effective action, Yad. Fiz. (Sov. Journ. Nucl. Phys.) 49 (1989) 1185.  Heat kernel and quantum gravity, (Springer-Verlag, 2000).
  * [5] E.V. Gorbar and I.L. Shapiro, Renormalization group and decoupling in curved space, JHEP 02 (2003) 021, hep-ph/0210388.
  * [6] E.V. Gorbar and I.L. Shapiro, Renormalization group and decoupling in curved space: II. The Standard Model and beyond, JHEP 06 (2003) 004, hep-ph/0303124.
  * [7] A.O. Barvinsky and G.A. Vilkovisky, The generalized Schwinger-DeWitt technique in gauge theories and quantum gravity, Phys. Repts. 119 (1985) 1.
  * [8] S.A. Franchino-Viñas, T. de Paula Netto, I.L. Shapiro, and O. Zanusso,  Form factors and decoupling of matter fields in four-dimensional gravity, Phys. Lett. B 790 (2019) 229, arXiv:1812.00460.
  * [9] A. Codello and O. Zanusso, On the non-local heat kernel expansion, J. Math. Phys. 54 (2013) 013513, arXiv:1203.2034.
  * [10] T. Appelquist and J. Carazzone, Infrared singularities and massive fields, Phys. Rev. D 11 (1975) 2856.
  * [11] I.L. Shapiro and J. Solà, On the possible running of the cosmological ’constant’, Phys. Lett. B 682 (2009) 105, hep-th/0910.4925.
  * [12] M. Maggiore, L. Hollenstein, M. Jaccard, and E. Mitsou, Early dark energy from zero-point quantum fluctuations. Phys. Lett. B 704 (2011) 102, arXiv:1104.3797;
L. Hollenstein, M. Jaccard, M. Maggiore, and E. Mitsou, Zero-point quantum
fluctuations in cosmology, Phys. Rev. D 85 (2012) 124031, arXiv:1111.5575.

  * [13] E.V. Gorbar and I.L. Shapiro, Renormalization group and decoupling in curved space: III. The case of spontaneous symmetry breaking, JHEP 02 (2004) 060, hep-ph/0311190.
  * [14] M. Asorey, P.M. Lavrov, B.J. Ribeiro, and I.L. Shapiro, Vacuum stress-tensor in SSB theories, Phys. Rev. D 85 (2012) 104001, arXiv:1202.4235.
  * [15] J.F. Donoghue, Non-local partner to the cosmological constant, arXiv:2201.12217.
  * [16] L. Brown, Quantum Field Theory, (Cambridge University Press, Cambridge, 1994).
  * [17] I.L. Buchbinder and I.L. Shapiro, Introduction to Quantum Field Theory with Applications to Quantum Gravity, (Oxford University Press, 2021).
  * [18] R.P. Woodard, Cosmology is not a renormalization group flow, Phys. Rev. Lett. 101 (2008) 081301, arXiv:0805.3089 [gr-qc].
  * [19] I. Antoniadis and E. Mottola, 4-D quantum gravity in the conformal sector, Phys. Rev. D 45 (1992) 2013.
  * [20] I.L. Shapiro and J. Solà, Scaling behavior of the cosmological constant: Interface between quantum field theory and cosmology, JHEP 02 (2002) 006, hep-th/0012227.
  * [21] V.K. Onemli and R.P. Woodard, Quantum effects can render $w<-1$ on cosmological scales, Phys. Rev. D 70 (2004) 107301, gr-qc/0406098.
  * [22] A.A. Starobinsky and J. Yokoyama, Equilibrium state of a selfinteracting scalar field in the De Sitter background, Phys. Rev. D 50 (1994) 6357, astro-ph/9407016.
  * [23] E.A. dos Reis, G. Krein, T. de Paula Netto, and I. L. Shapiro, Stochastic quantization of a self-interacting nonminimal scalar field in semiclassical gravity, Phys. Lett. B 798 (2019) 134925, arXiv:1804.04569.
  * [24] M.J. Duff,  Observations on conformal anomalies, Nucl. Phys. B 125 (1977) 334.
  * [25] S. Deser, M.J. Duff, and C. Isham,  Nonlocal conformal anomalies, Nucl. Phys. B 111 (1976) 45.
  * [26] M.V. Fischetti, J.B. Hartle, and B.L. Hu, Quantum effects in the early universe. I. Influence of trace anomalies on homogeneous, isotropic, classical geometries, Phys. Rev. D 20 (1979) 1757.
  * [27] A.M. Pelinson and I.L. Shapiro, On the scaling rules for the anomaly-induced effective action of metric and electromagnetic field, Phys. Lett. B 694 (2011) 467, arXiv:1005.1313.
  * [28] I.L. Shapiro, Effective action of vacuum: semiclassical approach, Class. Quant. Grav. 25 (2008) 103001, arXiv:0801.0216.
  * [29] T.d.P. Netto, A.M. Pelinson, I.L. Shapiro, and A.A. Starobinsky, From stable to unstable anomaly-induced inflation, Eur. Phys. J. C 76 (2016) 544, arXiv:1509.08882.
  * [30] J. Bezerra-Sobrinho and L. G. Medeiros, Modified Starobinsky inflation by nonlocal terms, arXiv:2202.13308.
  * [31] S. Deser and R. P. Woodard, Nonlocal cosmology, Phys. Rev. Lett. 99 (2007) 111301, arXiv:0706.2151.
  * [32] C. Deffayet and R. P. Woodard, Reconstructing the distortion function for nonlocal cosmology, JCAP 0908 (2009) 023, arXiv:0904.0961.
  * [33] A.A. Starobinski, A new type of isotropic cosmological models without singularity, Phys. Lett. B 91 (1980) 99.