# Mono-STAR: Mono-camera Scene-level Tracking and Reconstruction

Haonan Chang1, Dhruv Metha Ramesh1, Shijie Geng1, Yuqiu Gan, Abdeslam
Boularias1 1 Authors are with the Department of Computer Science, Rutgers
University, 08854 New Brunswick, USA. This work is supported by NSF awards
1734492, 1846043, and 2132972.

###### Abstract

We present _Mono-STAR_ , the first real-time 3D reconstruction system that
simultaneously supports semantic fusion, fast motion tracking, non-rigid
object deformation, and topological change under a unified framework. The
proposed system solves a new optimization problem incorporating optical-flow-
based 2D constraints to deal with fast motion and a novel semantic-aware
deformation graph (SAD-graph) for handling topology change. We test the
proposed system under various challenging scenes and demonstrate that it
significantly outperforms existing state-of-the-art methods. Supplementary
material, including videos, can be found at
https://github.com/changhaonan/Mono-STAR-demo.

##  I INTRODUCTION

Real-time perception is a crucial component of modern robotic manipulation
systems. Recently, You Demonstrate Only Once [1] has demonstrated that given
the geometry model and 6D-pose trajectory of a manipulated object during an
expert demonstration, a robot can quickly learn complex and contact-rich
manipulation skills. Such progress shows the importance of geometric 3D
reconstruction and tracking systems for robotic manipulation.

However, a perception system that can perform both tracking and reconstruction
simultaneously is notoriously difficult to build because reconstruction and
tracking inherently depend on each other. For example, tracking algorithms
usually require geometry models, while dynamic scene reconstruction relies on
accurate tracking for producing those geometry models. Scene-level Tracking
and Reconstruction (_STAR_) [2] refers to a category of perception systems
that generate both the geometry and the pose of every visible object in a
scene.

This problem is related to the multiple-instance dynamic SLAM problem, where
all movable objects in the scene are assumed to be rigid so that the problem
can be decomposed into multiple dense-SLAM sub-problems. This approach was
proposed in Co-Fusion [3] and MaskFusion [4], where a semantic neural network
was employed first to decompose the scene into multiple objects and then deal
with each object individually. This approach requires every object in the
scene to be rigid or quasi-rigid. The same problem was investigated in
MidFusion [5], where an octree was used to improve reconstruction and
tracking. However, these systems are limited to scenes of rigid objects with
slow motions.

Instead of dealing with each object individually based on their semantic
labels, one can also reconstruct all the objects in the scene as one large
non-rigid object and segment them ulteriorly. This approach was however very
challenging to apply until the introduction of the first real-time non-rigid
reconstruction DynamicFusion [6], where the non-rigid reconstruction problem
was decomposed into two sub-problems, (1) building the geometry at the initial
frame, and (2) computing the deformation using an embedded deformation graph,
namely ED-graph. This paradigm was also followed in OcclusionFusion [7].
Inspired by these previous efforts, a solution to the general STAR problem was
recently proposed in STAR-no-prior [2]. In contrast to SLAM-based methods,
STAR-no-prior reverses the order of segmentation and reconstruction. The
entire scene is first reconstructed and then segmented into different objects
based on topology. By doing so, STAR-no-prior outperforms previous state-of-
the-art methods such as [4] and MidFusion [5].

However, a major limitation of STAR-no-prior is its reliance on a system of
multiple cameras surrounding the scene, making it impractical for a mobile
robot. To address this shortcoming, we propose Mono-STAR, a mono-camera STAR
solution. Switching from a multi-camera system to a mono-camera setting
requires solving several non-trivial problems. Notably, STAR-no-prior relies
on the multi-camera system to overcome the plane-based-ICP constraint that it
inherits from DynamicFusion, which supports tracking of only slow motion along
the camera view. The use of multiple cameras can guarantee that any motion has
at least one non-zero projection to a camera view. However, the mono-camera
setting does not have such a guarantee and therefore requires a new solution.
Occlusion Fusion [7] adds a 2D constraint using optical-flow (RAFT [8] or GMA
[9]) to deal with fast motions. Inspired by this, we propose a new 2D loss to
track motions that are perpendicular to the camera view, which not only
stabilizes tracking performance under a single view but also improves our
system’s ability to handle fast motion.

Furthermore, STAR-no-prior does not take advantage of semantic labels. We,
therefore, combine the semantic information with the embedded deformation
graph mechanism and propose a Semantic-aware Adaptive Deformation graph, SAD-
graph, which is an extension of ED-graph. With just little extra computation,
SAD-graph can easily handle topology changes across distinct semantic classes
and assign different levels of rigidness for each type of object. To the best
of our knowledge, Mono-STAR is the first single-view real-time 3D
reconstruction system that can simultaneously handle semantic fusion, fast
motion tracking, non-rigid object deformation, and topological change under
one unified framework.

Method Semantic Fast Non-rigid Topology Single motion objects change view
SLAM++ ✔ ✘ ✘ ✘ ✔ DynamicFusion [6] ✘ ✘ ✔ ✘ ✔ Volume Deform ✘ ✘ ✔ ✘ ✔
SurfelWarp [10] ✘ ✘ ✔ ✘ ✔ TCAFusion [11] ✘ ✘ ✔ ✔ ✘ Co-fusion [3] ✔ ✘ ✘ ✔ ✔
Fusion4D [12] ✘ ✔ ✔ ✔ ✘ Motion2Fusion [13] ✘ ✔ ✔ ✔ ✘ Functon4D [14] ✘ ✔ ✔ ✔ ✘
MaskFusion [4] ✔ ✘ ✘ ✔ ✔ RigidFusion [15] ✔ ✘ ✘ ✔ ✔ MidFusion [5] ✔ ✘ ✘ ✔ ✔
OcclusionFusion [7] ✘ ✔ ✔ ✘ ✔ STAR-no-prior [2] ✘ ✘ ✔ ✔ ✘ Mono-STAR ✔ ✔ ✔ ✔ ✔

TABLE I: Taxonomy of the-state-of-art scene-level fusion systems.

##  II RELATED WORKS

Simultaneous Tracking and Reconstruction. Simultaneous 6D tracking and 3D
reconstruction was typically regarded in previous works as a multiple-instance
dynamic SLAM problem. Many works such as Co-fusion [3], MaskFusion [4], and
RigidFusion [15] proposed to divide the scene into multiple rigid objects and
track each object individually. More recently, STAR-no-prior [2] formalized
the STAR problem as a scene-level non-rigid reconstruction problem. Our mono-
camera system eliminates the multi-camera requirement of STAR-no-prior by
adding a new optical-flow-based 2D constraint and a novel semantic-aware
adaptive deformation graph.

Dynamic Scene Reconstruction. Dynamic scene reconstruction [16, 17] is the
problem of reconstructing the geometry and recording the deformation of a
scene with moving objects. DynamicFusion [6] was the first real-time GPU-based
solution for solving this problem. It adopts a TSDF-based geometry as the
canonical model and an embed-deformation graph (ED-graph) to describe the
deformation of the whole scene. A drawback of this method is that the
combination of TSDF and ED-graph cannot handle topology changes. Many recent
techniques such as Fusion4D [12], Motion2Fusion [13], [11, 18] have attempted
to address this problem. However, these methods require significantly more
computation or rely on expensive sensors. SurfelWarp [10] demonstrated that a
Surfel-based representation can be used to tackle topology changes. Therefore,
our proposed system also adopts a Surfel-based representation.

##  III problem formulation and background

###  III-A Problem formulation

Given a sequence of RGB-D images of a given dynamic scene taken from a single
fixed camera, we consider the problem of simultaneous tracking and
reconstruction of all the objects visible in the scene. The number of objects
is unknown. The objects can be non-rigid. Measurement, $M_{t}$ can defined as
set of measurement surfels $m_{i}$ at time-step $t$, generated from the RGB-D
input. $m_{i}=(v_{i},n_{i},c_{i})$, where $v,n,c$ are 3D coordinates, normal
and color respectively.

The proposed system returns at each time-step $t$ a Surfel-based geometry
$S_{t}$ (the reconstructed scene) for the entire scene and its corresponding
deformation graph $G_{t}$. Surfel-based geometry $S_{t}$ is a set of surfels
$s_{i}$. $s_{i}=(v_{i},n_{i},c_{i},r_{i},l_{s_{i}})$, where
$v_{i},n_{i},c_{i},r_{i},l_{s_{i}}$ are respectively the 3D coordinates,
normal, color, radius and semantic label of surfel $s_{i}\in S_{t}$. We assume
that there is a maximum of $H$ pre-defined different semantic categories
$\\{1,2,\ldots,H\\}$. If a surfel does not belong to any pre-defined category,
it will be labeled as $H+1$ (i.e., unrecognized). Deformation graph $G_{t}$ is
defined by a set of nodes $\\{g_{i}\\}$. Each node $g_{i}$ has a semantic
label $l_{g_{i}}$, and is connected to its nearest-neighbor nodes, denoted as
$N^{G}(g_{i})$, in the 3D space. Deformation graph $G_{t}$ is associated with
a warp field $W_{t}$, defined as
$W=\\{[p_{i}\in\mathbb{R}^{3},\delta_{i}\in\mathbb{R}^{+},T_{i}\in SE(3)]\\}$,
wherein $i$ is the index of a node in $G_{t}$, $p_{i}$ is the 3D point that
corresponds to node $g_{i}$, $\delta_{i}$ is the node’s radius of influence,
and $T_{i}$ is the 6D transformation defined on node $g_{i}$. $T_{i}$ is
represented by a dual quaternion $q_{i}$ for smooth interpolation [19]. Warp
field $W$ describes the deformation between two consecutive time steps. For
each surfel $s=(v,n,c,r,l)\in S$, we compute its 6D transformation
$\bar{W}(s)$ based on warp field $W$,

$\vspace{-0.1cm}\bar{W}(s)=normalize(\sum_{k\in
N^{G}(s)}w(v,p_{k})q_{k}),\vspace{-0.1cm}$ (1)

wherein $N^{G}(s)$ denotes the neighbors nodes of surfel $s$, $w(s)$ is an
interpolation parameter, defined as
$w(s)=\exp\big{(}\left\|v-p_{k}\right\|^{2}_{2}/(2\delta_{k}^{2})\big{)}$, and
$v$ is the 3D position of surfel $s$. The local transformation $\bar{W}(s)$ is
then used to describe the deformation of surfel $s$ as follows:

$v_{warp}=\bar{W}(s)v\;\quad
n_{warp}=\textrm{rotation}\big{(}\bar{W}(s)\big{)}n.$ (2)

Here, $v,n$ are the vertex and normal of $s$ before warping, and
$v_{warp},n_{warp}$ are the vertex and normal after the deformation.

Symbol | Meaning | Definition
---|---|---
$M_{t}$ | Measurement at time t. |  III-A, IV-A1
$S_{t-1}$ | Surfel geometry from t-1. | III-A
$R^{a}_{t-1}$ | 2D maps rendered from $S_{t-1}$. | IV-B1
$S_{t-1}^{warp}$ | Warped geometry after non-rigid alignment. | IV-B4
$R^{g}_{t-1}$ | 2D maps rendered from $S_{t-1}^{warp}$. | IV-B1
TABLE II: Notation sheet.

##  IV PROPOSED APPROACH

Figure 1: Overview of the proposed system. The system runs in two parallel
threads, one for measurement and one for geometry. In each time-step $t$, the
measurement thread loads a measurement $M_{t}$ from images or a camera buffer.
Then, a segmentation network generates a set of semantic labels $L^{m}_{t}$.
Once the measurement is loaded on the GPU memory, $M_{t}$ and previous
alignment rendering $R^{a}_{t-1}$ are fed into an optical-flow network to
generate the optical-flow $OF_{t}$ from previous geometry $S_{t-1}$ to
measurement $M_{t}$. Optical-flow $OF_{t}$, geometry rendering $R_{t}$ and
measurement $M_{t}$ are used to compute warp-field $W_{t}$ with non-rigid
alignment. After the alignment, previous geometry $S_{t-1}$ will be warped to
$S_{t-1}^{warp}$. The fusion rendering map $R^{g}_{t-1}$ is then rendered from
$S_{t-1}^{warp}$. $R^{g}_{t-1}$, $S_{t-1}^{warp}$ and semantic labels
$L^{m}_{t}$ are used to generate the updated geometry $S_{t}$, deformation
graph $G_{t}$ and the surfel semantic label $L^{s}_{t}$.

An overview of the proposed method is shown in Fig. 1. Mono-STAR uses two
parallel threads, a measurement thread, and a geometry thread. The first
thread is responsible for loading measurements $M_{t}$ and generating semantic
labels $L^{m}_{t}$. The geometry thread uses this measurement $M_{t}$ and the
alignment rendering $R^{a}_{t-1}$ to compute an optical-flow $OF_{t}$. Then,
$M_{t},R^{a}_{t-1},OF_{t}$ are given to the optimization module that then
computes the non-rigid deformation $W_{t}$. After the non-rigid alignment,
previous geometry $S_{t-1}$ is warped to $S_{t-1}^{warp}$, and the geometry
rendering $R^{g}_{t-1}$ is generated. Finally, $R^{g}_{t-1}$, deformation
$W_{t}$, warped geometry $S_{t-1}^{warp}$, and semantic labels $L^{m}_{t}$ are
combined to generate the latest geometry $S_{t}$.

Noticeably, loading $M_{t}$ takes less time than updating geometry $S_{t-1}$
and rendering $R^{a}_{t-1}$. Semantic segmentation is also faster than the
combined process of generating optical-flow $OF_{t}$ and the non-rigid
alignment. Thus, the geometry thread fully hides the latency of the
measurement thread.

###  IV-A Measurement Thread

####  IV-A1 Measurement

We use one Intel RealSense-415 camera to collect RGB-D images. Depth images
are denoised with a Gaussian filter. The maximum frame rate for this module is
limited to $20$ fps to coordinate with other modules. We use a double-buffer
strategy to hide latency. Specifically, we use two buffers $B_{0}$ and $B_{1}$
to store measurements. When $B_{0}$ is used by other threads, $B_{1}$ can read
images simultaneously. The filtered images are used to construct three maps,
$V_{t}^{m}$, $N_{t}^{m}$, $C_{t}^{m}$, storing 3D coordinates $v_{i}^{m}$,
normal $n_{i}^{m}$, and color $c_{i}^{m}$, respectively, for surfel $m_{i}$ of
each pixel in the measurement.

####  IV-A2 Segmentation

The segmentation module receives the color map $C_{t}^{m}$ and returns a
semantic label map $L_{t}^{m}$ of $H$ pre-defined semantic classes. Here, we
use two different segmentation models, a transformer-based Segmenter Mask
[20], and a more traditional MaskRCNN [21]. The two models are pre-trained on
two different datasets, ADE20K [22, 23] and COCO-Stuff [24] respectively. We
do not further train these models on any other dataset. We select which one to
use based on the types of objects in the scene.

###  IV-B Geometry Thread

####  IV-B1 Geometry Rendering

The input of the geometry rendering pipeline is a geometry $S$, and the output
is the rasterized rendering $R$ for geometry $S$ from the current camera view.
The rendering process to generate $R$ follows the classical point cloud
rasterizing process [25], where every surfel is projected to its nearest pixel
position on the camera plane based on its 3D coordinates. Each rendering map
$R$ is composed of four 2D maps. $R_{t}=\\{C_{t},V_{t},N_{t},I_{t}\\}$, where
$V_{t},N_{t},C_{t},I_{t}$ are respectively the vertex map, the normal map, the
color map and the index map. These maps store the 3D coordinates $v_{i}$, the
normal $n_{i}$, the color $c_{i}$ and the surfel index $i$ of the projected
surfel $s_{i}$ at each pixel.

At each time-step, the geometry rendering pipeline is called twice; once to
generate $R^{a}_{t-1}$ from previous geometry $S_{t-1}$ for non-rigid
alignment, and once to get $R^{g}_{t-1}$ with warped geometry $S^{warp}_{t-1}$
for updating the geometry. Rendering $R_{t-1}^{g}$ used for updating the
geometry operates on surfel-level granularity, whereas $R_{t-1}^{a}$ used for
geometry alignment rendering operates on deformation node granularity. Another
difference between $R_{t-1}^{g}$ and $R_{t-1}^{a}$ is resolution,
$R_{t-1}^{g}$ is up-sampled by $4\times 4$ compared to $R_{t-1}^{a}$ to
prevent different surfels from being projected onto the same pixel.
$R_{t-1}^{g}$ requires a higher resolution for accurate geometry update and
$R_{t-1}^{a}$ has a lower resolution for faster optimization.

####  IV-B2 Optical Flow

The optical flow module receives $C^{a}_{t-1},V^{a}_{t-1}$ from geometry
rendering $R^{a}_{t-1}$, and $C^{m}_{t-1},V^{m}_{t-1}$ from measurement
$M_{t}$, and generates an optical-flow map $OF_{t}$. $OF_{t}$ predicts the
optical-flow from previous geometry $S_{t-1}$ to the latest measurement
$M_{t}$. This prediction is later used for registration through non-rigid
alignment. We generate $OF_{t}$ using a neural network based on the RAFT
architecture [8], along with additional global motion features as performed in
GMA [9]. The global motion features provide stability for predicting motion
features, even in occluded scenes. Both RAFT and GMA models were originally
trained using only RGB images. The optical flow model used in [7] shows that
using RGB-D images for training provides a far more stable flow, even with
motion blurring. Thus, our model is also trained on RGB-D images from the
datasets FlyingThings3D [26], Monkaa [26] and Sintel [27, 28].

####  IV-B3 Non-rigid alignment

Non-rigid alignment is performed in order to compute non-rigid deformation
$W_{t}$. This step solves a massive optimization problem to warp the previous
geometry $S_{t-1}$ to a geometry $S_{t-1}^{warp}$ that fits current
measurement $M_{t}$. We use a Gauss-Seidel solver implemented with CUDA to
solve this problem, which is summarized as

$\displaystyle min_{W}E_{total}(W)=w_{picp}E_{picp}(W)+$ $\displaystyle
w_{2D}E_{2D}(W)$ $\displaystyle+$ $\displaystyle w_{areg}E_{areg}(W),$

where $w_{picp},w_{of},w_{areg}$ are the weights of terms $E_{picp}(W)$,
$E_{2D}(W)$ and $E_{areg}(W)$, explained in the following.

Registration. Let $u=(x,y)$ be a pixel in measurement map $u$, and let
$m_{i}=M(u)$ bet its associated surfel. Let $(m_{i},s_{\Pi(i)})$ denote a pair
of registered measurement and geometry surfel. $\Pi(i)$ is defined as
$\Pi(i)\coloneqq I^{a}_{t-1}(x-of^{x}_{t},y-of^{y}_{t})$, wherein
$(x,y)=u,(of^{x}_{t},of^{y}_{t})=OF_{t}(u)$. $I_{t-1}^{a}\in R_{t-1}^{a}$ is
the index map of the rendered geometry.

PICP Loss. Point-to-point ICP loss is sensitive to disturbance and outliers,
which limits its utility in real-world applications. Instead, we use a plane-
based ICP (PICP) loss to align the differences along the depth direction as
follows,

$E_{picp}(W)=\sum_{m_{i}\in
M}n_{i}^{m}\cdot(\bar{W}(s_{\Pi(i)})v^{s}_{\Pi(i)}-v_{i}^{m}),$ (3)

wherein $v^{s}_{\Pi(i)}$ is the 3D coordinates of surfel $s_{\Pi(i)}$,
$v_{i}^{m},n_{i}^{m}$ are the 3D coordinates and normal of measurement surfel
$m_{i}$. $\bar{W}$ is defined in Eq. 2.

2D Loss. One limitation of the PICP loss is that it cannot correctly capture
motions within the same plane, such as the moving calendar shown in Fig. 6. We
thus add to the objective function a 2D loss $E_{2D}$ defined as follows,

$\displaystyle P$ $\displaystyle=\begin{pmatrix}1&0&0\\\ 0&1&0\end{pmatrix}$
(4) $\displaystyle E_{2D}(W)$ $\displaystyle=\sum_{m_{i}\in M}\lVert
P(\bar{W}(s_{\Pi(i)})v^{s}_{\Pi(i)}-v_{i}^{m})\rVert_{2}.$ (5)

Here, $P$ is a projection matrix, projecting the 3D difference to the camera
X-Y plane. This term constrains $m_{i}$ and $s_{\Pi(i)}$ to be as close as
possible on the camera X-Y plane. It is worth noting that our proposed 2D loss
is different from the one proposed in OcclusionFusion [7], where pixel
differences are used to calculate the 2D loss. The influence of pixel
differences scales with the distance to the camera, which makes the
optimization parameters harder to tune.

Semantic-aware Adaptive Deformation Graph. The traditional Embedded
Deformation graph (ED-graph) has been widely used in non-rigid tracking and
non-rigid reconstruction. It can describe complicated warping fields with a
simple data structure and an interpolation strategy. Moreover, the as-rigid-
as-possible (ASAP) regulation term defined on deformation nodes provides a
continuity guarantee for neighboring nodes. However, the ED-graph cannot
handle the topology changes of different nodes. For example, if we use an ED
graph to describe a cup being lifted up from a table, as shown in Fig. 2 (a),
the motion of the cup’s nodes also influences and propagates to the table’s
nodes. Many previous works have attempted to overcome this limitation of the
ED-graph by proposing a dual deformation graph [18] or a level-set-based TSDF
fusion mechanism [11]. However, these approaches are too complicated or
introduce too much overhead computation. We propose the Semantic-aware
Adaptive Deformation Graph (SAD-graph) to address this issue of topological
changes among objects with different semantic classes. More importantly, the
proposed algorithm is intuitive and requires little extra computation compared
with ED-graph. Another advantage is that existing ED-graph based approaches
can be easily upgraded to support SAD-graph.

The core idea of SAD-graph is that instead of imposing a uniform regulation
continuity constraint on all deformation nodes, constraints of varying
adaptive strengths are imposed on different edges. A variable weight
$\omega_{i,j}$ is associated with the constraint (defined in Eq. 7) between
neighboring nodes $g_{i}$ and $g_{j}$, and the strength of the constraint is
systematically adjusted. Constraint weight $\omega_{g_{i},g_{j}}$ is a
function of $l_{g_{i}},l_{g_{j}}$, the semantic labels of nodes $g_{i}$ and
$g_{j}$. It is defined as follows,

$\omega_{g_{i},g_{j}}(l_{g_{i}},l_{g_{j}})=\begin{cases}0.1,&\text{if
}l_{g_{i}}\neq l_{g_{j}}\\\ \delta^{k},&\text{if
}l_{g_{i}}=l_{g_{j}}=k,k\in[1,\ldots,H+1]\end{cases}$ (6)

where $\delta^{k}$ is a constant describing the average rigidness of objects
belonging to semantic category $k$, e.g., $\delta^{table}=1.0$,
$\delta^{human}=0.3$, etc. For example, in Fig. 2 (b), since the internal
rigidity constraint within the cup or the table is much larger than the
constraint between them, their geometries can be accurately reconstructed
during the topology separation. It is important to note that these constraints
are not hard because the semantic labels obtained from a neural network
detector are error-prone.

Figure 2: Illustration of the proposed Semantic-aware Adaptive Deformation
Graph (SAD-graph). The scene describes a cup (brown) being lifted up from the
table (green). Black edges indicate strong continuity constraints, while grey
edges indicate weak constraints.

Adaptive Regulation Loss. We introduce a deformation graph, SAD-graph, and a
new regulation term, adaptive regulation $E_{areg}(W)$. A semantic-related
connection weight $\omega$ is used to adjust the regulation strength among and
within different semantic classes as follows,

$E_{areg}(W)=\sum_{g_{j}\in G}\sum_{g_{i}\in
N^{G}(g_{j})}\omega_{g_{i},g_{j}}\left\|T_{j}p_{j}-T_{i}p_{i}\right\|^{2}_{2},$
(7)

wherein $G$ is the deformation graph, $N^{G}(g_{j})$ refers to the set of
neighbors of node $g_{j}$ in the deformation graph, $T_{j}$ and $T_{i}$ are
the transformations defined on nodes $g_{i}$ and $g_{j}$. $p_{i}$ and $p_{j}$
are the 3D coordinate of $g_{i}$ and $g_{j}$, and $\omega_{g_{i},g_{j}}$ is
the weight of the connection between nodes $g_{i}$ and $g_{j}$, defined in Eq.
6.

####  IV-B4 Geometry and Graph Update

Once the non-rigid deformation is computed, the geometry update process of
Mono-STAR is similar to SurfelWarp [10]. Thus, we only briefly describe that
process and we focus on the semantic update. This step returns the updated
geometry $S_{t}$ (the reconstructed scene) and the updated graph $G_{t}$, both
of which are needed for processing the scene in the next time-step.

Updating the Geometry. The previous geometry $S_{t-1}$ is warped to
$S_{t-1}^{warp}$ after the non-rigid alignment step. Although $S_{t-1}^{warp}$
is already close enough to measurement $M_{t}$, there still exists a
discrepancy between them due to measurement noises, emerging surfaces,
topology changes, or even tracking failures. The geometry is updated to
address this gap between the warped geometry $S_{t-1}^{warp}$ and measurement
$M_{t}$. There are four steps in total in this process.

1\. Registration: A projective registration is made between measurement
$M_{t}$ and warped geometry $S_{t-1}^{warp}$ according to rendering map
$R^{g}_{t-1}$.

2\. Fusion: If a surfel $m_{i}\in M_{t}$ is mapped to $s_{j}\in
S_{t-1}^{warp}$ in the registration, $m_{i}$ is merged into $s_{j}$ to average
measurement noises. The semantic label $l_{s_{j}}$ of $s_{j}$ is defined as a
probability distribution $p_{s_{j}}$. When $m_{i}$ is fused into $s_{j}$,
$p_{s_{j}}$ is also updated by $l_{m_{i}}$. The update formula for $p_{s_{j}}$
is:

$\displaystyle
p_{s_{j}}(k)=(p_{s_{j}}(k)+\delta_{m})/\sum_{k^{\prime}}{p_{s_{j}}(k^{\prime})},\text{if
}k=l_{m_{i}}$ (8) $\displaystyle
p_{s_{j}}(k)=p_{s_{j}}(k)/\sum_{k^{\prime}}{p_{s_{j}}}(k^{\prime}),\text{otherwise}.$
(9)

Here, $\delta_{m}$ is the confidence of the measurement.

3\. Append: If there are no surfels in $S_{t-1}^{warp}$ that can be registered
to $m_{i}$, $m_{i}$ must belong to a newly observed surface or be noise. In
the first case, $m_{i}$ will be appended to $S_{t-1}^{warp}$. The semantic
label distribution $p^{s}_{i}$ of $m_{i}$ is initialized as:

$p^{s}_{i}(k)=\delta_{m},\text{if
}k=l_{m_{i}};p^{s}_{i}(k)=0,\text{otherwise},$ (10)

4\. Removal: After each $m_{i}\in M_{t}$ is either fused or appended, some
surfels $s_{j}\in S_{t-1}^{warp}$ are left with no correspondence. A geometry
violation test is performed on the remaining surfels, and those that fail the
test are removed [10].

After the four steps given above, we get the updated geometry $S_{t}$ for
time-step $t$.

Updating the Graph. The update of the SAD-graph is identical to the update of
the traditional ED-graph. The update appends new nodes but does not remove
existing ones. Let $S^{append}$ be the set of the appended surfels during the
geometry update. We first compute the distances between every surfel $s\in
S^{append}$ and every node $g\in G_{t-1}$. Let $D(s,G)=min_{g\in G}\textrm{
distance(s, g)}$. A surfel $s$ is said to be unsupported if $D(s,G)>\sigma$,
for some threshold $\sigma$. We perform a spatially uniform sampling from all
the unsupported surfels. Sampled surfels are appended to graph $G$ as new
nodes. The semantic label of node $g_{i}$, $l_{g_{i}}$ is updated according to
the semantic labels of $N^{S}(g_{i})$, neighbor surfels of $g_{i}$.
$l_{g_{i}}=argmax_{k}\\{\sum_{s_{j}\in N^{s}(g_{i})}\delta(l_{s_{j}},k)\\}$.
Here $\delta(l_{s_{j}},k)=1,\text{if
}l_{s_{j}}=k;\delta(l^{s}_{j},k)=0,\text{otherwise}$.

##  V EXPERIMENTS

We test our technique on a dataset we collected and a public dataset
VolumeDeform [29]. An ablation study and comparisons with SoTA methods such as
STAR-no-prior [2] and MaskFusion [4] on challenging scenes are presented in
this section. Since collecting ground-truth geometry and deformation for non-
rigid objects is extremely challenging, experiments and comparisons in this
area are limited to qualitative results [29]. Supplementary results and
resources can be found at https://github.com/changhaonan/Mono-STAR-demo.

###  V-A Performance

We tested our system on a desktop machine with a GeForce RTX 3090 and an AMD-
Ryzen 9 5900X. On average, measurement loading takes $4$ ms and segmentation
costs 10 ms (Segmenter Mask [20]). The optimization module uses $20$ ms. The
geometry update uses $7$ ms. The major bottleneck is the optical-flow network,
which takes $60$ ms. Since the latency for the measurement thread is fully
hidden by the geometry thread, our entire system runs in 11 Hz. If the
optical-flow runs on a separate graphic card, it would take only $26$ ms [7],
which would double the speed of our system.

###  V-B Qualitative Results

####  V-B1 Soft objects

Fig. 3 illustrates the non-rigid deformation ability of MaskFusion and Mono-
STAR. We can clearly see that MaskFusion fails to track the deformations of
the pillow and umbrella, while Mono-STAR correctly captures both of them in
the reconstructed model, which shows the advantage of our technique over
MaskFusion in handling non-rigid deformation.

Figure 3: Comparison on deformable objects with MaskFusion on our recorded
dataset (top) and VolumeDeform dataset (bottom).

####  V-B2 Fast Motion

Fig. 4 demonstrates Mono-STAR’s ability to handle fast motions. The top scene
in Fig. 4 shows an accident that was recorded during our data collection.
While we were pushing a cup on the table, the cup hit a bump and fell down.
The bottom scene is about passing a basketball between two hands. Objects in
both scenes moved very fast. One is 18 frames, and the other is 30 frames.
Significant motion blur can be observed in both middle images. However, Mono-
STAR can still capture these fast motions and correctly reconstruct the
objects at each frame.

Figure 4: Experiment on fast motion. Pushing down a coffee cup (top). Passing
a basketball between two hands (bottom). The second and fourth row are our 3D
scene reconstruction results.

####  V-B3 Resilience to Semantic Segmentation Noises

Fig. 5 shows how our proposed method can resist noise in semantic
segmentation. The figures on the left are the RGB measurement from the
beginning and the end frames. The right side compares the segmentation from
the measurement and the segmentation from our reconstruction. Although the
ground-truth measurement suffers from major segmentation errors, where the cup
label is completely lost for $t>0$, Mono-STAR still maintains the correct
semantic labels in its reconstruction result through semantic fusion.

Figure 5: Resilience to semantic label noise. The top sequence is the
segmentation map $L^{m}_{t}$ of the measurement. The bottom sequence is the
segmentation map $L^{s}_{t}$ from our reconstruction technique.

###  V-C Ablation Study

####  V-C1 2D Loss

We test Mono-STAR with and without the 2D loss on the “adventcalender” dataset
from VolumeDeform. Fig. 6 shows that the proposed 2D loss $E_{2D}$ can
efficiently track the motions within a plane. In contrast, tracking without
$E_{2D}$ fails in this type of motion, which clearly shows the effectiveness
of the proposed 2D loss.

Figure 6: Ablation study on the 2D loss.

####  V-C2 SAD-graph

In Fig. 7, we compare the ED-graph with the topology-aware ED-graph (STAR-no-
prior) and the SAD-graph (Mono-STAR). We can see that the ED-graph fails to
support the topology change that results from lifting the object from the
table. Topology-aware ED-graph can separate the topology, but it also
generates many outliers on the table. With the help of the proposed SAD-graph,
Mono-STAR can conduct a smoother and cleaner separation.

Figure 7: Comparing ED-graph (left), topology-aware ED-graph (middle, STAR-no-
prior [2]), and SAD-graph (right, ours). The scene shows a plushy toy being
lifted up from the table.

###  V-D Discussion of Limitations

Although Mono-STAR shows great potential in many different aspects, it still
has two limitations. First, it relies on the optical flow to track fast
motions. However, even the state-of-art optical flow detector GMA [9] is not
always accurate, especially when the motion is too fast and the tracked
surfaces are heavily occluded. Our system can tolerate some noise from the GMA
optical-flow module. However, if the optical flow provides inaccurate
predictions for multiple consecutive frames, the tracking of the corresponding
object may still fail. Another drawback of our system is the incompleteness of
the reconstructed geometry. Our reconstructed geometries usually have holes
and are not as smooth as TSDF-based geometry. The reason is that Surfel-based
geometry, unlike TSDF-based geometry, is discrete by default. Therefore, it is
difficult to maintain the smoothness of Surfel-based geometry in highly
dynamic scenes. These two challenges can be addressed in future works.

##  VI CONCLUSION

We presented Mono-STAR, a single-view solution for the semantic-aware STAR
problem. Mono-STAR uses a novel semantic-aware and adaptive deformation graph
for simultaneous tracking and reconstruction, and can handle topology changes
as well as semantic fusion. Experiments show that Mono-STAR achieves promising
results in non-rigid object reconstruction, while resisting to semantic
segmentation errors, and capturing fast motions on various challenging scenes.
We believe that this system can inspire and boost more future research on
imitation learning, dexterous manipulation, and many other relevant robotics
problems.

## References

  * [1] B. Wen, W. Lian, K. Bekris, and S. Schaal, “You only demonstrate once: Category-level manipulation from single visual demonstration,” _arXiv preprint arXiv:2201.12716_ , 2022.
  * [2] H. Chang and A. Boularias, “Scene-level tracking and reconstruction without object priors,” 2022. [Online]. Available: http://rl.cs.rutgers.edu/publications/HaonanIROS2022.pdf
  * [3] M. Runz and L. Agapito, “Co-fusion: Real-time segmentation, tracking and fusion of multiple objects,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 4471–4478, 2017.
  * [4] M. Runz, M. Buffier, and L. Agapito, “MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects,” _Proceedings of the 2018 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2018_ , pp. 10–20, 2019.
  * [5] B. Xu, W. Li, D. Tzoumanikas, M. Bloesch, A. Davison, and S. Leutenegger, “Mid-fusion: Octree-based object-level multi-instance dynamic slam,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 5231–5237.
  * [6] R. A. Newcombe, D. Fox, and S. M. Seitz, “DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time,” Tech. Rep.
  * [7] W. Lin, C. Zheng, J.-H. Yong, and F. Xu, “Occlusionfusion: Occlusion-aware motion estimation for real-time dynamic 3d reconstruction,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 1736–1745.
  * [8] Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” 2020. [Online]. Available: https://arxiv.org/abs/2003.12039
  * [9] S. Jiang, D. Campbell, Y. Lu, H. Li, and R. Hartley, “Learning to estimate hidden motions with global motion aggregation,” 2021. [Online]. Available: https://arxiv.org/abs/2104.02409
  * [10] W. Gao and R. Tedrake, “SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction,” apr 2019. [Online]. Available: http://arxiv.org/abs/1904.13073
  * [11] C. Li and X. Guo, “Topology-Change-Aware Volumetric Fusion for Dynamic Scene Reconstruction,” jul 2020. [Online]. Available: http://arxiv.org/abs/2007.06853
  * [12] M. Dou, S. Khamis, Y. Degtyarev, P. Davidson, S. R. Fanello, A. Kowdle, S. O. Escolano, C. Rhemann, D. Kim, J. Taylor, P. Kohli, V. Tankovich, and S. Izadi, “Fusion4D: Real-time performance capture of challenging scenes,” in _ACM Transactions on Graphics_ , vol. 35, no. 4. Association for Computing Machinery, jul 2016.
  * [13] M. Dou, P. Davidson, S. R. Fanello, S. Khamis, A. Kow-Dle, C. Rhemann, V. Tankovich, S. Izadi, and A. Kowdle, “Motion2Fusion: Real-time Volumetric Performance Capture. 1, 1, Article 246,” Tech. Rep., 2017. [Online]. Available: https://doi.org/10.475/1234
  * [14] T. Yu, Z. Zheng, K. Guo, P. Liu, Q. Dai, and Y. Liu, “Function4D.”
  * [15] Y.-S. Wong, C. Li, M. Nießner, N. J. Mitra, and A. Research, “RigidFusion: RGB-D Scene Reconstruction with Rigidly-moving Objects,” Tech. Rep. 2, 2021.
  * [16] A. Collet, M. Chuang, P. Sweeney, D. Gillett, D. Evseev, H. Hoppe, A. Kirk, S. Sullivan, and D. Calabrese, “High-Quality Streamable Free-Viewpoint Video.”
  * [17] M. Dou, J. Taylor, H. Fuchs, A. Fitzgibbon, and S. Izadi, “3d scanning deformable objects with a single rgbd sensor,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2015, pp. 493–501.
  * [18] K. Zampogiannis, C. Fermuller, and Y. Aloimonos, “Topology-Aware Non-Rigid Point Cloud Registration,” nov 2018. [Online]. Available: http://arxiv.org/abs/1811.07014http://dx.doi.org/10.1109/TPAMI.2019.2940655
  * [19] L. Kavan, S. Collins, J. Žára, and C. O’Sullivan, “Skinning with dual quaternions,” _Proceedings - I3D 2007, ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games_ , pp. 39–46, 2007.
  * [20] R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Transformer for semantic segmentation,” 2021. [Online]. Available: https://arxiv.org/abs/2105.05633
  * [21] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” 2017. [Online]. Available: https://arxiv.org/abs/1703.06870
  * [22] B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ade20k dataset,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017.
  * [23] ——, “Semantic understanding of scenes through the ade20k dataset,” _arXiv preprint arXiv:1608.05442_ , 2016.
  * [24] H. Caesar, J. Uijlings, and V. Ferrari, “Coco-stuff: Thing and stuff classes in context,” in _Computer vision and pattern recognition (CVPR), 2018 IEEE conference on_. IEEE, 2018.
  * [25] M. Segal and K. Akeley, “The opengl graphics system: a specification (version 4.6),” pp. 468–470, 05 2022.
  * [26] N. Mayer, E. Ilg, P. Häusser, P. Fischer, D. Cremers, A. Dosovitskiy, and T. Brox, “A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation,” in _IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, arXiv:1512.02134. [Online]. Available: http://lmb.informatik.uni-freiburg.de/Publications/2016/MIFDB16
  * [27] D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in _European Conf. on Computer Vision (ECCV)_ , ser. Part IV, LNCS 7577, A. Fitzgibbon et al. (Eds.), Ed. Springer-Verlag, Oct. 2012, pp. 611–625.
  * [28] J. Wulff, D. J. Butler, G. B. Stanley, and M. J. Black, “Lessons and insights from creating a synthetic optical flow benchmark,” in _ECCV Workshop on Unsolved Problems in Optical Flow and Stereo Estimation_ , ser. Part II, LNCS 7584, A. Fusiello et al. (Eds.), Ed. Springer-Verlag, Oct. 2012, pp. 168–177.
  * [29] M. Innmann, M. Zollhöfer, M. Nießner, C. Theobalt, and M. Stamminger, “VolumeDeform: Real-time Volumetric Non-rigid Reconstruction,” mar 2016. [Online]. Available: http://arxiv.org/abs/1603.08161


On the energy of Schwarzschild spacetime with the post-Newtonian approximation
I-Ching Yang<EMAIL_ADDRESS>
Department of Applied Science, National Taitung University,

Taitung 95002, Taiwan (R.O.C.)

ABSTRACT

With the post-Newtonian approxination, the energy of Schwarzschild spacetime
in the Weinberg prescription is obtained. The energy for the first post-
Newtonian approximation $E^{(1)}=m$ gives the Newtonian treatment of
Schwarzschild spacetime. However, for the second post-Newtonian approximation,
the erergy is shown that $E^{(1)}$ adds extra terms $E^{(2)}$ which consist of
the energy stored in the configuration $E_{\rm config}$, in the gravitational
field $E_{\rm field}$. and a term of surface integral. These extra terms gives
post-Newtonian corrections to the Newtonian treatment.

PACS No.: 04.20.Cv, 04.25.Nx
Keywords: post-Newtonian approximation of the Schwarzschild metric, the
Weinberg energy-momentum complex.

In special relativity, the physical quantity “mass” has two meanings. The
“invariant mass” $m_{0}$ (also called rest mass) is, in effect, the resistance
that a body of matter offers to change in its velocity upon the application of
a force, and the “relativistic mass” $m_{\rm ref}$ depends on the observer’s
inertial frame of reference for an object moving at a velocity $\vec{v}$
relative to the observer. Although, general relativity does not offer a single
definition of the term mass, but offers several different definitions that are
applicable under different circumstances, like ADM [1] and Bondi masses [2] in
asymptotically flat spacetimes, and Komar mass [3] in stationary spacetimes,
etc. Because of mass-energy equivalence, the concepts of enerrgy and mass are
indistinguishable. So, similar tricky situation also would come about the
definition of energy in general relativity.

There are two famous ideas on the definition of the localized energy-momentum
associated with the gravitational field, which are energy-mometum complex and
quasilocal energy. One idea, called energy-momentum complex, is introduced

$\Theta^{\mu}_{\nu}=\sqrt{-g}\left(T^{\mu}_{\nu}+t^{\mu}_{\nu}\right)$ (1)

which satisfies the differential conservation form
$\partial_{\nu}\Theta^{\mu}_{\nu}=0$. Here $T^{\mu}_{\nu}$ is the energy-
momentum tensor of matter and $t^{\mu}_{\nu}$ is the energy-momentum
pseudotensor from the gravitational field. There have been many proposals of
energy-momentum complex including Einstein [4], Tolman [5], Papapetrou [6],
Landau-Lifshitz [7], Møller [8] and Weinberg [9] tec. Another ideaa,
quasilocal energy, associated for a finite region is determined by the
integral

$H({\bf N},\Sigma)=\int_{\Sigma}N^{\mu}{\cal
H}_{\mu}+\oint_{\partial\Sigma}{\cal B}({\bf N}),$ (2)

which generates the spacetime displacement of a finite spacelike hypersurface
$\Sigma$ along a vector field $N^{\mu}$. Various physicists , such as Hawkin
[10], Penrose [11], Brown and York [12], Hayward [13] and Nester et al. [14]
etc., had given different definitions for quasilocal energy. Furthermore, some
studies were given the mathermatical relations [15, 16, 17] and Legendre
transformation [18, 19, 20, 21] between Einstein and Møller energy-momentum
complexes in specific spacetime Im my recent study, according to the first
post-Newtonian approximation of Schwarzschild metric, the energy components of
the Einstein and Møller energy-momentum complex were obtained, which involve
the rest-mass energy $m$, the energy stored in the configuration and that in
the gravitational field. Thus, in this article, I will estimate the energy
component of the Weinberg energy-momentum complex with the first and second
post-Newtonian approxination of Schwarzschild metric.

In oder to suppose a coordinate system that is quasi-Minkowskian, the metric
was given as

$g_{\mu\nu}=\eta_{\mu\nu}+h_{\mu\nu},$ (3)

where $\eta_{\mu\nu}$ is the Minkowski metric. Weinberg adopt the energy-
momentum pseudotensor of the gravitational field

$t_{\mu\nu}=\frac{1}{8\pi}\left[R_{\mu\nu}-\frac{1}{2}g_{\mu\nu}R^{\lambda}_{\lambda}-{R^{(1)}}_{\mu\nu}+\frac{1}{2}\eta_{\mu\nu}{R^{(1)}}^{\lambda}_{\lambda}\right],$
(4)

and the part of the Ricci tensor linear in $h_{\mu\nu}$

${R^{(1)}}_{\mu\nu}=\frac{1}{2}\left(\frac{\partial^{2}h^{\lambda}_{\lambda}}{\partial
x^{\mu}\partial x^{\nu}}-\frac{\partial^{2}h^{\lambda}_{\mu}}{\partial
x^{\lambda}\partial x^{\nu}}-\frac{\partial^{2}h^{\lambda}_{\nu}}{\partial
x^{\mu}\partial x^{\lambda}}+\frac{\partial^{2}h^{\mu}_{\nu}}{\partial
x^{\lambda}\partial x^{\lambda}}\right).$ (5)

Hence the Winberg energy-momentum complex [9] can be written as

$\Theta^{\mu\nu}={R^{(1)}}^{\mu\nu}-\frac{1}{2}\eta^{\mu\nu}{R^{(1)}}^{\lambda}_{\lambda}=\partial_{\rho}Q^{\rho\mu\nu}$
(6)

with Winberg’s superpotential is

$Q^{\rho\mu\nu}=\frac{1}{2}\left[\frac{\partial
h^{\lambda}_{\lambda}}{\partial x^{\mu}}\eta^{\rho\nu}-\frac{\partial
h^{\lambda}_{\lambda}}{\partial x^{\rho}}\eta^{\mu\nu}-\frac{\partial
h^{\lambda}_{\mu}}{\partial x^{\lambda}}\eta^{\rho\nu}+\frac{\partial
h^{\lambda}_{\rho}}{\partial x^{\mu}}\eta^{\mu\nu}+\frac{\partial
h^{\mu}_{\nu}}{\partial x^{\rho}}-\frac{\partial h^{\rho}_{\nu}}{\partial
x^{\mu}}\right]$ (7)

Thus, the energy component of the Weinberg energy-momentum complex within the
region $\Sigma$ is shown

$\begin{split}E&=\frac{1}{8\pi}\int_{\Sigma}\Theta^{00}d^{3}x=\frac{1}{8\pi}\int_{\Sigma}\frac{\partial}{\partial
x^{i}}Q^{i00}d^{3}x,\\\ &=\frac{1}{16\pi}\int_{\Sigma}\frac{\partial}{\partial
x^{i}}\left(\frac{\partial h_{ik}}{\partial x^{k}}-\frac{\partial
h_{kk}}{\partial x^{i}}\right)d^{3}x.\end{split}$ (8)

For a weak-field case, the first post-Newtonian approximation of the
Schwarzschild metric [9] is studied

$\begin{split}g_{00}&=1-2\phi+{\cal O}(\phi^{2})\\\ g_{0i}&=0\\\
g_{ij}&=(-1-2\phi)\delta_{ij}+{\cal O}(\phi^{2}),\end{split}$ (9)

and the qualities $h_{\mu\nu}$ are given as

$\begin{split}h_{00}&=-2\phi+{\cal O}(\phi^{2})\\\
h_{ij}&=-2\phi\delta_{ij}+{\cal O}(\phi^{2}),\end{split}$ (10)

where $\phi$ is the Newton’s gravitational potential and normalized such that
$\phi(\infty)=0$. Then Winberg’s superpotential is obtained

$Q^{i00}=2\nabla\phi,$ (11)

and the energy component of the Weinberg energy-momentum complex is described
to

${}_{W}\Theta^{0}_{0}=\frac{1}{4\pi}\nabla^{2}\phi.$ (12)

Therefore, in the Winberg prescription, the energy within the region $\Sigma$
that includes mass distribution is given by

$E=\frac{1}{4\pi}\int_{\Sigma}\nabla^{2}\phi d^{3}x.$ (13)

Since the gravitational potential satisfes the field equation with a matter
density $\rho$

$\nabla^{2}\phi=4\pi\rho$ (14)

and the total mass is given by

$\int_{\Sigma}\rho d^{3}x=m.$ (15)

Such, in first post-Newtonian approximation, the energy in the Winberg
prescription can be rewritten

$E=m,$ (16)

and the energy is contributed by the “rest-mass energy” as the invariant mass
is the rest energy in special relativity.

To study the contribution other than the rest-mass energy, the second post-
Newtonian approximation of the Schwarzschild metric [9] is considered

$\begin{split}g_{00}&=1-2\phi+2\phi^{2}+{\cal O}(\phi^{3}),\\\ g_{0i}&=0,\\\
g_{ij}&=(-1-2\phi-\phi^{2})\delta_{ij}-\phi^{2}\frac{x_{i}x_{j}}{r^{2}}+{\cal
O}(\phi^{3}),\end{split}$ (17)

and the qualities $h_{\mu\nu}$ are obtained as

$\begin{split}h_{00}&=-2\phi+2\phi^{2}+{\cal O}(\phi^{3}),\\\
h_{ij}&=(-2\phi-\phi^{2})\delta_{ij}-\phi^{2}\frac{x_{i}x_{j}}{r^{2}}+{\cal
O}(\phi^{3}).\end{split}.$ (18)

Here the Winberg’s superpotential is computed

$Q^{i00}=2\nabla\phi+2\phi\nabla\phi-\frac{2\phi^{2}}{r}\hat{r}..$ (19)

The energy component of the Møller energy-momentum complex is obtained

${}_{W}\Theta^{0}_{0}=\frac{1}{4\pi}\nabla^{2}\phi+\frac{1}{4\pi}\nabla\cdot(\phi\nabla\phi)-\frac{1}{4\pi}\nabla\cdot\frac{\phi^{2}}{r}\hat{r}$
(20)

and the energy within the region $\Sigma$ is followed

$E=\frac{1}{4\pi}\int_{\Sigma}\nabla^{2}\phi
d^{3}x-\frac{1}{4\pi}\int_{\Sigma}\nabla\cdot(\phi\nabla\phi)d^{3}x.+\frac{1}{8\pi}\int_{\Sigma}\nabla\cdot\left(\frac{\phi^{2}}{r}\right)d^{3}x.$
(21)

Although, the second term of the right-hand side in Eq. (19) can be rewriiten
as

$\frac{1}{4\pi}\int_{\Sigma}\nabla\cdot(\phi\nabla\phi)d^{3}x=\frac{1}{4\pi}\int_{\Sigma}\phi\nabla^{2}\phi
d^{3}x+\frac{1}{4\pi}\int_{\Sigma}(\nabla\phi)^{2}d^{3}x.$ (22)

and the first term of right-hand side in Eq.(20) is the energy stored in the
configuration

$E_{\rm config}\equiv\frac{1}{8\pi}\int_{\Sigma}\phi\nabla^{2}\phi d^{3},$
(23)

and the second term is the energy stored in the gravitational field

$E_{\rm field}\equiv\frac{1}{8\pi}\int_{\Sigma}(\nabla\phi)^{2}d^{3}x.$ (24)

The third term of the right-hand side in Eq.(19) can be expressed in terms of
surface integral

$\frac{1}{8\pi}\int_{\Sigma}\nabla\cdot\left(\frac{\phi^{2}}{r}\right)d^{3}x=\frac{1}{8\pi}\oint_{\partial\Sigma}\frac{\phi^{2}}{r}\hat{r}\cdot
d\hat{a}.$ (25)

Consequently, in second post-Newtonian approximation, the energy in the
Winberg prescription within the region $\Sigma$ is shown as

$E=m+2E_{\rm config}+2E_{\rm
field}-\frac{1}{8\pi}\oint_{\partial\Sigma}\frac{\phi^{2}}{r}\hat{r}\cdot
d\hat{a},$ (26)

and resembles the energy in the Einstein prescription

$E_{\rm E}=m+E_{\rm config}+E_{\rm
field}+\frac{1}{8\pi}\oint_{\partial\Sigma}\frac{\phi^{2}}{r}\hat{r}\cdot
d\hat{a}.$ (27)

In summary, the energy of the Schwarzschild spacetime for the first post-
Newtionian approximation by using the Weinberg energy-momentum complex is
obtained

$E^{(1)}=m,$ (28)

and gives the Newtonian treatmenet of Schwarzschild spacetime [22]. The energy
for the second post-Newtionian approximation is that $E^{(1)}$ adds extra
terms

$E^{(2)}=2E_{\rm config}+2E_{\rm
field}-\frac{1}{8\pi}\oint_{\partial\Sigma}\frac{\phi^{2}}{r}\hat{r}\cdot
d\hat{a},$ (29)

and $E^{(2)}$ gives post-Newtonian corrections to the Newtonian treatemnt
[22]. Referrung to quantum electrodynamics, $E^{(1)}$ and $E^{(1)}+E^{(2)}$
behave like bare mass and experimentally observable masss, and $E^{(2)}$ is
the increase in mass owing to the interaction of the matter with the field.
Particularly, the term of surface integral in Eq.(29) could be suggested as
the flux of $(\phi^{2}/r)\hat{r}$ through a surface $\partial\Sigma$. Thus,
for energy-momentum complexes, to study the energy in higher-order post-
Newtonian approximation will be useful to clarify their physical meanings.

## References

  * [1] R.L. Arnowitt, S. Deser and C.W. Misner, in Gravitation: An Introduction to Current Research, ed. L. Witten (Wiley, New York 1962) 227; This article is now available as gr-qc/0405109.
  * [2] H. Bondi, M.G.J. van der Burg and A.W.K. Metzner, . Proc. Roy. Soc. Lond. A269 21 (1962)..
  * [3] A. Komar, .Phys. Rev. 113 934.(1959).
  * [4] A. Einstein, Sitzungsber. Preuss. Akad. Wiss. Berlin 47, 778 (1915); 47, 799 (1915).
  * [5] R.C. Tolman, Phys. Rev..35, 875 (1930).
  * [6] A. Papapetrou, Proc. Roy. Irish Acad. A52, 11 (1948).
  * [7] L.D. Landau and E,M, Lifshitz, The Classical Theory of Fields, 2nd edn. (Addison Wesley, 1962).
  * [8] C. Møller, Ann. Phys. (NY) 4, 347 (1958); ibid. 12, 118 (1961).
  * [9] S. Weinberg, Gravitation and Cosmology: Principles and Applications of General Theory of Relativity (John Wiley and Sons, Inc., New York, 1972, p. 165).
  * [10] S.W. Hawking, J. Math. Phys .9, 598 (1968).
  * [11] R. Penrose, Proc. R. Soc. London A381, 53 (1982).
  * [12] J.D. Brown and J.W. York, Jr., Phys. Rev. D47, 1407, 1420 (1993).
  * [13] S.A. Hayward, Phys. Rev. D49, 831 (1994).
  * [14] C.-M. Chen, J.M. Nester and R.-S. Tung, Phys. Lett .A203, 5 (1995).
  * [15] I.-C. Yang, Chin. J. Phys. 40, 42 (2004).
  * [16] E.C. Vagenas, :Mod. Phys. Lett. A21, 1947 (2006).
  * [17] I.-C. Yang, C.-L. Lin and I. Radinschi, Int. J. Theor. Phys. 48, 248 (2009).
  * [18] I.-C. Yang, Chin. J. Phys. 50, 544 (2012).
  * [19] I.-C. Yang, B.-A. Chen and C.-C. Tsai, Mod. Phts. Lett. A27, 1250169 (2012).
  * [20] I.-C. Yang and W.-K. Huang, Mod. Phys. Lett. A29, 1450126 (2014).
  * [21] I.-C. Yang, Adv. High Energy Phys. 2017, 2482940 (2017).
  * [22] C. W. Misner, K. S. Thorne, and J. A. Wheeler, Gravitation. (Freeman, 2000).


#  Probing the Supersymmetric Grand Unified Theories with Gravity Mediation at
the Future Proton-Proton Colliders and Hyper-Kamiokande Experiment

Waqas Ahmed<EMAIL_ADDRESS>School of Mathematics and Physics, Hubei
Polytechnic University, Huangshi 435003,China  Tianjun Li<EMAIL_ADDRESS>CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics,
Chinese Academy of Sciences, Beijing 100190, China  School of Physical
Sciences, University of Chinese Academy of Sciences, No. 19A Yuquan Road,
Beijing 100049, China  Shabbar Raza<EMAIL_ADDRESS>Department of
Physics, Federal Urdu University of Arts, Science and Technology, Karachi
75300, Pakistan

###### Abstract

With the grand desert hypothesis, we have proposed to probe the supersymmetric
Grand Unified Theories (GUTs) at the future proton-proton (pp) colliders and
Hyper-Kamiokande experiment previously. In this paper, we study the
supersymmetric GUTs with gravity mediated supersymmetry breaking in details.
First, considering the dimension-six proton decay via heavy gauge boson
exchange, we point out that we can probe the supersymmetric GUTs with GUT
scale $M_{GUT}$ up to $1.778\times 10^{16}$ GeV at the Hyper-Kamiokande
experiment. Second, for the supersymmetric GUTs with $M_{GUT}\geq 1.0\times
10^{16}$ GeV and $M_{GUT}\geq 1.2\times 10^{16}$ GeV, we show that the upper
bounds on the universal gaugino mass are $7.2$ TeV and 3.5 TeV, respectively,
and thus the corresponding upper bounds on gluino mass are 15 TeV and 8 TeV,
respectively. Also, we shall study the masses for charginos, neutralinos,
squarks, sleptons, and Higgs particles in details. In particular, the
supersymmetric GUTs with $M_{GUT}\leq 1.2\times 10^{16}$ GeV can be probed at
the Hyper-Kamiokande experiment, and the supersymmetric GUTs with $M_{GUT}\geq
1.2\times 10^{16}$ GeV can be probed at the future 100 TeV pp collider
experiments such as the ${\rm FCC}_{\rm hh}$ and SppC via gluino searches.
Thus, the supersymmetric GUTs with gravity mediation can be probed by the
${\rm FCC}_{\rm hh}$, SppC, and Hyper-Kamiokande experiments. In our previous
study, we have shown that the supersymmetric GUTs with anomaly and gauge
mediated supersymmetry breakings are well within the reaches of these
experiments. Therefore, our proposal provides the concrete scientific goal for
the ${\rm FCC}_{\rm hh}$, SppC, and Hyper-Kamiokande experiments: probing the
supersymmetric GUTs.

##  I Introduction

It is well known that supersymmetry (SUSY) provides a natural solution to the
gauge hierarchy problem in the Standard Model (SM). In the supersymmetric SMs
(SSMs) with $Z_{2}$ $R$-parity, we can achieve the gauge coupling unification
gaugeunification , have the Lightest Supersymmetric Particle (LSP) like
neutralino as a dark matter (DM) candidate Jungman:1995df , and break the
electroweak (EW) gauge symmetry radiatively due to the large top quark Yukawa
coupling, etc. In particular, gauge coupling unification strongly suggests the
Grand Unified Theories (GUTs) Georgi:1974sy ; Pati:1974yy ; Fritzsch:1974nn ;
Georgi:1974my , and the SUSY GUTs can be constructed from superstring theory,
which is the most competitive candidate for quantum gravity. Therefore,
supersymmetry is not only the most promising new physics beyond the SM, but
also a bridge between the low energy phenomenology and high-energy fundamental
physics.

However, after accumulation of data from the LHC Run-1 and Run-2, we have no
hints for the SSMs. Of course, with the help of theses data now we have
stronger bounds on the spectra of the supersymmetric particles (sparticles).
For instance, the lower mass bounds on gluino, the first two generation of
squarks, stop, and sbottom are 2.3 TeV, 1.9 TeV, 1.25 TeV, and 1.5 TeV,
respectively ATLAS-SUSY-Search ; Aad:2020sgw ; Aad:2019pfy ; CMS-SUSY-Search-I
; CMS-SUSY-Search-II . Thus, there might exist the EW fine-tuning problem, and
some promising solutions have been proposed Dimopoulos:1995mi ; Cohen:1996vb ;
Kitano:2006gv ; LeCompte:2011cn ; Fan:2011yu ; Kribs:2012gx ; Baer:2012mv ;
Ding:2015epa ; Batell:2015fma ; Leggett:2014hha ; Du:2015una ; Li:2015dil ;
Ellis:1986yg ; Barbieri:1987fn . These natural SSMs generically predicts some
relatively light sparticles, for instance, Bino, Higgsino, stop, gluino, and
sleptons, etc.

On the other hand, to probe the new physics beyond the SM, we have a few
proposals for the future proton-proton (pp) colliders, for example, the ${\rm
FCC}_{\rm hh}$ Benedikt:2018csr  and SppC CEPC-SPPCStudyGroup:2015csa . The
naive question is whether we can probe the supersymmetry at the ${\rm
FCC}_{\rm hh}$ and SppC, but the answer is no since the sparticles can be very
heavy. And then the interesting question is whether we can probe the
supersymmetric GUTs at the future experiments even if there does exist the
SUSY EWFT problem. In a previous study Ahmed:2020qdt , with the grand desert
hypothesis, we showed that the supersymmetric GUTs can be probed at the future
proton-proton (pp) colliders and Hyper-Kamiokande experiment. For the GUTs
with the GUT scale $M_{GUT}\leq 1.0\times 10^{16}$ GeV, the dimension-six
proton decay via heavy gauge boson exchange can be probed at the Hyper-
Kamiokande experiment. Also, for the GUTs with $M_{GUT}\geq 1.0\times 10^{16}$
GeV, we showed that the GUTs with anomaly and gauge mediated supersymmetry
breakings are well within the reaches of the future 100 TeV pp colliders such
as the ${\rm FCC}_{\rm hh}$ and SppC, and the supersymmetric GUTs with gravity
mediated supersymmetry breaking can be probed at the future 160 TeV pp
collider. Therefore, the remaining interesting question is whether we can
probe the supersymmetric GUTs with gravity mediated supersymmetry breaking at
the ${\rm FCC}_{\rm hh}$ and SppC.

In this paper, we shall study the supersymmetric GUTs with gravity mediated
supersymmetry breaking chams ; bbo ; cmssm  in details. First, considering the
dimension-six proton decay via heavy gauge boson exchange, we point out that
we can probe the supersymmetric GUTs with GUT scale $M_{GUT}$ up to
$1.778\times 10^{16}$ GeV at the Hyper-Kamiokande experiment. Second, for the
supersymmetric GUTs with $M_{GUT}\geq 1.0\times 10^{16}$ GeV and $M_{GUT}\geq
1.2\times 10^{16}$ GeV, we show that the upper bounds on the universal gaugino
mass are $7.2$ TeV and 3.5 TeV, respectively, and thus the corresponding upper
bounds on gluino mass are 15 TeV and 8 TeV, respectively. Also, we shall study
the masses for charginos, neutralinos, squarks, sleptons, and Higgs particles
in details. In particular, the supersymmetric GUTs with $M_{GUT}\leq 1.2\times
10^{16}$ GeV can be probed at the Hyper-Kamiokande experiment, and the
supersymmetric GUTs with $M_{GUT}\geq 1.2\times 10^{16}$ GeV can be probed at
the ${\rm FCC}_{\rm hh}$ and SppC experiments via gluino searches. Thus, the
supersymmetric GUTs with gravity mediation can be probed by the ${\rm
FCC}_{\rm hh}$, SppC, and Hyper-Kamiokande experiments. In our previous study,
we have shown that the supersymmetric GUTs with anomaly and gauge mediated
supersymmetry breakings are well within the reaches of these experiments
Ahmed:2020qdt . Therefore, we propose the concrete scientific goal for the
${\rm FCC}_{\rm hh}$, SppC, and Hyper-Kamiokande experiments: probing the
supersymmetric GUTs.

This paper is organized as follows. In Section II, we discuss the
supersymmetric GUT searches at the Hyper-Kamiokande experiment. In Section
III, we study the supersymmetric GUTs with gravity mediated supersymmetry
breaking in details, and their searches at the future proton-proton colliders.
Our conclusion is given in Section IV.

##  II Probing the Supersymmetric Grand Unified Theories at the Hyper-
Kamiokande Experiment

In the GUTs, the well-know prediction is the dimension-six proton decay $p\to
e^{+}\pi^{0}$ via heavy gauge boson exchange, and the proton lifetime is given
by Hisano:2000dg ; Dutta:2016jqn

$\displaystyle\tau_{p}(e^{+}\pi^{0})$ $\displaystyle\simeq$ $\displaystyle
1.0\times
10^{34}\times\left(\frac{2.5}{A_{R}}\right)^{2}\times\left(\frac{0.04}{\alpha_{\rm
GUT}}\right)^{2}$ (1) $\displaystyle\times\left(\frac{M_{\rm X/Y}}{1.0\times
10^{16}~{}{\rm GeV}}\right)^{4}~{}{\rm years}~{},~{}\,$

where $A_{R}$ is the dimensionless one-loop renormalization factor associated
with anomalous dimension of the relevant baryon-number violating operators,
$\alpha_{\rm GUT}$ is the unified gauge coupling, and $M_{\rm X/Y}$ is the
mass for the heavy gauge bosons $X_{\mu}/Y_{\mu}$. The current lower limit on
the proton lifetime from the Super-Kamiokande experiment is
$\tau_{p}>1.6\times 10^{34}$ years Miura:2016krn . Thus, we obtain $M_{\rm
X/Y}\geq 1.0\times 10^{16}~{}{\rm GeV}$. At the future Hyper-Kamiokande
experiment, we can probe the proton lifetime at least above $1.0\times
10^{35}$ years Abe:2018uyc . Thus, the GUTs with $M_{\rm X/Y}\leq 1.778\times
10^{16}~{}{\rm GeV}$ is within the reach of the future Hyper-Kamiokande
experiment. For more detail related GUTs and $M_{GUT}$ related bosons see
Hisano:2013cqa ; Ellis:2015jwa ; Pokorski:2019ete ; Tobe:2003yj ; Goto:1998qg
; Murayama:2001ur ; Liu:2013ula ; Hisano:2013exa ; Babu:2020ncc .

To clarify the subtle point, we want to emphasize that the mass of the heavy
gauge bosons $X_{\mu}/Y_{\mu}$ is smaller than or equal to the GUT scale
$M_{GUT}$. Thus, the supersymmetric GUTs with GUT scale up to $1.778\times
10^{16}~{}{\rm GeV}$ can be probed at the future Hyper-Kamiokande experiment.

##  III Probing the Supersymmetric Grand Unified Theories with Gravity
Mediation at the Future Proton-Proton Colliders

The supersymmetry searches at the 100 TeV pp colliders have been studied
previously Benedikt:2018csr ; CEPC-SPPCStudyGroup:2015csa ; Cohen:2013xda ;
Arkani-Hamed:2015vfh ; Fan:2017rse ; Golling:2016gvc . For the integrated
luminosity 30 ${\rm ab}^{-1}$, Wino via Bino decay, gluino ${\tilde{g}}$ via
heavy flavor decay, gluino via light flavor decay, first-two generation
squarks ${\tilde{q}}$, and stop can be discovered for their masses up to about
6.5 TeV, 11 TeV, 17 TeV, 14 TeV, and 11 TeV, respectively. Moreover, if the
gluino and first-two generation squark masses are similar, they can be probed
up to 20 TeV.

Figure 1: Plot in $M_{1/2}-M_{GUT}$ plane. Gray points are consistent with
REWSB and LSP neutralino. Orange points satisfy the mass bounds including
$m_{h}=125\pm 3\,{\rm GeV}$ and the constraints from rare $B-$ meson decays.
Blue points form a subset of orange points and satisfy $1\lesssim
M_{GUT}\lesssim 1\times 10^{16}\,{\rm GeV}$, while red points form a subset of
orange points and satisfy $M_{GUT}\gtrsim 1.2\times 10^{16}\,{\rm GeV}$. Two
horizontal blue and red lines represent $M_{GUT}=1\times 10^{16}\,{\rm GeV}$
and $M_{GUT}=1.2\times 10^{16}\,{\rm GeV}$, respectively. The first vertical
line shows the upper bound on $M_{1/2}$ for red points ($M_{1/2}=3.5$ TeV),
and the second vertical line shows the upper bound on $M_{1/2}$ for blue
points ($M_{1/2}=7.2$ TeV).

By the way, the correlations between the low energy SUSY spectra and the GUT
scale have been studied via the one-loop renormalization group equations
before, and it was found that the bound from dimension-six proton decay
already excludes the gluinos and Winos heavier than about 120 TeV and 40 TeV,
respectively, if their mass ratio $M_{3}/M_{2}$ is about 3 Pokorski:2017ueo .
In our paper, we employ the ISAJET 7.85 package ISAJET  to perform the scan,
which will give us more precise results. To be concrete, for the
supersymmetric GUTs with $M_{GUT}\leq 1.0\times 10^{16}$ GeV, we find that the
current bound from dimension-six proton decay excludes the gluinos and Winos
heavier than 15 TeV and 6 TeV, respectively. Because Winos might decay via
Higgsinos as the benchmark point 4 given in the following subsection B, we are
not sure whether Wino is within the reach of the ${\rm FCC}_{\rm hh}$ and SppC
experiments, which will be studied elsewhere. Moreover, we shall study the
masses for charginos, neutralinos, squarks, sleptons, and Higgs particles as
well.

###  III.1 Phenomenological Constraints and Scanning Procedure

We summarize our scanning procedure and the experimental constraints in this
part of our article. The package ISAJET 7.85 ISAJET  is employed to perform
the random scans over the parameter space of mSUGRA/CMSSM. In this code, the
weak scale values of the gauge and third generation Yukawa couplings are
evolved to $M_{\rm GUT}$ via the MSSM RGEs in the $\overline{DR}$
regularization scheme. It is to be noted that we do not strictly enforce the
unification condition $g_{3}=g_{1}=g_{2}$ at $M_{\rm GUT}$. It is because a
few percent deviation from unification can be assigned to the unknown GUT-
scale threshold corrections Hisano:1992jj . Moreover we do not allow $g_{3}$
to deviate from the unification by more than about $3\%$. We define the
boundary conditions of our model at $M_{\rm GUT}$ and then all the Soft
Supersymmetry Breaking (SSB) parameters, along with the gauge and Yukawa
couplings, are evolved back to the weak scale $M_{\rm Z}$.

In evaluating Yukawa couplings, the SUSY threshold corrections Pierce:1996zz
are taken into account at the common scale $M_{\rm
SUSY}=\sqrt{m_{\tilde{t}_{L}}m_{\tilde{t}_{R}}}$. The entire parameter set is
iteratively run between $M_{\rm Z}$ and $M_{\rm GUT}$ using the full two-loop
RGEs until a stable solution is obtained. To better account for the leading-
log corrections, one-loop step-beta functions are adopted for the gauge and
Yukawa couplings, and the SSB parameters $m_{i}$ are extracted from RGEs at
appropriate scales as $m_{i}=m_{i}(m_{i})$. The RGE-improved one-loop
effective potential is minimized at an optimized scale $M_{\rm SUSY}$, which
effectively accounts for the leading two-loop corrections. The full one-loop
radiative corrections are incorporated for all sparticles.

Note that we set $\mu>0$ and use $m_{t}=173.3\,{\rm GeV}$ :2009ec . We also
use $m_{b}^{\overline{DR}}(M_{\rm Z})=2.83$ GeV which is hard-coded into
ISAJET.

---
Figure 2: The gluino mass versus supersymmetry breaking soft terms and
$\tan\beta$. The color coding is the same as in Fig. 1. Horizontal red, black
and blue lines represent the gluino mass upper bounds of 8 TeV, 11 TeV, and 15
TeV corresponding to the red points, gluino discovery via heavy flavor decay
at 100 TeV pp collider Cohen:2013xda , and blue points, respectively. The
first vertical line shows the upper bound on $M_{1/2}$ for red points
($M_{1/2}=3.5$ TeV), and the second vertical line shows the upper bound on
$M_{1/2}$ for blue points ($M_{1/2}=7.2$ TeV).

The fundamental parameters of mSUGRA/CMSSM are restricted as follows

$\displaystyle 0\leq$ $\displaystyle m_{0}\leq 90~{}\rm{TeV}~{},~{}$
$\displaystyle 0\leq$ $\displaystyle M_{1/2}\leq 30~{}\rm{TeV}~{},~{}$
$\displaystyle-3\leq$ $\displaystyle A_{0}/m_{0}\leq 3~{},~{}$ $\displaystyle
2\leq$ $\displaystyle\tan\beta\leq 60~{}.~{}$

We would like to draw attention of the reader to the fact that the requirement
of radiative electroweak symmetry breaking (REWSB) Ibanez:1982fr  puts an
important theoretical constraint on the parameter space. Another important
constraint comes from the limits on the cosmological abundance of stable
charged particles Beringer:1900zz . They exclude the parameter space where the
charged SUSY particles, such as $\tilde{\tau}_{1}$ or $\tilde{t}_{1}$, become
the LSP.

The data points collected all satisfy the requirement of REWSB, with the
neutralino being the LSP. In addition, after collecting the data, we impose
the mass bounds on all the sparticles Agashe:2014kda , and the constraints
from rare decay processes $B_{s}\rightarrow\mu^{+}\mu^{-}$ Aaij:2012nna ,
$b\rightarrow s\gamma$ Amhis:2012bh , and $B_{u}\rightarrow\tau\nu_{\tau}$
Asner:2010qj . More explicitly, we set

$\displaystyle m_{h}=122-128~{}{\rm GeV}~{}~{}$ , (3) $\displaystyle
m_{\tilde{g}}\geq 2.2~{}{\rm TeV}~{},$ (4) $\displaystyle 0.8\times
10^{-9}\leq{\rm BR}(B_{s}\rightarrow\mu^{+}\mu^{-})\leq 6.2\times
10^{-9}\;(2\sigma)~{}~{}$ , (5) $\displaystyle 2.99\times 10^{-4}\leq{\rm
BR}(b\rightarrow s\gamma)\leq 3.87\times 10^{-4}\;(2\sigma)~{}~{}$ , (6)
$\displaystyle 0.15\leq\frac{{\rm BR}(B_{u}\rightarrow\tau\nu_{\tau})_{\rm
MSSM}}{{\rm BR}(B_{u}\rightarrow\tau\nu_{\tau})_{\rm SM}}\leq
2.41\;(3\sigma)~{}~{}$ . (7)

To be general, we do not require the relic abundance of the LSP neutralino to
satisfy the Planck 2018 bound within $5\sigma$: $0.114\leq\Omega_{\rm
CDM}h^{2}(\rm Planck)\leq 0.126$ Akrami:2018vks .

---
Figure 3: The gluino mass versus the left-handed slepton mass, light stau
mass, light stop mass, and CP-odd Higgs boson mass. The color coding is the
same as in Fig. 2.
---
Figure 4: The gluino mass versus the light chargino mass and lightest
neutralino mass, the light chargino mass versus the lightest neutralino mass,
and dark matter density versus the lightest neutralino mass. The color coding
is the same as in Fig. 2.

###  III.2 Scan Results

We shall discuss results from the systematical scans. In Fig. 1, we show plot
$M_{GUT}$ as a function of $M_{1/2}$. Gray points are consistent with REWSB
and LSP neutralino. Orange points satisfy the mass bounds including
$m_{h}=125\pm 3\,{\rm GeV}$ and the constraints from rare $B-$ meson decays.
Blue points form a subset of orange points and satisfy $1\lesssim
M_{U}\lesssim 1\times 10^{16}\,{\rm GeV}$, while red points form a subset of
orange points and satisfy $M_{GUT}\gtrsim 1.2\times 10^{16}\,{\rm GeV}$. Two
horizontal blue and red lines represent $M_{GUT}=1\times 10^{16}\,{\rm GeV}$
and $M_{GUT}=1.2\times 10^{16}\,{\rm GeV}$, respectively. The first vertical
line shows the upper bound on $M_{1/2}$ for red points ($M_{1/2}=3.5$ TeV),
while the second vertical line shows the upper bound on $M_{1/2}$ for blue
points ($M_{1/2}=7.2$ TeV). From the upper bounds on gaugino masses $M_{1/2}$
given by two vertical lines, we obtain that the upper bounds on gluino masses
are 8 TeV and 15 TeV respectively for the red and blue points. Therefore, we
clearly show that SUSY GUTs with $M_{GUT}\gtrsim 1.2\times 10^{16}$ GeV for
gravity mediated SUSY breaking scenario chams ; bbo ; cmssm , i.e. the red
points, can be probed by the future 100 TeV pp colliders such as the ${\rm
FCC}_{\rm hh}$ and SppC. Moreover, the blue points and orange points with
$M_{GUT}\leq 1.0\times 10^{16}\,{\rm GeV}$ can be explored by the Hyper-
Kamiokande experiment. In the latter part of the paper we see the impact of
these bounds on the fundamental parameters of the mSUGRA/CMSSM and sparticle
spectrum.

In Fig. 2, we display plots the mSUGRA/CMSSM fundamental parameters as
function of gluino mass. The color coding is the same as in Fig. 1. Horizontal
red, black and blue lines represent the gluino mass upper bounds of 8 TeV, 11
TeV, and 15 TeV corresponding to the red points, gluino discovery via heavy
flavor decay at 100 TeV pp collider Cohen:2013xda , and blue points,
respectively. The first vertical line shows the upper bound on $M_{1/2}$ for
red points ($M_{1/2}=3.5$ TeV), and the second vertical line shows the upper
bound on $M_{1/2}$ for blue points ($M_{1/2}=7.2$ TeV). In the top left corner
the plot in $M_{1/2}-m_{\tilde{g}}$ plane is shown. We see that the gluino
mass range for red points is from 2.3 TeV to 8 TeV. And the Hyper-Kamiokande
experiment can probe gluino with masses in the range starting from 3 TeV to 15
TeV. In the right panel we show plot in $m_{0}-m_{\tilde{g}}$ plane. We see
that $m_{0}$ can be as heavy as 64 TeV for red points as well as blue points.
This has important implications. Because $m_{\tilde{q}}^{2}\approx
m_{0}^{2}+(6-7)M_{1/2}^{2}$ , $m_{\tilde{e}_{L}}^{2}\approx
m_{0}^{2}+0.5M_{1/2}^{2}$, and $m_{\tilde{e}_{R}}^{2}\approx
m_{0}^{2}+0.15M_{1/2}^{2}$ Baer:2006rs , the large $m_{0}$ term will give the
dominate contributions to the squark and slepton masses. Plot in
$A_{0}/m_{0}-m_{\tilde{g}}$ plane is shown in lower left panel. Here we see
that for smaller values of $A_{0}/m_{0}$, gluino mass $m_{\tilde{g}}$ rises.
Plot is almost symmetric along $A_{0}/m_{0}=0$, and $m_{\tilde{g}}$ decreases
as as $|A_{0}/m_{0}|$ increases. Plot in $\tan\beta-m_{\tilde{g}}$ plane is
shown in the lower right corner. It is evident from the plot that the red
points, blue points and orange points can have $\tan\beta$ from 2 to 60.

In Fig. 3, we present the gluino mass versus the left-handed slepton mass,
light stau mass, light stop mass, and CP-odd Higgs boson mass. In the top
upper panel, we display plot in $m_{\tilde{e}_{L}}-m_{\tilde{g}}$ plane. As we
stated earlier, large $m_{0}$ term can give dominant contributions to the
squark and slepton masses, so this plot is very much similar to
$m_{0}-m_{\tilde{g}}$. We do not show the plot for
$m_{\tilde{e}_{R}}-m_{\tilde{g}}$ because it is similar to
$m_{\tilde{e}_{L}}-m_{\tilde{g}}$ plot. Similarly, we depict
$m_{\tilde{\tau}_{1}}-m_{\tilde{g}}$ plot in the top right panel in which also
have similar feature to $m_{\tilde{e}_{L}}-m_{\tilde{g}}$. Plot in
$m_{\tilde{t}_{1}}-m_{\tilde{g}}$ plane is shown in lower left panel. Here we
see the similar trend but the mass ranges are reduced. Plot in lower right
panel is shown in $m_{A}-m_{\tilde{g}}$ plane. Here we notice that $m_{A}$ can
be as heavy as 64 TeV for both red and blue points.

In Fig. 4, we show the gluino mass versus the light chargino mass and lightest
neutralino mass, the light chargino mass versus the lightest neutralino mass,
and dark matter density versus the lightest neutralino mass. Plot in the top
left panel shows a graph in $m_{\tilde{\chi}_{1}^{\pm}}-m_{\tilde{g}}$ plane.
It shows that for red points the chargino mass $m_{\tilde{\chi}_{1}^{\pm}}$
can be in the range from 0.1 TeV to 2 TeV but for the blue points the chargino
mass reaches up to 3.4 TeV. Ref. ATL-PHYS-PUB-2021-007  reports the exclusion
limits on $\tilde{\chi}_{1}^{+}\tilde{\chi}_{1}^{-}$ and
$\tilde{\chi}_{1}^{\pm}\tilde{\chi}_{2}^{0}$ productions with
$\tilde{l}$-mediated decays and the productions with the SM-boson-mediated
decays, which require $m_{\tilde{\chi}_{1}^{\pm}}\gtrsim$ 700 GeV. This means
that most of our points can satisfy these bounds. Moreover, we will show later
that for some of the lighter solutions $m_{\tilde{\chi}_{1}^{\pm}}$ and
$m_{\tilde{\chi}_{1}^{0}}$ are mass-degenerate and then can evade these
constraints as well. A plot in $m_{\tilde{\chi}_{1}^{0}}-m_{\tilde{g}}$ plane
is shown in the top right panel. This plot also has similar features to the
previous plot. Here, the neutralino mass can reached up to 1.1 TeV for red
points and 2 TeV for blue points. In the lower left panel, we show plot in
$m_{\tilde{\chi}_{1}^{0}}-m_{\tilde{\chi}_{1}^{\pm}}$ plane. The diagonal
black line is just to show the region of chargino-neutralino coannihilation
where chargino and neutralino masses are degenerate, and most of these points
might have the Higgsino LSP. For red and blue points, the coannihilation
region has the upper mass bounds on $m_{\tilde{\chi}_{1}^{0}}$ about 1 TeV and
1.9 TeV, respectively. The coannihilation region with lower mass range as
stated before is safe from collider constraints. In the right lower panel, we
displays the plot with the LSP neutralino mass versus dark matter relic
density $\Omega h^{2}$. Horizontal two black lines represent Planck 2018
5$\sigma$ bounds on cold dark matter relic density as shown above. It can be
seen that some of the red and blue points can satisfy the relic density
bounds. In the following, we present four benchmark points in Table 1 for the
red and blue points, which are consistent with the dark matter relic density
bounds.

| Point 1 | Point 2 | Point 3 | Point 4
---|---|---|---|---
$m_{0}$ | 7966 | 56830 | 13170 | 59140
$M_{1/2}$ | 2225 | 3056 | 4223 | 6457
$A_{0}/m_{0}$ | 0.5109 | 0.84 | -0.038 | 0.9733
$\tan\beta$ | 54.8 | 6.2 | 20.6 | 7.32
$M_{GUT}$ | 1.234$\times 10^{16}$ | 1.226$\times 10^{16}$ | 1.035$\times 10^{16}$ | 1.01 $\times 10^{16}$
$m_{h}$ | 123 | 125 | 125 | 126
$m_{H}$ | 3425 | 57565 | 12686 | 59618
$m_{A}$ | 3402 | 57189 | 12603 | 59229
$m_{H^{\pm}}$ | 3426 | 57565 | 12686 | 59619
$m_{\tilde{\chi}^{0}_{1,2}}$ | 791, 800 | 989, 994 | 870, 873 | 1061, 1063
$m_{\tilde{\chi}^{0}_{3,4}}$ | 1009, 1893 | 1424, 2705 | 1962, 3633 | 3070, 5703
$m_{\tilde{\chi}^{\pm}_{1,2}}$ | 820, 1857 | 1038, 2619 | 901, 3570 | 1113, 5557
$m_{\tilde{g}}$ | 4964 | 7399 | 9009 | 14286
$m_{\tilde{u}_{L,R}}$ | 8894,8839 | 56887, 57050 | 15042, 14898 | 59898, 59990
$m_{\tilde{t}_{1,2}}$ | 5874,6820 | 33309, 46976 | 10067, 12811 | 35365, 49562
$m_{\tilde{d}_{L,R}}$ | 8895,8833 | 56887, 57077 | 15042,14898 | 59898, 60007
$m_{\tilde{b}_{1,2}}$ | 6780, 7340 | 46856, 56860 | 12763, 14506 | 49431, 59701
$m_{\tilde{\nu}_{1}}$ | 8087 | 56842 | 13429 | 59259
$m_{\tilde{\nu}_{3}}$ | 6919 | 56842 | 13192 | 59085
$m_{\tilde{e}_{L,R}}$ | 8085, 8000 | 56815, 56796 | 13426,13247 | 59231, 59141
$m_{\tilde{\tau}_{1,2}}$ | 5374, 6911 | 56564,56688 | 12764, 13187 | 58796, 59048
$\sigma_{SI}({\rm cm}^{2})$ | 1.71$\times 10^{-45}$ | 6.64$\times 10^{-46}$ | 1.3$\times 10^{-46}$ | 4.36$\times 10^{-47}$
$\Omega_{CDM}h^{2}$ | 0.114 | 0.123 | 0.115 | 0.125

Table 1:  Four benchmark points for the red and blue points, which are
consistent with the dark matter relic density bounds. Sparticle and Higgs
masses are in GeV units.

##  IV Conclusion

We have studied the supersymmetric GUTs with gravity mediated supersymmetry
breaking in details. First, considering the dimension-six proton decay via
heavy gauge boson exchange, we pointed out that the supersymmetric GUTs with
GUT scale $M_{GUT}$ up to $1.778\times 10^{16}$ GeV can be probed at the
Hyper-Kamiokande experiment. Second, for the supersymmetric GUTs with
$M_{GUT}\geq 1.0\times 10^{16}$ GeV and $M_{GUT}\geq 1.2\times 10^{16}$ GeV,
we showed that the upper bounds on the universal gaugino mass are $7.2$ TeV
and 3.5 TeV, respectively, and thus the corresponding upper bounds on gluino
mass are 15 TeV and 8 TeV, respectively. In particular, the supersymmetric
GUTs with $M_{GUT}\leq 1.2\times 10^{16}$ GeV can be probed at the Hyper-
Kamiokande experiment, and the supersymmetric GUTs with $M_{GUT}\geq 1.2\times
10^{16}$ GeV can be probed at the ${\rm FCC}_{\rm hh}$ and SppC experiments
via gluino searches. Thus, the supersymmetric GUTs with gravity mediation can
be probed by the ${\rm FCC}_{\rm hh}$, SppC, and Hyper-Kamiokande experiments.
In our previous study, we have shown that the supersymmetric GUTs with anomaly
and gauge mediated supersymmetry breakings are well within the reaches of
these experiments. Therefore, we propose the concrete scientific goal for the
${\rm FCC}_{\rm hh}$, SppC, and Hyper-Kamiokande experiments: probing the
supersymmetric GUTs.

###### Acknowledgements.

This work is supported in part by the National Key Research and Development
Program of China Grant No. 2020YFC2201504, by the Projects No. 11875062 and
No. 11947302 supported by the National Natural Science Foundation of China, by
the Key Research Program of the Chinese Academy of Sciences, Grant NO. XDPB15,
as well as by the Scientific Instrument Developing Project of the Chinese
Academy of Sciences, Grant No. YJKYYQ20190049.

## References

  * (1) Phys. Rev. D 24 (1981) 1681; W. J. Marciano and G. Senjanovic, Phys. Rev. D 25, 3092 (1982); U. Amaldi, W. de Boer and H. Furstenau, Phys. Lett. B 260, 447 (1991); J. R. Ellis, S. Kelley and D. V. Nanopoulos, Phys. Lett. B 260 (1991) 131; P. Langacker and M. X. Luo, Phys. Rev. D 44 (1991) 817.
  * (2) G. Jungman, M. Kamionkowski, and K. Griest, Phys.Rept. 267, 195 (1996).
  * (3) H. Georgi and S. L. Glashow,  Phys. Rev. Lett. 32 (1974).
  * (4) J. C. Pati and A. Salam,  Phys. Rev. D10, 275 (1974),  [Erratum: Phys. Rev.D11,703(1975)].
  * (5) H. Fritzsch and P. Minkowski,  Annals Phys. 93 (1975).
  * (6) H. Georgi,  AIP Conf. Proc. 23, 575 (1975).
  * (7) ATLAS Collaboration, ATLAS-CONF-2019-040.
  * (8) G. Aad et al. [ATLAS], [arXiv:2004.14060 [hep-ex]].
  * (9) G. Aad et al. [ATLAS], JHEP 12, 060 (2019) [arXiv:1908.03122 [hep-ex]].
  * (10) CMS Collaboration, CMS PAS SUS-19-005.
  * (11) CMS Collaboration, CMS PAS SUS-19-006.
  * (12) S. Dimopoulos and G. F. Giudice, Phys. Lett. B 357, 573-578 (1995) [arXiv:hep-ph/9507282 [hep-ph]].
  * (13) A. G. Cohen, D. B. Kaplan and A. E. Nelson, Phys. Lett. B 388, 588-598 (1996) [arXiv:hep-ph/9607394 [hep-ph]].
  * (14) R. Kitano and Y. Nomura, Phys. Rev. D 73, 095004 (2006) [arXiv:hep-ph/0602096 [hep-ph]].
  * (15) T. J. LeCompte and S. P. Martin, Phys. Rev. D 84, 015004 (2011) [arXiv:1105.4304 [hep-ph]].
  * (16) J. Fan, M. Reece and J. T. Ruderman, JHEP 11, 012 (2011) [arXiv:1105.5135 [hep-ph]].
  * (17) G. D. Kribs and A. Martin, Phys. Rev. D 85, 115014 (2012) [arXiv:1203.4821 [hep-ph]].
  * (18) H. Baer, V. Barger, P. Huang, D. Mickelson, A. Mustafayev and X. Tata, Phys. Rev. D 87, no.3, 035017 (2013) [arXiv:1210.3019 [hep-ph]].
  * (19) R. Ding, T. Li, F. Staub and B. Zhu, Phys. Rev. D 93, no. 9, 095028 (2016).
  * (20) B. Batell, G. F. Giudice and M. McCullough, JHEP 1512, 162 (2015).
  * (21) T. Leggett, T. Li, J. A. Maxin, D. V. Nanopoulos and J. W. Walker, Phys. Lett. B 740, 66 (2015).
  * (22) G. Du, T. Li, D. V. Nanopoulos and S. Raza, Phys. Rev. D 92, no. 2, 025038 (2015) [arXiv:1502.06893 [hep-ph]].
  * (23) T. Li, S. Raza and X. C. Wang, Phys. Rev. D 93, no.11, 115014 (2016) [arXiv:1510.06851 [hep-ph]].
  * (24) J. R. Ellis, K. Enqvist, D. V. Nanopoulos and F. Zwirner, Mod. Phys. Lett. A 1, 57 (1986).
  * (25) R. Barbieri and G. F. Giudice, Nucl. Phys. B 306, 63-76 (1988).
  * (26) A. Abada et al. [FCC], Eur. Phys. J. ST 228, no.4, 755-1107 (2019)
  * (27) M. Ahmad et al., IHEP-CEPC-DR-2015-01.
  * (28) W. Ahmed, T. Li, S. Raza and F. Z. Xu, Phys. Lett. B 819, 136378 (2021) [arXiv:2007.15059 [hep-ph]].
  * (29) A. H. Chamseddine, R. L. Arnowitt and P. Nath, Phys. Rev. Lett. 49 (1982) 970.
  * (30) V. D. Barger, M. S. Berger and P. Ohmann, Phys. Rev. D 47, 1093-1113 (1993); [arXiv:hep-ph/9209232 [hep-ph]].
  * (31) G. L. Kane, C. F. Kolda, L. Roszkowski and J. D. Wells, Phys. Rev. D 49, 6173-6210 (1994) [arXiv:hep-ph/9312272 [hep-ph]].
  * (32) J. Hisano, [arXiv:hep-ph/0004266 [hep-ph]].
  * (33) B. Dutta, Y. Gao, T. Ghosh, I. Gogoladze, T. Li, Q. Shafi and J. W. Walker, [arXiv:1601.00866 [hep-ph]].
  * (34) K. Abe et al. [Super-Kamiokande], Phys. Rev. D 95, no.1, 012004 (2017) [arXiv:1610.03597 [hep-ex]].
  * (35) K. Abe et al. [Hyper-Kamiokande], [arXiv:1805.04163 [physics.ins-det]].
  * (36) J. Hisano, T. Kuwahara and N. Nagata, Phys. Lett. B 723, 324-329 (2013) [arXiv:1304.0343 [hep-ph]].
  * (37) S. A. R. Ellis and J. D. Wells, Phys. Rev. D 91, no.7, 075016 (2015) [arXiv:1502.01362 [hep-ph]].
  * (38) S. Pokorski, K. Rolbiecki, G. G. Ross and K. Sakurai, JHEP 04, 161 (2019) [arXiv:1902.06093 [hep-ph]].
  * (39) K. Tobe and J. D. Wells, Phys. Lett. B 588, 99-104 (2004) [arXiv:hep-ph/0312159 [hep-ph]].
  * (40) T. Goto and T. Nihei, Phys. Rev. D 59, 115009 (1999) [arXiv:hep-ph/9808255 [hep-ph]].
  * (41) H. Murayama and A. Pierce, Phys. Rev. D 65, 055009 (2002) [arXiv:hep-ph/0108104 [hep-ph]].
  * (42) M. Liu and P. Nath, Phys. Rev. D 87, no.9, 095012 (2013) [arXiv:1303.7472 [hep-ph]].
  * (43) J. Hisano, D. Kobayashi, T. Kuwahara and N. Nagata, JHEP 07, 038 (2013) [arXiv:1304.3651 [hep-ph]].
  * (44) K. S. Babu, I. Gogoladze and C. S. Un, [arXiv:2012.14411 [hep-ph]].
  * (45) T. Cohen, T. Golling, M. Hance, A. Henrichs, K. Howe, J. Loyal, S. Padhi and J. G. Wacker, JHEP 04, 117 (2014). [arXiv:1311.6480 [hep-ph]].
  * (46) H. Baer and X. Tata, Cambridge, UK: Univ. Pr. (2006) 537 p.
  * (47) N. Arkani-Hamed, T. Han, M. Mangano and L. T. Wang, Phys. Rept. 652, 1-49 (2016) [arXiv:1511.06495 [hep-ph]].
  * (48) J. Fan, P. Jaiswal and S. C. Leung, Phys. Rev. D 96, no.3, 036017 (2017). [arXiv:1704.03014 [hep-ph]].
  * (49) T. Golling et al., CERN Yellow Rep., no.3, 441-634 (2017) [arXiv:1606.00947 [hep-ph]].
  * (50) S. Pokorski, K. Rolbiecki and K. Sakurai, Phys. Rev. D 97, no.3, 035027 (2018) [arXiv:1707.06720 [hep-ph]].
  * (51) H. Baer, F. E. Paige, S. D. Protopopescu and X. Tata, arXiv:hep-ph/0001086.
  * (52) J. Hisano, H. Murayama, and T. Yanagida, Nucl. Phys. B402 (1993) 46. Y. Yamada, Z. Phys. C60 (1993) 83; J. L. Chkareuli and I. G. Gogoladze, Phys. Rev. D 58, 055011 (1998).
  * (53) D. M. Pierce, J. A. Bagger, K. T. Matchev and R. j. Zhang, Nucl. Phys. B 491 (1997) 3 [hep-ph/9606211].
  * (54) [Tevatron Electroweak Working Group and CDF Collaboration and D0 Collab], arXiv:0903.2503 [hep-ex].
  * (55) L. E. Ibanez and G. G. Ross, Phys. Lett. B110 (1982) 215; K. Inoue, A. Kakuto, H. Komatsu and S. Takeshita, Prog. Theor. Phys. 68, 927 (1982) [Erratum-ibid. 70, 330 (1983)]; L. E. Ibanez, Phys. Lett. B118 (1982) 73; J. R. Ellis, D. V. Nanopoulos, and K. Tamvakis, Phys. Lett. B121 (1983) 123; L. Alvarez-Gaume, J. Polchinski, and M. B. Wise, Nucl. Phys. B221 (1983) 495.
  * (56) J. Beringer et al. [Particle Data Group Collaboration], Phys. Rev. D 86, 010001 (2012).
  * (57) K. A. Olive et al. [Particle Data Group], Chin. Phys. C 38, 090001 (2014). doi:10.1088/1674-1137/38/9/090001
  * (58) R. Aaij et al. [LHCb Collaboration], Phys. Rev. Lett. 110, no. 2, 021801 (2013) [arXiv:1211.2674 [hep-ex]].
  * (59) Y. Amhis et al. [Heavy Flavor Averaging Group], arXiv:1207.1158 [hep-ex].
  * (60) D. Asner et al. [Heavy Flavor Averaging Group], arXiv:1010.1589 [hep-ex].
  * (61) Y. Akrami et al. [Planck Collaboration], arXiv:1807.06205 [astro-ph.CO].
  * (62) ATL-PHYS-PUB-2021-007