2023

[1]Michel Hayoz 1]ARTORG Center, University of Bern, Switzerland 2]Dept. of
Visceral Surgery and Medicine, Inselspital, Switzerland 3] Applied Research,
Intuitive Surgical, USA

# Learning How To Robustly Estimate Camera Pose in Endoscopic Videos

<EMAIL_ADDRESS>Christopher Hahne  Mathias Gallardo  Daniel Candinas
Thomas Kurmann  Maximilian Allan  Raphael Sznitman  [  [  [

###### Abstract

Purpose: Surgical scene understanding plays a critical role in the technology
stack of tomorrow’s intervention-assisting systems in endoscopic surgeries.
For this, tracking the endoscope pose is a key component, but remains
challenging due to illumination conditions, deforming tissues and the
breathing motion of organs. Method: We propose a solution for stereo
endoscopes that estimates depth and optical flow to minimize two geometric
losses for camera pose estimation. Most importantly, we introduce two learned
adaptive per-pixel weight mappings that balance contributions according to the
input image content. To do so, we train a Deep Declarative Network to take
advantage of the expressiveness of deep-learning and the robustness of a novel
geometric-based optimization approach. We validate our approach on the
publicly available SCARED dataset and introduce a new in-vivo dataset,
StereoMIS, which includes a wider spectrum of typically observed surgical
settings. Results: Our method outperforms state-of-the-art methods on average
and more importantly, in difficult scenarios where tissue deformations and
breathing motion are visible. We observed that our proposed weight mappings
attenuate the contribution of pixels on ambiguous regions of the images, such
as deforming tissues. Conclusion: We demonstrate the effectiveness of our
solution to robustly estimate the camera pose in challenging endoscopic
surgical scenes. Our contributions can be used to improve related tasks like
simultaneous localization and mapping (SLAM) or 3D reconstruction, therefore
advancing surgical scene understanding in minimally-invasive surgery.

###### keywords:

Camera pose estimation, Endoscopic surgery, Deep Declarative Network

##  1 Introduction

Camera pose estimation is a well established computer vision problem at the
core of numerous applications of medical robotic systems for minimally
invasive surgery (MIS). With a variety of methods proposed in recent years,
most approaches have focused on Simultaneous Localization and Mapping (SLAM)
and Visual Odometry (VO) frameworks to solve the pose estimation problem. Well
established techniques such as ORB-SLAM2 Mur-Artal2017  and ElasticFusion
Whelan2016  have shown great promise in rigid scenes. More recently, non-rigid
cases in MIS using monocular Lamarca2019 ; Gomez-Rodriguez2021 ; Liu2022  and
stereoscopic cameras Song2018 ; Zhou2021 ; Ruofeng2022  have also been
studied. Yet to this day, pose estimation in typical MIS settings remains
particularly difficult due to deformations caused by instruments and
breathing, self or instrument-based occlusions, textureless surfaces and
tissue specularities.

In this work, we tackle the problem of pose estimation in such difficult cases
when using a stereo endoscopic camera system.

This allows depth estimation to be computed from parallax, which has been
shown to improve robustness of SLAM methods Mur-Artal2017 . In contrast to
Song2018 ; Zhou2021  which assume the tissue is smooth and locally rigid,
respectively, we avoid making assumptions on the tissue deformation and
topology. Instead, we propose a dense stereo VO framework that handles tissue
deformations and the complexity of surgical scenes. To do this, our approach
leverages geometric pose optimization by infering where to look at in the
scene. At its core, our method uses a Deep Declarative Network (DDN) Gould2021
to enable back-propagation of gradients through the pose optimization.

More specifically, we propose to integrate two adaptive weight maps with the
role of balancing the contribution of two geometrical losses and the
contribution of each pixel towards each of these losses. We learn these
adaptive weight maps using a DDN with the goal of solving the pose estimation
problem, inspired by the recent DiffPoseNet Parameshwara  approach. Similarly
to theirs, our method exploits the expressiveness of neural networks while
leveraging robustness from the geometric optimization approach. This allows
our method to adapt the contribution of the image region depending on the
image content, for each loss but also between the two losses. We thoroughly
evaluate our approach by characterizing its performance in comparison to the
state-of-the-art on various practical scenarios: rigid scenes, breathing,
scanning and deforming sequences. This validation is performed on two
different datasets and we show that our method allows for more precise pose
estimation in a wide range of cases.

##  2 Method

In the following, we present our depth-based pose estimation approach from an
optimization perspective. We first derive our method in terms of context-
specific adaptive weight maps in the pose estimation optimization and then
show how to learn theset from data in an end-to-end way using DDNs to
facilitate differentiation Gould2021 . Our proposed method is illustrated in
Fig. 1 and we provide a notation overview in Tab. 1.

Figure 1: Overview of our proposed VO framework, which can be sub-divided into
3 parts (from left to right): (1) optical flow and depth are computed using
RAFT Teed2021 , (2) weight maps computed and used in (3) to estimate the
camera pose $\mathbf{p}^{\star}_{t}$. Weight maps
$(\omega_{\text{2D}}(\mathbf{x})$, $\omega_{\text{3D}}(\mathbf{x}))$ are
learned via back propagation using the Deep Declarative Network (DDN)
Gould2021 .

For all images in an image sequence, we first employ RAFT Teed2021  to
establish correspondences between frames for both the stereo and temporal
domain. This model allows disparity and optical flow estimation to be computed
simultaneously, based on the observation that both share similar constraints
on relative pixel displacements. From these estimates, we extract the
horizontal component of the parallax flow $\mathcal{F}_{t}^{\prime}$ at time
$t$ as the stereo disparity to compute depth maps $\mathcal{D}_{t}$. As we
would typically expect large vertical disparities in areas of low-texture or
for de-calibrated stereo endoscopes, we use this parallax flow
$\mathcal{F}_{t}^{\prime}$ as input for the weight map estimation described in
the following.

###  2.1 Pose estimation

In contrast to most existing VO methods, we estimate the camera motion
exclusively based on a geometric loss function given that photometric
consistency is entailed in optical flow estimation. We thus compute a 2D
residual function based on a single depth map by,

$\displaystyle
r_{\text{2D}}(\mathbf{p}_{t},\mathbf{x})=\sqrt{\frac{1}{XY}}\left\lVert\pi_{\text{2D}}\Big{(}\exp(\mathbf{p}_{t})\,\pi_{\text{3D}}\big{(}\mathcal{D}_{t},\mathbf{x}\big{)}\Big{)}-\big{(}\mathbf{x}+\mathcal{F}_{t}(\mathbf{x})\big{)}\right\rVert_{2}\,\,,$
(1)

where
$\pi_{\text{2D}}(\exp(\mathbf{p}_{t})\,\pi_{\text{3D}}(\mathcal{D}_{t},\mathbf{x}))$
is the pixel location based on depth projection and the relative camera pose
$p_{t}$ that aligns views $\mathcal{I}^{(l)}_{t}$ to
$\mathcal{I}^{(l)}_{t-1}$. We scale these residuals by the image dimensions
$X$ and $Y$ to make values independent of the image size. Note that we
normalize depth maps by the maximum expected depth value, such that rotation
and translation components of $\mathbf{p}_{t}$ have the same order of
magnitude and thus equally contribute to the residuals, which is important for
a well-conditioned optimization. Ideally, a projected pixel position coincides
with the optical flow when the observed scene is rigid and when flow and depth
maps are correct.

Table 1: Summary of used notation. Symbol | Description
---|---
$t\in\mathbb{Z}$ | time frame index
$\mathcal{I}^{(l)}_{t}\in\mathbb{R}^{X\times Y\times 3}$ | rectified left stereo image at time $t$
$\mathcal{D}_{t}\in\mathbb{R}^{X\times Y}$ | depth map w.r.t. to left image at time $t$
$\mathcal{F}_{t}\in\mathbb{R}^{X\times Y}$ | optical flow from $\mathcal{I}^{(l)}_{t}$ to $\mathcal{I}^{(l)}_{t-1}$
$\mathcal{F}^{\prime}_{t}\in\mathbb{R}^{X\times Y}$ | parallax flow displacement across stereo images
$\mathbf{x}\in\mathbb{Z}^{2}$ | pixel index in 2D Cartesian coordinate system
$\mathbf{p}_{t}\in\mathfrak{se}(3)\subset\mathbb{R}^{6}$ | relative pose from $t$ to $t-1$ in Lie algebra space
$\mathbf{p}_{t}^{\star}\in\mathfrak{se}(3)\subset\mathbb{R}^{6}$ | relative pose solution in Lie algebra space
$\exp(\mathbf{p}):\mathfrak{se}(3)\rightarrow\text{SE}(3)$ | matrix exponential from Lie algebra to Lie group
$\pi_{\text{2D}}(\mathbf{v}):\mathbb{R}^{4}\rightarrow\mathbb{R}^{2}$ | projection of homogeneous 3D to 2D coordinates
$\pi_{\text{3D}}(\mathcal{D}_{t},\mathbf{x}):\mathbb{R}^{X\times Y}\times\mathbb{Z}^{2}\rightarrow\mathbb{R}^{4}$ | re-projection of 2D to 3D homogeneous coordinates
$\mathcal{F}_{t}(\mathbf{x}):\mathbb{Z}^{2}\rightarrow\mathbb{R}^{2}$ | optical flow function across temporal domain
$\omega_{\text{2D}}(\mathbf{x}):\mathbb{Z}^{2}\rightarrow\left[0,1\right]$ | learned per-pixel weight for 2D residuals
$\omega_{\text{3D}}(\mathbf{x}):\mathbb{Z}^{2}\rightarrow\left[0,1\right]$ | learned per-pixel weight for 3D residuals
$\lVert\cdot\rVert_{n}:\mathbb{R}^{m}\rightarrow\mathbb{R}^{+}$ |  $\ell^{n}$ vector norm
|

While minimizing $r_{\text{2D}}(\mathbf{p}_{t},\mathbf{x})$ helps to reliably
estimate the camera motion in rigid scenes, detection of deformations is most
effective by looking at the displacement of points in 3D space. To address
this need, we propose to leverage the depth map at $t-1$ and introduce a 3D
residual function by,

$\displaystyle
r_{\text{3D}}(\mathbf{p}_{t},\mathbf{x})=\Big{\lVert}\exp(\mathbf{p}_{t})\,\pi_{\text{3D}}\big{(}\mathcal{D}_{t},\mathbf{x}\big{)}-\pi_{\text{3D}}\big{(}\mathcal{D}_{t-1},\mathbf{x}+\mathcal{F}_{t}(\mathbf{x})\big{)}\Big{\rVert}_{2}\,\,,$
(2)

which measures the point-to-point alignment of the re-projected depth maps. As
opposed to Whelan2016 , we avoid using a point-to-plane distance as it is less
constrained on planar surfaces such as organs (e.g., liver). While a known
weakness of the point-to-point distance is its sensitivity to noise in regions
with large perspectives, we mitigate this effect by combining 2D and 3D
residuals. Intuitively, we expect $r_{\text{2D}}(\mathbf{p}_{t},\mathbf{x})$
to be most accurate when camera motion is large and
$r_{\text{3D}}(\mathbf{p}_{t},\mathbf{x})$ when deformations are significant.
Similar to Teed2021 , we use bilinear sampling to warp point clouds from
$\pi_{\text{3D}}\big{(}\mathcal{D}_{t-1},\mathbf{x}\big{)}$ to
$\pi_{\text{3D}}\big{(}\mathcal{D}_{t-1},\mathbf{x}+\mathcal{F}_{t}(\mathbf{x})\big{)}$,
using our optical flow estimates.

In contrast to conventional scalar-weighted sum of residuals, we propose to
weigh each residual using a dedicated weight map that is inferred from the
image data. The final residual is computed as,

$\displaystyle
r(\mathbf{p}_{t},\mathbf{x})=\omega_{\text{2D}}(\mathbf{x})\,r_{\text{2D}}(\mathbf{p}_{t},\mathbf{x})+\omega_{\text{3D}}(\mathbf{x})\,r_{\text{3D}}(\mathbf{p}_{t},\mathbf{x})\,\,,$
(3)

where $\omega_{\text{2D}}(\mathbf{x})$ and $\omega_{\text{3D}}(\mathbf{x})$
are the per-pixel weight maps for the 2D and 3D residuals, respectively.

At its core, our hypothesis is that we can learn how the weight maps should
combine the contributions of both $\omega_{\text{2D}}(\mathbf{x})$ and
$\omega_{\text{3D}}(\mathbf{x})$ even in challenging situations where tissue
deformations are taking place. That is, the role of
($\omega_{\text{2D}}(\mathbf{x})$, $\omega_{\text{3D}}(\mathbf{x})$) is to (1)
weigh relative focus based on the context of tissue deformations, (2) weigh
reliable residual functions (2D vs 3D) given a motion pattern and (3) balance
the scale of the loss. In Sec. 2.2, we detail how we learn a model to infer
these weight maps.

Optimization: To compute the relative pose
$\mathbf{p}_{t}^{\star}\in\mathfrak{se}(3)$, we then optimize,

$\displaystyle\mathbf{p}_{t}^{\star}=\underset{\mathbf{p}_{t}}{\operatorname{arg\,min}}\,\left\\{\sum_{\mathbf{x}\in\boldsymbol{\Omega}}r(\mathbf{p}_{t},\mathbf{x})^{2}\right\\},$
(4)

in a Non-Linear Least-Squares (NLLS) problem. Here, $\boldsymbol{\Omega}$ is a
set containing all spatial image coordinates $\mathbf{x}$. We choose to
optimize the pose in the Lie algebra vector space $\mathfrak{se}(3)$ because
this is a unique representation of the pose and has the same number of
parameters as degrees of freedom. NLLS problems are typically solved
iteratively using a second-order optimizer. To do this, we use the quasi-
Newton method L-BFGS LBGFS  due to its fast convergence properties and
computational efficiency. Identical to Parameshwara , we simply chain relative
camera poses to obtain the full trajectory.

###  2.2 Learning the weight maps

In Eq. (3), we propose to learn residual weight maps
$\omega_{\text{2D}}(\mathbf{x})$ and $\omega_{\text{3D}}(\mathbf{x})$, as
determining these otherwise is not trivial. To this end, we train a separate
encoder-decoder network, denoted by $g(\cdot)$, for each weight map. The input
to these networks are all the elements used to compute residuals,

$\displaystyle\omega_{\text{2D}}(\mathbf{x})=$ $\displaystyle
g\big{(}\mathbf{x},\mathcal{I}^{(l)}_{t},\mathcal{D}_{t},\mathcal{F}_{t},\mathcal{F}^{\prime}_{t},\boldsymbol{\theta}_{\text{2D}}\big{)},$
(5) $\displaystyle\omega_{\text{3D}}(\mathbf{x})=$ $\displaystyle
g\big{(}\mathbf{x},\mathcal{I}^{(l)}_{t},\mathcal{D}_{t},\mathcal{F}_{t},\mathcal{F}^{\prime}_{t},\mathcal{I}^{(l)}_{t-1},\mathcal{D}_{t-1},\mathcal{F}^{\prime}_{t-1},\boldsymbol{\theta}_{\text{3D}}\big{)}\,,$
(6)

where $\boldsymbol{\theta}_{\text{2D}}$ and $\boldsymbol{\theta}_{\text{3D}}$
are the network parameters that are learned at training time. For $g(\cdot)$,
we employ a 3-layer UNet unet_2015  with Sigmoid activation function to ensure
outputs in $[0,1]$.

To train $g(\cdot)$, we aim to learn weight-maps that lead to improved pose
estimation by minimizing the $\ell^{1}$ supervised training loss,

$\displaystyle\mathcal{L}_{\text{train}}=\lVert\mathbf{p}^{\star}_{t}-\mathbf{p}^{(\text{gt})}_{t}\rVert_{1},$
(7)

where $\mathbf{p}^{(\text{gt})}_{t}$ is the ground-truth pose. Because the
pose optimization in Eq. (4) is not directly differentiable, we leverage a DDN
Gould2021  to enable end-to-end learning. This approach uses implicit
differentiation of Eq. (4) to compute the gradients of the weight map
parameters $(\boldsymbol{\theta}_{\text{2D}}$,
$\boldsymbol{\theta}_{\text{3D}})$ with respect to
$\mathcal{L}_{\text{train}}$. Therefore, the only requirements are that (1)
the function to be optimized
$\sum_{\mathbf{x}\in\boldsymbol{\Omega}}r(\mathbf{p}_{t},\mathbf{x})^{2}$ is
twice differentiable and (2) we find a local or global minimum in the forward
pass.

##  3 Experiments

###  3.1 Datasets

We evaluate our method on two separate stereo video datasets: one containing
rigid MIS scenes and another containing non-rigid scenes:

SCARED dataset Allan2021 : consists of 9 in-vivo porcine subjects with 4
sequences for each subject. The dataset contains a video stream captured using
a da Vinci Xi surgical robot and camera forward kinematics. All sequences show
rigid scenes without breathing motion or surgical instruments. We split the
dataset into training (d2, d3, d6, d7) and testing sequences (d1, d8, and d9)
where we exclude d4 and d5 due to bad camera calibrations.

StereoMIS: Additionally, we introduce a new in-vivo dataset also recorded
using a da Vinci Xi surgical robot. Similarly to Allan2021 , ground truth
camera poses are generated from the endoscope forward kinematics and
synchronized with the video feed. While we expect errors in the absolute
camera pose due to accumulated errors in the forward kinematics, relative
camera motions are expected to be accurate. It consists of 3 porcine (P1, P2,
and P3) and 3 human subjects (H1, H2, and H3) with a total of 16 recorded
sequences. We denote sequences with Px_y where Px is the subject and y the
sequence number. Sequence duration’s range from 50 seconds to 30 minutes. They
contain challenging scenes with breathing motions, tissue deformations,
resections, bleeding and presence of smoke. We assign P1 and H1 to the
training set and the rest is kept for testing.

To provide a finer grained performance characterization of methods with this
data, we extract from each video a number of short sequences that visually
depict one of several possible settings:

  * •

breathing: only depicts breathing deformations and contains no camera or tool
motion,

  * •

scanning: includes camera motion in addition to breathing deformations,

  * •

deforming: comprises tissue deformations due breathing and manipulation or
resection of tissue while the camera is static.

In practice, we select 88 different, non-overlapping, and 150-frames-long
sequences from P2, P3, H2, H3 and assigned each to one of the above categories
or surgical scenarios (see supplementary material for more information).

###  3.2 Implementation details

####  3.2.1 Segmentation of surgical instruments

For all experiments, we mask out surgical instrument pixels by setting
corresponding residuals to 0. To do this, we use the DeepLabv3+ architecture
Chen_2018_ECCV  trained on the EndoVis2018 segmentation dataset Allan2019  to
generate instrument masks for each frame. Additionally, we mask out
specularities, by means of maximum intensity detection, as they cause optical
flow estimations to be ill-defined.

####  3.2.2 Training and inference

First, we classify all training frames from the SCARED and StereoMIS training
sequences into ”moving” and ”static” based on the camera forward kinematics.
We then randomly sample 4’000 frames from each sequence keeping a balance
between moving and static frames. For each sampled frame, we generate a sample
pair with an interval of 1 to 5 frames. We use the forward kinematics of the
camera as the ground-truth pose change between two frames in a sample pair.
Note that forward kinematics entail minor deviations that may propagate during
training. We randomly assign $80\%$ of the sample pairs to the training set
and $20\%$ to the validation set.

For all experiments, we resize images to half resolution (512x640 pixels). We
use a batch size of 8 and the Adam optimizer with learning rate $10^{-5}$. We
train for 200 epochs and perform early stopping on the validation loss. We
implement our method using PyTorch, and train on a NVIDIA RTX3090 GPU. We
reach 6.5 frames per second at test time. RAFT is trained on the
FlyingThings3D dataset and we do not perform any fine-tuning.

###  3.3 Metrics and baseline methods

We use trajectory error metrics as defined in ozyoruk2020 , namely the
absolute trajectory error ATE-RMSE to evaluate the overall shape of the
trajectory and the relative pose errors, RPE-trans and RPE-rot, to evaluate
relative pose changes from frame to frame. The ATE-RMSE is sensitive to drift
and depends on the length of the sequence, whereas the RPE measures the
average performance for frame-to-frame pose estimation.

As no stereo SLAM method dedicated for MIS has open source code or is
evaluated on a public dataset with trajectory ground-truth, we compare our
method to two general state-of-the art rigid SLAM methods that contain loop
closure and are based on the rigid scene assumption:

  * •

ORB-SLAM2 Mur-Artal2017 , a sparse SLAM leveraging bundle adjustment to
compensate drift,

  * •

ElasticFusion Whelan2016 , a dense SLAM and as such closer to our proposed
method.

In addition, we compare, our method to Ruofeng2022  on the frames of the
SCARED dataset for which they reported performances. For fair comparison, we
use the same input depth maps for all methods.

##  4 Results

Surgical scenarios and ablation study: Tab. 2 reports the performance of our
approach on the StereoMIS surgical scenarios. To show the importance of
learning the weight maps we perform an ablation study where we evaluate the
impact of (1) constant weights, denoted by ours (w/o weight), where
$\omega_{\text{2D}}(\mathbf{x})=\omega_{\text{3D}}(\mathbf{x})=1$ for each
$\mathbf{x}$; (2) our method with only 2D-residuals, denoted by ours (only
2D); and (3) using only 3D-residuals, denoted by ours (only 3D).

Table 2: The ATE-RMSE (mean$\pm$std mm) for the different scenarios from the StereoMIS dataset with average over sequences (micro avg.) and scenarios (macro avg.). scenario | breathing | scanning | deforming | micro avg. | macro avg.
---|---|---|---|---|---
# sequences | 17 | 60 | 9 |  |
camera motion |  | ✓ |  |  |
breathing | ✓ | ✓ | ✓ |  |
tool interactions |  |  | ✓ |  |
ORB-SLAM2 Mur-Artal2017  | $2.35\pm 1.81$ | $3.26\pm 1.65$ | $4.29\pm 2.30$ | $3.19\pm 1.81$ | $3.30\pm 0.97$
ElasticFusion Whelan2016  | $1.94\pm 0.93$ | $4.04\pm 3.46$ | $6.47\pm 8.64$ | $3.88\pm 4.12$ | $4.15\pm 2.27$
ours (w/o weight) | $1.65\pm 0.97$ | $3.01\pm 1.60$ | $4.67\pm 2.13$ | $2.91\pm 1.74$ | $3.11\pm 1.51$
ours (only 2D) | $1.15\pm 0.72$ | $3.01\pm 1.66$ | $2.83\pm 1.41$ | $2.62\pm 1.66$ | $2.33\pm 1.03$
ours (only 3D) | $\mathbf{0.78\pm 2.03}$ | $7.02\pm 5.86$ | $2.72\pm 1.90$ | $5.34\pm 5.64$ | $3.51\pm 3.20$
ours (2D & 3D) | $1.01\pm 0.59$ | $\mathbf{2.89\pm 2.33}$ | $\mathbf{2.23\pm 1.07}$ | $\mathbf{2.45\pm 2.12}$ | $\mathbf{2.04\pm 0.95}$

Our proposed method outperforms the baselines in all scenarios. Improvements
in breathing and scanning are partly due to correct identification of errors
in the optical flow and depth estimation as well as optimal balancing of the
2D and 3D residuals. Indeed, exploiting the complementary properties of 2D and
3D residuals improves the average performance. The fact, that ours (only 3D)
outperforms ours (only 2D) in breathing and deforming supports our intuition
that it is easier to learn tissue deformations from the 3D residuals.
Contrarily, in scanning where the optical flow is dominated by the camera
motion, the 2D residuals lead to a more accurate pose estimation.

In general, it is not possible to detect or completely compensate the
breathing motion on a frame-to-frame basis in our proposed optimization scheme
as we cannot completely disambiguate the camera and tissue motion. However,
the method learns which regions are more affected by breathing deformations
and consequently assigns a smaller weight to those regions.

We note that the weight maps in Fig. 2 (see breathing rows) support our claims
that the weight maps have low values in the dark regions (A) where we expect
the optical flow to be inaccurate and where tissue is moving most (B). The
scanning example also illustrates that the weight maps have a different
response depending on the motion pattern and deformation. Note that the
presence of surgical instruments has no influence on the weight maps in
scanning, as no tissue interaction takes place. As expected, the largest
improvements can be seen in the deforming scenario. Inspecting the two last
rows in Fig. 2 reveals that regions where the instruments deforms tissue (C)
are correctly ignored in the pose estimation. Similarly, the region occluded
by smoke (D) has low values in the weight maps. Additionally, we observe that
$\omega_{\text{2D}}$ usually has 100 times larger magnitude than
$\omega_{\text{3D}}$ compensating for the different scale of the 2D and 3D
residuals.

Figure 2: Exemplary results for 5 different scenarios. Surgical instruments
and specularities are masked out. From left to right: left image, its depth
map, its optical flow displacement as well as its weights
$\omega_{\text{2D}}(\mathbf{x})$ and $\omega_{\text{3D}}(\mathbf{x})$. Weight
maps are normalized (lowest value in dark blue and highest value in yellow).
Best viewed in color.

Results on full test StereoMIS sequences:  Tab. 3 shows the pose estimation
performance on the complete sequences in the StereoMIS testset. As the
sequences are much longer than in the scenario experiment, accumulation of
drift results in a large ATE-RMSE for all methods. Even though our frame-to-
frame approach does not include any bundle-adjustment or regularization over
time, it still has the lowest ATE-RMSE on average. The reason for this good
performance is reflected in the relative metrics RPE-trans and RPE-rot, where
our method outperforms all others by almost a factor of three and five,
respectively. Our method robustly estimates the pose in challenging situations
whereas ORB-SLAM2 fails in two sequences (H2_0, P2_5). Fig. 3 shows two
example trajectories. P2_7 does not include any tool-tissue interactions and
consists of smooth camera motions. Its trajectory illustrates the drift of our
method which results in an ATE-RMSE of $9.28$ versus $3.76$mm for ORB-SLAM2.
On the other hand, P3_0 consists of strong tissue deformations and abrupt
camera movements. Despite visible drift, we can see that our method is able to
follow the abrupt movements. The small scale oscillations in the trajectories
are due to breathing motion. The trajectories of all test sequences and
evaluation results excluding frames where the SLAM methods fail can be found
in the supplementary materials.

Table 3: Pose estimation results on full StereoMIS test sequences for ORB-SLAM2 Mur-Artal2017 , ElasticFusion Whelan2016 , and ours. Metrics are reported in (mean$\pm$std) when applicable. | H2 | H3 | P2 | P3 | macro avg.
---|---|---|---|---|---
ATE-RMSE (mm)
ORB-SLAM2 Mur-Artal2017  | $18.0$ | $\mathbf{9.1}$ | $14.0$ | $21.4$ | $15.6\pm 5.3$
ElasticFusion Whelan2016  | $30.8$ | $72.1$ | $33.6$ | $37.7$ | $43.6\pm 19.3$
ours | $\mathbf{10.9}$ | $21.2$ | $\mathbf{13.8}$ | $\mathbf{8.8}$ | $\mathbf{13.7\pm 5.4}$
RPE-trans (mm)
ORB-SLAM2 Mur-Artal2017  | $0.20\pm 0.43$ | $0.24\pm 0.25$ | $0.35\pm 0.46$ | $0.54\pm 0.47$ | $0.33\pm 0.13$
ElasticFusion Whelan2016  | $0.87\pm 1.11$ | $0.56\pm 1.03$ | $0.81\pm 1.11$ | $0.71\pm 0.79$ | $0.74\pm 0.12$
ours | $\mathbf{0.10\pm 0.27}$ | $\mathbf{0.10\pm 0.18}$ | $\mathbf{0.16\pm 0.32}$ | $\mathbf{0.19\pm 0.31}$ | $\mathbf{0.14\pm 0.04}$
RPE-rot (deg)
ORB-SLAM2 Mur-Artal2017  | $0.16\pm 0.36$ | $0.16\pm 0.22$ | $0.19\pm 0.24$ | $0.28\pm 0.27$ | $0.20\pm 0.05$
ElasticFusion Whelan2016  | $0.73\pm 1.06$ | $0.41\pm 0.96$ | $0.50\pm 1.11$ | $0.38\pm 0.40$ | $0.51\pm 0.14$
ours | $\mathbf{0.04\pm 0.20}$ | $\mathbf{0.04\pm 0.13}$ | $\mathbf{0.07\pm 0.14}$ | $\mathbf{0.05\pm 0.10}$ | $\mathbf{0.05\pm 0.01}$
|  |  |  |  |

Figure 3: Two example trajectories of the StereoMIS test set. Top:  trajectory
of P2_7. Bottom:  trajectory of P3_0.

Results on SCARED dataset:  Wei et al. reported ATE-RMSE results for rigid
surgical scenes of the SCARED dataset in a frame-to-model approach Ruofeng2022
. For the sake of fair comparison, we extend our method to SLAM by
accommodating a surfel map model denoted by ours (frame2model), which is
equivalent to that used in Ruofeng2022 . Specifically, we replace input images
$\mathcal{I}^{(l)}_{t-1},\mathcal{I}^{(r)}_{t-1}$ by rendered images from the
surfel map. Note, we can only adopt this approach for the SCARED dataset, as
the surfel map model assumes scene rigidity. Results are provided in Tab. 4.

Table 4: The ATE-RMSE in mm for SCARED sequences reported by Ruofeng2022  and micro average over all SCARED test sequences (SCARED avg.) using surfel maps.  | d1_k2 | d8_k1 | d9_k1 | d9_k3 | avg. | SCARED avg.
---|---|---|---|---|---|---
ORB-SLAM2 Mur-Artal2017  | $0.91$ | $2.97$ | $4.33$ | $3.79$ | $3.00$ | $2.34\pm 1.24$
ElasticFusion Whelan2016  | $1.02$ | $3.62$ | $4.30$ | $3.36$ | $3.08$ | $2.91\pm 1.77$
Wei et al. Ruofeng2022  | $0.74$ | $2.47$ | $4.07$ | $1.54$ | $2.21$ | -
ours (frame2model) | $\mathbf{0.37}$ | $\mathbf{2.08}$ | $\mathbf{2.04}$ | $\mathbf{0.84}$ | $\mathbf{1.33}$ | $\mathbf{1.38\pm 0.93}$
|  |  |  |  |  |

##  5 Conclusion

We proposed a visual odometry method for robust pose estimation in the
challenging context of endoscopic surgeries. To do so, we learn adaptive
weight maps for two geometrical residuals to leverage pose estimation
performance on common surgical scenes including breathing motion and tissue
deformations. Thanks to a performance analysis in common scenarios, we
observed the complementary action of the 2D/3D residuals and the strong
contribution of their specific weighting at pixel level. This results in
better performances compared to state-of-the-art methods, on average and in
the most challenging cases. We believe that our contributions are beneficial
for some SLAM components, e.g. map building, and therefore downstream
applications in MIS. Future work will focus on drift and breathing
compensation.

Supplementary information Appendix A: details on StereoMIS. Appendix B:
trajectories of StereoMIS test set. Appendix C: Results on full StereoMIS
sequences. Appendix D: trajectories of SCARED test set.

## Declarations

  * •

Funding: This work was supported by InnoSuisse grant # 50204.1 IP-LS.

  * •

Competing interests: Authors declare that they have no conflict of interest.

  * •

Ethics approval: All applicable international, national, and institutional
guidelines for the care and use of animals were followed. All procedures
performed in studies involving animals were in accordance with the ethical
standards of the institution at which the studies were conducted.

  * •

Consent: N/A

  * •

Availability of data, materials, and code:
StereoMIS porcine data - https://doi.org/10.5281/zenodo.7727692.
Code and models - https://github.com/aimi-lab/robust-pose-estimator

## References

  *   * (1) Mur-Artal, R., Tardos, J.D.: ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras. T-RO 2016. https://doi.org/10.1109/TRO.2017.2705103
  * (2) Whelan, T., Salas-Moreno, R.F., Glocker, B., Davison, A.J., Leutenegger, S.: Elasticfusion: Real-time dense slam and light source estimation. IJRR 2016. https://doi.org/10.1177/0278364916669237
  * (3) Lamarca, J., Montiel, J.M.M.: Camera tracking for slam in deformable maps. In: ECCV 2018. https://doi.org/10.1007/978-3-030-11009-3_45
  * (4) Gomez-Rodriguez, J.J., Lamarca, J., Morlana, J., Tardos, J.D., Montiel, J.M.M.: SD-DefSLAM: Semi-Direct Monocular SLAM for Deformable and Intracorporeal Scenes. In: ICRA 2021. https://doi.org/10.1109/ICRA48506.2021.9561512
  * (5) Liu, X., Li, Z., Ishii, M., Hager, G.D., Taylor, R.H., Unberath, M.: SAGE: SLAM with Appearance and Geometry Prior for Endoscopy. In: ICRA 2022. https://doi.org/10.1109/ICRA46639.2022.9812257
  * (6) Song, J., Wang, J., Zhao, L., Huang, S., Dissanayake, G.: MIS-SLAM: Real-Time Large-Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing. RA-L 2018. https://doi.org/10.1109/LRA.2018.2856519
  * (7) Zhou, H., Jayender, J.: EMDQ-SLAM: Real-Time High-Resolution Reconstruction of Soft Tissue Surface from Stereo Laparoscopy Videos. In: MICCAI 2021. https://doi.org/10.1007/978-3-030-87202-1_32
  * (8) Wei, R., Li, B., Mo, H., Lu, B., Long, Y., Yang, B., Dou, Q., Liu, Y., Sun, D.: Stereo Dense Scene Reconstruction and Accurate Localization for Learning-Based Navigation of Laparoscope in Minimally Invasive Surgery. TBME 2022. https://doi.org/10.1109/TBME.2022.3195027
  * (9) Gould, S., Hartley, R., Campbell, D.: Deep declarative networks. PAMI 2022. https://doi.org/10.1109/TPAMI.2021.3059462
  * (10) Parameshwara, C.M., Hari, G., Fermüller, C., Sanket, N.J., Aloimonos, Y.: Diffposenet: Direct differentiable camera pose estimation. In: CVPR 2022. https://doi.org/10.1109/CVPR52688.2022.00672
  * (11) Teed, Z., Deng, J.: RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. In: ECCV 2020. https://doi.org/10.1007/978-3-030-58536-5_24
  * (12) Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. In: Mathematical Programming 1989. https://doi.org/10.1007/BF01589116
  * (13) Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomedical Image Segmentation. In: MICCAI 2015. https://doi.org/10.1007/978-3-319-24574-4_28
  * (14) Allan, M., McLeod, A.J., Wang, C.C., Rosenthal, J., Hu, Z., Gard, N., Eisert, P., Fu, K.X., Zeffiro, T., Xia, W., Zhu, Z., Luo, H., Jia, F., Zhang, X., Li, X., Sharan, L., Kurmann, T., Schmid, S., Sznitman, R., Psychogyios, D., Azizian, M., Stoyanov, D., Maier-Hein, L., Speidel, S.: Stereo Correspondence and Reconstruction of Endoscopic Data Challenge. CoRR 2021
  * (15) Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. In: ECCV 2018. https://doi.org/10.1007/978-3-030-01234-2_49
  * (16) Allan, M., Kondo, S., Bodenstedt, S., Leger, S., Kadkhodamohammadi, R., Luengo, I., Fuentes, F., Flouty, E., Mohammed, A., Pedersen, M., Kori, A., Alex, V., Krishnamurthi, G., Rauber, D., Mendel, R., Palm, C., Bano, S., Saibro, G., Shih, C., Speidel, S.: 2018 Robotic Scene Segmentation Challenge (2019). https://doi.org/10.13140/RG.2.2.26015.07843
  * (17) Ozyoruk, K.B., Gokceler, G.I., Bobrow, T.L., Coskun, G., Incetan, K., Almalioglu, Y., Mahmood, F., Curto, E., Perdigoto, L., Oliveira, M., Sahin, H., Araujo, H., Alexandrino, H., Durr, N.J., Gilbert, H.B., Turan, M.: Endoslam dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Medical Image Analysis 2021. https://doi.org/10.1016/j.media.2021.102058


# Rigidity results for Riemann solitons

Willian Tokura111https://orcid.org/0000-0001-9363-793X<EMAIL_ADDRESS>Marcelo Barboza222https://orcid.org/0000-0003-4258-4676<EMAIL_ADDRESS>Elismar
Batista333https://orcid.org/0000-0002-4391-4238<EMAIL_ADDRESS>Ilton
Menezes444https://orcid.org/0000-0002-9590-6731<EMAIL_ADDRESS>Universidade Federal do Acre, CCET, 69920-900, Rio Branco - AC, Brazil
Universidade Federal de Goiás, IME, 74001-970, Goiânia - GO, Brazil

###### Abstract

In this paper we take a look at conditions that make a Riemann soliton
trivial, compacity being one of them. We also show that the behaviour at
infinity of the gradient field of a non-compact gradient Riemann soliton might
cause the soliton to be an Einstein manifold. Finally, we obtain scalar
curvature estimates for complete shrinking or steady gradient Riemann solitons
whose scalar curvature is bounded from below at infinity.

###### keywords:

Riemann flow, Riemann solitons, gradient Riemann solitons, rigidity results,
gradient almost Ricci solitons.

###### MSC:

[2010] 53C21, 53C25, 53C26

##  1 Introduction and main results

The Riemann flow, which was introduced by Udriste [1, 2], is the flow
associated with the evolution equation

$\frac{\partial}{\partial t}G(t)=-2Rm_{g(t)},$ (1)

for Riemannian metrics on a given differentiable manifold $M^{n}$, where
$Rm_{g(t)}$ is the Riemann curvature $(0,4)$-tensor of the metric $g(t)$,
$G=\frac{1}{2}g\odot g$ and $\odot$ is the Kulkarni-Nomizu product. Recall
that, for symmetric $(0,2)$-tensors $A$ and $B$, the Kukarni-Nomizu product
$A\odot B$ is defined by (see [3]):

$\displaystyle(A\odot B)(X_{1},X_{2},X_{3},X_{4})$
$\displaystyle=A(X_{1},X_{3})B(X_{2},X_{4})+A(X_{2},X_{4})B(X_{1},X_{3})$
$\displaystyle\qquad-A(X_{1},X_{4})B(X_{2},X_{3})-A(X_{2},X_{3})B(X_{1},X_{4}).$

Short-time existence and uniqueness of the flow (1) is guaranteed by Hirica
and Udriste on compact manifolds in [4], where the authors utilize the Ricci
flow to construct short-time solutions of (1).

Just like it happens with Ricci solitons, critical metrics for the Riemann
flow are self similar solutions of (1), that is, it evolves overs time from a
given Riemannian metric on the manifold by diffeomorphisms and dilatations
[5].

###### Definition 1

A Riemannian manifold $(M^{n},g)$, $n\geqslant 3$, is a Riemann soliton if
there exist a vector field $X\in\mathfrak{X}(M)$ and a real number
$\lambda\in\mathbb{R}$ satisfying:

$Rm+\frac{1}{2}\mathcal{L}_{X}g\odot g=\lambda G,$ (2)

where $\mathcal{L}_{X}g$ is the Lie derivative of $g$ in the direction of $X$.
We write the soliton in (2) as $(M^{n},g,X,\lambda)$ for the sake of
simplicity. It may happen that $X=\nabla f$ for some $f\in C^{\infty}(M)$, in
which case we say that $(M^{n},g,\nabla f,\lambda)$ is a gradient Riemann
soliton. Notice that equation (2) for gradient Riemann solitons reads:

$Rm+\nabla^{2}{f}\odot g=\lambda G.$ (3)

A Riemann soliton is classified into one of three types according to the sign
of $\lambda$, expanding if $\lambda<0$, steady if $\lambda=0$ and, finally,
shrinking if $\lambda>0$. Also, a Riemann soliton is called trivial if either
$\mathcal{L}_{X}g=0$ or $\nabla^{2}f=0$.

One might start picturing Riemann solitons by noticing that the trivial ones
satisfy $Rm=\frac{\lambda}{2}g\odot g$, thus having constant sectional
curvature. Also, in [6] Catino and Mastrolia define what they call $X$-space
and $f$-space forms. Interestingly, $X$-space forms with constant $div(X)$ are
Riemann solitons.

###### Example 1

Let $(\mathbb{Q}^{n}(c),g)$ be a Riemannian manifold of constant sectional
curvature $c\in\\{-1,0,1\\}$. Then, $(\mathbb{Q}^{n}(c),g,f,c)$ is a trivial
Riemann soliton for any constant function $f$ on $\mathbb{Q}^{n}(c)$.

###### Example 2

(Gaussian soliton) The Gaussian soliton on $\mathbb{R}^{n}$, which is given by

$g_{ij}=\delta_{ij}\quad\mbox{and}\quad f(x)=\frac{\lambda}{4}|x|^{2},$

is a Riemann soliton since

$Rm=0\quad\mbox{and}\quad\nabla^{2}f=\frac{\lambda}{2}g.$

###### Example 3

Let $(M^{n+1},g)$ be the Riemannian product manifold of $(0,\infty)$ and
$\mathbb{S}^{n}$. Then, $(M^{n+1},g,\nabla f,\lambda)$ is a gradient Riemann
soliton for

$f:(0,\infty)\times\mathbb{S}^{n}\longrightarrow\mathbb{R},\quad(r,p)\longmapsto\frac{\lambda}{4r},$

where $\lambda\in\mathbb{R}$.

###### Example 4

Let $(M^{n+1},g)$ be the product manifold $(0,\infty)\times\mathbb{S}^{n}$
equipped with the metric tensor given by

$g_{(r,p)}(t_{1}\oplus v_{1},t_{2}\oplus
v_{2})=r\left(t_{1}t_{2}+v_{1}v_{2}\right),$

for every $(r,p)\in(0,\infty)\times\mathbb{S}^{n}$ and $t_{1}\oplus
v_{1},t_{2}\oplus v_{2}\in\mathbb{R}\oplus T_{p}\mathbb{S}^{n}\cong
T_{(r,p)}\left((0,\infty)\times\mathbb{S}^{n}\right)$. Then, the quadruple
$(M^{n+1},g,\nabla f,\lambda)$ is a gradient Riemann soliton for

$f:(0,\infty)\times\mathbb{S}^{n}\longrightarrow\mathbb{R},\quad(r,p)\longmapsto\ln{\left(r^{2}\right)}+\frac{r^{2}}{4}.$

Examples (3) and (4) both are locally conformally flat gradient Riemann
solitons invariant by the left action of

$\mbox{SO}(n,\mathbb{R})=\left\\{A\in\mbox{GL}(n,\mathbb{R}):A^{-1}=A^{t}\right\\},$

on $(0,\infty)\times\mathbb{S}^{n}$, given by

$\mbox{SO}(n,\mathbb{R})\times\left((0,\infty)\times\mathbb{S}^{n}\right)\longrightarrow(0,\infty)\times\mathbb{S}^{n},\quad(A,(r,p))\longmapsto(r,A\cdot
p).$

We refer the reader to [9, 11, 12, 8, 10, 7, tokura2021invariant] for examples
of other symmetric solitons. When we first met Riemann solitons it occurred to
us the results of Ivey [13], showing that expanding or steady compact gradient
Ricci solitons are rigid, of Hsu [14], showing that compact gradient Yamabe
solitons are trivial, of Barros et al. [15], showing that compact almost Ricci
solitons with constant scalar curvature are Einstein manifolds, of Catino et
al. [16], showing that compact $\rho$-Einstein solitons are trivial, and even
one of our own, Tokura et al. [17], showing that compact gradient $k$-Yamabe
solitons are trivial. On Riemann solitons, specifically, we found a result due
to Blaga [18], showing that compact Riemann solitons with potential vector
field $X$ of constant length are trivial. Our first result extends Blaga’s
result as it doesn’t impose any additional conditions on the potential vector
field.

###### Theorem 1

Compact Riemann solitons $(M^{n},g,X,\lambda)$ are trivial.

Also, we show that non-compact Riemann solitons are Einstein manifolds as long
as it has constant scalar curvature and a certain weighted integral of
$|\nabla f|^{2}$ is finite.

###### Theorem 2

Let $(M^{n},g,\nabla f,\lambda)$ be a complete non-compact gradient Riemann
soliton with constant scalar curvature satisfying

$\int_{M^{n}\setminus B_{r}(x_{0})}\frac{|\nabla
f|^{2}}{d(x,x_{0})^{2}}\,dv_{g}<\infty,$

for some $r>0$, where $d$ is the $g$-distance function on $M^{n}$ and
$B_{r}(x_{0})$ then is the $d$-ball of radius $r$ centered at $x_{0}$. Then,
$(M^{n},g)$ is an Einstein manifold.

Next, we show that for Riemann solitons the sign os the scalar curvature at
infinity determines the sign of the scalar curvature as a whole.

###### Theorem 3

Let $(M^{n},g,X,\lambda)$ be a complete non-compact shrinking or steady
Riemann soliton. Assume that $\liminf_{x\to\infty}S(x)\geqslant 0$. Then, $M$
has nonnegative scalar curvature. Furthermore, if $M$ is not scalar flat, then
$S>n\lambda$.

Our last result classifies scalar flat and steady gradient Riemann solitons.

###### Theorem 4

Let $(M^{n},g,\nabla f,\lambda)$ be a complete steady gradient Riemann
soliton. If $M^{n}$ is scalar flat and $f$ is not constant, then $M^{n}$ is
isometric to a cylinder $\mathbb{R}\times\Sigma^{n-1}$ where $\Sigma^{n-1}$ is
a Ricci flat hypersurface of $M^{n}$ and $f(t,x)=at+b$, for certain
$a,b\in\mathbb{R}$, $a\neq 0$.

##  2 Key lemmas and proofs

Our first lemma provides a link between Riemann solitons and almost Ricci
solitons. This relation was also noted by Hirica and Udriste [5] and Blaga
[18].

###### Lemma 5

The Riemannian manifold $(M^{n},g)$ admits a Riemannian soliton structure
$(M^{n},g,X,\lambda)$ if and only if $(M^{n},g)$ has null Weyl tensor and
admits an almost Ricci soliton structure with soliton vector field $(n-2)X$
satisfying

$Rc_{jk}+\frac{n-2}{2}(\mathcal{L}_{X}g)_{jk}=\left[\frac{(n-2)\lambda}{2}+\frac{S}{2(n-1)}\right]g_{jk},$
(4)

where $Rc$ is the Ricci tensor and $S$ the scalar curvature of $(M^{n},g)$.

###### Proof of Lemma 5 1

From the fundamental equation (2), we have

$\begin{split}Rm_{ijkl}+\frac{1}{2}[g_{il}(\mathcal{L}_{X}g)_{jk}+g_{jk}(\mathcal{L}_{X}g)_{il}-g_{ik}(\mathcal{L}_{X}g)_{jl}-g_{jl}(\mathcal{L}_{X}g)_{ik}]=\lambda[g_{il}g_{jk}-g_{ik}g_{jl}],\end{split}$

which by contraction over $i$ and $l$, gives

$Rc_{jk}+\frac{n-2}{2}(\mathcal{L}_{X}g)_{jk}=[(n-1)\lambda-div(X)]g_{jk},$
(5)

or equivalently

$Rc_{jk}+\frac{n-2}{2}(\mathcal{L}_{X}g)_{jk}=\left[\frac{(n-2)\lambda}{2}+\frac{S}{2(n-1)}\right]g_{jk}.$
(6)

Now, substituting equation (6) into the decomposition of the Riemann tensor
(see [3]):

$Rm_{ijkl}=\left[\frac{1}{n-2}\left(Rc_{g}-\frac{S}{2(n-1)}g\right)\odot
g\right]_{ijkl}+W_{ijkl},$ (7)

and using the Riemann soliton equation (2), we deduce that $W_{ijkl}=0$.

On the other hand, if $(M^{n},g)$ has null Weyl $W=0$ and admits an almost
Ricci soliton structure

$Rc_{jk}+\frac{n-2}{2}(\mathcal{L}_{X}g)_{jk}=\left[\frac{(n-2)\lambda}{2}+\frac{S}{2(n-1)}\right]g_{jk}.$

Then from (7) we have

$\begin{split}Rm_{ijkl}&=\left[\frac{1}{n-2}\left(Rc_{g}-\frac{S}{2(n-1)}g\right)\odot
g\right]_{ijkl}=\frac{1}{2}\left[\left(-\mathcal{L}_{X}g+\lambda g\right)\odot
g\right]_{ijkl}.\end{split}$

Therefore, $(M^{n},g)$ admits a Riemann solitons structure

$Rm_{ijkl}+\frac{1}{2}\left(\mathcal{L}_{X}g\odot
g\right)_{ijkl}=\left(\frac{\lambda}{2}g\odot g\right)_{ijkl},$

###### Remark 1

In the particular case in which $X$ is a gradient vector field, we have that
equation (4) produces a Schouten soliton [16]. Therefore, $(M^{n},g)$ is a
gradient Riemann soliton with potential $X$ if and only if $(M^{n},g)$ is a
Schouten soliton with potential $(n-2)X$ and $W_{ijkl}=0$.

###### Lemma 6

Let $(M^{n},g,X,\lambda)$ be a Riemann soliton. Then the following formulas
hold

$div(X)=\frac{n\lambda}{2}-\frac{S}{2(n-1)}$ (8)
$Rc_{lj}X_{l}+\frac{S_{j}}{2(n-1)}+X_{jii}=0$ (9)
$\frac{n-2}{2}X_{j}S_{j}+\frac{n-2}{2}\lambda
S+\frac{S^{2}}{2(n-1)}-|Rc|^{2}=0$ (10)

###### Proof of Lemma 6 1

In order to obtain (8) it is enough to contract equation (4). Now, for
equation (9), we remember the Ricci identity:

$X_{ijk}-X_{ikj}=X_{t}Rm_{tijk}.$ (11)

Contracting (11) with respect $i$, $k$, we have

$X_{iji}-X_{iij}=Rm_{liji}X_{l}=Rc_{lj}X_{l}.$

Next, differentiating (4) we get

$\begin{split}Rc_{ij,i}&=-\frac{n-2}{2}(X_{iji}+X_{jii})+\frac{S_{i}}{2(n-1)}g_{ij}\\\
&=-\frac{n-2}{2}(X_{iji}-X_{iij}+X_{iij}+X_{jii})+\frac{S_{i}}{2(n-1)}g_{ij}\\\
&=-\frac{n-2}{2}Rc_{lj}X_{l}-\frac{n-2}{2}(X_{iij}+X_{jii})+\frac{S_{i}}{2(n-1)}g_{ij}.\end{split}$

Using the twice contracted second Bianchi identity and equation (8), we deduce

$\frac{1}{2}S_{j}=Rc_{ij,i}=-\frac{n-2}{2}Rc_{lj}X_{l}-\frac{n-2}{2}\left[-\frac{S_{j}}{2(n-1)}+X_{jii}\right]+\frac{S_{j}}{2(n-1)},$

which enables us obtain equation (9).

Finally, from Lemma 5, the Riemann soliton $(M^{n},g)$ admits a gradient
almost Ricci solitons structure with vector field $(n-2)X$ and soliton
function $\frac{(n-2)\lambda}{2}+\frac{S}{2(n-1)}$. Then, according to Lemma 3
of [15], we have

$\begin{split}\Delta_{(n-2)X}Rc_{ik}=\left[(n-2)\lambda+\frac{S}{(n-1)}\right]Rc_{ik}-2Rm_{ijks}Rc_{js}+\frac{n-2}{2}Rc_{is}(X_{sk}-X_{ks})\\\
+\frac{n-2}{2}Rc_{sk}(X_{si}-X_{is})+\frac{S_{ik}}{2}+\frac{\Delta
S}{2(n-1)}g_{ki}-\frac{S_{ij}}{2(n-1)}g_{kj}.\end{split}$ (12)

Computing the trace of identity (12), we deduce

$-\frac{n-2}{2}X_{j}S_{j}=\left[\frac{n-2}{2}\lambda+\frac{S}{2(n-1)}\right]S-|Rc|^{2},$

which is equation (10).

###### Proof of Theorem 1 1

Let $p,q\in M$ be the points where $S$ attains its maximum and minimum in $M$,
i.e.,

$S(q)=\min_{x\in M}S(x),\quad S(p)=\max_{x\in M}S(x),\quad\nabla S(p)=0=\nabla
S(q),$

and

$\lambda-\frac{S(q)}{n(n-1)}\geqslant\lambda-\frac{S(x)}{n(n-1)}\geqslant\lambda-\frac{S(p)}{n(n-1)}.$

From (10) and the fact $|\textup{Rc}|^{2}\geqslant\frac{1}{n}S^{2}$, we have

$0=\frac{n-2}{2}\lambda
S(q)+\frac{S(q)^{2}}{2(n-1)}-|\textup{Rc}|^{2}(q)\leqslant\frac{n-2}{2}S(q)\left(\lambda-\frac{S(q)}{n(n-1)}\right),$
(13)

and

$\begin{split}0=\frac{n-2}{2}\lambda
S(p)+\frac{S(p)^{2}}{2(n-1)}-|\textup{Rc}|^{2}(p)\leqslant\frac{n-2}{2}S(p)\left(\lambda-\frac{S(p)}{n(n-1)}\right).\end{split}$
(14)

On the other hand, since $M$ is compact, integrating equation (8), we obtain
the identity

$\lambda=\frac{1}{\textup{Vol}(M)}\int_{M}\frac{S}{n(n-1)}dv_{g},$ (15)

from what we see that

$\lambda-\frac{S(q)}{n(n-1)}\geqslant
0,\qquad\lambda-\frac{S(p)}{n(n-1)}\leqslant 0.$ (16)

Now, we state that

$\lambda-\frac{S(q)}{n(n-1)}=0.$

In fact, if $\lambda-\frac{S(q)}{n(n-1)}>0$, then from (13), we have
$S(q)\geqslant 0$ and $\lambda>0$. Consequently, $S(p)>0$ from (16). Now,
since $S(p)>0$ we deduce from (14) and (16) that $n(n-1)\lambda=S(p)$ and
$S\equiv S(p)$ which is an absurd from $\lambda-\frac{S(q)}{n(n-1)}>0$.
Therefore, $\lambda-\frac{S(q)}{n(n-1)}=0$. However, this implies $S\equiv
S(q)$ and $\lambda-\frac{S}{n(n-1)}=0$. Hence, from (4) we deduce that
$(M^{n},g)$ is a Ricci soliton. From Perelman work’s [19] we known that any
compact Ricci soliton is a gradient Ricci soliton. Then there exist a smooth
function $f$ on $M$ such that $div(X)=\Delta f=0$. The compacteness of $M$
implies that $f$ is constant and the soliton is trivial.

###### Proof of Theorem 2 1

We follow the argument in [20]. The Bochner formula states that ([21]):

$\frac{1}{2}\Delta|\nabla f|^{2}=|\nabla^{2}f|^{2}+\langle\nabla
f,\nabla\Delta f\rangle+Rc(\nabla f,\nabla f).$

From equation (9) we conclude that

$\begin{split}Rc_{lj}f_{l}&=-\frac{S_{j}}{2(n-1)}-(\nabla^{2}f)_{ji,i}\\\
&=-\frac{S_{j}}{2(n-1)}+\frac{Rc_{ji,i}}{n-2}-\frac{S_{j}}{2(n-1)(n-2)}\\\
&=-\frac{S_{j}}{2(n-1)}+\frac{S_{j}}{2(n-2)}-\frac{S_{j}}{2(n-1)(n-2)}\\\
&=0.\end{split}$ (17)

Since $S$ is constant, we have that $div(X)=\Delta f$ is constant from (8).
Hence, from (17), we arrive at

$\frac{1}{2}\Delta|\nabla f|^{2}=|\nabla^{2}f|^{2}.$ (18)

We choose a cut-off function $\psi_{r}$ on $B_{2r}(x_{0})$ such that
$0\leqslant\psi_{r}\leqslant 1$, $\mathrm{supp}(\psi_{r})\subset
B_{2r}(x_{0})$ and

$\psi_{r}=1\quad\mbox{in}\quad
B_{r}(x_{0}),\quad|\nabla\psi_{r}|^{2}\leqslant\frac{C}{r^{2}},\quad\Delta\psi_{r}\leqslant\frac{C}{r^{2}}.$

Then, multiplying (18) by $\psi_{r}$ and integrating it over $B_{r}(x_{0})$,
we get

$\begin{split}\int_{B_{r}(x_{0})}|\nabla^{2}f|^{2}\psi_{r}^{2}&=\int_{B_{r}(x_{0})}\frac{1}{2}\Delta|\nabla
f|^{2}\psi_{r}^{2}\\\ &=\int_{B_{r}(x_{0})}\frac{1}{2}|\nabla
f|^{2}\Delta\psi_{r}^{2}\\\ &\leqslant\int_{B_{2r}(x_{0})\setminus
B_{r}(x_{0})}\frac{1}{2}\frac{C}{r^{2}}|\nabla f|^{2}\longrightarrow
0\quad\mbox{as}\quad r\longrightarrow\infty.\end{split}$

Hence

$\int_{M}|\nabla^{2}f|^{2}=0.$

Therefore $\nabla^{2}f=0$, and from Lemma 5 $(M^{n},g)$ is Einstein.

###### Proof of Theorem 3 1

From equation (10) of Lemma 6 and the inequality
$|Rc|^{2}\geqslant\frac{S^{2}}{n}$, we deduce that

$\langle X,\nabla S\rangle+\lambda S-\frac{S^{2}}{n(n-1)}\geqslant 0.$ (19)

Using the maximum principle we can prove Theorem 3. In fact, suppose that
$\inf_{M}S(x)<0$, then by the assumption $\liminf_{x\to\infty}S(x)\geqslant
0$, there exists some point $z\in M$ such that

$S(z)=\inf_{M}S(x)<0.$

Hence,

$\Delta S(z)\geqslant 0,\qquad\nabla S(z)=0.$

Then, from (19) we deduce that

$\lambda S(z)-\frac{S(z)^{2}}{n(n-1)}\geqslant 0.$

This is impossible since $\lambda S(z)-\frac{S(z)^{2}}{n(n-1)}<0$ for
$\lambda\geqslant 0$. Therefore $S(x)\geqslant 0$. From the maximum principle
we have that either $S\equiv 0$ or $S(x)>n(n-1)\lambda$.

The next Lemma is a particular case of theorem 6.3 of Catino et al. [22]. For
the sake of completeness we will prove here.

###### Lemma 7

Let $(M^{n},g,\nabla f,\lambda)$ be a complete gradient Riemann soliton,
$c\in\mathbb{R}$ be a regular value of $f$ and $\Sigma_{c}=f^{-1}(c)$ be its
level surface. Then

  1. 1.

$|\nabla f|^{2}$ is constant on a connected component of $\Sigma_{c}$.

  2. 2.

$\Sigma_{c}$ is totally umbilical and the scalar curvature $S$ is constant on
$\Sigma_{c}$

  3. 3.

the mean curvature $H$ is constant on $\Sigma_{c}$.

  4. 4.

$\Sigma_{c}$ is Einstein with respect to the induced metric.

  5. 5.

in any open set of $\Sigma_{c}$ in which $f$ has non critical points, the
metric $g$ can be written as

$g=dr^{2}+\psi(r)^{2}g_{\Sigma},$

where $g_{\Sigma}$ is the metric induced by $g$ in $\Sigma_{c}$ and
$\psi(r)=e^{\frac{1}{n-1}\int_{r_{0}}^{r}H(s)ds}$.

###### Proof of Lemma 7 1

Let $\\{e_{1},e_{2},\dots,e_{n}\\}$ be a local orthonormal frame in a
neighborhood of a regular point $p\in\Sigma_{c}$ such that
$\\{e_{2},\dots,e_{n}\\}$ are tangent to $\Sigma_{c}$ and $e_{1}=\frac{\nabla
f}{|\nabla f|}$. In order to prove (a), note that for any $X\bot\nabla f$ we
have

$\nabla_{X}|\nabla f|^{2}=2\nabla^{2}f(X,\nabla f)=\frac{2}{n-2}Rc(X,\nabla
f)=0.$

Now, from Lemma 5 the Weyl tensor $W$ vanishes, then since Riemann solitons
are in the class of almost Ricci solitons, we can apply Theorem 4.4 and
Proposition 6.1 of [22] with $\alpha=1$, $\beta=n-2$, $\mu=0$ to deduce that

$h_{ab}=\frac{H}{n-1}g_{ab},\qquad|\nabla S|=0,\quad\text{on}\quad\Sigma_{c},$

where $h_{ab}$ and $H$ are respectively the second fundamental form and the
mean curvature of $\Sigma_{c}$. This proves (b).

For item (c) we use the Codazzi equation

$Rm_{1cab}=\nabla^{\Sigma_{c}}_{a}h_{bc}-\nabla^{\Sigma_{c}}_{b}h_{ac},\qquad
a,b,c\in\\{2,\dots,n\\}.$ (20)

Tracing over $b$ and $c$ in (20), we obtain

$Rc_{1a}=\nabla^{\Sigma_{c}}_{a}H-\nabla^{\Sigma_{c}}_{b}h_{ab}=\left(1-\frac{1}{n-1}\right)\nabla_{a}H.$
(21)

Then (c) follows since $Rc_{1a}=0$.

Now, using (4) we deduce the following expression for the second fundamental
form on $\Sigma_{c}$

$h_{ab}=\langle\nabla_{a}e_{1},e_{b}\rangle=\frac{(\nabla^{2}f)_{ab}}{|\nabla
f|}=\frac{\left[\frac{\lambda}{2}+\frac{S}{2(n-1)(n-2)}\right]g_{ab}-\frac{Rc_{ab}}{n-2}}{|\nabla
f|}=\frac{H}{n-1}g_{ab}.$ (22)

Hence

$Rc_{ab}=\left[\frac{\lambda(n-2)}{2}+\frac{S}{2(n-1)}-\frac{H(n-2)}{n-1}|\nabla
f|\right]g_{ab}.$ (23)

Since $S$, $H$ and $|\nabla f|$ are constant on $\Sigma_{c}$, we have that
$\Sigma_{c}$ is Einstein.

Finally, since $\nabla f$ and the level surface of $f$ are orthogonal to each
other, we can express the metric $g$ in the form

$g=dr^{2}+g_{ab}(r,\theta)d\theta^{a}d\theta^{b},$

where $\theta=(\theta^{2},\dots,\theta^{n})$ denotes a local coordinates on
$\Sigma_{c_{0}}$ and $r(x)=\int\frac{df}{|\nabla f|}$. A good survey on level
set structures can be found in [23, 24, 22, 25].

Fixing local coordinates system

$(x^{1},\dots,x^{n})=(r,\theta^{1},\cdots,\theta^{n}),$

we can express the second fundamental form of $\Sigma_{c}$ in terms of the
Christoffel symbol $\Gamma_{ab}^{1}$, that is

$h_{ab}=-\langle\partial_{r},\nabla_{a}\partial_{b}\rangle=-\langle\partial_{r},\Gamma_{ab}^{1}\partial_{r}\rangle=-\Gamma_{ab}^{1}=-\frac{g^{11}}{2}\left(-\frac{\partial
g_{ab}}{\partial r}\right)=\frac{1}{2}\frac{\partial g_{ab}}{\partial r}.$

Then

$\frac{1}{2}\frac{\partial g_{ab}}{\partial r}=\frac{H}{n-1}g_{ab},$

which implies that

$g_{ab}(r,\theta)=e^{\frac{2}{n-1}\int_{r_{0}}^{r}H(s)ds}g_{ab}(r_{0},\theta).$

Here the level surface $\\{r=r_{0}\\}$ corresponds to $\Sigma_{c_{0}}$.
Therefore,

$g=dr^{2}+e^{\frac{2}{n-1}\int_{r_{0}}^{r}H(s)ds}g_{ab}(r_{0},\theta)d\theta^{a}d\theta^{b},$

which proves item (e).

###### Proof of Theorem 4 1

From equation (10) of Lemma 6, we deduce that $Rc=0$, so, from (4) we deduce
that $\nabla^{2}f=0$. In particular $|\nabla f|$ is constant. Then either $f$
is constant, or $f$ has no critical point at all. Since $f$ is not constant,
this latter case occurs. Now, Lemma 7 implies that $(M^{n},g)$ is isometric to
the warped product $\mathbb{R}\times_{\psi}\Sigma^{n-1}$ of the entire real
line with a $(n-1)$–dimensional complete Riemannian manifold.

We now prove that $\psi$ is constant and $\Sigma^{n-1}$ is Ricci flat. Indeed,
since $\nabla^{2}f=0$, we have from equation (22) that

$h_{ab}=\langle\nabla_{a}e_{1},e_{b}\rangle=\frac{(\nabla^{2}f)_{ab}}{|\nabla
f|}=0,$

which implies that $H=0$. Then

$\psi(r)^{2}=e^{\frac{2}{n-1}\int_{r_{0}}^{r}H(s)ds}=1.$

This fact shows that $M^{n}=\mathbb{R}\times_{1}\Sigma^{n-1}$ is a Riemannian
product. On the other hand, from the expression (23), we have that
$Rc_{ab}=0$, i.e., $\Sigma^{n-1}$ is Ricci flat. Furthermore, since
$\nabla^{2}f=0$ on $\mathbb{R}\times\Sigma^{n-1}$, we derive that
$f(t,x)=at+b$, $a,b\in\mathbb{R}$, $a\neq 0$.

## Conflict of interest

The authors declare that there is no conflict of interest.

## Data Availability Statements

Data sharing not applicable to this article as no datasets were generated or
analysed during the current study.

## References

  * [1] C. Udriste, Riemann flow and Riemann wave, An. Univ. Vest, Timisoara, Ser. Mat. Inf 48 (2010) 1–2.
  * [2] C. Udriste, Riemann flow and Riemann wave via bialternate product Riemannian metric, arXiv preprint arXiv:1112.4279 (2011).
  * [3] A. L. Besse, Einstein manifolds, Springer Science & Business Media, 2007.
  * [4] I. E. Hirica, C. Udriste, Basic evolution PDEs in Riemannian geometry, Balkan J. Geom. Appl 17 (1) (2012) 30–40.
  * [5] I. E. Hirica, C. Udriste, Ricci and Riemann solitons, Balkan J. Geom. Appl 21 (2) (2016) 35–44.
  * [6] G. Catino, P. Mastrolia, A potential generalization of some canonical riemannian metrics, Annals of Global Analysis and Geometry 55 (4) (2019) 719–748.
  * [7] W. Tokura, L. Adriano, R. Pina, M. Barboza, On warped product gradient yamabe solitons, Journal of Mathematical Analysis and Applications 473 (1) (2019) 201–214.
  * [8] B. L. Neto, K. Tenenblat, On gradient yamabe solitons conformal to a pseudo-euclidian space, Journal of Geometry and Physics 123 (2018) 284–291.
  * [9] E. Barbosa, R. Pina, K. Tenenblat, On gradient ricci solitons conformal to a pseudo-euclidean space, Israel Journal of Mathematics 200 (1) (2014) 213–224.
  * [10] R. Pina, I. Menezes, On gradient schouten solitons conformal to a pseudo-euclidean space, manuscripta mathematica 163 (3) (2020) 395–406.
  * [11] E. Batista, L. Adriano, W. Tokura, On warped product gradient ricci-harmonic soliton, arXiv preprint arXiv:1906.11933 (2019).
  * [12] B. Leandro, J. P. dos Santos, Reduction of gradient ricci soliton equation, Ann. Acad. Sci. Fenn. Math 45 (2) (2020) 1003–1011.
  * [13] T. Ivey, Ricci solitons on compact three-manifolds, Differential Geometry and its Applications 3 (4) (1993) 301–307.
  * [14] S. Y. Hsu, A note on compact gradient Yamabe solitons, Journal of Mathematical Analysis and Applications 388 (2) (2012) 725–726.
  * [15] A. Barros, R. Batista, E. Ribeiro, Compact almost Ricci solitons with constant scalar curvature are gradient, Monatshefte für Mathematik 174 (1) (2014) 29–39.
  * [16] G. Catino, L. Mazzieri, Gradient Einstein solitons, Nonlinear Analysis 132 (2016) 66–94.
  * [17] W. Tokura, E. Batista, Triviality results for compact $k$-Yamabe solitons, Journal of Mathematical Analysis and Applications 502 (2) (2021) 125274.
  * [18] A. M. Blaga, Remarks on almost Riemann solitons with gradient or torse-forming vector field, Bulletin of the Malaysian Mathematical Sciences Society 44 (5) (2021) 3215–3227.
  * [19] G. Perelman, The entropy formula for the Ricci flow and its geometric applications, arXiv preprint math/0211159 (2002).
  * [20] L. Ma, V. Miquel, Remarks on scalar curvature of Yamabe solitons, Annals of Global Analysis and Geometry 42 (2) (2012) 195–205.
  * [21] S. Bochner, K. Yano, Curvature and Betti numbers, Princeton University Press, 1954\.
  * [22] G. Catino, P. Mastrolia, D. Monticelli, M. Rigoli, On the geometry of gradient Einstein-type manifolds, Pacific Journal of Mathematics 286 (1) (2016) 39–67.
  * [23] H. D. Cao, Q. Chen, On locally conformally flat gradient steady Ricci solitons, Transactions of the American Mathematical Society 364 (5) (2012) 2377–2391.
  * [24] H. D. Cao, X. Sun, Y. Zhang, On the structure of gradient Yamabe solitons, Mathematical Research Letters 19 (4) (2012) 767–774.
  * [25] B. Leandro, Vanishing conditions on Weyl tensor for Einstein-type manifolds, Pacific Journal of Mathematics 314 (1) (2021) 99–113.