#  : Motion-Aware Fast And Robust Camera
Localization for Dynamic NeRF

Mert Asim Karaoglu1,3∗, Hannah Schieber2∗, Nicolas Schischka1∗, Melih
Gorgulu1∗, Florian Grötzner1,
Alexander Ladikos3, Daniel Roth4, Nassir Navab1,5 and Benjamin Busam1 $*$equal
contribution1Mert Asim Karaoglu, Benjamin Busam, Nassir Navab, Nicolas
Schischka, Melih Gorgulu, Florian Grötzner are with Technical University of
Munich, Munich, Germany<EMAIL_ADDRESS>Schieber is with the
Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany
<EMAIL_ADDRESS>Asim Karaoglu and Alexander Ladikos are with
ImFusion GmbH, Munich, Germany4Daniel Roth is with Technical University of
Munich, School of Medicine and Health, Klinikum rechts der Isar, Orthopaedics
and Sports Orthopaedics, Munich, Germany5Nassir Navab is with Johns Hopkins
University, Baltimore, MD, USA

###### Abstract

Dynamic reconstruction with neural radiance fields (NeRF) requires accurate
camera poses. These are often hard to retrieve with existing structure-from-
motion (SfM) pipelines as both camera and scene content can change. We propose
DynaMoN that leverages simultaneous localization and mapping (SLAM) jointly
with motion masking to handle dynamic scene content. Our robust SLAM-based
tracking module significantly accelerates the training process of the dynamic
NeRF while improving the quality of synthesized views at the same time.
Extensive experimental validation on TUM RGB-D, BONN RGB-D Dynamic and the
DyCheck’s iPhone dataset, three real-world datasets, shows the advantages of
DynaMoN both for camera pose estimation and novel view synthesis.

##  I INTRODUCTION

Enabling novel view synthesis on dynamic scenes often requires multi-camera
setups [1, 2]. However, everyday dynamic scenes are often captured by a single
camera, restricting the field of view [1]. Deep learning’s success in novel
view synthesis has the potential to transcend these restrictions. In
comparison to the commonly used voxel, surfel or truncated signed distance
fields (TSDF) output of Simultaneous Localization and Mapping (SLAM)
approaches, neural radiance fields (NeRF) can address incomplete 3D
reconstructions and enable novel view synthesis from new, reasonable camera
positions [3]. The success in dynamic NeRFs [4, 5, 6, 7] allows novel view
synthesis not only for static captures but also for dynamic ones.

To enable novel view synthesis using a NeRF, accurate camera poses, usually
retrieved via COLMAP [8] or the Apple ARKit, are essential. The structure-
from-motion (SfM)-based approach often demands hours of computation to
estimate reasonable camera poses and is challenged by large scale dynamic
scenes.

To overcome the limitation of slow SfM approaches, recent works combined SLAM
and NeRF [9, 10, 11, 3]. SLAM provides faster results for the camera
trajectory than classic SfM. However, by nature, scenes are more often dynamic
than static. Existing NeRF-SLAM [9, 10, 11, 3] methods assume a static scene,
making them challenged by dynamic ones, see Fig 1.

Figure 1: NeRF-SLAM approaches rely on static scenes (left). However,
approaches like InstantNGP [12] used for visualization in NeRF-SLAM [3] or
Orbeez-SLAM [11] enable masking out dynamic pixels (left). Still when using
the masking the scene representation lacks in quality (left). Dynamic NeRF
approaches rely on SfM which are offline and sometimes not successful on
highly dynamic scenes (center). Considering dynamics already in the SLAM
approach (right) enables better camera tracking and enables novel view
synthesis with higher quality.

In this paper, we present DynaMoN, a motion-aware camera localization and
visualization approach which can handle highly dynamic scenes. We utilize
motion and semantic segmentation during our dynamic camera localization
process to enable a robust tracking of the camera path in a dynamic
environment. We combine our camera localization approach with a dynamic NeRF
to enable a decent visualization of novel views of the scenes. To demonstrate
our robustness we evaluate our approach on three challenging datasets, namely
the dynamic subset of the TUM RGB-D [13] dataset, the BONN RGB-D Dynamic [14]
dataset and the DyCheck’s iPhone [15] dataset.

In summary we contribute:

  * •

DynaMoN, a motion-aware fast and robust camera localization approach for
dynamic novel view synthesis.

  * •

State-of-the-art camera localization and novel view synthesis results on TUM
RGB-D, BONN RGB-D Dynamic and the DyCheck’s iPhone dataset.

##  II RELATED WORKS

Our approach DynaMoN combines a dynamic NeRF with a fast and robust motion-
aware camera localization, thus, our related works are grouped under the
topics of camera localization, their employment for novel view synthesis and
neural representation for dynamic scenes.

###  II-A Camera Localization and Scene Dynamics

SfM and SLAM [16, 17, 18, 19] are the two most common approaches for robust
camera localization. More specifically, SLAM often targets real-time
localization of a camera coupled with mapping. In addition to the visual
information, some approaches can utilize additional sensory signals like depth
and IMU to further improve the accuracy. A common branch of work employs ORB
descriptors to build and track on sparse 3D maps [16, 19, 17].

In addition to classical or partially learnable SLAM approaches, Teed and Deng
[18] introduce the end-to-end learnable DROID-SLAM. DROID-SLAM consists of a
Dense Bundle Adjustment layer enabling recurrent iterative updates of camera
poses and pixel-wise depth.

Initial camera localization approaches rely on static scenes. However, natural
scenes are dynamic [20, 21, 22, 23]. To address this, Yu et al. [20] introduce
a dynamic SLAM approach building upon ORB-SLAM2 [19]. Their DS-SLAM integrates
a semantic segmentation network and a moving consistency check in the tracking
process.

Rünz et al. [24] enrich RGB-D SLAM with object-awareness and semantic
segmentation capabilities. DynaSLAM [25] combines geometric and deep learning-
based segmentation techniques to effectively mask out dynamic objects.

Dai et al. [22] build upon ORB-SLAM2 [19] and introduce the use of semantic
segmentation using point correlation to split dynamic and static points. Ye et
al. [23] divide the scene representation in a static flow field and a dynamic
flow field.

###  II-B Camera Localization and NeRF

Using the retrieved camera poses, a captured environment can be represented as
a 3D scene representation. One possible scene representation is the use of a
multilayer perceptron (MLP) [26]. Mildenhall et al. [26] introduce the use of
a MLP as neural representation for novel view synthesis, denoted as NeRF. A
NeRF relies on accurate camera poses, typically captured with COLMAP [8].
However, this slows down the overall pipeline. An alternative to SfM is the
use of SLAM to retrieve the camera poses.

Sucar et al. [27] introduce the use of an MLP as scene representation in a
SLAM approach. Building upon the use of a single MLP, Zhu et al. [9] introduce
the use of multiple NeRFs in a hierarchical feature grid to learn the camera
pose and scene representation. NICER-SLAM [10] introduces a similar approach
for RGB input including depth estimation.

Other approaches purely relying on RGB and combining SLAM and NeRF [3, 11]
utilize the well-known hash-based InstantNGP [12] for the scene
representation. While NeRF-SLAM [3] builds upon an adaption of Droid-SLAM
[18], Orbeez-SLAM [11] leverages visual odometry [19].

###  II-C Neural Representation of Dynamic Scenes

In real-world, scenes are inherently dynamic in contrast to the assumptions of
the more common static representation [9, 10, 27, 11, 3, 12, 26]. To represent
dynamics in a NeRF, the use of grid structures has been a popular approach [4,
5]. HexPlane [4] utilizes a 4D space-time grid divided into six feature
planes. The feature vector is a 4D point in space-time which is projected onto
each feature plane. Similarly, Tensor4d [5] builds upon a 4D tensor field
using time-aware volumes projected onto nine 2D planes. TiNeuVox [7]
represents dynamic scenes using time-aware voxel features and enables faster
training.

Optimizing the camera poses and the view synthesis has been explored in static
scenes [28, 29, 30, 31, 32] and often improved the novel view synthesis. Liu
et al. [33] introduce this optimization to a dynamic NeRF. To model the
dynamic scene, they use one static and one dynamic NeRF, where the static NeRF
is optimized using the camera poses.

##  III Method

In our setup we consider our input to be a video captured in a dynamic
environment using a moving camera without knowing its poses. This leads to two
objectives: first, to estimate the camera poses along the dynamic input, and
second, to build a 4D representation enabling novel views using the input
images and predicted camera poses.

Figure 2: Our DynaMoN architecture. We utilize segmentation
($M_{ss}\rightarrow S_{S_{i}}$) and motion masks ($M_{MS}\rightarrow
S_{M_{i}}$) on the input images ($I$) to enable a motion-aware fast and robust
camera localization. Based on the predicted camera poses $G_{i},G_{i-1,...}$
and the camera intrinsic ($K_{i}$) we produce a 4D output ($\Theta$) using
dynamic NeRF ($M_{N}$).

###  III-A Camera Localization

Our work builds upon DROID-SLAM [18]. To enhance its capacity of coping with
highly dynamic scenes, we take advantage of motion and semantic segmentation,
see Figure 2. Our underlying architecture [18] solves a dense bundle
adjustment problem in every iteration for a set of keyframes to acquire their
corresponding poses G and depths d. It is weighted by the confidences $w_{ij}$
of the optical flow calculation by RAFT [34]:

$\textbf{E}(\textbf{G},\textbf{d})=\sum_{(i,j)\in\epsilon}\left\|p_{ij}^{*}-\Pi_{C}(G_{ij}\circ\Pi_{C}^{-1}(p_{i},d_{i}))\right\|^{2}_{\Sigma_{ij}},$
(1)

where $\Sigma_{ij}=diag(w_{ij})$, $p_{ij}^{*}$ is the estimated optical flow,
$G_{ij}$ is the motion between the poses $G_{i}$ and $G_{j}$ and $p_{i}$ as
well as $d_{i}$ represent the pixel grid and inverse depth map of frame $i$
[18]. $(i,j)\in\epsilon$ means that there exists an edge between the images
$i$ and $j$ in the frame graph of the SLAM method, i.e. they have overlapping
fields of view. $\circ$ represents the retraction on the SE3 manifold and
$\Pi_{C}$ as well as $\Pi_{C}^{-1}$ the projection function from 3D to the
image plane and inverse projection function, respectively. The remaining poses
that do not belong to keyframes are filled in with a motion-only bundle
adjustment in the end. To introduce our proposed masking of dynamics, we set
the weight $w_{ij}$ equal to $0$, wherever one of the motion ($M_{MS}$) or the
segmentation mask ($M_{SS}$) evaluates to true. Due to the nature of the
Mahalanobis distance, potentially dynamic pixels will then not be considered
for optimization.

Our semantic segmentation module predicts masks for the predefined classes
person, cat and dog, since these appear in the used datasets. We use a pre-
trained version of the state-of-the-art semantic segmentation network
DeepLabV3 [35] with a ResNet50-backbone [36]. Additionally, we utilize a
motion-based filtering of pixels belonging to dynamic objects to enhance the
segmentation of dynamics ($M_{MS}$). This enables our approach to deal with a
greater variety of dynamics independent of previously known categories. We
refine the motion mask and the estimated camera movement between two adjacent
frames twice with an incremental increase of the segmentation threshold [37].

###  III-B Dynamic Scene Representation

While our dynamic camera localization module enables the retrieval of camera
poses in dynamic scenes, a 4D scene representation and novel view generation
is achieved with dynamic NeRF. To represent a scene in 4D (3D$+t$) we follow
the combination of implicit and explicit representation presented by Cao and
Johnson [4]. Our NeRF consists of six features planes, each pair for one
coordinate axis (e.g. XY, XZ, YT) [4].

The 3D scene is represented as a 4D feature volume $V\in\mathbf{R}^{X\times
Y\times Z\times T\times F}$ [4]:

$\begin{split}\sum_{r=1}^{R_{1}}\mathbf{M}_{r}^{X\times
Y}\otimes\mathbf{M}_{r}^{Z\times
T}\otimes\mathbf{v}_{r}^{1}&+\sum_{r=1}^{R_{2}}\mathbf{M}_{r}^{X\times
Z}\otimes\mathbf{M}_{r}^{Y\times T}\otimes\mathbf{v}_{r}^{2}\\\
&+\sum_{r=1}^{R_{3}}\mathbf{M}_{r}^{Y\times Z}\otimes\mathbf{M}_{r}^{X\times
T}\otimes\mathbf{v}_{r}^{3},\end{split}$ (2)

where each $\textbf{M}_{r}^{A\times B}\in\mathbf{R}^{A\times B}$ is one of the
six learned feature planes, $\mathbf{v}_{r}^{i}$ are vectors along the $F$
axis and $\otimes$ represents the outer product. Each pair of feature planes
(e.g. $\textbf{M}_{r}^{X\times Y},\textbf{M}_{r}^{Z\times T}$) has one spatial
and one spatio-temporal plane. Color and density features acquired from the
feature planes are fed into tiny MLPs to yield the final density and RGB color
values.

To optimize our NeRF module we sample batches of rays $\mathcal{R}$ for which
we compute the mean squared error between the rendered RGB values $C$ and the
ground truth $\hat{C}$:

$L_{RGB}=\frac{1}{|\mathcal{R}|}\sum_{r\in\mathcal{R}}\left\|C(r)-\hat{C}(r)\right\|_{2}^{2}.$
(3)

In addition, we apply the Total Variation (TV) loss [38, 4] as regularization
term for the features $x$ of each feature plane. For the multi-spatial planes
this results in

$\begin{split}L_{TV,s}&=\frac{2}{b}\sum_{i,j}(\frac{1}{N}\left(x_{i,j}-x_{i+1,j}\right)^{2}\\\
&+\frac{1}{M}\left(x_{i,j}-x_{i,j+1}\right)^{2})\end{split}$ (4)

and for the spatio-temporal planes in

$\begin{split}L_{TV,ts}&=\frac{2}{b}\sum_{i,j}(\frac{1}{N}\left(x_{i,j}-x_{i+1,j}\right)^{2}\\\
&+\frac{\lambda_{ts}}{M}\left(x_{i,j}-x_{i,j+1}\right)^{2}).\end{split}$ (5)

Here, $N$ is the total number of differences in the first plane dimension, $M$
is the total number of differences in the second dimension and b is the batch
size. In total, $L_{TV,\sigma}$ is the sum of the TV losses over all feature
planes for the density and $L_{TV,RGB}$ the same for the color. Thus, the
overall loss function for training the NeRF, which is done in a coarse-to-fine
manner, is characterized by the following:

$L_{NeRF}=L_{RGB}+\lambda_{TV}(L_{TV,\sigma}+wL_{TV,RGB}).$ (6)

TABLE I: Absolute Trajectory Error (RMS) [m] on TUM RGB-D [13].

| Sequences |  | RGB-D |  |  |  |  | Monocular |  |
---|---|---|---|---|---|---|---|---|---|---
| DVO-SLAM [39] | ORB-SLAM2 [19] | PointCorr [22] |  | DytanVO [37] | DeFlowSLAM [23] | DROID-SLAM [18] | OURS (MS&SS) | OURS (MS)
slightly dynamic | fr2/desk-person | 0.104 | 0.006 | 0.008 |  | 1.166 | 0.013 | 0.006 | 0.007 | 0.006
fr3/sitting-static | 0.012 | 0.008 | 0.010 |  | 0.016 | 0.007 | 0.005 | 0.005 | 0.005
fr3/sitting-xyz | 0.242 | 0.010 | 0.009 |  | 0.260 | 0.015 | 0.009 | 0.010 | 0.009
fr3/sitting-rpy | 0.176 | 0.025 | 0.023 |  | 0.046 | 0.027 | 0.022 | 0.024 | 0.021
fr3/sitting-halfsphere | 0.220 | 0.025 | 0.024 |  | 0.310 | 0.025 | 0.014 | 0.023 | 0.019
highly dynamic | fr3/walking-static | 0.752 | 0.408 | 0.011 |  | 0.021 | 0.007 | 0.012 | 0.007 | 0.014
fr3/walking-xyz | 1.383 | 0.722 | 0.087 |  | 0.028 | 0.018 | 0.016 | 0.014 | 0.014
fr3/walking-rpy | 1.292 | 0.805 | 0.161 |  | 0.155 | 0.057 | 0.040 | 0.031 | 0.039
fr3/walking-halfsphere | 1.014 | 0.723 | 0.035 |  | 0.385 | 0.420 | 0.022 | 0.019 | 0.020
| Mean/Max | 0.577/1.383 | 0.304/0.805 | 0.041/0.161 |  | 0.265/1.166 | 0.065/0.420 |  0.016/0.040 |  0.016/0.031 |  0.016/0.039

TABLE II: Absolute Trajectory Error (RMS) [m] on DyCheck’s iPhone [15]. The -
denotes that the camera pose could not be regressed.

Sequences | COLMAP [8] |  | DROID-SLAM [18] | OURS | OURS
---|---|---|---|---|---
| (MS&SS) | (MS)
Apple | - |  | 0.004 | 0.007 | 0.004
Backpack | - |  | 0.008 | 0.006 | 0.008
Block | - |  | 0.052 | 0.005 | 0.005
Creeper | 0.031 |  | 0.004 | 0.004 | 0.004
Handwavy | - |  | 0.002 | 0.001 | 0.001
Haru | - |  | 0.008 | 0.007 | 0.008
Mochi | - |  | 0.002 | 0.002 | 0.002
Paper Windmill | 0.012 |  | 0.002 | 0.002 | 0.002
Pillow | - |  | 0.058 | 0.020 | 0.023
Space Out | 0.057 |  | 0.008 | 0.008 | 0.008
Spin | 0.015 |  | 0.005 | 0.004 | 0.004
Sriracha | - |  | 0.002 | 0.002 | 0.002
Teddy | - |  | 0.015 | 0.011 | 0.010
Wheel | - |  | 0.005 | 0.004 | 0.063
Mean/Max | - |  | 0.013/0.058 | 0.006/0.020 | 0.010/0.063

##  IV Evaluation

DynaMoN is assessed on three tough datasets, examining camera localization and
novel view synthesis quality.

TABLE III: Absolute Trajectory Error (RMS) [m] on BONN RGB-D [14]

Sequences |  |  | RGB-D |  |  |  | Monocular |
---|---|---|---|---|---|---|---|---
ReFusion [14] | StaticFusion [40] | DynaSLAM (G) [25] | DynaSLAM (N+G) [25] |  | DROID-SLAM [18] | OURS (MS&SS) | OURS (MS)
balloon | 0.175 | 0.233 | 0.050 | 0.030 |  | 0.075 | 0.028 | 0.068
balloon2 | 0.254 | 0.293 | 0.142 | 0.029 |  | 0.041 | 0.027 | 0.038
balloon_tracking | 0.302 | 0.221 | 0.156 | 0.049 |  | 0.035 | 0.034 | 0.036
balloon_tracking2 | 0.322 | 0.366 | 0.192 | 0.035 |  | 0.026 | 0.032 | 0.028
crowd | 0.204 | 3.586 | 1.065 | 0.016 |  | 0.052 | 0.035 | 0.061
crowd2 | 0.155 | 0.215 | 1.217 | 0.031 |  | 0.065 | 0.028 | 0.056
crowd3 | 0.137 | 0.168 | 0.835 | 0.038 |  | 0.046 | 0.032 | 0.055
kidnapping_box | 0.148 | 0.336 | 0.026 | 0.029 |  | 0.020 | 0.021 | 0.020
kidnapping_box2 | 0.161 | 0.263 | 0.033 | 0.035 |  | 0.017 | 0.017 | 0.017
moving_no_box | 0.071 | 0.141 | 0.317 | 0.232 |  | 0.023 | 0.013 | 0.014
moving_no_box2 | 0.179 | 0.364 | 0.052 | 0.039 |  | 0.040 | 0.027 | 0.026
moving_o_box | 0.343 | 0.331 | 0.544 | 0.044 |  | 0.177 | 0.152 | 0.167
moving_o_box2 | 0.528 | 0.309 | 0.589 | 0.263 |  | 0.236 | 0.175 | 0.176
person_tracking | 0.289 | 0.484 | 0.714 | 0.061 |  | 0.043 | 0.148 | 0.024
person_tracking2 | 0.463 | 0.626 | 0.817 | 0.078 |  | 0.054 | 0.022 | 0.035
placing_no_box | 0.106 | 0.125 | 0.645 | 0.575 |  | 0.078 | 0.021 | 0.018
placing_no_box2 | 0.141 | 0.177 | 0.027 | 0.021 |  | 0.030 | 0.020 | 0.020
placing_no_box3 | 0.174 | 0.256 | 0.327 | 0.058 |  | 0.025 | 0.022 | 0.022
placing_o_box | 0.571 | 0.330 | 0.267 | 0.255 |  | 0.127 | 0.172 | 0.117
removing_no_box | 0.041 | 0.136 | 0.016 | 0.016 |  | 0.016 | 0.015 | 0.015
removing_no_box2 | 0.111 | 0.129 | 0.022 | 0.021 |  | 0.020 | 0.019 | 0.021
removing_o_box | 0.222 | 0.334 | 0.362 | 0.291 |  | 0.189 | 0.177 | 0.181
synchronous | 0.441 | 0.446 | 0.977 | 0.015 |  | 0.006 | 0.007 | 0.006
synchronous2 | 0.022 | 0.027 | 0.887 | 0.009 |  | 0.012 | 0.006 | 0.011
Mean/Max | 0.232/0.571 | 0.412/3.586 | 0.428/1.217 | 0.095/0.575 |  | 0.061/0.236 | 0.052/0.177 |  0.051/0.181

###  IV-A Datasets

To assess DynaMoN, we compare both the camera localization and the novel view
synthesis part with state-of-the-art approaches on three challenging datasets.

TUM RGB-D - Dynamic subset [13]: The TUM RGB-D dataset has a resolution of
$640\times 480$. While the dataset mainly focuses on static parts, five
sequences were rated _slightly dynamic_ and four _highly dynamic_.

BONN RGB-D Dynamic [14]: The BONN RGB-D Dynamic consists of 24 dynamic
sequences and 2 static sequences. The number of recordings per scene varies.
The images have a resolution of $640\times 480$.

DyCheck’s iPhone [15]: The iPhone dataset consists of 14 scenes, seven
captured with multi cameras and another seven scenes captured with a single
camera. The sequences are taken with an iPhone including a lidar scanner for
depth. The resolution of the images is $720\times 960$.

For the novel view synthesis results we use every 8th frame of the TUM RGB-D
dataset and BONN RGB-D Dynamic dataset for testing. On the iPhone dataset we
follow the official evaluation split.

###  IV-B Implementation Details

We implemented our approach in Python using PyTorch [41] as the deep learning
framework. While we use the reported image sizes for the novel view synthesis
experiments, we use half of the resolution for the camera localization to be
consistent with existent trajectory evaluations of [18]. For our motion mask
module we introduce a threshold to avoid false positives. We set the initial
threshold to 0.95 and increase it to 0.98 throughout the motion mask
refinement. For cases where the camera motion is higher than in the training
sequences, the motion segmentation module results in an increase of false
positives, as a counter-measurement we introduce a thresholding for the number
of pixels that contain motion. Consequently, masks exceeding this threshold
are discarded. This ensures that there is always enough pixels for the dense
bundle adjustment.

For the loss functions we set $w$ for $L_{NeRF}$ to 0.1, $\lambda_{ts}$ to 20
and $\lambda_{TV}$ initially to 0.005.

To compare our camera pose retrieval with the classic way of using COLMAP, we
followed the implementation of InstantNGP [12] to retrieve the COLMAP ground
truth.

Besides the COLMAP comparison, we compare our approach with RoDynRF [33]. For
the TUM RGB-D dataset, we train their approach on a NVIDIA RTX 3090 with 24GB
of VRAM. We follow their training guideline and do not employ prior pose
initialization.

TABLE IV: Novel view synthesis results on TUM RGB-D [13]. The - denotes that
the camera pose could not be regressed.

Sequences | PSNR$\uparrow$/SSIM$\uparrow$
---|---
RoDynRF [33] |  | Ours
---
(COLMAP)
| Ours
---
(SLAM)
slightly dynamic | fr2/desk-person | 5.87 / 0.018 | - | 23.04 / 0.680
fr3/sitting-static | 9.43 / 0.151 | - | 28.55 / 0.899
fr3/sitting-xyz | 11.26 / 0.213 | 19.98 / 0.589 | 25.90 / 0.848
fr3/sitting-rpy | 8.94 / 0.197 | 16.63 / 0.472 | 25.76 / 0.843
fr3/sitting-halfsphere | 8.07 / 0.216 | 24.25 / 0.793 | 24.05 / 0.792
highly dynamic | fr3/walking-static | 5.99 / 0.009 | 20.27 / 0.650 | 25.84 / 0.849
fr3/walking-xyz | 10.07 / 0.183 | - | 24.05 / 0.797
fr3/walking-rpy | 10.08 / 0.207 | - | 24.18 / 0.790
fr3/walking-halfsphere | 9.60 / 0.253 | 22.01 / 0.692 | 24.05 / 0.781
Mean |  | 8.81 / 0.161 | - | 25.05 / 0.809

###  IV-C Evaluation Metrics

To evaluate our approach, we consider two aspects. First, we consider the
translational trajectory error using RMS in meters. Second, we analyze the
rendering quality of the novel dynamic views by reporting PSNR and SSIM [42]
values.

TABLE V: Novel view synthesis results on BONN RGB-D Dynamic [14]. The - denotes that the camera pose could not be regressed. Sequences | PSNR$\uparrow$/SSIM$\uparrow$
---|---
| Ours
---
(COLMAP)
| Ours
---
(SLAM)
balloon | - | 29.83 / 0.868
balloon2 | - | 27.61 / 0.841
balloon_tracking | - | 29.27 / 0.847
balloon_tracking2 | - | 29.68 / 0.860
crowd | - | 27.92 / 0.831
crowd2 | - | 26.68 / 0.825
crowd3 | - | 26.57 / 0.815
kidnapping_box | - | 30.43 / 0.870
kidnapping_box2 | 30.87 / 0.881 | 30.28 / 0.860
moving_no_box | 30.67 / 0.881 | 30.32 / 0.866
moving_no_box2 | 31.14 / 0.890 | 30.57 / 0.874
moving_o_box | 29.21 / 0.889 | 30.26 / 0.884
moving_o_box2 | - | 29.82 / 0.875
person_tracking | 26.57 / 0.808 | 27.78 / 0.843
person_tracking2 | - | 27.47 / 0.837
placing_no_box | 25.73 / 0.780 | 30.94 / 0.869
placing_no_box2 | - | 31.18 / 0.885
placing_no_box3 | - | 30.79 / 0.879
placing_o_box | - | 30.42 / 0.886
removing_no_box | - | 30.66 / 0.879
removing_no_box2 | 31.13 / 0.887 | 30.73 / 0.871
removing_o_box | 31.13 / 0.887 | 30.30 / 0.880
synchronous | - | 28.42 / 0.835
synchronous2 | - | 28.85 / 0.887
Mean | - | 29.45 / 0.861

###  IV-D Camera Localization Quality

Our results on all datasets show a lower trajectory error compared to the
state-of-the-art approaches. For the TUM RGB-D dataset, the camera trajectory
error is denoted in Table I. We compare RGB-D-based and monocular SLAM
approaches with our proposed camera retrieval method using only motion masks
(MS) as well as using motion and semantic segmentation masks in combination
(MS&SS). For the mean value of the trajectory error, our approach performs
equally compared to DROID-SLAM. However, for the maximum error value our
approach performs best among the compared results.

Additionally, we evaluate our approach on the BONN RGB-D Dynamic dataset in
Table III. This dataset contains more dynamic scenes compared to the dynamic
sequences of the TUM RGB-D dataset. On this dataset, our approach outperforms
the state-of-the-art approaches in terms of mean and max trajectory error.

Testing our approach on camera localization datasets we compare our method
with DROID-SLAM and COLMAP on the DyCheck’s iPhone dataset [15], see Table II.
DynaMoN outperforms DROID-SLAM on the iPhone dataset. COLMAP fails to generate
camera poses for the majority of the sequences.

TABLE VI: Novel view synthesis results on the iPhone dataset [15] Sequences | PSNR$\uparrow$/SSIM$\uparrow$
---|---
RoDynRF [33] |  | Ours
---
(SLAM)
Apple | 18.73 / 0.722 | 15.92 / 0.694
Block |  18.73 / 0.634 | 17.17 / 0.655
Paper Windmill | 16.71 / 0.321 | 18.28 / 0.380
Space Out | 18.56 / 0.594 | 17.08 / 0.590
Spin |  17.41 / 0.484 | 15.94 / 0.544
Teddy |  14.33 / 0.536 | 13.56 / 0.544
Wheel |  15.20 / 0.449 | 14.81 / 0.532
Mean |  17.09 / 0.534 | 16.11 / 0.563
Figure 3: Novel views from DynaMoN (Ours) compared to the Ground Truth and
COLMAP-based dynamic NeRF on the BONN RGB-D dynamic dataset (left). On the TUM
RGB-D dataset we compare DynaMoN (Ours) and the ground truth.

###  IV-E Novel View Synthesis Quality

To evaluate our approach from a NeRF perspective we consider two aspects.
First, the classic way of generating camera ground truth when using NeRF is
the use of COLMAP. Thus, we compare our DynaMoN using the same dynamic NeRF
but with camera poses from COLMAP. Second, we compare our approach with
RoDynRF [33] which regresses the camera pose along with a dynamic NeRF.

The results for the TUM RGB-D dataset, denoted in Table IV show that DynaMoN
leads to a higher novel view synthesis quality compared to RoDynRF or when
using the classic COLMAP for the camera poses. Examples of our renderings can
be found in Fig. 3.

On the BONN RGB-D dataset, see Table V, COLMAP is challenged on several
sequences. First, our DynaMoN can retrieve all camera poses, see Table III,
and second, our approach performs well on all sequences of the BONN RGB-D
dataset for the novel view synthesis. Examples can be seen in Fig. 3. However,
if COLMAP was successful, the accurate camera poses also lead to decent novel
view synthesis results.

In addition to the SLAM datasets, we compare our approach with the most
similar NeRF approach RoDynRF [33] on the multi-camera scenes of the DyCheck’s
iPhone dataset, see Table VI. We achieve similar but slightly worse
performance on this dataset in PSNR. However, for SSIM our approach shows
better results.

##  V DISCUSSION

In comparison to more sparse reconstructions, our approach addresses camera
localization and novel view synthesis in combination. While this leads to more
pleasing 3D visualizations and novel views, the computing time is higher
compared to traditional camera localization and 3D representation methods.

Our evaluation considers two aspects, the robustness of the camera pose
estimation and the novel view synthesis results. For the camera pose
estimation, our approach shows an improved performance compared to the state-
of-the-art approaches. Furthermore, our used motion masks already provide an
improved performance. However, on scenes with large motions the combination
with semantic segmentation masks provide an improved camera localization
result. Moreover, we demonstrate that our approach also achieve a robust
performance on scenes with less camera motion, normally used for novel view
synthesis, see Table II.

According to the results in Table IV, existing camera regression and novel
view synthesis results are challenged by more dynamic camera motions and
dynamic scenes.

Although our approach has demonstrated competitive results on the TUM RGB-D
and BONN RGB-D datasets, it faced challenges when applied to the multi-camera
iPhone dataset due to its narrower field of view. Additionally, on the BONN
RGB-D Dynamic dataset, our approach struggled with handling strong light
influences near the camera, as illustrated in Fig. 4. Future work could focus
on enhancing motion mask prediction by eliminating the need for predefined
classes.

Figure 4: Failure cases. Our DynaMoN (center and right bottom) is challenged
by dynamics with a high light influence.

##  VI CONCLUSION

We present DynaMoN, a motion aware fast and robust camera localization
approach for novel view synthesis. DynaMoN can handle not only the motion of
known objects using semantic segmentation masks but also that of unknown
objects using a motion segmentation mask. Furthermore, it retrieves the camera
poses faster and more robust compared to classical SfM approaches enabling a
more accurate 4D scene representation. Compared to the state-of-the-art,
DynaMoN outperforms other dynamic camera localization approaches and shows
better results for novel view synthesis.

## Acknowledgment

We thank d.hip for providing a campus stipend and gratefully acknowledge their
support.

## References

  * [1] M. Broxton, J. Flynn, R. Overbeck, D. Erickson, P. Hedman, M. Duvall, J. Dourgarian, J. Busch, M. Whalen, and P. Debevec, “Immersive light field video with a layered mesh representation,” _ACM Trans. Graph._ , vol. 39, no. 4, aug 2020. [Online]. Available: https://doi.org/10.1145/3386569.3392485
  * [2] S. Orts-Escolano, C. Rhemann, S. Fanello, W. Chang, A. Kowdle, Y. Degtyarev, D. Kim, P. L. Davidson, S. Khamis, M. Dou _et al._ , “Holoportation: Virtual 3d teleportation in real-time,” in _Proceedings of the 29th annual symposium on user interface software and technology_ , 2016, pp. 741–754.
  * [3] A. Rosinol, J. J. Leonard, and L. Carlone, “Nerf-slam: Real-time dense monocular slam with neural radiance fields,” _arXiv preprint arXiv:2210.13641_ , 2022.
  * [4] A. Cao and J. Johnson, “Hexplane: A fast representation for dynamic scenes,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 130–141.
  * [5] R. Shao, Z. Zheng, H. Tu, B. Liu, H. Zhang, and Y. Liu, “Tensor4d: Efficient neural 4d decomposition for high-fidelity dynamic reconstruction and rendering,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 16 632–16 642.
  * [6] L. Song, A. Chen, Z. Li, Z. Chen, L. Chen, J. Yuan, Y. Xu, and A. Geiger, “Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields,” _IEEE Transactions on Visualization and Computer Graphics_ , pp. 2732–2742, 2023.
  * [7] J. Fang, T. Yi, X. Wang, L. Xie, X. Zhang, W. Liu, M. Nießner, and Q. Tian, “Fast dynamic radiance fields with time-aware neural voxels,” in _SIGGRAPH Asia 2022 Conference Papers_ , 2022.
  * [8] J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 4104–4113.
  * [9] Z. Zhu, S. Peng, V. Larsson, W. Xu, H. Bao, Z. Cui, M. R. Oswald, and M. Pollefeys, “Nice-slam: Neural implicit scalable encoding for slam,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 12 786–12 796.
  * [10] Z. Zhu, S. Peng, V. Larsson, Z. Cui, M. R. Oswald, A. Geiger, and M. Pollefeys, “Nicer-slam: Neural implicit scene encoding for rgb slam,” _arXiv preprint arXiv:2302.03594_ , 2023.
  * [11] C.-M. Chung, Y.-C. Tseng, Y.-C. Hsu, X.-Q. Shi, Y.-H. Hua, J.-F. Yeh, W.-C. Chen, Y.-T. Chen, and W. H. Hsu, “Orbeez-slam: A real-time monocular visual slam with orb features and nerf-realized mapping,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 9400–9406.
  * [12] T. Müller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” _ACM Transactions on Graphics (ToG)_ , vol. 41, no. 4, pp. 1–15, 2022.
  * [13] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in _Proc. of the International Conference on Intelligent Robot Systems (IROS)_ , Oct. 2012.
  * [14] E. Palazzolo, J. Behley, P. Lottes, P. Giguère, and C. Stachniss, “ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals,” in _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2019, pp. 7855–7862.
  * [15] H. Gao, R. Li, S. Tulsiani, B. Russell, and A. Kanazawa, “Monocular dynamic view synthesis: A reality check,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 33 768–33 780, 2022.
  * [16] R. Mur-Artal, J. M. M. Montiel, and J. D. Tardos, “Orb-slam: a versatile and accurate monocular slam system,” _IEEE transactions on robotics_ , vol. 31, no. 5, pp. 1147–1163, 2015.
  * [17] C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. Montiel, and J. D. Tardós, “Orb-slam3: An accurate open-source library for visual, visual–inertial, and multimap slam,” _IEEE Transactions on Robotics_ , vol. 37, no. 6, pp. 1874–1890, 2021.
  * [18] Z. Teed and J. Deng, “Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras,” _Advances in neural information processing systems_ , vol. 34, pp. 16 558–16 569, 2021.
  * [19] R. Mur-Artal and J. D. Tardós, “Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras,” _IEEE transactions on robotics_ , vol. 33, no. 5, pp. 1255–1262, 2017.
  * [20] C. Yu, Z. Liu, X.-J. Liu, F. Xie, Y. Yang, Q. Wei, and Q. Fei, “Ds-slam: A semantic visual slam towards dynamic environments,” in _2018 IEEE/RSJ international conference on intelligent robots and systems (IROS)_. IEEE, 2018, pp. 1168–1174.
  * [21] M. Henein, J. Zhang, R. Mahony, and V. Ila, “Dynamic slam: The need for speed,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 2123–2129.
  * [22] W. Dai, Y. Zhang, P. Li, Z. Fang, and S. Scherer, “Rgb-d slam in dynamic environments using point correlations,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 1, pp. 373–389, 2020.
  * [23] W. Ye, X. Yu, X. Lan, Y. Ming, J. Li, H. Bao, Z. Cui, and G. Zhang, “Deflowslam: Self-supervised scene motion decomposition for dynamic dense slam,” _arXiv preprint arXiv:2207.08794_ , 2022.
  * [24] M. Runz, M. Buffier, and L. Agapito, “Maskfusion: Real-time recognition, tracking and reconstruction of multiple moving objects,” in _2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)_. IEEE, 2018, pp. 10–20.
  * [25] B. Bescos, J. M. Fácil, J. Civera, and J. Neira, “Dynaslam: Tracking, mapping, and inpainting in dynamic scenes,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 4076–4083, 2018.
  * [26] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing Scenes as Neural Radiance Fields for View Synthesis,” in _European Conference on Computer Vision_. Springer, 2020, pp. 405–421.
  * [27] E. Sucar, S. Liu, J. Ortiz, and A. J. Davison, “imap: Implicit mapping and positioning in real-time,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 6229–6238.
  * [28] Z. Wang, S. Wu, W. Xie, M. Chen, and V. A. Prisacariu, “NeRF–: Neural radiance fields without known camera parameters,” _arXiv preprint arXiv:2102.07064_ , 2021.
  * [29] Q. Meng, A. Chen, H. Luo, M. Wu, H. Su, L. Xu, X. He, and J. Yu, “GNeRF: GAN-Based Neural Radiance Field Without Posed Camera,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , Oct. 2021, pp. 6351–6361.
  * [30] L. Yen-Chen, P. Florence, J. T. Barron, A. Rodriguez, P. Isola, and T.-Y. Lin, “iNeRF: Inverting Neural Radiance Fields for Pose Estimation,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2021, pp. 1323–1330.
  * [31] W. Bian, Z. Wang, K. Li, J. Bian, and V. A. Prisacariu, “NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior,” in _CVPR_.
  * [32] H. Schieber, F. Deuser, B. Egger, N. Oswald, and D. Roth, “Nerftrinsic four: An end-to-end trainable nerf jointly optimizing diverse intrinsic and extrinsic camera parameters,” 2023.
  * [33] Y.-L. Liu, C. Gao, A. Meuleman, H.-Y. Tseng, A. Saraf, C. Kim, Y.-Y. Chuang, J. Kopf, and J.-B. Huang, “Robust dynamic radiance fields,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 13–23.
  * [34] Z. Teed and J. Deng, “Raft: Recurrent all-pairs field transforms for optical flow,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16_. Springer, 2020, pp. 402–419.
  * [35] L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” _arXiv preprint arXiv:1706.05587_ , 2017.
  * [36] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [37] S. Shen, Y. Cai, W. Wang, and S. Scherer, “Dytanvo: Joint refinement of visual odometry and motion segmentation in dynamic environments,” in _2023 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2023, pp. 4048–4055.
  * [38] A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su, “Tensorf: Tensorial radiance fields,” in _European Conference on Computer Vision_. Springer, 2022, pp. 333–350.
  * [39] C. Kerl, J. Sturm, and D. Cremers, “Robust odometry estimation for rgb-d cameras,” in _2013 IEEE international conference on robotics and automation_. IEEE, 2013, pp. 3748–3754.
  * [40] R. Scona, M. Jaimez, Y. R. Petillot, M. Fallon, and D. Cremers, “Staticfusion: Background reconstruction for dense rgb-d slam in dynamic environments,” in _2018 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2018, pp. 3849–3856.
  * [41] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
  * [42] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE transactions on image processing_ , vol. 13, no. 4, pp. 600–612, 2004, publisher: IEEE.


N})\sqsubseteq(T(\langle c_{i}\rangle_{i\in
N}))^{*}R^{p,\mathbf{\Omega}}\circ\mathrm{Sp}^{A}(T\circ
F^{N},\sigma\mathbin{\bullet}(T\circ\tau^{N}))\circ
L^{p^{N},\langle\mathbf{\Omega}\rangle_{N}}(\langle\bigsqcup_{\langle
P_{i}\rangle_{i\in N}\in\mathcal{V}}P_{i}\rangle_{i\in N})$
$\displaystyle\phantom{\text{if and only if }\bigsqcup_{\langle
P_{i}\rangle_{i\in N}\in\mathcal{V}}[T,\sigma]^{\mathbf{\Omega}}(\langle
P_{i}\rangle_{i\in N})}\,=\bigsqcap_{a\in A,\langle
k_{i}\in\mathbb{E}(\bigsqcup_{\langle
P_{i}\rangle_{i}\in\mathcal{V}}P_{i},\mathbf{\Omega})\rangle_{i\in
N}}\big{(}\sigma\circ T(\langle\tau_{a}\circ Fpk_{i}\circ c_{i}\rangle_{i\in
N})\big{)}^{*}\mathbf{\Omega}$ if and only if $\displaystyle\forall\langle
P_{i}\rangle_{i\in N}\in\mathcal{V}.~{}\forall a\in A.~{}\forall\langle
k_{i}\colon\bigsqcup_{\langle
P_{i}^{\prime}\rangle_{i}\in\mathcal{V}}P_{i}^{\prime}\dot{\rightarrow}\mathbf{\Omega}\rangle_{i\in
N}.~{}[T,\sigma]^{\mathbf{\Omega}}(\langle P_{i}\rangle_{i\in
N})\sqsubseteq\big{(}\sigma\circ T(\langle\tau_{a}\circ Fk_{i}\circ
c_{i}\rangle_{i\in N})\big{)}^{*}\mathbf{\Omega}$ $\displaystyle\text{if and
only if }\forall\langle P_{i}\rangle_{i\in N}\in\mathcal{V}.~{}\forall a\in
A.~{}\forall\langle k_{i}\in\mathbb{C}(X_{i},\Omega)\rangle_{i\in N}.$
$\displaystyle\phantom{\text{if and only if }}\bigl{(}\forall\langle
P_{i}^{\prime}\rangle_{i\in N}\in\mathcal{V}.~{}\forall i\in N~{}k_{i}\colon
P_{i}^{\prime}\dot{\rightarrow}\mathbf{\Omega}\Rightarrow[T,\sigma]^{\mathbf{\Omega}}(\langle
P_{i}\rangle_{i\in N})\sqsubseteq\big{(}\sigma\circ T(\langle\tau_{a}\circ
Fk_{i}\circ c_{i}\rangle_{i\in N})\big{)}^{*}\mathbf{\Omega}\bigr{)}$
$\displaystyle\text{if and only if }\forall\langle P_{i}\rangle_{i\in
N}\in\mathcal{V}.~{}\forall a\in A.~{}\forall\langle
k_{i}\in\mathbb{C}(X_{i},\Omega)\rangle_{i\in N}.$
$\displaystyle\phantom{\text{if and only if
}}\bigl{(}[T,\sigma]^{\mathbf{\Omega}}(\langle P_{i}\rangle_{i\in
N})\not\sqsubseteq\big{(}\sigma\circ T(\langle\tau_{a}\circ Fk_{i}\circ
c_{i}\rangle_{i\in N})\big{)}^{*}\mathbf{\Omega}\Rightarrow\exists\langle
P_{i}^{\prime}\rangle_{i\in N}\in\mathcal{V}.~{}\exists i\in N~{}k_{i}\colon
P_{i}^{\prime}\dot{\not\rightarrow}\mathbf{\Omega}\bigr{)}.$

∎

######  Lemma A.10.

Let $\mathcal{V}\subseteq\prod_{i\in N}\mathrm{Obj}(\mathbb{E}_{X_{i}})$ be a
set. If a set $\mathcal{V}$ is an invariant of a codensity game
$\mathcal{G}_{c}$ for $\mathbf{D}$ then so is $\overline{\mathcal{V}}$.

###### Proof.

Suppose $\mathcal{V}$ is an invariant of $\mathcal{G}_{c}$ for $\mathbf{D}$.
To show that $\overline{\mathcal{V}}$ is an invariant of $\mathcal{G}_{c}$ for
$\mathbf{D}$, we need to check: for each $(a\in A,k\colon X\to\Omega(a))$ such
that $\tau_{a}\circ Fk\circ
c\colon\bigsqcup_{P\in\mathcal{V}}P\dot{\not{\rightarrow}}\mathbf{\Omega}(a)$,
there exists $P^{\prime}\in\mathcal{V}$ such that $k\colon
P^{\prime}\dot{\not{\rightarrow}}\Omega(a)$. It is true because $\tau_{a}\circ
Fk\circ
c\colon\bigsqcup_{P\in\mathcal{V}}P\dot{\not{\rightarrow}}\mathbf{\Omega}(a)$
implies $\exists~{}P^{\prime}\in\mathcal{V}.~{}\tau_{a}\circ Fk\circ c\colon
P^{\prime}\dot{\not{\rightarrow}}\mathbf{\Omega}(a)$ and $\mathcal{V}$ is an
invariant. ∎

###### Proof of Proposition 8.7.

Assume that each $\mathcal{V}_{i}\subseteq\mathbb{E}_{X_{i}}$ is an invariant
of $\mathcal{G}_{c_{i}}$ for $\mathbf{D}$. For any $a\in A$ and
$\left({k_{i}\in\mathbb{C}(X_{i},\Omega)}\right)_{i\in N}$,

$\displaystyle\begin{split}[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{(\tau_{a}\circ
Fk_{i}\circ
c_{i})^{*}\mathbf{\Omega}(a)}})&=\bigsqcap_{\begin{subarray}{c}a^{\prime}\in
A,\left({k^{\prime}_{i}\colon(\tau_{a}\circ Fk_{i}\circ
c_{i})^{*}\mathbf{\Omega}\to\mathbf{\Omega}}\right)_{i\in
N}\end{subarray}}\big{(}\sigma_{a^{\prime}}\circ
T(\overrightarrow{{pk^{\prime}_{i}}})\big{)}^{*}\mathbf{\Omega}(a^{\prime})\\\
&\sqsubseteq\big{(}\sigma_{a}\circ T(\overrightarrow{{\tau_{a}\circ
Fk_{i}\circ c_{i}}})\big{)}^{*}\mathbf{\Omega}(a)\end{split}$ (10)

For arbitrary $\overrightarrow{{P_{i}}}\in\prod_{i\in N}\mathcal{V}_{i}$,
$a\in A$, and $\left({k_{i}\in\mathbb{C}(X_{i},\Omega)}\right)_{i\in N}$, if
$[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{P_{i}}})\not\sqsubseteq\big{(}\sigma_{a}\circ
T(\overrightarrow{{\tau_{a}\circ Fk_{i}\circ
c_{i}}})\big{)}^{*}\mathbf{\Omega}(a)$, we have $\exists i\in
N.~{}P_{i}\not\sqsubseteq(\tau_{a}\circ Fk_{i}\circ
c_{i})^{*}\mathbf{\Omega}(a)$ by (10). Since $\mathcal{V}_{i}$ is an invariant
for $\mathbf{D}$, $\exists P^{\prime}_{i}\in\mathcal{V}_{i}.~{}k_{i}\colon
P^{\prime}_{i}\dot{\not\rightarrow}\mathbf{\Omega}(a)$. Thus we have
$\overrightarrow{{P^{\prime}_{i}}}\in\mathcal{V}$ such that $\exists i\in
N~{}k_{i}\colon P_{i}^{\prime}\dot{\not\rightarrow}\mathbf{\Omega}(a)$. ∎

###### Proof of Theorem 8.9.

(1) If
$\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{P_{i}}})$
is a $[F,\tau]^{\mathbf{\Omega}}$-bisimulation on
${T}_{\lambda}(\overrightarrow{{c_{i}}})$, then $\mathcal{V}$ satisfies the
inequality in Lemma A.9 because
$\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{P_{i}}})\sqsubseteq({T}_{\lambda}(\overrightarrow{{c_{i}}}))^{*}[F,\tau]^{\mathbf{\Omega}}\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{P_{i}}})\sqsubseteq({T}_{\lambda}(\overrightarrow{{c_{i}}}))^{*}[F,\tau]^{\mathbf{\Omega}}[T,\sigma]^{\mathbf{\Omega}}(\left({\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}P_{i}}\right)_{i\in
N})$. Therefore $\mathcal{V}$ is an invariant by Lemma A.9.

(2) Since
$\left({\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}P_{i}}\right)_{i\in
N}\in\overline{\mathcal{V}}$ and
$[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{P_{i}}})\sqsubseteq[T,\sigma]^{\mathbf{\Omega}}(\left({\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}P_{i}}\right)_{i\in
N})$ for each $\overrightarrow{{P_{i}}}\in\overline{\mathcal{V}}$,
$\bigsqcup_{\overrightarrow{{P_{i}}}\in\overline{\mathcal{V}}}[T,\sigma]^{\mathbf{\Omega}}(\overrightarrow{{P_{i}}})=[T,\sigma]^{\mathbf{\Omega}}(\left({\bigsqcup_{\overrightarrow{{P_{i}}}\in\mathcal{V}}P_{i}}\right)_{i\in
N})=[T,\sigma]^{\mathbf{\Omega}}(\left({\bigsqcup_{\overrightarrow{{P_{i}}}\in\overline{\mathcal{V}}}P_{i}}\right)_{i\in
N})$. Lemma A.9 concludes the proof. ∎


$\displaystyle\leq\bigg{(}\int_{\mathbb{R}^{2d}}|y-z|^{r^{\prime}}\,m(\mathop{}\\!\mathrm{d}y,\mathop{}\\!\mathrm{d}z)\bigg{)}^{1/r^{\prime}}\,\bigg{(}\int_{\mathbb{R}^{2d}}|MDb(x-y)+MDb(x-z)|^{r}\,m(\mathop{}\\!\mathrm{d}y,\mathop{}\\!\mathrm{d}z)\bigg{)}^{1/r}.$

Moreover we have

$\displaystyle\int_{\mathbb{R}^{d}}|MDb(x-y)|^{r}\,m(\mathop{}\\!\mathrm{d}y,\mathop{}\\!\mathrm{d}z)=\int_{\mathbb{R}^{d}}|MDb(x-y)|^{r}\mu(\mathrm{d}y)\leq\|MDb\|_{L^{rq^{\prime}}}^{r}\|\mu\|_{L^{q}}\lesssim\|Db\|_{L^{rq^{\prime}}}^{r}\|\mu\|_{L^{q}};$

since $rq^{\prime}=p$, applying a similar estimate to $MDb(x-z)$ we obtain
that

$\sup_{x\in\mathbb{R}^{d}}|b\ast(\mu-\nu)(x)|\lesssim\|b\|_{W^{1,p}}\,\big{(}\|\mu\|_{L^{q}}^{1/r}+\|\nu\|_{L^{q}}^{1/r}\big{)}\,\bigg{(}\int_{\mathbb{R}^{2d}}|y-z|^{r^{\prime}}\,m(\mathop{}\\!\mathrm{d}y,\mathop{}\\!\mathrm{d}z)\bigg{)}^{1/r^{\prime}}.$

Minimizing over all possible $m\in\Pi(\mu,\nu)$ we obtain the conclusion. ∎

Inequality (B.1) is an example of a family of possibly asymmetric inequalities
involving $Mb$, see [11] for a general discussion. It was shown in [12, Lemma
5.1] that, under the stricter requirement that $b\in
W^{1,p}(\mathbb{R}^{d};\mathbb{R}^{d})$ for $p>d$, there exists a function
$g\in L^{p}(\mathbb{R}^{d})$ (which can be taken as
$g=c_{d,p}\,(M|Db|^{\tilde{p}})^{1/\tilde{p}}$ for any $\tilde{p}\in(d,p)$)
and a Lebesgue-negligible set $N\subset\mathbb{R}^{d}$ such that
$\|g\|_{L^{p}}\lesssim_{d,p}\|f\|_{W^{1,p}}$ and

$|f(x)-f(y)|\leq g(x)\,|x-y|\quad\forall\,x,y\in\mathbb{R}^{d}\setminus N.$
(B.2)

Contrary to Lemma B.2, which requires both $\mu,\nu\in L^{q}_{x}$, the one-
sided inequality (B.2) provides the following estimate.

######  Lemma B.3.

Let $(p,q,r)\in(1,\infty)^{3}$ be parameters satisfying

$p>d,\quad\frac{r}{p}+\frac{1}{q}\leq 1.$ (B.3)

Then there exists a constant $C=C(d,p,q,r)$ such that, for any $b\in
W^{1,p}_{x}$ and $\mu,\nu\in\mathcal{P}$ with $\mu\in L^{q}_{x}$, it holds
that

$|(b\ast\mu)(x)-(b\ast\nu)(y)|\leq
C\|b\|_{W^{1,p}}(1+\|\mu\|_{L^{q}})\,\big{(}|x-y|+d_{r^{\prime}}(\mu,\nu)\big{)}.$

###### Proof.

As in the proof of Lemma B.2, we only need to consider the case
$d_{r^{\prime}}(\mu,\nu)<\infty$ and may take $b$ to be smooth and $r/p+1/q=1$
if needed.

Since $p>d$, by the Sobolev embedding $W^{1,p}\hookrightarrow C^{0}_{b}$, so
$b\ast\mu$ and $b\ast\nu$ are well defined for any
$\mu,\nu\in\mathcal{P}(\mathbb{R}^{d})$; by Young’s inequality

$|(b\ast\mu)(x)-(b\ast\mu)(y)|\leq\|b\ast\mu\|_{W^{1,\infty}}|x-y|\lesssim\|b\|_{W^{1,p}}(1+\|\mu\|_{L^{q}})|x-y|.$

Let $m\in\Pi(\mu,\nu)$, then by virtue of (B.2) (as before the negligible set
$N$ doesn’t play any role), going through similar calculations to Lemma B.2 we
have

$\displaystyle|(b\ast\mu)(y)-(b\ast\nu)(y)|$
$\displaystyle\leq\bigg{(}\int_{\mathbb{R}^{d}}|g|^{r}(y-z_{1})\,\mathop{}\\!\mathrm{d}\mu(z_{1})\bigg{)}^{1/r}\,\bigg{(}\int_{\mathbb{R}^{2d}}|z_{1}-z_{2}|^{r^{\prime}}\,\mathop{}\\!\mathrm{d}m(z_{1},z_{2})\bigg{)}^{1/r^{\prime}}$
$\displaystyle\leq\|g\|_{L^{rq^{\prime}}}\|\mu\|_{L^{q}}^{1/r}\,\bigg{(}\int_{\mathbb{R}^{2d}}|z_{1}-z_{2}|^{r^{\prime}}\,\mathop{}\\!\mathrm{d}m(z_{1},z_{2})\bigg{)}^{1/r^{\prime}}$
$\displaystyle\lesssim\|b\|_{W^{1,p}}(1+\|\mu\|_{L^{q}})\,\bigg{(}\int_{\mathbb{R}^{2d}}|z_{1}-z_{2}|^{r^{\prime}}\,\mathop{}\\!\mathrm{d}m(z_{1},z_{2})\bigg{)}^{1/r^{\prime}}$

where in the last passage we used the fact that $rq^{\prime}=p$ and
$\|g\|_{L^{p}}\lesssim\|b\|_{W^{1,p}}$. Combining the two estimates above and
taking the infimum over all possible couplings $m\in\Pi(\mu,\nu)$ yields the
conclusion. ∎

## References

  * [1] Luigi Ambrosio, Nicola Gigli, and Giuseppe Savare.  Gradient Flows: In Metric Spaces and in the Space of Probability Measures.  Birkhäuser, 2008.
  * [2] Ismaël Bailleul, Rémi Catellier, and François Delarue.  Propagation of chaos for mean field rough differential equations.  The Annals of Probability, 49(2):944–996, 2021.
  * [3] Martin Bauer, Thilo Meyer-Brandis, and Frank Proske.  Strong solutions of mean-field stochastic differential equations with irregular drift.  Electronic Journal of Probability, 23(none):1 – 35, 2018.
  * [4] P. R. Beesack.  On integral inequalities of Bihari type.  Acta Math. Acad. Sci. Hungar., 28(1-2):81–88, 1976.
  * [5] Emmanuel Boissard.  Simple bounds for the convergence of empirical and occupation measures in $1$-Wasserstein distance.  Electronic Journal of Probability, 16:2296–2333, 2011.
  * [6] François Bolley.  Quantitative concentration inequalities on sample path space for mean field interaction.  ESAIM: Probability and Statistics, 14:192–209, 2010.
  * [7] François Bolley, José A Canizo, and José A Carrillo.  Stochastic mean-field limit: non-Lipschitz forces and swarming.  Mathematical Models and Methods in Applied Sciences, 21(11):2179–2210, 2011.
  * [8] François Bolley, Arnaud Guillin, and Cédric Villani.  Quantitative concentration inequalities for empirical measures on non-compact spaces.  Probability Theory and Related Fields, 137(3-4):541–593, 2007.
  * [9] Timothee Bonnefoi, Ajay Chandra, Augustin Moinat, and Hendrik Weber.  A priori bounds for rough differential equations with a non-linear damping term.  arXiv preprint arXiv:2011.06645, 2020.
  * [10] Didier Bresch, Pierre-Emmanuel Jabin, and Zhenfu Wang.  On mean-field limits and quantitative estimates with a large class of singular kernels: Application to the Patlak–Keller–Segel model.  Comptes Rendus Mathematique, 357(9):708–720, 2019.
  * [11] Elia Bruè, Maria Colombo, and Camillo De Lellis.  Positive solutions of transport equations and classical nonuniqueness of characteristic curves.  Archive for Rational Mechanics and Analysis, pages 1–36, 2021.
  * [12] Laura Caravenna and Gianluca Crippa.  A directional Lipschitz extension lemma, with applications to uniqueness and Lagrangianity for the continuity equation.  Communications in Partial Differential Equations, pages 1–33, 2021\.
  * [13] Jose A. Carrillo, Jingwei Hu, Li Wang, and Jeremy Wu.  A particle method for the homogeneous Landau equation.  Journal of Computational Physics: X, 7:100066, 2020.
  * [14] Thomas Cass and Terry Lyons.  Evolving communities with individual preferences.  Proceedings of the London Mathematical Society, 110(1):83–107, 08 2014.
  * [15] Ştefan Cobzaş, Radu Miculescu, and Adriana Nicolae.  Lipschitz functions, volume 2241.  Springer, 2019.
  * [16] Michele Coghi, Jean-Dominique Deuschel, Peter K. Friz, and Mario Maurelli.  Pathwise McKean–Vlasov theory with additive noise.  Annals of Applied Probability, 30(5):2355–2392, 2020.
  * [17] Sonja Cox, Martin Hutzenthaler, and Arnulf Jentzen.  Local Lipschitz continuity in the initial value and strong completeness for nonlinear stochastic differential equations.  arXiv preprint arXiv:1309.5595, 2013.
  * [18] Gianluca Crippa and Camillo De Lellis.  Estimates and regularity results for the DiPerna–Lions flow.  Journal für die reine und angewandte Mathematik, 2008(616):15–46, 2008.
  * [19] P.E. Chaudru de Raynal.  Strong well posedness of McKean–Vlasov stochastic differential equations with Hölder drift.  Stochastic Processes and their Applications, 130(1):79–107, 2020\.
  * [20] Sylvain Delattre, Giambattista Giacomin, and Eric Luçon.  A note on dynamical models on random graphs and Fokker–Planck equations.  Journal of Statistical Physics, 165(4):785–798, 2016.
  * [21] Steffen Dereich, Michael Scheutzow, and Reik Schottstedt.  Constructive quantization: Approximation by empirical measures.  In Annales de l’IHP Probabilités et statistiques, volume 49, pages 1183–1203, 2013.
  * [22] Roland L’vovich Dobrushin.  Vlasov equations.  Functional Analysis and Its Applications, 13(2):115–123, 1979.
  * [23] Gonçalo Dos Reis, William Salkeld, Julian Tugaut, et al.  Freidlin–Wentzell LDP in path space for McKean–Vlasov equations and the functional iterated logarithm law.  The Annals of Applied Probability, 29(3):1487–1540, 2019.
  * [24] Xavier Erny.  Well-posedness and propagation of chaos for McKean–Vlasov equations with jumps and locally Lipschitz coefficients, 2021.
  * [25] Nicolas Fournier and Arnaud Guillin.  On the rate of convergence in Wasserstein distance of the empirical measure.  Probability Theory and Related Fields, 162(3):707–738, 2015.
  * [26] Nicolas Fournier and Arnaud Guillin.  From a Kac-like particle system to the Landau equation for hard potentials and Maxwell molecules.  Ann. Sci. Éc. Norm. Supér. (4), 50(1):157–199, 2017.
  * [27] Nicolas Fournier, Maxime Hauray, and Stéphane Mischler.  Propagation of chaos for the 2D viscous vortex model.  Journal of the European Mathematical Society, 16(7):1423–1466, 2014\.
  * [28] L. Galeati, F. Harang, and A. Mayorcas.  Distribution dependent SDEs driven by additive fBm, 2021.  Work in progress.
  * [29] Jürgen Gärtner.  On the McKean-Vlasov limit for interacting diffusions.  Math. Nachr., 137:197–248, 1988.
  * [30] William Hammersley, David Šiška, and Lukasz Szpruch.  McKean-Vlasov SDEs under measure dependent Lyapunov conditions.  arXiv preprint arXiv:1802.03974, 2018.
  * [31] Maxime Hauray and Pierre Emmanuel Jabin.  N-particles approximation of the Vlasov equations with singular potential.  Archive for Rational Mechanics and Analysis, 183(3):489–524, 2007\.
  * [32] Juha Heinonen.  Lectures on analysis on metric spaces.  Springer Science & Business Media, 2012.
  * [33] Pierre Emmanuel Jabin and Maxime Hauray.  Particles approximations of Vlasov equations with singular forces: Propagation of chaos.  In Annales scientifiques de l’École Normale Supérieure, 2015\.
  * [34] Pierre-Emmanuel Jabin and Zhenfu Wang.  Mean field limit for stochastic particle systems.  In Active Particles, Volume 1, pages 379–402. Springer, 2017.
  * [35] Pierre-Emmanuel Jabin and Zhenfu Wang.  Quantitative estimates of propagation of chaos for stochastic systems with ${W}^{-1,\infty}$ kernels.  Inventiones mathematicae, 214(1):523–591, 2018.
  * [36] Daniel Lacker.  On a strong form of propagation of chaos for McKean–Vlasov equations.  Electronic Communications in Probability, 23, 2018.
  * [37] Jean-Michel Lasry and Pierre-Louis Lions.  Mean field games.  Japanese journal of mathematics, 2(1):229–260, 2007.
  * [38] H. P. Jr. McKean.  A class of Markov processes associated with nonlinear parabolic equations.  Proceedings of the National Academy of Sciences of the United States of America, 56(6):1907–1911, 1966.
  * [39] A. V. Medvedev.  On a concave differentiable majorant of a modulus of continuity.  Real Analysis Exchange, 27(1):123–130, 2001.
  * [40] Sima Mehri and Wilhelm Stannat.  Weak solutions to Vlasov–McKean equations under Lyapunov-type conditions.  Stochastics and Dynamics, 19(06):1950042, 2019.
  * [41] Yuliya S. Mishura and Alexander Yu Veretennikov.  Existence and uniqueness theorems for solutions of McKean–Vlasov stochastic equations.  arXiv preprint arXiv:1603.02212, 2016.
  * [42] W. F. Osgood.  Beweis der Existenz einer Lösung der Differentialgleichung $\frac{{dy}}{{dx}}=f({x,y})$ ohne Hinzunahme der Cauchy-Lipschitz’schen Bedingung.  Monatsh. Math. Phys., 9(1):331–345, 1898.
  * [43] Sebastian Riedel and Michael Scheutzow.  Rough differential equations with unbounded drift term.  Journal of Differential Equations, 262(1):283–312, 2017.
  * [44] Michael Röckner and Xicheng Zhang.  Well-posedness of distribution dependent SDEs with singular drifts.  arXiv preprint arXiv:1809.02216, 2018.
  * [45] Michael Scheutzow.  Uniqueness and non-uniqueness of solutions of Vlasov–McKean equations.  Journal of the Australian Mathematical Society, 43(2):246–256, 1987\.
  * [46] Sylvia Serfaty.  Mean field limits of the Gross-Pitaevskii and parabolic Ginzburg-Landau equations.  J. Amer. Math. Soc, 30(3):713–768, 2017.
  * [47] Sylvia Serfaty.  Mean field limit for Coulomb-type flows.  Duke Mathematical Journal, 169(15):2887–2935, 2020.
  * [48] Elias M. Stein.  Singular integrals and differentiability properties of functions, volume 2.  Princeton university press, 1970.
  * [49] Alain-Sol Sznitman.  Topics in propagation of chaos.  In Ecole d’été de probabilités de Saint-Flour XIX-1989, pages 165–251. Springer, 1991.
  * [50] Hiroshi Tanaka.  Limit theorems for certain diffusion processes with interaction.  In Kiyosi Itô, editor, Stochastic Analysis, volume 32 of North-Holland Mathematical Library, pages 469 – 488. Elsevier, 1984.
  * [51] Cédric Villani.  Optimal transport: old and new, volume 338.  Springer Science & Business Media, 2008.
  * [52] Feng-Yu Wang.  Distribution dependent SDEs for Landau type equations.  Stochastic Processes and their Applications, 128(2):595–621, 2018\.
  * [53] Toshio Yamada and Shinzo Watanabe.  On the uniqueness of solutions of stochastic differential equations.  Journal of Mathematics of Kyoto University, 11(1):155–167, 1971\.


-continued Star Name Teff logg [Fe/H] [ls/Fe] [hs/Fe] [hsls] MBa M${}^{ini}_{AGB}$ $\chi^{2}$ dil N Type Ref (K) (cgs) (M⊙) (M⊙) HD 95193 5000 2.70 0.04 0.61 0.54 -0.070 2.60 3.0 1.16 0.20525 7 m 2, 3 HD 95345 4500 1.80 -0.16 0.01 0.23 0.220 1.50 5.0 0.54 0.11124 5 m 2 HD 100503 4000 2.00 -0.72 1.18 1.67 0.490 - 2.0 6.43 0.00100 12 s 4 HD 102762 4400 1.70 -0.17 0.85 1.51 0.650 2.30 - - - 5 s 2 HD 107264 4500 1.50 -0.19 0.74 1.23 0.480 2.60 - - - 5 s 2 HD 107270 5400 2.70 0.05 0.69 0.32 -0.370 4.00 - - - 5 m 2 HD 107541 5000 3.20 -0.63 1.44 2.07 0.630 0.80 - - - 5 s 2 HD 107574 6400 3.60 -0.55 0.84 1.19 0.350 1.20 2.5 1.66 0.65343 14 s 1 HD 109061 4700 2.00 -0.56 0.56 0.75 0.190 0.90 4.0 0.16 0.19367 5 s 2 HD 110483 4900 2.60 -0.04 0.78 1.17 0.390 1.90 - - - 5 s 2 HD 110591 4700 1.80 -0.56 0.41 0.86 0.450 0.90 2.0 0.38 0.91031 5 s 2 HD 111315 4900 2.00 0.04 0.47 0.62 0.150 3.50 - - - 5 s 2 HD 113195 4700 2.10 -0.15 0.57 0.47 -0.100 1.40 2.0 0.11 0.50865 5 m 2 HD 113291 4700 2.60 -0.02 1.04 1.24 0.200 2.60 - - - 5 s 2 HD 114678 5200 2.80 -0.50 1.04 1.40 0.360 1.90 2.5 0.37 0.35642 5 s 2 HD 115277 4800 2.40 -0.03 0.52 0.29 -0.230 1.80 3.0 0.09 0.50414 5 m 2 HD 116869 4892 2.60 -0.44 0.87 1.19 0.320 1.40 2.5 3.63 0.59315 15 s 1, 2, 4 HD 119185 4800 2.00 -0.43 0.26 0.53 0.270 1.30 2.5 0.05 1.39963 5 m 2 HD 119650 4500 1.60 -0.10 0.29 0.28 -0.010 2.00 4.0 0.19 0.25877 5 m 2 HD 120571 4600 1.70 -0.39 0.42 0.83 0.410 2.00 - - - 5 s 2 HD 120620 4831 3.00 -0.30 1.41 1.66 0.250 0.80 - - - 11 s 2, 4 HD 121447 4000 1.00 -0.90 1.38 2.13 0.750 0.80 - - - - s 4 HD 122687 5000 2.60 -0.07 0.86 1.22 0.350 2.50 - - - 5 s 2 HD 123396 4360 1.40 -0.99 0.64 1.23 0.600 0.80 2.0 3.93 0.54064 14 s 1, 2 HD 123585 6350 4.20 -0.48 1.27 1.64 0.370 1.40 2.5 1.02 0.18789 14 s 1 HD 123701 5000 2.50 -0.44 0.83 1.30 0.470 1.70 2.0 0.64 0.40272 5 s 2 HD 123949 4378 1.80 -0.31 1.08 1.44 0.360 - 2.5 4.02 0.13418 11 s 4 HD 126313 4900 2.20 -0.10 0.76 1.21 0.460 4.00 - - - 5 s 2 HD 130255 4400 1.50 -1.11 0.34 0.42 0.080 - 4.0 0.27 0.47583 5 m 2 HD 131670 4700 2.20 -0.04 0.61 0.74 0.130 1.60 3.0 3.23 0.06881 5 s 2 HD 134698 4500 1.70 -0.52 0.57 0.53 -0.050 0.90 4.0 0.50 0.37469 5 m 2 HD 136636 4900 2.50 -0.04 0.91 1.29 0.390 1.70 - - - 5 s 2 HD 139266 4300 1.50 -0.27 0.78 1.17 0.390 1.60 2.0 2.43 0.20975 5 s 2 HD 139409 4700 2.10 -0.51 0.57 0.46 -0.110 0.80 4.0 0.95 0.43531 5 m 2 HD 142491 4800 2.00 0.02 0.02 0.26 0.240 4.50 - - - 5 m 2 HD 142751 4600 2.00 -0.10 0.72 0.78 0.060 1.80 2.0 0.74 0.18853 5 s 2 HD 143899 5000 2.50 -0.27 0.71 0.84 0.120 1.60 3.0 0.29 0.36915 5 s 2 HD 147136 4900 2.20 -0.01 -0.03 0.25 0.280 2.80 4.0 0.95 0.81585 5 m 2 HD 147609 6350 3.50 -0.28 1.19 1.25 0.060 1.50 2.5 1.41 0.08444 10 s 1, 5 HD 147884 5100 3.00 -0.09 0.90 0.90 0.010 2.50 - - - 5 s 2 HD 148177 4400 1.60 -0.15 0.69 0.78 0.090 2.50 2.0 1.32 0.20523 5 s 2 HD 148892 5000 2.30 -0.15 0.73 0.87 0.140 2.80 - - - 5 s 2 HD 150862 6310 4.60 -0.10 1.00 0.68 -0.320 1.20 3.0 0.75 0.19437 14 s 1 HD 154276 5820 4.30 -0.10 -0.08 0.25 0.330 - 5.0 1.39 0.06317 8 m 5 HD 154430 4200 1.20 -0.36 0.95 1.55 0.600 - 2.0 2.20 0.01517 5 s 2 HD 162806 4500 1.70 -0.26 0.72 1.11 0.390 1.80 2.0 0.28 0.10093 5 s 2 HD 168214 5300 3.20 -0.08 0.99 0.88 -0.110 1.40 - - - 5 s 2 HD 168560 4400 1.60 -0.13 0.42 0.65 0.240 1.70 2.0 1.42 0.45179 5 s 2 HD 168791 4400 1.70 -0.23 0.79 1.07 0.290 2.80 3.0 1.90 0.27214 5 s 2 HD 169106 4900 2.20 0.01 0.41 0.35 -0.050 4.00 - - - 5 m 2 HD 176105 4500 1.60 -0.14 0.46 0.49 0.030 1.90 2.0 0.19 0.56867 5 m 2 HD 177192 4700 1.70 -0.17 0.65 0.48 -0.170 1.20 2.0 0.31 0.44651 5 m 2 HD 178717 3800 1.00 -0.52 0.89 0.89 -0.010 0.80 2.5 5.88 0.76176 12 s 4 HD 179832 4780 2.70 0.23 0.52 0.43 -0.100 - 4.0 43.14 0.19279 11 m 5 HD 180622 4600 2.20 0.03 0.51 0.39 -0.120 1.50 4.0 0.29 0.03012 5 m 2 HD 180996 4900 2.60 0.06 0.55 0.49 -0.060 2.10 3.0 0.72 0.31511 5 m 2 HD 182300 5000 2.70 0.06 0.84 1.10 0.260 2.60 - - - 5 s 2 HD 183915 4500 1.60 -0.39 0.78 1.58 0.800 1.70 2.0 0.07 0.09492 5 s 2

-continued Star Name Teff logg [Fe/H] [ls/Fe] [hs/Fe] [hsls] MBa M${}^{ini}_{AGB}$ $\chi^{2}$ dil N Type Ref (K) (cgs) (M⊙) (M⊙) HD 184001 5000 2.50 -0.21 0.60 0.61 0.010 2.20 2.5 0.04 0.81789 5 s 2 HD 187308 4900 2.50 -0.08 0.65 0.70 0.060 2.10 3.0 1.93 0.08263 5 s 2 HD 187762 4800 2.40 -0.30 0.23 0.54 0.310 1.50 - - - 5 m 2 HD 188985 6090 4.30 -0.30 1.08 1.16 0.070 0.90 2.5 0.71 0.46841 14 s 1 HD 193530 4400 1.60 -0.17 0.66 0.84 0.180 1.80 2.0 1.77 0.17296 5 s 2 HD 196445 4400 1.40 -0.19 0.99 1.42 0.430 1.90 - - - 5 s 2 HD 199435 5000 2.60 -0.39 0.86 1.46 0.590 1.70 3.0 0.02 0.09638 5 s 2 HD 199939 4710 2.40 -0.22 1.39 1.84 0.450 2.30 - - - 11 s 4 HD 200063 4100 1.10 -0.34 0.75 1.05 0.300 1.20 3.0 0.83 0.33877 5 s 2 HD 200995 4600 2.10 -0.03 0.60 0.82 0.220 2.10 3.0 3.55 0.02106 5 s 2 HD 201657 4700 2.20 -0.34 0.85 1.54 0.690 1.30 2.0 0.06 0.09499 5 s 2 HD 201824 4900 2.30 -0.33 0.89 1.47 0.580 1.70 2.0 0.55 0.09220 5 s 2 HD 203137 3609 0.60 0.07 -0.00 0.56 0.560 - 4.0 2.63 0.45154 5 m 3 HD 204075 5300 1.50 0.06 1.10 0.86 -0.240 4.00 - - - 5 s 2 HD 204886 4600 2.10 0.04 0.66 0.94 0.280 2.20 - - - 5 s 2 HD 207277 4600 2.00 -0.13 0.60 0.94 0.340 2.00 - - - 5 s 2 HD 207585 5800 3.80 -0.39 1.29 1.66 0.370 1.00 - - - 9 s 5 HD 210030 4700 1.90 -0.03 0.35 0.34 -0.010 1.60 4.0 0.22 0.18726 6 m 2, 3 HD 210709 4630 2.40 -0.04 0.55 0.70 0.150 1.10 3.0 4.53 0.09912 5 s 1, 2 HD 210910 4570 2.70 0.04 0.04 0.32 0.280 1.30 4.0 2.39 0.81279 14 m 1 HD 210946 4800 2.10 -0.12 0.67 0.69 0.030 1.80 2.0 0.38 0.28203 5 s 2 HD 211173 4900 2.60 -0.17 0.58 0.75 0.170 1.60 2.0 19.22 0.02981 11 s 5 HD 211221 4800 2.30 -0.11 -0.10 0.18 0.280 1.60 6.0 0.68 0.00099 5 m 2 HD 211594 4900 2.40 -0.43 1.19 1.79 0.600 1.30 2.5 2.52 0.04023 5 s 2 HD 211954 4400 1.70 -0.51 1.08 1.71 0.630 - 2.0 0.74 0.03212 5 s 2 HD 212320 4880 2.60 -0.11 0.26 1.15 0.890 2.30 - - - 5 s 3 HD 213084 5000 2.80 -0.09 0.76 0.97 0.210 2.40 - - - 5 s 2 HD 214579 4400 1.60 -0.26 0.64 0.94 0.300 1.30 2.0 0.58 0.42933 5 s 2 HD 214889 4900 2.60 -0.17 0.61 0.59 -0.020 2.10 - - - 5 m 2 HD 215555 5200 3.20 -0.08 0.90 0.79 -0.100 1.30 - - - 5 s 2 HD 216809 4400 1.20 -0.04 0.57 0.49 -0.080 1.70 2.0 1.17 0.22005 5 m 2 HD 217143 4600 2.10 -0.35 0.74 1.28 0.540 1.10 3.0 1.46 0.17751 5 s 2 HD 217447 5000 2.50 -0.17 0.77 0.98 0.210 2.10 - - - 5 s 2 HD 218356 4374 1.50 -0.26 0.42 1.01 0.590 2.40 3.0 1.85 0.56672 5 s 3 HD 219116 5050 2.50 -0.45 0.71 1.32 0.610 2.00 3.0 1.33 0.10027 10 s 5 HD 221879 4300 1.30 -0.10 0.88 0.66 -0.220 1.30 3.0 0.63 0.22572 5 s 2 HD 222349 6130 3.90 -0.63 1.05 1.35 0.300 0.90 2.5 1.83 0.47279 14 s 1 HD 223586 4700 2.90 -0.08 0.85 0.92 0.070 1.00 - - - 5 s 2 HD 223617 4700 2.30 -0.18 0.71 0.77 0.060 1.40 2.0 0.59 0.19449 5 s 2 HD 223938 4970 3.10 -0.13 0.58 0.87 0.300 1.40 2.0 2.88 0.31487 14 s 1, 2 HD 224276 4910 2.60 0.08 0.20 0.76 0.560 3.00 4.0 3.99 0.04711 5 s 3 HD 252117 4600 1.80 -0.14 0.88 1.34 0.460 2.00 - - - 5 s 2 HD 273845 4800 2.50 -0.15 0.82 1.41 0.590 1.70 - - - 5 s 2 HD 288174 4800 2.30 -0.05 0.75 0.90 0.150 2.30 - - - 5 s 2 MFU112 4900 2.40 -0.43 1.12 1.80 0.670 1.00 2.5 3.47 0.06293 5 s 2 MFU214 4800 2.40 0.00 0.45 0.20 -0.260 1.90 3.0 0.17 0.64404 5 m 2 MFU229 4900 2.60 -0.01 0.61 0.49 -0.120 2.20 3.0 0.49 0.26493 5 m 2 Our Work Star Name Teff logg [Fe/H] [ls/Fe] [hs/Fe] [hsls] MBa M${}^{ini}_{AGB}$ $\chi^{2}$ dil N Type Ref (K) (cgs) (M⊙) (M⊙) HD 75 348 4840 2.00 -0.42 1.16 1.75 0.59 1.20 2.5 1.20 0.01118 12 s 6 BD+09 3019 4220 2.00 -0.55 1.49 2.08 0.59 1.00 - - - 10 s 6 HD 238020 5150 2.10 -0.68 -0.04 0.30 0.34 2.20 3.0 2.37 1.26402 10 m 6

Table 14. Initial masses of the AGB companions of CEMP-s stars. Star Name Teff
log$g$ [Fe/H] M${}^{ini}_{AGB}$ $\chi^{2}$ dil. N (K) (cgs) (M⊙) BD+04 2466
5100 1.80 -1.92 3.0 3.61 0.26884 7 BS 16077-0077 5900 3.19 -2.05 2.0 1.89
1.14484 9 CD-27 14351 4320 0.50 -2.71 1.3 8.02 0.24628 8 CD-38 2151 4600 0.90
-2.03 3.0 6.52 0.14247 9 CS 22880-0074 5850 3.80 -1.93 3.0 2.74 0.34538 11 CS
22942-0019 5000 2.40 -2.64 1.3 5.30 0.55686 10 CS 22949-0008 6000 3.74 -2.45
2.0 7.42 0.87952 7 CS 29512-0073 5560 3.44 -2.06 2.0 1.03 0.08517 11 CS
30301-0015 4750 0.80 -2.64 1.5 1.96 1.30710 10 G 24-25 5828 3.86 -1.40 2.0
0.74 0.29380 12 HD 145777 4160 0.90 -2.17 2.0 0.58 0.00111 10 HD 196944 5250
1.80 -2.25 3.0 0.45 0.60809 12 HD 198269 4800 1.30 -1.30 2.5 3.99 0.07653 9 HE
0024-2523 6625 4.30 -2.72 2.0 9.47 0.39594 4 HE 0202-2204 5280 1.65 -1.98 2.0
0.53 0.38639 9 HE 0231-4016 5972 3.59 -2.08 2.0 0.19 0.18554 6 HE 0253-6024
4640 1.20 -2.10 2.5 6.37 0.00567 7 HE 0317-4705 4730 1.30 -2.30 1.5 5.78
0.14257 7 HE 0336+0113 5700 3.50 -2.68 1.5 0.94 0.22047 7 HE 0430-4404 6214
4.27 -2.07 2.0 0.94 0.15989 4 HE 1031-0020 5080 2.20 -2.86 2.0 24.74 1.75142 7
HE 1135+0139 5487 1.80 -2.33 1.5 0.36 0.55062 8 HE 1152-0355 4000 1.00 -1.29
1.5 2.99 0.35113 6 HE 1430-1123 5915 3.75 -2.71 2.0 3.71 0.85928 4 HE
1434-1442 5420 3.15 -2.39 2.0 7.13 1.61944 4 HE 1509-0806 5185 2.50 -2.91 2.0
0.93 0.35547 7 HE 2138-3336 5850 3.60 -2.79 2.0 10.82 0.61524 11 HE 2150-0825
5960 3.67 -1.98 2.0 0.35 0.06557 7 HE 2158-0348 5215 2.50 -2.70 2.0 1.14
0.72601 9 HE 2158-5134 4950 1.90 -3.00 1.3 12.46 0.00100 5 HE 2240-0412 5750
3.50 -1.85 3.0 1.90 0.31705 6 HE 2258-4427 4560 1.00 -2.10 2.5 2.24 0.10845 8
Our Work Star Name Teff log$g$ [Fe/H] M${}^{ini}_{AGB}$ $\chi^{2}$ dil. N (K)
(cgs) (M⊙) HE 0319-0215 4650 0.50 -2.57 1.5 2.44 0.48308 10 HE 0507-1653 4970
2.20 -1.44 2.0 1.85 0.09699 10 HE 0930-0018 4190 2.65 -1.39 1.5 0.71 0.38533 9
HE 1023-1504 4440 0.50 -1.66 2.0 4.45 0.06614 9

Figure 14:  Spectral synthesis plots for a few Fe lines of the spectrum of HE
1023–1504.  Figure 15:  Sample spectra of the Co, Cu and Zn lines of HE
0319–0215. Figure 16:  Sample spectra of the Eu II line at 6645.064 Å in the
stars BD+09 3019, HE 0507–1653 and HE 0930–0018. This line is very weak (not
usable) in the star HE 0930–0018.