# SANPO
A Scene Understanding, Accessibility, Navigation, Pathfinding, Obstacle
Avoidance Dataset

Sagar M. Waghmare
Google ResearchFor correspondence please email at<EMAIL_ADDRESS>Kimberly Wilber11footnotemark: 1
Dave Hawkey11footnotemark: 1
Xuan Yang11footnotemark: 1
Matthew Wilson11footnotemark: 1
Stephanie Debats11footnotemark: 1
Cattalyya Nuengsigkapian11footnotemark: 1
Astuti Sharma11footnotemark: 1
Lars Pandikow
Parallel Domain Huisheng Wang11footnotemark: 1
Hartwig Adam11footnotemark: 1
Mikhail Sirotenko11footnotemark: 1

###### Abstract

We introduce SANPO111https://google-research-
datasets.github.io/sanpo_dataset/, a large-scale egocentric video dataset
focused on dense prediction in outdoor environments. It contains stereo video
sessions collected across diverse outdoor environments, as well as rendered
synthetic video sessions 222Synthetic data was provided by Parallel Domain.
All sessions have (dense) depth and odometry labels. All synthetic sessions
and a subset of real sessions have temporally consistent dense panoptic
segmentation labels. To our knowledge this is the first human egocentric video
dataset with both large scale dense panoptic segmentation and depth
annotations.

In addition to the dataset we also provide zero-shot baselines and SANPO
benchmarks for future research. We hope that the challenging nature of SANPO
will help advance the state-of-the-art in video segmentation, depth
estimation, multi-task visual modeling, and synthetic-to-real domain
adaptation, while enabling human navigation systems.

##  1 Introduction

Figure 1: SANPO is the only human-egocentric dataset with panoptic masks,
multi-view stereo, depth, camera pose, and both real and synthetic data. SANPO
has the largest number of panoptic frames among related work and a respectable
number of depth annotations. (Note: 1: multi-view, 2: partial coverage, 3:
sparse depth)

Egocentric scene understanding is an important research area with many
applications in robotics, autonomous driving, augmented reality, and
accessibility. It includes a range of tasks, such as video semantic and
panoptic segmentation, depth estimation, object tracking among others. To
advance this field, the community needs high-quality, large-scale datasets. In
the last 10 years, growing interest in autonomous driving has resulted in the
creation of several large-scale video datasets Kang et al. (2019); Mao et al.
(2022); Wilson et al. (2023) that have panoptic segmentation masks, depth
maps, camera poses, and other related annotations. However, outside of the
autonomous driving domain, to the best of our knowledge, there is no publicly
available video dataset annotated with both panoptic segmentation and depth
maps. Autonomous driving datasets, though plenty, have limited generalization
to egocentric human scene understanding. Videos taken from the human
perspective have their own challenges, such as unorthodox viewpoints, motion
artifacts, and dynamic or unpredictable interactions between other humans and
objects in the scene. Unlike cars, humans operate in environments that are
more cluttered, unpredictable, and less regulated. We believe that a
comprehensive human egocentric dataset should not only help to build systems
for related applications, but also _serve as a challenging benchmark for the
scene understanding community._

This work introduces SANPO, a dataset built to support research in outdoor
human egocentric scene understanding. Although we focus on human navigation
tasks, SANPO supports a wide variety of dense prediction tasks in outdoor
environments and is challenging enough to be beyond the capabilities of
current models. SANPO includes both real and synthetic data, with 112K and
113K video panoptic masks, respectively. It also includes 617K and 113K of
real and synthetic depth maps, respectively. The dataset was collected in
various locations in the United States and covers different environments with
varying weather conditions, times of day, and types of egomotion. Each real
session also has videos from two stereo cameras, which can help to advance
multi-view methods.

In addition to the dataset, we also set baselines for monocular depth
estimation, semantic and panoptic segmentation, using state-of-the-art models.

##  2 Related Work

Figure 2: Data capture methodology for SANPO. SANPO contains a mix of both
real and synthetic data. The real data is captured from a chest-mounted camera
and a head-mounted camera, while the synthetic data comes from a virtual
environment. Our videos have depth maps and panoptic segmentations.

(a)

(b)

(c)

(d)

Figure 3: SANPO Real Sample. Top row shows a stereo left frame from a session
along with its ML depth and segmentation annotation. Bottom row shows the 3D
scene of the session built using the annotations we provide. Points from
several seconds of video are accumulated and aligned with ICP.

The closest publicly available datasets to ours are SCAND Karnan et al.
(2022), MuSoHu Nguyen et al. (2023), and Ego4D Grauman et al. (2022), which
are collected with a human egocentric perspective. SCAND is an autonomous
robot navigation dataset collected with a front facing stereo camera, among
other sensors, fitted on robots which are teleoperated. MuSoHu is collected
with human ego motion with front facing stereo camera along with Lidar,
microphone array and a $360\text{\,}\mathrm{\SIUnitSymbolDegree}$ camera.
SCAND and MuSoHu provide depth and odometry labels. MuSoHu also exhibits the
camera motion artifacts caused by human motion. Ego4D is large and showcases a
wide variety of activities. But MuSoHu, SCAND, and Ego4D lack semantic
segmentation labels, and the first two are primarily developed for enabling
robot navigation in social environments.

MOTSynth Fabbri et al. (2021) is another dataset that comes relatively close.
It is a synthetic dataset for pedestrian detection and tracking, and it has
both segmentation and depth annotations. However, this dataset has some
limitations: (a) It only includes pedestrian segmentation and tracking
annotations. (b) Only a small portion of the samples provide an egocentric
view similar to what you would expect in egocentric human navigation.

Autonomous navigation is a well researched field Wen and Jo (2022); Shi et al.
(2017) and the literature is teeming with various real-world Qiao et al.
(2020); Kang et al. (2019); Wilson et al. (2023); Karnan et al. (2022); Nguyen
et al. (2023); Cordts et al. (2016); Liao et al. (2022); Lin et al. (2014); Xu
et al. (2018); Caelles et al. (2019); Brostow et al. (2009); Caesar et al.
(2019) and synthetic datasets Mao et al. (2022); Richter et al. (2017); Fabbri
et al. (2021). The majority of the datasets available fall in either self
driving car category Mei et al. (2022); Mao et al. (2022); Wilson et al.
(2023); Cordts et al. (2016); Liao et al. (2022); Richter et al. (2017);
Caesar et al. (2019); Pham et al. (2020) or general purpose scene
understanding category Grauman et al. (2022); Lin et al. (2014); Xu et al.
(2018); Caelles et al. (2019); Brostow et al. (2009). The well known
Cityscapes dataset Cordts et al. (2016); Qiao et al. (2020) is a daytime
stereo video dataset with vehicle ego motion and segmentation & depth labels.
Similarly, Wilson et al. (2023) is self driving car dataset with stereo video
but with only 3D object detection labels. The datasets Richter et al. (2017);
Mei et al. (2022); Mao et al. (2022); Brostow et al. (2009) are also self
driving car video datasets with segmentation labels, except Mao et al. (2022),
which includes 3D object detection labels instead.

Other existing datasets, such as MSCOCO Lin et al. (2014), DAVIS-2017 Caelles
et al. (2019), and YouTube-VOS Xu et al. (2018), are either general-purpose
scene understanding or domain-specific datasets, but they are not specifically
designed for human navigation. MSCOCO Lin et al. (2014) is an image-based
dataset, whereas DAVIS-2017 and Youtube-VOS Caelles et al. (2019); Xu et al.
(2018) are video datasets. All of them are segmentation and/or object
detection datasets but are not relevant to human navigation.

While there are many datasets available (see the supplementary material for an
overview), there is a clear need for a challenging human egocentric dataset
featuring unconstrained environments and comprehensive dense prediction
annotations.

##  3 SANPO Dataset

SANPO dataset consists of two parts - SANPO-Real and SANPO-Synthetic. In this
section we give an overview of both parts and describe how the dataset was
collected and labeled.

###  3.1 SANPO-Real

This dataset consists of 701 sessions recorded from two stereo cameras
simultaneously (thus each session has four RGB streams in total). Each video
is approximately 30 seconds long with a frame rate of 15 frames per second
(FPS). at 15 FPS. 597 sessions are recorded at a resolution of $2208\times
1242$ pixels, and the remainder are recorded at a resolution of $1920\times
1080$ pixels. We provided all videos in a lossless format to help facilitate
stereo vision research. All videos were rectified using ZED software.

Each session is annotated with high-level attributes such as human traffic,
vehicular traffic, number of obstacles, environment type, camera information
and intrinsics, etc.333Please see the appendix for additional details.. Every
stereo camera recording has camera poses provided by the ZED software using
fused IMU and VIO measurements.

Each camera has both a sparse depth map from the ZED SDK and a dense depth map
from CREStereo Li et al. (2022a), a recent ML-based stereo depth model. This
model converts stereo frames to disparity maps444We compute disparity before
blurring the sensitive information because blurry patches can create
inaccurate or misleading results., which we then convert to depth using camera
intrinsics and clip to 0-80 meters. Note that these CREStereo depth maps have
a resolution of 1280$\times$720 pixels; this is smaller than the RGB stream,
but is the maximum resolution that pre-trained CREStereo supports Li et al.
(2022a).

We provide semantic segmentation annotations for a of 237 videos: 146 long-
range ZED 2i videos and not-from-same-session 91 wide-angle ZED M videos. Our
segmentation taxonomy covers 31 categories: 15 _“thing”_ classes and 16
_“stuff”_ classes. We developed this taxonomy with a focus on egocentric scene
understanding, balancing annotation practicality with the desire to be
maximally useful for understanding the navigation environment.

A detailed taxonomy of these categories is provided in the appendix. The
SANPO-Real dataset contains a total of 975,207 masks, including 195,187 human-
annotated masks and 780,020 propagated masks (more details in the following
section).Figure 3 shows an example of a SANPO-Real session.

####  3.1.1 SANPO-Real Data Collection

In order to collect the real data, we designed a custom data collection rig
(see supplementary material for details). Our volunteers wear a head-mounted
ZED-M stereo camera and a chest-mounted ZED-2i stereo camera, as well as a
backpack full of supporting hardware.

The chest-mounted ZED-2i captured 308,957 stereo frames with its 4mm lens,
providing long range depth at a stable mounting point to mitigate motion blur.
The lightweight head-mounted ZED-M provided wide range video and depth for
308,451 stereo frames. A team of volunteers collected data from various
geographic locations across the United States covering different environments,
including urban, suburban, city streets, and parks. Volunteers ran through
different weather conditions (including snow and rain), times of the day
(excluding low light conditions), ground types, obstacles, run/walk speeds,
traffic levels, etc. We asked each volunteer to prefer diverse, dynamic
scenarios and rare instances and events.

(a)

(b)

(c)

(d)

Figure 4:  Temporally Consistent Segmentation Annotation. Top and bottom rows:
Human-annotated segmentation masks for consecutive frames. Middle two rows:
AOT-propagated segmentation masks for the intermediate frames (out of five)
that were skipped during human annotation.

####  3.1.2 Panoptic Segmentation Annotation

Our segmentation annotation protocol is as follows: We divide each video into
30-second sub-videos and annotate every fifth frame for a total of 90 frames
per sub-video. To make process more efficient and less error-prone we use two
techniques. For dealing with a large number of classes we use cascaded
annotation approach. We split all the labels in our taxonomy into five
mutually exclusive subsets of co-occurring labels. A given sub-video is
annotated for each subset in a prescribed order. When annotating a subset, all
the annotations from the previous subset(s) are frozen and shown to the
annotator. This approach helps reduce annotation time while improving boundary
precision. We include all the labels in the last subset to facilitate the
annotation of any missing regions from the previous subsets. We use AOT Yang
et al. (2021) to both propagate masks from the previous frame to the next one
during the annotation process and to infer the segmentation annotations for
the intermediate frames, using the manually annotated preceding and following
frames. This approach ensures that the annotations are temporally consistent
for up to 30 seconds. We also provide information on whether each frame was
annotated by a human or propagated by machine. The figure 4 shows an example
of human annotated preceding and following frames along with the AOT
propagated intermediate frames.

####  3.1.3 Privacy

All data collection is done in compliance with local, state, and city laws.
Every volunteer was able to review each video in the data collection app
before uploading it. All videos are processed to remove personally
identifiable information (PII) such as faces and license plates before sending
them for annotation.

(a)

(b)

(c)

(d)

Figure 5: SANPO-Synthetic Sample. Top row shows a single frame from a
synthetic session along with its depth and segmentation annotation. Bottom row
shows the 3D scene of the session built using the annotations we provide.
Points come from the accumulated depth maps and camera locations across many
frames. Figure 6: Synthetic vs real. A sample of SANPO-Real and SANPO-
Synthetic data. _How quickly can you tell which of these images is synthetic?_
Answer key in base64: ‘c3ludGg6IEFCRUZILCByZWFsOiBDREdJ’

###  3.2 SANPO-Synthetic

Data captured and annotated under real-world conditions unfortunately has
imperfect ground truth labels. These imperfections come from hardware (for
example, motion blur), algorithms (i.e. depth from stereo), and human rating
mistakes. Whereas, synthetic data has near perfect ground truth and can have
any predefined properties. We partnered with _Parallel Domain_ to supplement
SANPO-Real with high-quality synthetic training and evaluation data. The
synthetic environment was optimized to match real-world capture conditions as
closely as possible, including camera parameters, placement and scenery.
_SANPO-Synthetic and SANPO-Real are intended to be drop-in replacements for
each other_ , so researchers can study domain transfer tasks or take advantage
of synthetic data during training without changing many domain-specific
assumptions.

SANPO-Synthetic has 113,794 monocular and single-view video frames across 1961
sessions. 960 sessions are synthesized with a simulated chest-level ZED-2i
camera and the other 1001 are taken with a simulated head-mounted ZED-M. Each
virtual camera parameters match corresponding ZED camera parameters. Frame
rate varies between 5 FPS, 14.28 FPS, and 33.33 FPS. Each synthetic video has
dense depth maps and panoptic segmentation maps using the same taxonomy as
SANPO-Real.

One advantage of synthetic data is its pixel-perfect instance segmentations,
even with many small and distant instances. This is particularly beneficial
for developing a challenging dataset to mimic the complexity of real-world
scenes. _Over half of the synthetic frames contain $\geq$60 unique instance
segmentations_, and a sixth of the data has $\geq$150 instances. Most of these
masks are challenging: _80% of SANPO-Synthetic masks have less than $32^{2}$
pixels_, compared to $8.1\%$ of masks in SANPO-Real. Instance IDs persist
across frames and occlusions, which may be useful for tracking/reacquisition
studies. Overall, there are 393,000 unique instance IDs in the synthetic data.

##  4 Experiments

In this section, we establish SANPO baselines in two evaluation settings:

  1. 1.

Zero shot baseline: In this setting, we evaluate and report the generalization
capability of published model checkpoints to SANPO dataset.

  2. 2.

SANPO benchmark: We report and establish a baseline for a couple of state-of-
the-art architectures on dense prediction tasks using SANPO dataset.

####  4.0.1 Metrics

We report mean intersection over union (mIoU) and panoptic quality (PQ) for
semantic segmentation and panoptic segmentation, respectively, as in Yu et al.
(2023). For depth, we report depth inliers
($\mathbb{E}\left[\max(\frac{y}{y^{\prime}},\frac{y^{\prime}}{y})\leq
1.25\right]$, denoted as $\delta_{\leq 1.25}$) as in Bhat et al. (2023). All
metrics are computed per image and then averaged over all images. Higher
values are better for all metrics.

| Depth |  | Prompt Based
---
Instance Segmentation
Semantic Segmentation
Dataset | DPT | ZoeDepth | SAM |  | Kmax-Deeplab
---
ConvNeXt-L
| Mask2Former
---
Swin-L
|  Ransac $\delta_{\leq 1.25}\uparrow$ | Instance mIoU$\uparrow$ | mIoU$\uparrow$
SANPO-Real | 0.6703 | 0.6978 | 0.4896 | 0.3234 | 0.497
SANPO-Synthetic | 0.7955 | 0.8032 | 0.5121 | 0.4639 | 0.535
Table 1: Zero-shot evaluation. In this setting, we evaluated the ability of
state-of-the-art models trained on other relevant datasets to generalize to
the SANPO test set for depth estimation and semantic segmentation. SANPO
challenges these models’ generalization capabilities.

###  4.1 Zero shot evaluation

We intend for SANPO to be representative of outdoor human navigation tasks
from an egocentric perspective. Human-centric tasks are distinct from other
well-studied domains, such as autonomous driving. Our objective with this
evaluation is to establish zero-shot baseline while evaluating how challenging
our dataset is for zero-shot prediction. To this end, we evaluate various
existing models on both depth estimation and semantic segmentation tasks.

For depth estimation, we used the publicly released checkpoints for DPT Ranftl
et al. (2021) and ZoeDepth-M12 NK Bhat et al. (2023), which, according to the
authors, were trained on a collection of both proprietary and public datasets.
SANPO is a metric depth dataset, but for this zero-shot comparison, we found
it necessary to give both these models the best possible advantage by
calculating $\delta_{\leq 1.25}$ in a scale-invariant way: we used RANSAC to
find alignment coefficients $\alpha,\beta$ that best aligned each image $x$
with its groundtruth $y$; namely, $\arg\min_{\alpha,\beta}\|\alpha
f(x)+\beta-y\|^{2}$, taking $y^{\prime}=\alpha f(x)+\beta$ as the output for
each model.

For semantic segmentation, we used Kmax-Deeplab Yu et al. (2023) and
Mask2Former Cheng et al. (2022) checkpoints trained on the Cityscapes dataset
Cordts et al. (2016). For a fair comparison, we mapped Cityscapes labelmap to
the SANPO labelmap and excluded the SANPO classes (18 in total) that do not
have a one-to-one correspondence. We do not report panoptic quality for this
baseline because the SANPO _“thing”_ labels differ from those of
Cityscapes555Details about the SANPO labelmap, its mapping to and from the
Cityscapes labelmap, and the list of ignored labels are provided in the
supplementary material..

We also included SAM Kirillov et al. (2023), a recent foundation model. For
SAM, we used the center point prompt and reported instance-level mIoU,
adhering to the conventional evaluation procedure for interactive segmentation
Sofiiuk et al. (2022).

Figure 7: Segment Anything Model (SAM) on SANPO. We evaluated SAM on SANPO
images. The middle column shows sample instance masks in SANPO and the
selected point used to prompt SAM. The last column shows the predicted masks
generated by SAM.

For the purpose of ensuring a streamlined evaluation process, we excluded very
small instances which were less than 2% of image in size.

Our findings are summarized in Table 1. Overall, SANPO seems to be a
challenging dataset for both depth and segmentation models. For example, DPT
reports good depth estimation performance ($\delta_{\leq 1.25}$$>0.9$) on
KITTI, but we observe $\sim 0.67$ on SANPO-Real and $\sim 0.8$ on SANPO-
Synthetic.

ZoeDepth Bhat et al. (2023) is designed to estimate metric depth for out-of-
domain datasets, but still requires alignment on this data (unaligned
$\delta_{\leq 1.25}\approx 0.2$ on SANPO-Real). The performance difference may
be due to the lack of metric depth data available to the community.
ZoeD-M12-NK was trained on total of 12 datasets, only two of which (NYUv2 and
KITTI) are metric depth datasets.

On the segmentation side, Mask2Former (Swin-L) achieves an mIoU of 0.83 on
Cityscapes validation set but $\sim 0.49$ on SANPO-Real.

In general, SANPO is a challenging and novel dataset that focuses on the
domain of egocentric human navigation, with plenty of headroom.

| Panoptic Segmentation | Depth Estimation
---|---|---
Dataset |  | Kmax-Deeplab-R50
---
SANPO-Real
| Kmax-Deeplab-R50
---
SANPO-Combined
| BinsFormer
---
SANPO-Real
| BinsFormer
---
SANPO-Combined
| mIoU$\uparrow$ | PQ$\uparrow$ | mIoU$\uparrow$ | PQ$\uparrow$ | $\delta_{\leq 1.25}\uparrow$
| Initialized with random weights
SANPO-Real | 0.3416 | 0.3173 | 0.3409 | 0.3210 | 0.4523 | 0.4702
SANPO-Synth | 0.2735 | 0.2277 | 0.5549 | 0.4483 | 0.2744 | 0.8546
| Pretrained with Cityscapes | Pretrained with Cityscapes-DVPS
SANPO-Real | 0.4370 | 0.4298 | 0.4381 | 0.4234 | 0.4524 | 0.4862
SANPO-Synth | 0.3900 | 0.3387 | 0.7109 | 0.5714 | 0.3235 | 0.8639
Table 2: SANPO Benchmark. Baseline performance of Kmax-Deeplab and BinsFormer,
using ResNet-50 backbone, on SANPO for panoptic segmentation and depth
estimation with limited training budget and standard hyperparameters.

###  4.2 SANPO Benchmark

In these experiments, we evaluated two state-of-the-art architectures:
BinsFormer Li et al. (2022b) for depth estimation and Kmax-Deeplab Yu et al.
(2023) for panoptic segmentation. Our objective is to establish baseline
performance on the SANPO dataset for future research.

####  4.2.1 Experimental Setup

We trained the models on the SANPO train sets and evaluated them on the test
sets. For the SANPO-Real experiments, we trained on $\sim$494K samples from
the SANPO-Real train set. We evaluated depth estimation on both the real and
synthetic test sets. For panoptic segmentation, we trained on $\sim$89K
samples from the SANPO-Real train set and evaluated on the SANPO real and
synthetic test sets. For the SANPO-Combined experiments, we combined the
SANPO-Real and SANPO-Synthetic train sets for training and used the test sets
for evaluation.

We resized the data to 1025$\times$2049 (with padding to maintain the aspect
ratio) for training and evaluation. We trained two sets of models using a
ResNet-50 backbone architecture:

  1. 1.

Models initialized with random weights.

  2. 2.

Models initialized with weights from models trained on the Cityscapes dataset
for panoptic segmentation, and the Cityscapes-DVPS dataset 666Cityscapes-DVPS
Qiao et al. (2020) is based on Cordts et al. (2016) for depth estimation.

To ensure fair comparison and reproducibility, we limited the training budget
to 50,000 steps with a batch size of 32 and used the standard hyperparameters
as defined in Weber et al. (2021). For reference, this training budget results
in:

  1. 1.

$\sim$540 epochs of the Cityscapes panoptic segmentation dataset.

  2. 2.

$\sim$18 epochs of the SANPO-Real panoptic segmentation dataset.

  3. 3.

$\sim$3.3 epochs of the SANPO-Real depth estimation dataset.

Table 2 shows the baseline performance for panoptic segmentation and depth
estimation on SANPO. Similar to the zero-shot evaluation, we observe that
SANPO is a challenging dataset for both dense prediction tasks.

Additionally, we also observe, here and in the zero-shot experiments, that
synthetic data has higher accuracy than real data. This performance gap
between the real and synthetic sets could be attributed to two factors:

  1. 1.

Complexity of the environments & domain gap: Real-world environments are more
complex than synthetic data, with more variation in objects, backgrounds, and
their interactions. The synthetic data also differs from the real data in
appearance and lighting, although it can sometimes be hard to tell.

  2. 2.

Accuracy of the segmentation annotations: Segmentation annotations are more
precise in the synthetic data than in the real data.

Exact quantification of these factors would require additional domain
adaptation experiments, which are beyond the scope of this work. We built the
SANPO-Synthetic dataset to facilitate this line of research.

##  5 Conclusion

We presented the SANPO dataset, a large-scale video dataset for egocentric
human navigation. It consists of 617k real stereo frames and 113k synthetic
frames. All real frames have dense depth annotations, and $\sim 20\%$ of them
have dense segmentation annotations. All synthetic frames have both depth and
segmentation annotations. In addition to the depth and segmentation
annotations, we also provide visual odometry readings (camera/ego-person
poses).

This work also evaluated the dataset and presented benchmarks for cross-
dataset zero-shot generalization and training on some state-of-the-art
architectures. We hope that this dataset will help fellow researchers build
visual navigation systems for the visually impaired and push the frontiers of
visual scene understanding.

## References

  * Kang et al. (2019) Yue Kang, Hang Yin, and Christian Berger.  Test your self-driving algorithm: An overview of publicly available driving datasets and virtual testing environments.  _IEEE Transactions on Intelligent Vehicles_ , 4(2):171–185, 2019.
  * Mao et al. (2022) Ruiqing Mao, Jingyu Guo, Yukuan Jia, Yuxuan Sun, Sheng Zhou, and Zhisheng Niu.  Dolphins: Dataset for collaborative perception enabled harmonious and interconnected self-driving.  In _Proceedings of the Asian Conference on Computer Vision (ACCV)_ , pages 4361–4377, December 2022.
  * Wilson et al. (2023) Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, et al.  Argoverse 2: Next generation datasets for self-driving perception and forecasting.  _arXiv preprint arXiv:2301.00493_ , 2023.
  * Karnan et al. (2022) Haresh Karnan, Anirudh Nair, Xuesu Xiao, Garrett Warnell, Sören Pirk, Alexander Toshev, Justin Hart, Joydeep Biswas, and Peter Stone.  Socially compliant navigation dataset (scand): A large-scale dataset of demonstrations for social navigation.  _IEEE Robotics and Automation Letters_ , 7(4):11807–11814, 2022.
  * Nguyen et al. (2023) Duc M Nguyen, Mohammad Nazeri, Amirreza Payandeh, Aniket Datar, and Xuesu Xiao.  Toward human-like social robot navigation: A large-scale, multi-modal, social human navigation dataset.  _arXiv preprint arXiv:2303.14880_ , 2023.
  * Grauman et al. (2022) Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al.  Ego4d: Around the world in 3,000 hours of egocentric video.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 18995–19012, 2022.
  * Fabbri et al. (2021) Matteo Fabbri, Guillem Brasó, Gianluca Maugeri, Aljoša Ošep, Riccardo Gasparini, Orcun Cetintas, Simone Calderara, Laura Leal-Taixé, and Rita Cucchiara.  Motsynth: How can synthetic data help pedestrian detection and tracking?  In _International Conference on Computer Vision (ICCV)_ , 2021.
  * Wen and Jo (2022) Li-Hua Wen and Kang-Hyun Jo.  Deep learning-based perception systems for autonomous driving: A comprehensive survey.  _Neurocomputing_ , 489:255–270, 2022.
  * Shi et al. (2017) Weijing Shi, Mohamed Baker Alawieh, Xin Li, and Huafeng Yu.  Algorithm and hardware implementation for visual perception system in autonomous vehicle: A survey.  _Integration_ , 59:148–156, 2017.
  * Qiao et al. (2020) Siyuan Qiao, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen.  Vip-deeplab: Learning visual perception with depth-aware video panoptic segmentation.  _arXiv preprint arXiv:2012.05258_ , 2020.
  * Cordts et al. (2016) Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.  The cityscapes dataset for semantic urban scene understanding.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016.
  * Liao et al. (2022) Yiyi Liao, Jun Xie, and Andreas Geiger.  KITTI-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d.  _Pattern Analysis and Machine Intelligence (PAMI)_ , 2022.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_ , pages 740–755. Springer, 2014.
  * Xu et al. (2018) Ning Xu, Linjie Yang, Yuchen Fan, Dingcheng Yue, Yuchen Liang, Jianchao Yang, and Thomas Huang.  Youtube-vos: A large-scale video object segmentation benchmark.  _arXiv preprint arXiv:1809.03327_ , 2018.
  * Caelles et al. (2019) Sergi Caelles, Jordi Pont-Tuset, Federico Perazzi, Alberto Montes, Kevis-Kokitsi Maninis, and Luc Van Gool.  The 2019 davis challenge on vos: Unsupervised multi-object segmentation.  _arXiv:1905.00737_ , 2019.
  * Brostow et al. (2009) Gabriel J. Brostow, Julien Fauqueur, and Roberto Cipolla.  Semantic object classes in video: A high-definition ground truth database.  _Pattern Recognition Letters_ , 30(2):88–97, 2009\.  ISSN 0167-8655.  doi: https://doi.org/10.1016/j.patrec.2008.04.005.  URL https://www.sciencedirect.com/science/article/pii/S0167865508001220.  Video-based Object and Event Analysis.
  * Caesar et al. (2019) Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.  nuscenes: A multimodal dataset for autonomous driving.  _arXiv preprint arXiv:1903.11027_ , 2019.
  * Richter et al. (2017) Stephan R Richter, Zeeshan Hayder, and Vladlen Koltun.  Playing for benchmarks.  In _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , pages 2213–2222, 2017.
  * Mei et al. (2022) Jieru Mei, Alex Zihao Zhu, Xinchen Yan, Hang Yan, Siyuan Qiao, Yukun Zhu, Liang-Chieh Chen, Henrik Kretzschmar, and Dragomir Anguelov.  Waymo open dataset: Panoramic video panoptic segmentation, 2022.
  * Pham et al. (2020) Quang-Hieu Pham, Pierre Sevestre, Ramanpreet Singh Pahwa, Huijing Zhan, Chun Ho Pang, Yuda Chen, Armin Mustafa, Vijay Chandrasekhar, and Jie Lin.  A*3d dataset: Towards autonomous driving in challenging environments.  In _Proceedings of The International Conference in Robotics and Automation (ICRA)_ , 2020.
  * Li et al. (2022a) Jiankun Li, Peisen Wang, Pengfei Xiong, Tao Cai, Ziwei Yan, Lei Yang, Jiangyu Liu, Haoqiang Fan, and Shuaicheng Liu.  Practical stereo matching via cascaded recurrent network with adaptive correlation, 2022a.
  * Yang et al. (2021) Zongxin Yang, Yunchao Wei, and Yi Yang.  Associating objects with transformers for video object segmentation.  In _Advances in Neural Information Processing Systems (NeurIPS)_ , 2021.
  * Yu et al. (2023) Qihang Yu, Huiyu Wang, Siyuan Qiao, Maxwell Collins, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen.  kmax-deeplab: k-means mask transformer, 2023.
  * Bhat et al. (2023) Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias Müller.  Zoedepth: Zero-shot transfer by combining relative and metric depth, 2023\.
  * Ranftl et al. (2021) René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.  Vision transformers for dense prediction, 2021.
  * Cheng et al. (2022) Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar.  Masked-attention mask transformer for universal image segmentation.  In _CVPR_ , 2022.
  * Kirillov et al. (2023) Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick.  Segment anything, 2023.
  * Sofiiuk et al. (2022) Konstantin Sofiiuk, Ilya A Petrov, and Anton Konushin.  Reviving iterative training with mask guidance for interactive segmentation.  In _2022 IEEE International Conference on Image Processing (ICIP)_ , pages 3141–3145. IEEE, 2022.
  * Li et al. (2022b) Zhenyu Li, Xuyang Wang, Xianming Liu, and Junjun Jiang.  Binsformer: Revisiting adaptive bins for monocular depth estimation, 2022b.
  * Weber et al. (2021) Mark Weber, Huiyu Wang, Siyuan Qiao, Jun Xie, Maxwell D. Collins, Yukun Zhu, Liangzhe Yuan, Dahun Kim, Qihang Yu, Daniel Cremers, Laura Leal-Taixe, Alan L. Yuille, Florian Schroff, Hartwig Adam, and Liang-Chieh Chen.  DeepLab2: A TensorFlow Library for Deep Labeling.  _arXiv: 2106.09748_ , 2021.
  * Huang et al. (2018) Xinyu Huang, Xinjing Cheng, Qichuan Geng, Binbin Cao, Dingfu Zhou, Peng Wang, Yuanqing Lin, and Ruigang Yang.  The apolloscape dataset for autonomous driving.  _arXiv: 1803.06184_ , 2018.
  * Guizilini et al. (2020) Vitor Guizilini, Rares Ambrus, Sudeep Pillai, Allan Raventos, and Adrien Gaidon.  3d packing for self-supervised monocular depth estimation.  In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020.

##  Appendix A Dataset comparison

There are many outdoor video datasets created to study particular tasks
ranging from robot navigation to autonomous driving to video segmentation and
more. Tab. 3 compares our neighbors in this space. However, SANPO fills an
important gap: to our knowledge, ours is the only human-egocentric dataset
annotated with panoptic labels and depth maps that has both real and synthetic
data.

Dataset |  | Domain
---
Environment | Quantity | # Sem Masks | # Depth Maps
| SCAND
---
Karnan et al. (2022)
|  |  $+$
---
$\sim$522 minutes |  |  $\sim$522 minutes
| MuSoHu
---
Nguyen et al. (2023)
|  |  $+$
---
$\sim$600 minutes |  |  $\sim$600 minutes
| Playing for Benchmark
---
(Synthetic)
Richter et al. (2017)
|  | 250K frames | 250K (Dense) |
| Cityscapes-DVPS
---
Qiao et al. (2020)
|  $+$ | 3K frames | 3K (Dense) | 3K (Dense)
| KITTI-360
---
Liao et al. (2022)
|  | 320K frames |  | 2x78K (Dense)
---
(left & right))
| 2x78K (Dense)
---
(left & right))
| Panoptic-nuScenes
---
Caesar et al. (2019)
|  | 1.4M frames | 40K (Dense & 3D) |
| Waymo Open Dataset
---
-Panoramic
Mei et al. (2022)
|  | 390K Frames | 100K Frames |
| A${}^{*}3$D
---
Pham et al. (2020)
|  | 39K frames | 39K (3D Bbox) |
| ApolloScape-SceneParsing
---
Huang et al. (2018)
|  | 140K frames | 140K (Dense) | 140K (Dense)
| DDAD
---
Guizilini et al. (2020)
|  | 21K frames |  | 21K (Dense)
| DOLPHINS
---
(Synthetic)
Mao et al. (2022)
|  | 42K frames | 42K (3D Bbox) |
| Argoverse2-Sensor Data
---
Wilson et al. (2023)
|  $+$ | 1000 videos | 3D Bbox |
| CamVid
---
Brostow et al. (2009)
|  | 5 videos | 700 (Dense) |
| Youtube-VOS
---
Xu et al. (2018)
V.O.S. |  $+$ |  |  $\sim$340 minutes,
---
$\sim$20K frames
$\sim$4K |
| DAVIS-2017
---
Caelles et al. (2019)
V.O.S. |  $+$ |  | 150 videos,
---
10K frames
10K |
| MOTSynth
---
(Synthetic)
Fabbri et al. (2021)
| Pedestrain
---
Detection
$+$ | 1.3M frames | 1.3M | 1.3M
|  SANPO-
---
Real
|  $+$ |  | 2x617K frames
---
(left & right)
112K (Dense) | 617K (Dense)
|  SANPO-
---
Synthetic
|  | 113K frames | 113K (Dense) | 113K (Dense)
|  : Robot navigation,  : Self-driving,  : Egocentric navigation, V.O.S: Video
object segmentation,
---
: Indoor, : Outdoor, : Stereo
Table 3: SANPO is a unique video dataset that fills a gap in the existing
landscape of video datasets. Most existing video datasets are focused on self-
driving cars or general video object segmentation (VOS), and many of them are
too constrained and not diverse enough. SANPO is the first dataset
specifically focused on egocentric human navigation, and it has the following
features: i) It is large-scale, challenging, and diverse. ii) It includes both
real and synthetic data. iii) The real data is a multi-view stereo dataset.

###  A.1 Composition and statistics

Fig. 9 shows how often the high-level segmentation labels (described below)
occur throughout SANPO imagery. Fig. 8 shows how many instances tend to occur
in the real and synthetic portions of the dataset.

Figure 8: Histogram of SANPO instance counts per image. Real imagery tends to
have fewer instances per image, with almost no real-world images having more
than 40 annotated instances. Synthetic images, on the other hand, feature many
more instances. This is because each object that populates the virtual world
can have its own instance mask. Figure 9:  Semantic class occurrences in
SANPO, shown as the fraction of frames that contain pixels labeled with each
class. Common semantic classes like Building, Obstacle, Pole, Sky, and Tree
appear in almost all frames. Other classes like Railway track, Bus stops, and
Animal (including the horse that interrupted one volunteer in Central Park)
are less common.

##  Appendix B Data Collection

(a)

(b)

(c)

(d)

Figure 10: The SANPO data collection system for collecting real-world data.
See text for details.

Capture rig. Figure 10 shows all the all components and the assembled rig. The
top row of images shows the system’s components: (i) a backpack with ZED 2i
and ZED-M cameras for data collection, (ii) the inside of the backpack showing
the ZED box and battery pack mounted on a 3D printed container, and (iii) an
Android app showing the live feed from the Zed cameras. The bottom row shows
the system in use.

##  Appendix C Data Annotation

###  C.1 Session Attributes

Each real session is annotated with the following high level attributes.

  1. 1.

Human Traffic

  2. 2.

Vehicular Traffic

  3. 3.

Number of Obstacles

  4. 4.

Environment Type

  5. 5.

Weather Condition

  6. 6.

Visibility

  7. 7.

Speed of Ego motion

  8. 8.

Elevation Change

  9. 9.

Ground Appearances

  10. 10.

Motion Blur

###  C.2 SANPO Taxonomy

Below SANPO taxonomy along with whether a class label is _stuff_ or _thing_
(i.e. with instance ids).

  1. 0.

unlabeled ${\rightarrow}$ stuff

  2. 1.

road ${\rightarrow}$ stuff

  3. 2.

curb ${\rightarrow}$ stuff

  4. 3.

sidewalk ${\rightarrow}$ stuff

  5. 4.

guard rail/road barrier ${\rightarrow}$ stuff

  6. 5.

crosswalk ${\rightarrow}$ thing

  7. 6.

paved trail ${\rightarrow}$ stuff

  8. 7.

building ${\rightarrow}$ stuff

  9. 8.

wall/fence ${\rightarrow}$ stuff

  10. 9.

hand rail ${\rightarrow}$ stuff

  11. 10.

opening-door ${\rightarrow}$ thing

  12. 11.

opening-gate ${\rightarrow}$ thing

  13. 12.

pedestrian ${\rightarrow}$ thing

  14. 13.

rider ${\rightarrow}$ thing

  15. 14.

animal ${\rightarrow}$ thing

  16. 15.

stairs ${\rightarrow}$ thing

  17. 16.

water body ${\rightarrow}$ stuff

  18. 17.

other walkable surface ${\rightarrow}$ stuff

  19. 18.

inaccessible surface ${\rightarrow}$ stuff

  20. 19.

railway track ${\rightarrow}$ stuff

  21. 20.

obstacle ${\rightarrow}$ thing

  22. 21.

vehicle ${\rightarrow}$ thing

  23. 22.

traffic sign ${\rightarrow}$ thing

  24. 23.

traffic light ${\rightarrow}$ thing

  25. 24.

pole ${\rightarrow}$ thing

  26. 25.

bus stop ${\rightarrow}$ thing

  27. 26.

bike rack ${\rightarrow}$ thing

  28. 27.

sky ${\rightarrow}$ stuff

  29. 28.

tree ${\rightarrow}$ thing

  30. 29.

vegetation ${\rightarrow}$ stuff

  31. 30.

terrain ${\rightarrow}$ stuff

###  C.3 Cityscape-19 ${\rightarrow}$ SANPO Mapping

To ensure a fair comparison, we map Cityscape-19 labels to SANPO labels
wherever possible. Below mapping from Cityscape-19 to SANPO taxonomy:

  1. 1.

road ${\rightarrow}$ road

  2. 2.

sidewalk ${\rightarrow}$ sidewalk

  3. 3.

building ${\rightarrow}$ building

  4. 4.

wall ${\rightarrow}$ wall/fence

  5. 5.

fence ${\rightarrow}$ wall/fence

  6. 6.

pole ${\rightarrow}$ pole

  7. 7.

traffic light ${\rightarrow}$ traffic light

  8. 8.

traffic sign ${\rightarrow}$ traffic sign

  9. 9.

vegetation ${\rightarrow}$ vegetation

  10. 10.

terrain ${\rightarrow}$ terrain

  11. 11.

sky ${\rightarrow}$ sky

  12. 12.

person ${\rightarrow}$ pedestrian

  13. 13.

rider ${\rightarrow}$ rider

  14. 14.

car ${\rightarrow}$ vehicle

  15. 15.

truck ${\rightarrow}$ vehicle

  16. 16.

bus ${\rightarrow}$ vehicle

  17. 17.

train ${\rightarrow}$ vehicle

  18. 18.

motorcycle ${\rightarrow}$ vehicle

  19. 19.

bicycle ${\rightarrow}$ vehicle

For all SANPO labels without an appropriate mapping from Cityscape-19, we
treat the corresponding pixels as unlabeled and exclude them from the mIoU
metric computation in the zero-shot experiments. The following SANPO labels
were excluded:

  * •

curb

  * •

guard rail/road barrier

  * •

crosswalk

  * •

paved trail

  * •

hand rail

  * •

opening-door

  * •

opening-gate

  * •

animal

  * •

stairs

  * •

water body

  * •

other walkable surface

  * •

inaccessible surface

  * •

railway track

  * •

obstacle

  * •

bus stop

  * •

bike rack

  * •

tree

##  Appendix D Evaluation

(a)

(b)

(c)

(d)

Figure 11: Mask2Former Zero Shot Evaluation. An egocentric human navigation
needs to accurately distinguish between road and sidewalk (a walkable
surface), as one is not safe for human navigation and the other is safer. The
top row (SANPO synthetic sample) and the bottom row (SANPO real sample)
demonstrate how Mask2Former (trained on Cityscapes) confuses sidewalk as road.
The visualization is generated with Mask2Former tool.

###  D.1 Zero-shot Mask2Former Evaluation

Mask2Former Swin-L model is reported to have mIoU semantic segmentation score
0.833 on Cityscapes validation set, but $\sim 0.497$ on SANPO-Real and $\sim
0.535$ on SANPO-Syn as reported in the Table 4. Fig 11 shows qualitative
evaluation on a SANPO synthetic and real sample.

###  D.2 SANPO SAM Evaluation

We have captured very fine details with high quality, exemplified by semantic
masks for even the smallest instances within the image. Some examples are
shown in Fig 12. As an image can have many such instances, therefore, in order
to streamline our experiment with SAM on SANPO we report evaluation with masks
of size greater than 2% of the image area.

Figure 12: SANPO images and masks with fine details. Right column shows masks
(purple) even for the smallest instances within the images.

###  D.3 SANPO qualitative examples

We show some example images in Fig. 13, as well as ground truth and predicted
segmentation maps from kMax-Deeplab and ground truth and predicted depth maps
from Binsformer.

Figure 13: Qualitative examples on SANPO, showing (left to right): image, groundtruth and predicted segmentation maps, and groundtruth and predicted depth maps.  | mIoU
---|---
Labels | SANPO-Real | SANPO-Synthetic
road | 0.255 | 0.407
sidewalk | 0.120 | 0.262
building | 0.642 | 0.934
wall/fence | 0.448 | 0.087
pedestrian | 0.679 | 0.878
rider | 0.271 | 0.247
vehicle | 0.658 | 0.817
traffic sign | 0.212 | 0.240
traffic light | 0.127 | 0.344
pole | 0.310 | 0.586
sky | 0.658 | 0.919
vegetation | 0.654 | 0.303
terrain | 0.394 | 0.166
Average mIoU | 0.497 | 0.535
Table 4: Mask2Former Zero-Shot Evaluation: Per label breakdown of mIoU on the
Mask2Former zero-shot experiment.


# A Systematic Investigation of the Coupling between One-Dimensional Edge
States
of a Topological Crystalline Insulator

Johannes Jung  Physikalisches Institut, Experimentelle Physik II, Universität
Würzburg, Am Hubland, 97074 Würzburg, Germany  Artem Odobesko
<EMAIL_ADDRESS>Physikalisches Institut, Experimentelle
Physik II, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany  Robin
Boshuis  Physikalisches Institut, Experimentelle Physik II, Universität
Würzburg, Am Hubland, 97074 Würzburg, Germany  Andrzej Szczerbakow  Institute
of Physics, Polish Academy of Sciences, Aleja Lotników 32/46, 02-668 Warsaw,
Poland  Tomasz Story  Institute of Physics, Polish Academy of Sciences, Aleja
Lotników 32/46, 02-668 Warsaw, Poland  International Research Centre MagTop,
Institute of Physics, Polish Academy of Sciences, Aleja Lotnikow 32/46, 02-668
Warsaw, Poland  Matthias Bode  Physikalisches Institut, Experimentelle Physik
II, Universität Würzburg, Am Hubland, 97074 Würzburg, Germany

###### Abstract

The interaction of spin-polarized one-dimensional (1D) topological edge modes
localized along single-atomic steps of the topological crystalline insulator
Pb0.7Sn0.3Se(001) has been studied systematically by scanning tunneling
spectroscopy. Our results reveal that the coupling of adjacent edge modes sets
in at a step–to–step distance $d_{\rm ss}\leq 25$ nm, resulting in a
characteristic splitting of a single peak at the Dirac point in tunneling
spectra. Whereas the energy splitting exponentially increases with decreasing
$d_{\rm ss}$ for single-atomic steps running almost parallel, we find no
splitting for single-atomic step edges under an angle of $90^{\circ}$. The
results are discussed in terms of overlapping wave functions with
$p_{x},p_{y}$ orbital character.

Introduction — The discovery of two-dimensional (2D) topological insulators
(TIs), such as HgTe/CdTe König  _et al._ (2007) or InAs/GaSb quantum wells
Knez  _et al._ (2011), generated significant attention, especially due to the
existence of edge states protected by time-reversal symmetry. The
spin–momentum locking pertinent to these states mandates that charge carriers
with a given spin can only propagate in a predetermined direction, thereby
inhibiting disorder scattering and greatly reducing dissipation in electrical
transport Bauer and Bobisch (2016); Hou  _et al._ (2019); Liu  _et al._
(2019); Plucinski (2019); Liu  _et al._ (2020). Further material classes with
topologically protected band structures have been predicted and realized, such
as topological crystalline insulators (TCI) Fu (2011); Dziawa  _et al._
(2012); Xu  _et al._ (2012); Tanaka  _et al._ (2012) and Weyl semimetals Weng
_et al._ (2015); Xu  _et al._ (2015); Lv  _et al._ (2015a); Yang  _et al._
(2015); Lv  _et al._ (2015b); Sessi  _et al._ (2017). However, although the
ruggedness of the respective topological states is often praised as one of
their main advantages, the reliable realization of one-dimensional (1D)
topological conductance channels highly relevant for practical applications
remains a challenge. Some systems with topological 1D states, like higher-
order topological hinge states Drozdov  _et al._ (2014); Schindler  _et al._
(2018), or surface step edges of weak TIs Pauly  _et al._ (2015) or TCIs Sessi
_et al._ (2016), could be identified, but systematic studies which address the
interaction of adjacent channels remain elusive. As a result the question up
to which density topological 1D channels may be packed without compromising
the signal quality has not yet been answered.

In this context hybridization effects of topological states play a
particularly important role. For example, the close proximity to the
underlying substrate may lead to a significant interaction of topological edge
states with substrate bulk states Hirahara  _et al._ (2011); Yang  _et al._
(2012); Kim  _et al._ (2014). Calculations predict that—if tuned
appropriately—these effects may even be utilized, such as in topological-
insulator–ferromagnetic-metal (TI–FM) heterostructures, where hybridization-
induced interface states lead to a large spin-transfer torque Hsu  _et al._
(2017). Strong hybridization between the top and the bottom surfaces, however,
can also result in band crossings resulting in a trivial TCI phase Liu  _et
al._ (2014) or—for very thin films of three-dimensional TIs—to the complete
quenching of the topological surface state Zhang  _et al._ (2010).

In this contribution we systematically study the interactions of one-
dimensional topological states which arise at odd-atomic step edges of
(001)-terminated (Pb,Sn)Se single crystals. Whereas the rock salt structure
compound PbSe is a trivial insulator, the order of p-derived cation and anion
surface states bands is inverted for Pb1-xSnxSe at $x\gtrsim 0.2$, i.e., when
substituting a sufficient fraction of Pb atoms with Sn Dziawa  _et al._
(2012); Wojek  _et al._ (2014). Band inversion results in four Dirac cones per
surface Brillouin zone (SBZ) centered in close proximity to the $\overline{X}$
and $\overline{Y}$ points which are protected by $\langle 110\rangle$ mirror
symmetry Hsieh  _et al._ (2012); Liu  _et al._ (2013); Wang  _et al._ (2013);
Safaei  _et al._ (2013).

Clean (001) surfaces of Pb1-xSnxSe are typically created by cleaving bulk
crystals. As first shown in Ref. Okada  _et al._ , 2013, tunneling spectra of
flat (Pb,Sn)Se (001) surfaces exhibit a V-shaped dip with two symmetrically
arranged side peaks which were assigned to the Dirac point and two Van Hove
singularities caused by saddle points in the dispersion, respectively.
However, the cleaving process unavoidably also results in a variety of steps
of different height and shape. Whereas the electronic structure of double- or
other even-atomic step edges is virtually identical to a flat terrace, see
Ref. Sup  (Note 1), the observation of a 1D topological edge mode at odd-
atomic step edges, e.g., steps with a height of one or three atomic layers,
which are characterized by a strong peak in the LDOS at the Dirac energy, was
reported in Ref. Sessi  _et al._ , 2016 and confirmed in Ref. Iaia  _et al._ ,
2019. As detailed in Ref. Rechciński and Buczko, 2018, adjacent extended (001)
terraces of (Pb,Sn)Se must—due to the displacement by one atomic layer in a
two-atomic unit cell—be described by Hamiltonians with chiral symmetry and an
inverted order of states. The resulting surface states are divided into two
classes with an even or odd number of layers. Therefore, odd-atomic step edges
host a doubly-degenerate flat band at the Dirac energy which is protected by
the mirror plane symmetry of the film Rechciński and Buczko (2018).

In this paper we systematically investigate the interaction between closely
adjacent 1D edge states of the (001) surface of the TCI (Pb,Sn)Se.
Hybridization is observed for step–to–step distances $d_{\rm ss}\lesssim 25$
nm, resulting in the splitting of the single peak in tunneling spectra at the
Dirac point into a double-peak. Our results reveal that the energetic
splitting exponentially increases with decreasing $d_{\rm ss}$. It is largely
insensitive to the details of a particular configuration of step edges as long
as the steps run approximately in parallel. In contrast, for single-atomic
steps crossing at an almost perpendicular angle, we do not observe any
splitting.

Methods — Experiments have been performed in an ultra-high vacuum system
equipped with a cryogenic STM (operation temperature $T=4.9$ K). Pb0.7Sn0.3Se
p-doped single crystals grown by the self-selecting vapor growth method Dziawa
_et al._ (2012); Sessi  _et al._ (2016) (see Ref. Sup, , Note 2, for further
information) have been cleaved in UHV at a base pressure $p<10^{-10}$ mbar and
immediately transferred into the STM. Tunneling conductance
$\mathrm{d}I/\mathrm{d}U$ maps and spectra were measured by lock-in technique
with modulation voltage $U_{\textrm{mod}}=2.5$ mV at a frequency $f=790$ Hz.

Results —

Figure 1: (a) Topographic STM image of a cleaved Pb0.7Sn0.3Se surface with a
number of wedge-shaped terraces with a single- and double-atomic step edge
height; (b) A line profile measured along the green line in (a); (c) Typical
tunneling spectra measured on the plain terrace (1), double- (2) and single-
atomic (3) step edges. Spectrum (4) with a characteristic double-peak was
obtained at a single-atomic step edge close to the pointed end of of the
wedge-shaped plateau. Stabilization parameters: $U_{\textrm{set}}=200$ mV,
$I_{\textrm{set}}=0.2$ nA.

Figure 1(a) shows a typical STM topography image of the cleaved (001) surface
of the TCI Pb0.7Sn0.3Se. Numerous wedge-shaped atomically flat plateaus can be
recognized. The line profile measured along the green line in Fig. 1(a),
presented in Fig. 1(b), reveals that most step edges are about 3 Å high,
corresponding to half of the two-atomic unit cell. Fig. 1(c) shows typical
$\mathrm{d}I/\mathrm{d}U$ spectra measured on a flat terrace far away from any
step edge (1), on a double-atomic step edge (2), and at two single-atomic step
edges [points (3),(4)]. Spectrum (1) exhibits a minimum at the Dirac point
$E_{\rm D}=(125\pm 5)$ meV and two shoulders (L- and L+), which are associated
with the two saddle points in the dispersion Okada  _et al._ (2013); Liu  _et
al._ (2013). Spectrum (2) taken on a double-step edge is virtually
indistinguishable from (1), indicating the absence of any edge state at step
edges equivalent to an even number of atomic layers. In agreement with our
earlier publication Sessi  _et al._ (2016), a pronounced peak at the Dirac
energy indicative for the 1D edge state can only be found at odd-atomic step
edges, such as spectrum (3) measured on a single-atomic step edge. However, we
find that the spectra of single-atomic step edges at some locations deviate
from this simple single-peak structure. For example, the spectrum measured at
(4) close to the termination point of an acute-angled wedge-shaped plateau
reveals a pronounced double-peak feature in the vicinity of the Dirac point
with a peak splitting of about $25$ mV.

Figure 2: (a) Topographic STM image of a wedge-shaped plateau surrounded by
single-atomic step edges. Note, that the $x$\- and $y$-direction are scaled
differently, i.e., the wedge shape is even more acute-angled than it appears.
(b) $\mathrm{d}I/\mathrm{d}U$ map measured at the Dirac energy of the same
region shown in (a). An enhanced intensity at the position of the step edges
is clearly visible in the bottom part, indicating the presence of the
topologically protected edge state. As the wedge-shaped plateau becomes more
narrow, the intensity of the $\mathrm{d}I/\mathrm{d}U$ signal measured at both
step edges decreases (see region above arrows) and eventually vanishes close
to the termination point (ellipse). (c) Tunneling spectra measured at the left
(blue) and right (red) single-atomic step edge at various step–to–step
distances indicated in (a). (d) Color-coded raw $\mathrm{d}I/\mathrm{d}U$
spectra measured along the green hatched line in (a). (e) Numerically
calculated second derivative ($\mathrm{d^{3}}I/\mathrm{d}U^{3}$) of (d). (f)
Sketch of a model of two quantum wells localized along two converging single-
atomic step edges. The wave-functions extend beyond the walls and overlap. The
overlap area (green) indicates the coupling strength and determines the energy
splitting. (g) The peak energy splitting $\Delta E$ in dependence of
step–to–step distance im semilogarithmic scale. Stabilization parameters:
(a,b) $U_{\textrm{set}}=125$ mV (a), $I_{\textrm{set}}=0.2$ nA.

To understand the physical origin of this peak splitting we performed detailed
spectroscopic investigations at numerous single-atomic step edge structures on
a variety of topological Pb0.7Sn0.3Se surfaces. Our results indicate that the
peak splitting observed in Fig. 1(c) on the single-atomic step edge is not
caused by the presence of kinks, defects, or random variations of the local
doping level Beidenkopf  _et al._ (2011). Instead, the data reveal that the
splitting occurs when two single-atomic step edges are in close proximity. A
particularly illustrative case is displayed in Fig. 2. The topographic STM
image of Fig. 2(a) shows a wedge-shaped plateau bound by two single-atomic
step edges which converge under an acute angle of $3.4^{\circ}\pm
0.3^{\circ}$. As a result, the step–to–step distance $d_{\rm ss}$ slowly
decreases over a total length of about 500 nm from $d_{\rm ss}>30$ nm in the
bottom part to almost zero at the apex of the wedge. Both steps have rough
edges which clearly deviate from high symmetry directions. Nevertheless, the
$\mathrm{d}I/\mathrm{d}U$ map measured at a bias voltage corresponding to the
Dirac point presented in Fig. 2(b) shows several unique features. First, in
the bottom part of Fig. 2(b) where the two single-atomic step edges are far
apart an intense $\mathrm{d}I/\mathrm{d}U$ signal can be recognized along the
step edges, indicating the presence of the topological edge state. Second, as
$d_{\rm ss}$ falls below about 15 nm, indicated by two arrows in Fig. 2(b),
the intensity of the $\mathrm{d}I/\mathrm{d}U$ signal noticeably decreases.
Third, close to the top of the wedge where $d_{\rm ss}\lesssim 7$ nm (marked
by a hatched ellipse) it becomes essentially indistinguishable from the signal
measured on the surrounding terraces.

This modification of the LDOS at the Dirac energy correlates with a systematic
alteration of the local STS data. Some exemplary spectra measured at various
step–to–step distances ranging from $d_{\rm ss}=30$ nm down to $d_{\rm ss}=7$
nm are shown in Fig. 2(c). In each case we present the spectra measured at the
left and the right single-atomic step edge in blue and red, respectively. The
tunneling spectra measured at $d_{\rm ss}=30$ nm (curve \footnotesize1⃝)
exhibit a single intense peak at the Dirac energy $E_{\rm D}=125$ meV which is
fully consistent with the spectra measured at isolated step edges Sessi  _et
al._ (2016). At $d_{\rm ss}=22$ nm (curve \footnotesize2⃝) the peak height
slightly decreases and a weak shoulder (blue curve) or subtle double humps
(red curve) can be recognized. With $d_{\rm ss}$ reduced to 14 nm (curve
\footnotesize3⃝) a clear splitting in two peaks becomes apparent, which
amounts to $\Delta E\approx 18$ mV. The splitting further increases with
decreasing step–to–step distance reaching $\Delta E\approx 26$ meV at $d_{\rm
ss}=11$ nm (curve \footnotesize4⃝). At $d_{\rm ss}=7$ nm the peak intensities
have drastically dropped (curve \footnotesize5⃝), even though the splitting
still appears to increase ($\Delta E\approx 43$ meV). Eventually, at even
smaller $d_{\rm ss}$ the peaks become almost undistinguishable from the
spectra of the terrace (curve \footnotesize6⃝ and dashed line, respectively).

Figure 2(d) represents the complete data set of color-coded raw
$\mathrm{d}I/\mathrm{d}U$ spectra measured along the green hatched line in
Fig. 2(a). This line section follows the contour of the left single-atomic
step edge for about 500 nm from the bottom of Fig. 2(a) ($L=0$) all the way to
its termination point at the upper part of the the wedge-shaped plateau. For
better contrast the numerically calculated second derivative of the
$\mathrm{d}I/\mathrm{d}U$ signal, i.e., $\mathrm{d^{3}}I/\mathrm{d}U^{3}$, is
presented in Fig. 2(e). At 0 nm $\leq L\leq 200$ nm the data confirm the
existence of a single peak at the Dirac energy $E_{\rm D}=eU\approx 125$ meV.
Local fluctuations by about $\pm 5$ mV are probably caused by statistical
variations of the substitutional dopant concentration Beidenkopf  _et al._
(2011); Storz  _et al._ (2016). As indicated by arrows in Fig. 2(d) and (e),
for $L\geq 250$ nm the single peak which is known to represent the topological
edge state of (Pb,Sn)Se Sessi  _et al._ (2016) splits into two diverging
branches. The data presented in Fig. 2(e) reveal the peak splitting increases
with decreasing step–to–step distance up to $L\leq 400$ nm where $\Delta
E\approx 28$ meV at $d_{\rm ss}\approx 10$ nm. Above this line both peaks
experience significant peak-broadening and strongly decrease in intensity,
possibly due to the onset of attenuation. As presented in detail in Ref. Sup,
(Note 3), the results obtained on the right step edge are fully consistent
with the results presented for the left step edge in Fig. 2. Furthermore,
these main findings are independent of the specific type and shape of the step
edges. For example, results presented in Ref. Sup,  (Notes 4 and 5) confirm
that the peak splitting also exists for wedge-shaped vacancy islands and
nearby parallel steps, respectively.

Discussion — Similar observations of a coupling-dependent energy splitting
have been made for zero-dimensional quantum dots Nilsson  _et al._ (2018) and
two-dimensional layered van der Waals heterostructures Coy Diaz  _et al._
(2015). In the latter case the opening of band-gaps was particularly prominent
for out-of-plane–oriented MoS2 orbitals which strongly overlap with graphene
$\pi$-bands Coy Diaz  _et al._ (2015). In fact, periodic boundary calculations
presented in Fig. 6 of Ref. Rechciński and Buczko, 2018 suggest that the
topological 1D edge state of (Pb,Sn)Se remains largely unaffected for terrace
width $d_{\rm ss}\geq 86$ nm, in reasonable agreement with our experimental
data, cf. Fig. 2. Similar to the tight binding calculations presented in Ref.
Sessi  _et al._ , 2016, a weak splitting may be visible for $d_{\rm ss}\geq
43$ nm Rechciński and Buczko (2018). For narrower terraces, however, a
collapse of the double Dirac cone into a single cone with a crossing at the
$\overline{X}$ point was predicted Rechciński and Buczko (2018), in clear
disagreement with our results which show the persistence of a split peak well
below $d_{\rm ss}\leq 10$ nm, cf. Fig. 2.

To simulate the observed behavior, we assume that the formation of the edge
state results in an increased local density of states, crammed in a narrow
region around the step edge. This regions of accumulated charge are—in a
first-order approach—modeled by narrow identical quantum wells with a widths
$\sim 1$ nm perpendicular to the step edge and much larger along the step
edge, determined as the intra-row coherence length. For a perfect step edge
the topological edge mode would be completely delocalized along the step edge.
The fact that the energy splitting observed at different points along the step
edge depends on the step–to–step distance suggests that we don’t deal with a
single, infinitely extended quantum state, but with electron wave functions
which are localized to a certain extent due to their finite coherence length,
potentially induced by disorder. Two quantum wells are separated by a distance
$d$ and affect the edge state, see Fig. 2(f). The well depth is assumed to
$\sim 100$ meV, corresponding to the bulk band gap of the Pb0.7Sn0.3Se 111 We
would like to note that we cannot exclude that similar or even better results
can be obtained by more elaborate models, where the electrostatic interaction
becomes less relevant and is instead replaced by a behavior governed by
spatially varying topological indices. As shown in detail in Ref. Sup,  (Note
6), both quantum wells host wave functions which extend beyond their
respective boundaries and decay exponentially with $\exp(-\frac{x}{\xi})$
outside the well, with the decay length $\xi$.

When the wave functions overlap at low $d$, see green area in Fig. 2(f), the
exponentially decaying tails of the wave functions couple, resulting in an
energy splitting $\Delta E$. Indeed, a plot of $\Delta E$ values extracted
from Fig. 2(e) versus $d_{\rm ss}$ confirms the expected exponential
dependence, see Fig. 2(g). The decay length of the 1D edge mode perpendicular
to the step direction is determined to $\xi=(10\pm 1)$ nm. By taking
$\hbar\delta k\propto\hbar/2\xi$ and $\delta E\approx 10$ meV as the width of
the single peak in spectrum \footnotesize1⃝ of Fig. 2(c), we estimate the
group velocity to $\upsilon_{\rm gr}\approx\frac{\delta E}{\hbar\delta
k}\approx 4.8\cdot 10^{4}$ m/s, in good agreement with earlier band dispersion
measurements Wang  _et al._ (2013); Zeljkovic  _et al._ (2014).

Figure 3:  (a) Topography and (b) $\mathrm{d}I/\mathrm{d}U$ map measured at
the Dirac point ($U_{\rm set}=120$ mV)) of two intersecting at $90^{\circ}$
single-step edges, both running along the sample’s high symmetry directions;
(c) Color-coded raw $\mathrm{d}I/\mathrm{d}U$ spectra measured along step-edge
running in [100] marked with the hatched line in (a); (d) Second derivative
($\mathrm{d^{3}}I/\mathrm{d}U^{3}$) of the same signal in (c); Stabilization
parameters: $U_{\textrm{set}}=120$ mV, $I_{\textrm{set}}=0.2$ nA.

Orthogonal step edges — Our investigations of wedge-shaped islands like those
presented in Figs. 1 and 2 suggest that wedge angles are always $\lesssim
30^{\circ}$. In contrast, much larger adjacent angles we regularly observed at
points where two single-atomic steps intersect. For example, the STM data of
Fig. 3(a) show two steps oriented along the planar high-symmetry $\langle
100\rangle$ directions intersecting under an angle of $\approx 90^{\circ}$.
Again, we find a strongly enhanced $\mathrm{d}I/\mathrm{d}U$ signal at the
Dirac energy at the location of odd-atomic step edges and a strong reduction
at the step crossing point, Fig. 3(b). In contrast to the situation observed
for wedge-shaped step edges in Fig. 2, however, the tunneling spectra measured
along the white arrow in panel Fig. 3(a) show no hints for any peak splitting,
see Fig. 3(c) and (d) for raw spectra and $\mathrm{d^{3}}I/\mathrm{d}U^{3}$,
respectively. The full data set is presented in Ref. Sup,  (Note 7).

The results presented in Figs. 2, 3 and Ref. Sup,  show that the topological
edge state characteristic for odd-atomic step edges of the (Pb,Sn)Se (001)
surface splits for adjacent steps ($d_{\rm ss}\lesssim 25$ nm) running in
parallel or under an acute angle. In contrast, for step edges crossing
perpendicularly no such peak splitting is observed. This latter behavior hints
at wave functions which are orthogonal such that the solution of the
Schrödinger equation becomes $\int\Phi^{\ast}_{n}\Phi_{m}dV=\delta_{n,m}$,
where the indices $n,m$ stand for quantum numbers and $\delta_{n,m}$ is the
Kronecker symbol. Obvious choices for this quantum number might be the spin or
the orbital moment of the edge state. We are convinced that we can exclude the
spin since results obtained within a tight binding Sessi  _et al._ (2016) or
envelope function model Rechciński and Buczko (2018) indicate an out-of-plane
or vanishing spin polarization of the edge state, respectively, both
insufficient to explain the absence of a peak splitting for orthogonal edge
states. In contrast, the contribution of a Se $p_{x}$ and $p_{y}$ orbital
momentum Dirac points at $\overline{X}$ and $\overline{Y}$ has been described
theoretically Wojek  _et al._ (2013); Zeljkovic  _et al._ (2014); Rechciński
and Buczko (2018) and nicely corresponds with our experimental findings. In
order to test this interpretation we investigated intersections of two single-
atomic steps at angles below but close to $90^{\circ}$. Indeed, for single-
atomic steps that cross under a $60^{\circ}$ angle we observe a very weak but
detectable splitting, see Ref. Sup,  (Note 8). It is not clear, however, if
the spin-orbit coupling in (Pb,Sn)Se plays any role. This question remains to
be clarified by future theoretical investigations.

Conclusion— In conclusion, we have studied the interaction of adjacent 1D
spin-polarized edge states which are localized at single-atomic step-edges of
the TCI Pb0.7Sn0.3Se(001). Coupling at a step–to-step distance $d_{\rm
ss}\lesssim 25$ nm leads to the energy splitting of the single peak in LDOS at
the Dirac point into two. The energy splitting increases exponentially with
the reducing $d_{\rm ss}$ and eveals the localized character of the
topological edge mode, instead of single quantum state. At a very small
separation ($d_{\rm ss}\lesssim 10$ nm) the two peaks strongly decrease in
intensity. No peak splitting is found for perpendicular single-atomic step
edges, suggesting an important role of the Se $p_{x}$ and $p_{y}$ orbital
momentum of the edge state.

###### Acknowledgements.

We acknowledge the contribution of P. Dziawa, J. Korczak, and R. Minikayev to
crystal growth and characterization. The work was supported by the DFG through
SFB 1180 (project A02). We acknowledge financial support by the Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s
Excellence Strategy through Würzburg-Dresden Cluster of Excellence on
Complexity and Topology in Quantum Matter – ct.qmat (EXC 2147, project-id
390858490). TS acknowledges the support from Foundation for Polish Science
through IRA Programme co-financed by EU within Smart Growth Operational
Programme.

## References

  * König  _et al._ (2007) M. König, S. Wiedmann, C. Brüne, A. Roth, H. Buhmann, L. W. Molenkamp, X.-L. Qi, and S.-C. Zhang, “Quantum spin hall insulator state in HgTe quantum wells,” Science 318, 766–770 (2007).
  * Knez  _et al._ (2011) I. Knez, R.-R. Du, and G. Sullivan, “Evidence for helical edge modes in inverted $\mathrm{InAs}/\mathrm{GaSb}$ quantum wells,” Phys. Rev. Lett. 107, 136603 (2011).
  * Bauer and Bobisch (2016) S. Bauer and C. A. Bobisch, “Nanoscale electron transport at the surface of a topological insulator,” Nature Comm. 7, 11381 (2016).
  * Hou  _et al._ (2019) Y. Hou, R. Wang, R. Xiao, L. McClintock, H. C. Travaglini, J. P. Francia, H. Fetsch, O. Erten, S. Y. Savrasov, B. Wang, A. Rossi, I. Vishik, E. Rotenberg, and D. Yu, “Millimetre-long transport of photogenerated carriers in topological insulators,” Nature Comm. 10, 5723 (2019).
  * Liu  _et al._ (2019) P. Liu, J. R. Williams, and J. J. Cha, “Topological nanomaterials,” Nature Rev. Mater. 4, 479–496 (2019).
  * Plucinski (2019) L. Plucinski, “Band structure engineering in 3D topological insulators,” Jour. Phys.: Cond. Matter 31, 183001 (2019).
  * Liu  _et al._ (2020) C.-W. Liu, Z. Wang, R. L. J. Qiu, and X. P. A. Gao, “Development of topological insulator and topological crystalline insulator nanostructures,” Nanotechn. 31, 192001 (2020).
  * Fu (2011) L. Fu, “Topological crystalline insulators,” Phys. Rev. Lett. 106, 106802 (2011).
  * Dziawa  _et al._ (2012) P. Dziawa, B. J. Kowalski, K. Dybko, R. Buczko, A. Szczerbakow, M. Szot, E. Lusakowska, T. Balasubramanian, B. M. Wojek, M. H. Berntsen, O. Tjernberg, and T. Story, “Topological crystalline insulator states in Pb1-xSnxSe,” Nature Mater. 11, 1023 (2012).
  * Xu  _et al._ (2012) S.-Y. Xu, C. Liu, N. A., M. Neupane, D. Qian, I. Belopolski, J.D. Denlinger, Y.J. Wang, H. Lin, L.A. Wray, G. Landolt, B. Slomski, J.H. Dil, A. Marcinkova, E. Morosan, Q. Gibson, R. Sankar, F.C. Chou, R.J. Cava, A. Bansil, and M.Z. Hasan, “Observation of a topological crystalline insulator phase and topological phase transition in Pb1-xSnxTe,” Nature Comm. 3, 1192 (2012).
  * Tanaka  _et al._ (2012) Y. Tanaka, Z. Ren, T. Sato, K. Nakayama, S. Souma, T. Takahashi, K. Segawa, and Y. Ando, “Experimental realization of a topological crystalline insulator in SnTe,” Nature Phys. 8, 800–803 (2012).
  * Weng  _et al._ (2015) H. Weng, C. Fang, Z. Fang, B. A. Bernevig, and X. Dai, “Weyl semimetal phase in noncentrosymmetric transition-metal monophosphides,” Phys. Rev. X 5, 011029 (2015).
  * Xu  _et al._ (2015) S.-Y. Xu, I. Belopolski, N. Alidoust, M. Neupane, G. Bian, C. Zhang, R. Sankar, G. Chang, Z. Yuan, C.-C. Lee, S.-M. Huang, H. Zheng, J. Ma, D. S. Sanchez, B. K. Wang, A. Bansil, F. Chou, P. P. Shibayev, H. Lin, S. Jia, and M. Z. Hasan, “Discovery of a Weyl fermion semimetal and topological Fermi arcs,” Science 349, 613–617 (2015).
  * Lv  _et al._ (2015a) B. Q. Lv, H. M. Weng, B. B. Fu, X. P. Wang, H. Miao, J. Ma, P. Richard, X. C. Huang, L. X. Zhao, G. F. Chen, Z. Fang, X. Dai, T. Qian, and H. Ding, “Experimental discovery of Weyl semimetal TaAs,” Phys. Rev. X 5, 031013 (2015a).
  * Yang  _et al._ (2015) L. X. Yang, Z. K. Liu, Y. Sun, H. Peng, H. F. Yang, T. Zhang, B. Zhou, Y. Zhang, Y. F. Guo, M. Rahn, D. Prabhakaran, Z. Hussain, S. K. Mo, C. Felser, B. Yan, and Y. L. Chen, “Weyl semimetal phase in the non-centrosymmetric compound TaAs,” Nature Phys. 11, 728–732 (2015).
  * Lv  _et al._ (2015b) B. Q. Lv, N. Xu, H. M. Weng, J. Z. Ma, P. Richard, X. C. Huang, L. X. Zhao, G. F. Chen, C. E. Matt, F. Bisti, V. N. Strocov, J. Mesot, Z. Fang, X. Dai, T. Qian, M. Shi, and H. Ding, “Observation of Weyl nodes in TaAs,” Nature Phys. 11, 724–727 (2015b).
  * Sessi  _et al._ (2017) P. Sessi, Y. Sun, T. Bathon, F. Glott, Z. Li, H. Chen, L. Guo, X. Chen, M. Schmidt, C. Felser, B. Yan, and M. Bode, “Impurity screening and stability of Fermi arcs against Coulomb and magnetic scattering in a Weyl monopnictide,” Phys. Rev. B 95, 035114 (2017).
  * Drozdov  _et al._ (2014) I. K. Drozdov, A. Alexandradinata, S. Jeon, S. Nadj-Perge, H. Ji, R. J. Cava, A. B. Bernevig, and A. Yazdani, “One-dimensional topological edge states of bismuth bilayers,” Nature Physics 10, 664–669 (2014).
  * Schindler  _et al._ (2018) F. Schindler, Z. Wang, M. G. Vergniory, A. M. Cook, A. Murani, S. Sengupta, A. Y. Kasumov, R. Deblock, S. Jeon, I. Drozdov, H. Bouchiat, S. Guéron, A. Yazdani, B. A. Bernevig, and T. Neupert, “Higher-order topology in bismuth,” Nature Phys. 14, 918–924 (2018).
  * Pauly  _et al._ (2015) C. Pauly, B. Rasche, K. Koepernik, M. Liebmann, M. Pratzer, M. Richter, J. Kellner, M. Eschbach, B. Kaufmann, L. Plucinski, C. M. Schneider, M. Ruck, J. van den Brink, and M. Morgenstern, “Subnanometre-wide electron channels protected by topology,” Nature Physics 11, 338–343 (2015).
  * Sessi  _et al._ (2016) P. Sessi, D. Di Sante, A. Szczerbakow, F. Glott, S. Wilfert, H. Schmidt, T. Bathon, P. Dziawa, M. Greiter, T. Neupert, G. Sangiovanni, T. Story, R. Thomale, and M. Bode, “Robust spin-polarized midgap states at step edges of topological crystalline insulators,” Science 354, 1269–1273 (2016).
  * Hirahara  _et al._ (2011) T. Hirahara, G. Bihlmayer, Y. Sakamoto, M. Yamada, H. Miyazaki, S. Kimura, S. Blügel, and S. Hasegawa, “Interfacing 2D and 3D topological insulators: Bi(111) bilayer on ${\mathrm{Bi}}_{2}{\mathrm{Te}}_{3}$,” Phys. Rev. Lett. 107, 166801 (2011).
  * Yang  _et al._ (2012) F. Yang, L. Miao, Z. F. Wang, M.-Y. Yao, F. Zhu, Y. R. Song, M.-X. Wang, J.-P. Xu, A. V. Fedorov, Z. Sun, G. B. Zhang, C. Liu, F. Liu, D. Qian, C. L. Gao, and J.-F. Jia, “Spatial and energy distribution of topological edge states in single Bi(111) bilayer,” Phys. Rev. Lett. 109, 016801 (2012).
  * Kim  _et al._ (2014) S. H. Kim, K.-H. Jin, J. Park, J. S. Kim, S.-H. Jhi, T.-H. Kim, and H. W. Yeom, “Edge and interfacial states in a two-dimensional topological insulator: Bi(111) bilayer on ${\mathrm{Bi}}_{2}{\mathrm{Te}}_{2}\mathrm{Se}$,” Phys. Rev. B 89, 155436 (2014).
  * Hsu  _et al._ (2017) Y.-T. Hsu, K. Park, and E.-A. Kim, “Hybridization-induced interface states in a topological-insulator–ferromagnetic-metal heterostructure,” Phys. Rev. B 96, 235433 (2017).
  * Liu  _et al._ (2014) J. Liu, T. H. Hsieh, P. Wei, W. Duan, J. Moodera, and L. Fu, “Spin-filtered edge states with an electrically tunable gap in a two-dimensional topological crystalline insulator,” Nature Mater. 13, 178–183 (2014).
  * Zhang  _et al._ (2010) Y. Zhang, K. He, C.-Z. Chang, C.-L. Song, L.-L. Wang, X. Chen, J.-F. Jia, Z. Fang, X. Dai, W.-Y. Shan, S.-Q. Shen, Q. Niu, X.-L. Qi, S.-C. Zhang, X.-C. Ma, and Q.-K. Xue, “Crossover of the three-dimensional topological insulator Bi2Se3 to the two-dimensional limit,” Nature Phys. 6, 584–588 (2010).
  * Wojek  _et al._ (2014) B. M. Wojek, P. Dziawa, B. J. Kowalski, A. Szczerbakow, A. M. Black-Schaffer, M. H. Berntsen, T. Balasubramanian, T. Story, and O. Tjernberg, “Band inversion and the topological phase transition in (Pb,Sn)Se,” Phys. Rev. B 90, 161202 (2014).
  * Hsieh  _et al._ (2012) T. H. Hsieh, H. Lin, J. Liu, W. Duan, A. Bansil, and L. Fu, “Topological crystalline insulators in the SnTe material class,” Nature Comm. 3, 982 (2012).
  * Liu  _et al._ (2013) J. Liu, W. Duan, and L. Fu, “Two types of surface states in topological crystalline insulators,” Phys. Rev. B 88, 241303 (2013).
  * Wang  _et al._ (2013) Y. J. Wang, W.-F. Tsai, H. Lin, S.-Y. Xu, M. Neupane, M. Z. Hasan, and A. Bansil, “Nontrivial spin texture of the coaxial Dirac cones on the surface of topological crystalline insulator SnTe,” Phys. Rev. B 87, 235317 (2013).
  * Safaei  _et al._ (2013) S. Safaei, P. Kacman, and R. Buczko, “Topological crystalline insulator (Pb,Sn)Te: Surface states and their spin polarization,” Phys. Rev. B 88, 045305 (2013).
  * Okada  _et al._ (2013) Y. Okada, M. Serbyn, H. Lin, D. Walkup, W. Zhou, C. Dhital, M. Neupane, S. Xu, Y. J. Wang, R. Sankar, F. Chou, A. Bansil, M. Z. Hasan, S. D. Wilson, L. Fu, and V. Madhavan, “Observation of Dirac node formation and mass acquisition in a topological crystalline insulator,” Science 341, 1496–1499 (2013).
  * (34) See supplemental material for detailed information regarding the electronic structure of odd- and even-atomic step edges, details regarding crystal synthesis and characterization, further data obtained on wedge-shaped islands and depressions, the asymmetric energy splitting observed for two parallel single-atomic steps, our model of two interacting 1D edge modes, single-atomic steps crossing under angles of about $60^{\circ}$ and $90^{\circ}$, and fluctuations of the Dirac energy.
  * Iaia  _et al._ (2019) D. Iaia, C.-Y. Wang, Y. Maximenko, D. Walkup, R. Sankar, F. Chou, Y.-M. Lu, and V. Madhavan, “Topological nature of step-edge states on the surface of the topological crystalline insulator ${\mathrm{Pb}}_{0.7}{\mathrm{Sn}}_{0.3}\mathrm{Se}$,” Phys. Rev. B 99, 155116 (2019).
  * Rechciński and Buczko (2018) R. Rechciński and R. Buczko, “Topological states on uneven (Pb,Sn)Se (001) surfaces,” Phys. Rev. B 98, 245302 (2018).
  * Beidenkopf  _et al._ (2011) H. Beidenkopf, P. Roushan, J. Seo, L. Gorman, I. Drozdov, Y. S. Hor, R. J. Cava, and A. Yazdani, “Spatial fluctuations of helical Dirac fermions on the surface of topological insulators,” Nature Phys. 7, 939–943 (2011).
  * Storz  _et al._ (2016) O. Storz, A. Cortijo, S. Wilfert, K. A. Kokh, O. E. Tereshchenko, M. A. H. Vozmediano, M. Bode, F. Guinea, and P. Sessi, “Mapping the effect of defect-induced strain disorder on the Dirac states of topological insulators,” Phys. Rev. B 94, 121301 (2016).
  * Nilsson  _et al._ (2018) M. Nilsson, F. V. Boström, S. Lehmann, K. A. Dick, M. Leijnse, and C. Thelander, “Tuning the two-electron hybridization and spin states in parallel-coupled InAs quantum dots,” Phys. Rev. Lett. 121, 156802 (2018).
  * Coy Diaz  _et al._ (2015) H. Coy Diaz, J. Avila, C. Chen, R. Addou, M. C. Asensio, and M. Batzill, “Direct observation of interlayer hybridization and Dirac relativistic carriers in graphene/MoS2 van der Waals heterostructures,” Nano Letters 15, 1135–1140 (2015).
  * Note (1) We would like to note that we cannot exclude that similar or even better results can be obtained by more elaborate models, where the electrostatic interaction becomes less relevant and is instead replaced by a behavior governed by spatially varying topological indices.
  * Zeljkovic  _et al._ (2014) I. Zeljkovic, Y. Okada, C.-Y. Huang, R. Sankar, D. Walkup, W. Zhou, M. Serbyn, F. Chou, W.-F. Tsai, H. Lin, A. Bansil, L. Fu, M. Z. Hasan, and V. Madhavan, “Mapping the unconventional orbital texture in topological crystalline insulators,” Nature Phys. 10, 572–577 (2014).
  * Wojek  _et al._ (2013) B. M. Wojek, R. Buczko, S. Safaei, P. Dziawa, B. J. Kowalski, M. H. Berntsen, T. Balasubramanian, M. Leandersson, A. Szczerbakow, P. Kacman, T. Story, and O. Tjernberg, “Spin-polarized (001) surface states of the topological crystalline insulator Pb0.73Sn0.27Se,” Phys. Rev. B 87, 115106 (2013).