# Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal
State Regularization

Youngwoon Lee1 Joseph J. Lim1 Anima Anandkumar2,3 Yuke Zhu2,4
1University of Southern California 2NVIDIA
3California Institute of Technology 4The University of Texas at Austin Work
done during an internship at NVIDIA

###### Abstract

Skill chaining is a promising approach for synthesizing complex behaviors by
sequentially combining previously learned skills. Yet, a naive composition of
skills fails when a policy encounters a starting state never seen during its
training. For successful skill chaining, prior approaches attempt to widen the
policy’s starting state distribution. However, these approaches require larger
state distributions to be covered as more policies are sequenced, and thus are
limited to short skill sequences. In this paper, we propose to chain multiple
policies without excessively large initial state distributions by regularizing
the terminal state distributions in an adversarial learning framework. We
evaluate our approach on two complex long-horizon manipulation tasks of
furniture assembly. Our results have shown that our method establishes the
first model-free reinforcement learning algorithm to solve these tasks;
whereas prior skill chaining approaches fail. The code and videos are
available at https://clvrai.com/skill-chaining.

> Keywords: Long-Horizon Manipulation, Skill Chaining, Reinforcement Learning

##  1 Introduction

Deep reinforcement learning (RL) presents a promising framework for learning
impressive robot behaviors [1, 2, 3, 4]. Yet, learning a complex long-horizon
task using a single control policy is still challenging mainly due to its high
computational costs and the exploration burdens of RL models [5]. A more
practical solution is to decompose a whole task into smaller chunks of
subtasks, learn a policy for each subtask, and sequentially execute the
subtasks to accomplish the entire task [6, 7, 8, 9].

However, naively executing one policy after another would fail when the
subtask policy encounters a starting state never seen during training [6, 7,
9]. In other words, a terminal state of one subtask may fall outside of the
set of starting states that the next subtask policy can handle, and thus fail
to accomplish the subtask, as illustrated in Figure 1(a). Especially in robot
manipulation, complex interactions between a high-DoF robot and multiple
objects could lead to a wide range of robot and object configurations, which
are infeasible to be covered by a single policy [10]. Therefore, skill
chaining with policies with limited capability is not trivial and requires
adapting the policies to make them suitable for sequential execution.

To resolve the mismatch between the terminal state distribution (i.e.
termination set) of one policy and the initial state distribution (i.e.
initiation set) of the next policy, prior skill chaining approaches have
attempted to learn to bring an agent to a suitable starting state [7],
discover a chain of options [11, 12], jointly fine-tune policies to
accommodate larger initiation sets that encompass terminal states of the
preceding policy [6], or utilize modulated skills for smooth transition
between skills [13, 14, 15, 16, 9]. Although these approaches can widen the
initiation sets to smoothly sequence several subtask policies, it quickly
becomes infeasible as the larger initiation set often leads to an even larger
termination set, which is cascaded along the chain of policies, as illustrated
in Figure 1(c).

Instead of enlarging the initiation set to encompass a termination set
modelled as a simple Gaussian distribution [6], we propose to keep the
termination set small and near the initiation set of the next policy. This can
prevent the termination sets from becoming too large to be covered by the
subsequent policies, especially when executing a long sequence of skills;
thus, fine-tuning of subtask polices becomes more sample efficient. As a
result, small changes in the initiation sets of subsequent policies are
sufficient to successfully execute a series of subtask policies.

To this end, we devise an adversarial learning framework that learns an
initiation set discriminator to distinguish the initiation set of the policy
that follows from terminal states, and uses it as a regularization for
encouraging the terminal states of a policy to be near the initiation set of
the next policy. With this terminal state regularization, the pretrained
subtask policies are iteratively fine-tuned to solve the subtask with the new
initiation set while keep the terminal states of a policy small enough to be
covered by the initiation set of the subsequent policy. As a result, our model
is capable of chaining a sequence of closed-loop policies to accomplish a
collection of multi-stage IKEA furniture assembly tasks [17] that require
high-dimensional continuous control under contact-rich dynamics.

In summary, our contributions are threefold:

  * •

We propose a novel adversarial skill chaining framework with Terminal STAte
Regularization, T-STAR, for learning long-horizon and hierarchical
manipulation tasks.

  * •

We demonstrate that the proposed method can learn a long-horizon manipulation
task, furniture assembly of two different types of furniture. Our terminal
state regularization algorithms improves the success rate of the policy
sequencing method from 0% to 56% for chair_ingolf and from 59% to 87% for
table_lack. To our best knowledge, this is the first empirical result of a
model-free RL method that solves these furniture assembly tasks without manual
engineering.

  * •

We present comprehensive comparisons with prior skill composition methods and
qualitative visualizations to analyze our model performances.

(a) Naive execution of $\pi_{i}$

(b) Widen the initiation set of $\pi_{i}$ (Policy Sequencing)

(c) Update $\pi_{i}$ with terminal state regularization (Ours)

Figure 1:  We aim to solve a long-horizon task, e.g., furniture assembly,
using independently trained subtask policies. (a) Each subtask policy,
$\pi_{i}$, works successfully only on its initiation set (green),
$\mathcal{I}_{i}$, and results in its termination set (pink), $\beta_{i}$;
thus, it fails when performed outside of $\mathcal{I}_{i}$ (red curve). (b) To
enable sequencing policies, a subsequent policy $\pi_{i}$ needs to widen its
initiation set to cover the termination set of the prior policy $\beta_{i-1}$.
But this can result in an increase of its termination set $\beta_{i}$, which
makes fine-tuning of the following policy $\pi_{i+1}$ even more challenging.
This effect is exacerbated when more policies are chained together. (c) During
fine-tuning of a policy $\pi_{i}$, we regularize the terminal state
distribution $\beta_{i}$ to be close to the initiation set of the next policy
$\mathcal{I}_{i+1}$. In contrast to the boundless increase of
$\tilde{\beta}_{i}$ in (b), our approach effectively keeps the required
initiation set small over the chain of policies with the terminal state
regularization.

##  2 Related Work

Deep reinforcement learning (RL) for continuous control [18, 19, 20] is an
active research area. While some complex tasks can be solved based on a reward
function, undesired behaviors often emerge [21] when tasks require several
different primitive skills. Moreover, learning such complex tasks becomes
computationally impractical as the tasks become long and complicated due to
the credit assignment problem and the large exploration space.

Imitation learning aims to reduce this complexity of exploration and the
difficulty of learning from reward signal. Behavioral cloning approaches [22,
23, 24, 25] greedily imitate the expert policy and therefore suffer from
accumulated errors, causing a drift away from states seen in the
demonstrations. On the other hand, inverse reinforcement learning [26, 27, 28]
and adversarial imitation learning approaches [29, 30] encourage the agent to
imitate expert trajectories with a learned reward function, which can better
handle the compounding errors. Specifically, generative adversarial imitation
learning (GAIL) [29] and its variants [30, 31] show improved demonstration
efficiency by training a discriminator to distinguish expert versus agent
transitions and using the discriminator output as a reward for policy
training. Although these imitation learning methods can learn simple
locomotion behaviors [29] and a handful of short-horizon manipulation tasks
[32, 31], these methods easily overfit to local optima and still suffer from
temporal credit assignment and accumulated errors for long-horizon tasks.

Instead of learning an entire task using a single policy, we can tackle the
task by decomposing it into easier and reusable subtasks. Hierarchical
reinforcement learning does this by decomposing a task into a sequence of
temporally extended macro actions. It often consists of one high-level policy
and a set of low-level policies, such as in the options framework [33], in
which the high-level policy decides which low-level policy to activate and the
chosen low-level policy generates a sequence of atomic actions until the high-
level policy switches it to another low-level policy. Options can be
discovered without supervision [34, 35, 36, 37, 11, 12], learned from data
[38, 39, 40], meta-learned [41], pre-defined [7, 42, 43, 44], or attained
through task structure supervision [45, 10]. For long-horizon planning
capability, the high-level policy can be trained using traditional planning
methods [46, 47].

To synthesize complex motor skills with a set of predefined skills, Lee et al.
[7] learns to find smooth transitions between subtask policies. This method
assumes the subtask policies are fixed, which makes learning such transitions
feasible but, at the same time, leads to failure of smooth transition when an
external state has to be changed or the end state of the prior subtask policy
is too far away from the initiation set of the following subtask policy. On
the other hand, Clegg et al. [6] proposes to sequentially improve subtask
policies to cover the terminal states of previous subtask policies. While this
method fails when the termination set of the prior policy becomes extremely
large, our method prevents such boundless expansion of the termination set.

Closely related to our work, prior skill chaining methods [11, 12, 38] have
proposed to discover a new option that ends with an initiation set of the
previous option. With newly discovered options, the agent can reach the goal
from more initial states. However, these methods have a similar issue with
Clegg et al. [6] that discovering a new option requires a large enough
initiation set of the previous option, which is cascaded along the chain of
options.

Furniture assembly is a challenging robotics task requiring reliable 3D
perception, high-level planning, and sophisticated control. Prior works [48,
49, 50] tackle this problem by manually programming the high-level plan and
learning only a subset of low-level controls. In this paper, we propose to
tackle the furniture assembly task in simulation [17] by learning all the low-
level control skills and then effectively sequencing these skills.

##  3 Approach

In this paper, we aim to address the problem of chaining multiple policies for
long-horizon complex manipulation tasks, especially in furniture assembly
[17]. Sequentially executing skills often fails when a policy encounters a
starting state (i.e. a terminal state of the preceding policy) never seen
during its training. The subsequent policy can learn to address these new
states, but this may require even larger states to be covered by the following
policies. To chain multiple skills without requiring boundlessly large
initiation sets of subtask policies, we introduce a novel adversarial skill
chaining framework that constrains the terminal state distribution, as
illustrated in Figure 2. Our approach first learns each subtask using subtask
rewards and demonstrations (Section 3.2); it then adjusts the subtask policies
to effectively chain them to complete the whole task via the terminal state
regularization (Section 3.3).

###  3.1 Preliminaries

We formulate our learning problem as a Markov Decision Process [51] defined
through a tuple $(\mathcal{S},\mathcal{A},R,P,\rho_{0},\gamma)$ for the state
space $\mathcal{S}$, action space $\mathcal{A}$, reward function
$R(s,a,s^{\prime})$, transition distribution $P(s^{\prime}\lvert s,a)$,
initial state distribution $\rho_{0}$, and discount factor $\gamma$. We define
a policy $\pi:\mathcal{S}\rightarrow\mathcal{A}$ that maps from states to
actions and correspondingly moves an agent to a new state according to the
transition probabilities. The policy is trained to maximize the expected sum
of discounted rewards
$\mathbb{E}_{(s,a)\sim\pi}\left[\sum_{t=0}^{T-1}\gamma^{t}R(s_{t},a_{t},s_{t+1})\right]$,
where $T$ is the episode horizon. Each policy comes with an initiation set
$\mathcal{I}\subset\mathcal{S}$ and termination set $\beta\subset\mathcal{S}$,
where the initiation set $\mathcal{I}$ contains all initial states that lead
to successful execution of the policy and the termination set $\beta$ consists
of all final states of successful executions. We assume the environment
provides a success indicator of each subtask and this can be easily inferred
from the final state, e.g., two parts are aligned and the connect action is
activated. In addition to the reward function, we assume the learner receives
a fixed set of expert demonstrations,
$\mathcal{D}^{e}=\\{\tau_{1}^{e},\dots,\tau_{N}^{e}\\}$, where a demonstration
is a sequence of state-action pairs,
$\tau_{j}^{e}=(s_{0},a_{0},\dots,s_{T_{j}-1},a_{T_{j}-1},s_{T_{j}})$.

Figure 2:  Our adversarial skill chaining framework regularizes the terminal
state distribution to be close to the initiation set of the subsequent
subtask. The initiation set discriminator models the initiation set
distribution by discerning the initiation set and states in agent
trajectories, while the policy learns to reach states close to the initiation
set by augmenting the reward with the discriminator output, dubbed terminal
state regularization. Our method jointly trains all policies and initiation
set discriminators, pushing the termination set close to the initiation set of
the next policy, which leads to smaller changes required for the policies that
follow, especially effective in a long chain of skills.

###  3.2 Learning Subtask Policies

To solve a complicated long-horizon task (e.g. assembling a table), we
decompose the task into smaller subtasks (e.g. assembling a table leg to a
table top), learn a policy $\pi^{i}_{\theta}$ for each subtask
$\mathcal{M}_{i}$, and chain these subtask policies. Learning each subtask
solely from reward signals is still challenging in robot manipulation due to a
huge state space of the robot and objects to explore, and complex robot
control and dynamics. Thus, for efficient exploration, we use an adversarial
imitation learning approach, GAIL [29], which encourages the agent to stay
near the expert trajectories using a learned reward by discerning expert and
agent behaviors. Together with reinforcement learning, the policy can
efficiently learn to solve the subtask even on states not covered by the
demonstrations.

More specifically, we train each subtask policy $\pi^{i}_{\theta}$ on $R^{i}$
following the reward formulation proposed in [52, 53], which uses the weighted
sum of the environment and GAIL rewards:

$R^{i}(s_{t},a_{t},s_{t+1};\phi)=\lambda_{1}R^{i}_{ENV}(s_{t},a_{t},s_{t+1})+\lambda_{2}R^{i}_{GAIL}(s_{t},a_{t};\phi),$
(1)

where
$R^{i}_{GAIL}(s_{t},a_{t};\phi)=1-0.25\cdot[f^{i}_{\phi}(s_{t},a_{t})-1]^{2}$
is the predicted reward by the GAIL discriminator $f^{i}_{\phi}$ [53], and
$\lambda_{1}$ and $\lambda_{2}$ are hyperparameters that balance between the
reinforcement learning and imitation learning objectives. We found this reward
formulation [53] works most stable among variants of GAIL [29] in our
experiments thanks to its bounded reward between $[0,1]$. The discriminator is
trained using the following objective:
$\min_{f_{\phi}^{i}}\mathbb{E}_{(s)\sim\mathcal{D}^{e}}\left[(f^{i}_{\phi}(s)-1)^{2}\right]+\mathbb{E}_{(s)\sim\pi^{i}}\left[(f^{i}_{\phi}(s)+1)^{2}\right]$.

Due to computational limitations, training a subtask policy for all possible
initial states is impractical, and hence it can cause failure on states unseen
during training. Instead of indefinitely increasing the initiation set, we
first train the policy on a limited set of initial states (e.g. predefined
initial states with small noise), and later fine-tune the policy on the set of
initial states required for skill chaining as described in the following
section. This pretraining of subtask policies ensures the quality of the
pretrained skills and makes the fine-tuning stage of our method easy and
efficient.

Algorithm 1 T-STAR: Skill chaining via terminal state regularization

0: Expert demonstrations $\mathcal{D}^{e}_{1},\dots,\mathcal{D}^{e}_{K}$,
subtask MDPs $\mathcal{M}_{1},\dots,\mathcal{M}_{K}$

1: Initialize subtask policies $\pi^{1}_{\theta},\dots,\pi^{K}_{\theta}$, GAIL
discriminators $f^{1}_{\phi},\dots,f^{K}_{\phi}$, initiation set
discriminators $D^{1}_{\omega},\dots,D^{K}_{\omega}$, initial state buffers
$\mathcal{B}_{\mathcal{I}}^{1},\dots,\mathcal{B}_{\mathcal{I}}^{K}$, and
terminal state buffers $\mathcal{B}_{\beta}^{1},\dots,\mathcal{B}_{\beta}^{K}$

2: for each subtask $i=1,...,K$ do

3: while until convergence of $\pi_{\theta}^{i}$ do

4: Rollout trajectories $\tau=(s_{0},a_{0},r_{0},\dots,s_{T})$ with
$\pi^{i}_{\theta}$

5: Update $f^{i}_{\phi}$ with $\tau$ and $\tau^{e}\sim\mathcal{D}^{e}_{i}$
$\triangleright$ Train GAIL discriminator

6: Update $\pi^{i}_{\theta}$ using $R^{i}(s_{t},a_{t},s_{t+1};\phi)$ in
Equation (1) $\triangleright$ Train subtask policy

7: end while

8: end for

9: for iteration $m=0,1,...,M$ do

10: for each subtask $i=1,...,K$ do

11: Sample $s_{0}$ from environment or $\mathcal{B}_{\beta}^{i-1}$

12: Rollout trajectories $\tau=(s_{0},a_{0},r_{0},\dots,s_{T})$ with
$\pi^{i}_{\theta}$

13: if $\tau$ is successful then

14: $\mathcal{B}_{\mathcal{I}}^{i}\leftarrow\mathcal{B}_{\mathcal{I}}^{i}\cup
s_{0},\mathcal{B}_{\beta}^{i}\leftarrow\mathcal{B}_{\beta}^{i}\cup s_{T}$
$\triangleright$ Collect initial and terminal states of successful
trajectories

15: end if

16: Update $f^{i}_{\phi}$ with $\tau$ and $\tau^{e}\sim\mathcal{D}^{e}_{i}$
$\triangleright$ Fine-tune GAIL discriminator

17: Update $D^{i}_{\omega}$ with $s_{\beta}\sim\mathcal{B}_{\beta}^{i-1}$ and
$s_{\mathcal{I}}\sim\mathcal{B}_{\mathcal{I}}^{i}$ $\triangleright$ Train
initiation set discriminator

18: Update $\pi^{i}_{\theta}$ using $R^{i}(s_{t},a_{t},s_{t+1};{\phi,\omega})$
in Equation (3) $\triangleright$ Fine-tune subtask policy with terminal state
regularization

19: end for

20: end for

###  3.3 Skill Chaining with Terminal State Regularization

Once subtask polices are acquired, one can sequentially execute the subtask
policies to complete more complex tasks. However, naively executing the
polices one-by-one would fail since the policies are not trained to be
smoothly connected. As can be seen in Figure 1(a), independently trained
subtask policies only work on a limited range of initial states. Therefore,
the execution of a policy $\pi_{i}$ fails on the terminal states of the
preceding policy $\beta_{i-1}$ outside of its initiation set
$\mathcal{I}_{i}$.

For successful sequential execution of $\pi_{i-1}$ and $\pi_{i}$, the terminal
states of the preceding policy should be included in the initiation set of the
subsequent policy, $\beta_{i-1}\subset\mathcal{I}_{i}$. This can be achieved
either by widening the initiation set of the subsequent policy or by shifting
the terminal state distribution of the preceding policy. However, in robot
manipulation, the set of valid terminal states can be huge with freely located
objects (e.g. a robot can mess up the workplace by moving or throwing other
parts away). This issue is cascaded along the chain of policies, which leads
to the boundlessly large initiation set required for policies, as illustrated
in Figure 1(c).

Therefore, a policy needs to not only increase the initiation set (e.g.
assemble the table leg in diverse configurations) but also regularize the
termination set to be bounded and close to the initiation set of the
subsequent policies (e.g. keep the workplace organized), as described in
Figure 1(c). To this end, we devise an adversarial framework which jointly
trains an initiation set discriminator, $D^{i}_{\omega}(s_{t})$, to
distinguish the terminal states of the preceding policy and the initiation set
of the corresponding policy, and a policy to reach the initiation set of the
subsequent policy with the guidance of the initiation set discriminator. We
train the initiation set discriminator for each policy to minimize the
following objective:
$\mathcal{L}_{i}(\omega)=\mathbb{E}_{s_{\mathcal{I}}\sim\mathcal{I}_{i}}\big{[}D^{i}_{\omega}(s_{\mathcal{I}})-1\big{]}^{2}+\mathbb{E}_{s_{T}\sim\beta_{i-1}}\big{[}D^{i}_{\omega}(s_{T})\big{]}^{2}$.

With the initiation set discriminator, we regularize the terminal state
distribution of the policy by encouraging the policy to reach a terminal state
close to the initiation set of the following policy. The terminal state
regularization can be formulated as following:

$R^{i}_{TSR}(s;\omega)=\mathbbm{1}_{s\in\beta_{i}}D^{i+1}_{\omega}(s)$ (2)

Then, we can rewrite the reward function with the terminal state
regularization:

$\displaystyle
R^{i}(s_{t},a_{t},s_{t+1};\phi,\omega)=\lambda_{1}R^{i}_{ENV}(s_{t},a_{t},s_{t+1})+\lambda_{2}R^{i}_{GAIL}(s_{t},a_{t};\phi)+\lambda_{3}R^{i}_{TSR}(s_{t+1};\omega),$
(3)

where $\lambda_{3}$ is a weighting factor for the terminal state
regularization. The first two terms of this reward function guide a policy to
accomplish the subtask while the terminal state regularization term forces the
termination set to be closer to the initiation set of the following policy.

With this reward function incorporating the terminal state regularization,
subtask policies and GAIL discriminators can be trained to cover unseen
initial states while keeping the termination set closer to the initiation set
of the next policy. Once subtask policies are updated, we collect terminal
states and initiation sets with the updated policies, and train the initiation
set discriminators. We alternate these procedures to smoothly chain subtask
policies, as summarized in Algorithm 1, where changes of our algorithm with
respect to Clegg et al. [6] are marked in red.

##  4 Experiments

In this paper, we propose a skill chaining approach with the terminal state
regularization, which encourages to match the terminal state distribution of
the prior skill with suitable starting states of the following skill. Through
our experiments, we aim to verify our hypothesis that the policy sequencing
fails due to unbounded terminal states, which is cascaded along the sequence
of skills, and show the effectiveness of our framework on learning a long
sequence of complex manipulation skills.

###  4.1 Baselines

We compare our method to the state-of-the-art prior works in reinforcement
learning, imitation learning, and skill composition, which are listed below:

  * •

BC [22] fits a policy to the demonstration actions with supervised learning.

  * •

PPO [54] is a model-free on-policy RL method that learns a policy from the
environment reward.

  * •

GAIL [29] is an adversarial imitation learning approach with a discriminator
trained to distinguish expert and agent state-action pairs $(s,a)$.

  * •

GAIL + PPO uses both the environment and GAIL reward (Equation (1)) to
optimize a policy.

  * •

SPiRL [55] is a hierarchical RL approach that learns fixed-length skills and
skill prior from the dataset, and then learns a downstream task using skill-
prior-regularized RL.

  * •

Policy Sequencing [6] first trains a policy for each subtask independently and
finetunes the policies to cover the terminal states of the previous policy.

  * •

T-STAR (Ours) learns subtask policies simultaneously, leading them to be
smoothly connected using the terminal state regularization.

(a) table_lack

(b) chair_ingolf

Figure 3:  Two furniture assembly tasks [17] consist of four subtasks (four
table legs for table_lack; two seat supports, chair seat, and front legs for
chiar_ingolf).

###  4.2 Tasks

We test our method and baselines with two furniture models, table_lack and
chiar_ingolf, from the IKEA furniture assembly environment [17] as illustrated
in Figure 3:

  * •

table_lack: Four table legs need to be picked up and aligned to the corners of
the table top.

  * •

chair_ingolf: Two chair supports and front legs need to be attached to the
chair seat. Then, the chair seat needs to be attached to the chair back while
avoiding collision to each other.

In our experiments, we define a subtask as assembling one part to another;
thus, we have four subtasks for each task. Subtasks are independently trained
on the initial (object) states sampled from the environment with random noise
ranging from [$-2$cm, $2$cm] and [$-3$°, $3$°] in the $(x,y)$-plane. This
subtask decomposition is given, i.e., the environment can be initialized for
each subtask and the agent is informed whether the subtask is completed and
successful.

For the robotic agent, we use the 7-DoF Rethink Sawyer robot operated via
joint velocity control. For imitation learning, we collected 200
demonstrations for each furniture part assembly with a programmatic assembly
policy. Each demonstration for single-part assembly consists of around
$200$-$900$ steps long due to the long-horizon nature of the task.

The observation space includes robot observations ($29$ dim), object
observations ($35$ dim), and task phase information ($8$ dim). The object
observations contain the positions ($3$ dim) and quaternions ($4$ dim) of all
five furniture pieces in the scene. Once two parts are attached, the
corresponding subtask is completed and the robot arm moves back to its initial
pose in the center of the workplace. Our approach can in theory solve the task
without resetting the robot; however, this resetting behavior effectively
reduces the gap in robot states when switching skills and is available in most
robots.

Table 1:  Average progress of the furniture assembly task. Each subtask amounts to 0.25 progress. Hence, 1 represents successful execution of all four subtasks while 0 means the agent does not achieve any subtask. Our method learns to complete all four subtasks in sequence and outperforms the policy sequencing baseline and standard RL and IL methods. We report the mean and standard deviation across 5 seeds.  | table_lack | chair_ingolf
---|---|---
BC [22] | 0.03 $\pm$ 0.00 | 0.04 $\pm$ 0.01
PPO [54] | 0.09 $\pm$ 0.11 | 0.14 $\pm$ 0.03
GAIL [29] | 0.00 $\pm$ 0.00 | 0.00 $\pm$ 0.00
GAIL + PPO [52] | 0.21 $\pm$ 0.11 | 0.22 $\pm$ 0.08
SPiRL [55] | 0.05 $\pm$ 0.00 | 0.03 $\pm$ 0.00
Policy Sequencing [6] | 0.63 $\pm$ 0.28 | 0.77 $\pm$ 0.12
T-STAR (Ours) | 0.90 $\pm$ 0.07 | 0.89 $\pm$ 0.04

###  4.3 Results

The results in Table 1 show the average progress of the furniture assembly
tasks across 200 testing episodes for 5 different seeds. Since each task
consists of assembling furniture parts four times, completing one furniture
part assembly amounts to task progress of 0.25. Even though the environment
has small noises in furniture initialization, both BC [22] and GAIL [29]
baselines move the robot arm near furniture pieces but struggle at picking up
even one furniture piece. This shows the limitation of BC and GAIL in dealing
with compounding errors in long-horizon tasks with the large state space and
continuous action space. Similarly, the hierarchical skill-based learning
approach, SPiRL, also struggles at learning picking up a single furniture
piece. This can be due to the insufficient amount of data to cover a long
sequence of skills on the large state space, e.g., many freely located
objects. Moreover, due to the difficulty of exploration, the model-free RL
baseline, PPO, rarely learns to assemble one part. On the other hand, the GAIL
+ PPO baseline can consistently learn one-part assembly, but cannot learn to
assemble further parts due to the exploration challenge and temporal credit
assignment problem. These baselines are trained for 200M environment steps (5M
for off-policy SPiRL).

By utilizing pretrained subtask policies with GAIL + PPO (25M steps for each
subtask), skill chaining approaches could achieve improved performance
compared to the single-policy baselines. We train the policy sequencing
baseline and our method for additional 100M steps, which requires in total
200M steps including the pretraining stage. The policy sequencing baseline [6]
achieves 0.63 and 0.77 average task progress, whereas our method achieves 0.90
and 0.89 average task progress on table_lack and chair_ingolf, respectively.
The performance gain of our method comes from the reduced discrepancy between
the termination set and the initiation set of the next subtask thanks to the
terminal state regularization.

We can observe this performance gain even clearer in the success rates. Our
terminal state regularization improves the success rate of policy sequencing
from 0% to 56% for chair_ingolf and from 59% to 87% for table_lack. We observe
that with more skills to be chained, the success rate of the newly chained
skill decreases, especially in chair_ingolf; the policy sequencing baseline
learns to complete the first three subtasks, but fails to learn the last
subtask due to excessively large and shifted initial state distribution, i.e.,
terminal states of the preceding subtask.

###  4.4 Qualitative Results

To analyze the effect of the proposed terminal state regularization, we
visualize the changes in termination sets over training. We first collect 300
terminal states of the third subtask policy on chair_ingolf both for our
method and the policy sequencing baseline for 36M, 39M, 42M, and 45M training
steps. Then, we apply PCA on the object state information in the terminal
states and use the first two principal components to reduce the data
dimension.

Figure 4 shows that our method effectively constrains the terminal state
distribution of a subtask policy. Before 36M training steps, the policy cannot
solve the third subtask due to the shifted initial state distribution by the
second subtask policy. With additional training, at 42M and 45M training
steps, the policy learns to solve the subtask on newly added initial states
for both methods. From 36M to 45M steps of training, the termination set of
the policy sequencing baseline (red) spreads out horizontally after 39M
training steps and vertically after 42M steps. For successful skill chaining,
this wide termination set has to be covered by the following policy, requiring
a large change in the policy and potentially causing an even larger
termination set. In contrast, the terminal state distribution of our method
(blue) does not excessively increase but actually shrinks in this experiment,
which leads to successful execution and efficient adaptation of subsequent
subtask policies.

(a) 36M steps

(b) 39M steps

(c) 42M steps

(d) 45M steps

Figure 4:  To demonstrate the benefit of our terminal state regularization, we
visualize the changes in termination sets over training of the third subtask
policy on chair_ingolf. We plot each terminal state by projecting its object
configuration into 2D space using PCA. Through 36M to 45M training steps, both
the policy sequencing baseline [6] and our method successfully learn to cover
most terminal states from the second subtask. However, without regularization,
the policy sequencing method (red) shows the increasing size of the
termination set (e.g. spread over horizontally at 39M and vertically at 42M
steps) as more initial states are covered by the policy. In contrast, in our
approach (blue), the terminal state distribution is bounded, which shows that
the terminal state regularization can effectively prevent the terminal state
distribution diverging. This bounded termination set makes learning of the
following skills efficient, and thus helps chaining a long sequence of skills.

##  5 Conclusion

We propose T-STAR, a novel adversarial skill chaining framework that addresses
the problem of the increasing size of initiation sets required for executing a
long chain of manipulation skills. To prevent excessively large initiation
sets to be learned, we regularize the terminal state distribution of a subtask
policy to be close to the initiation set of the following subtask policy.
Through terminal state regularization, our approach jointly trains all subtask
policies to ensure that the final state of one policy is a good initial state
for the policy that follows. We demonstrate the effectiveness of our approach
on the challenging furniture assembly tasks, where prior skill chaining
approaches fail. These results are promising and motivate future work on
chaining more skills with diverse skill combinations to tackle complex long-
horizon problems. Another interesting research direction is eliminating
subtask supervision required in our work and discovering subtask decomposition
from large data in a unsupervised learning manner. Finally, transferring our
framework to real robot systems, which involves learning a vision-based
policy, improving sample efficiency, and closing simulation-to-real gaps, is
our definite future work.

#### Acknowledgments

This work was initiated when Youngwoon Lee worked at NVIDIA Research as an
intern. This research is also supported by the Annenberg Fellowship from USC
and the Google Cloud Research Credits program with the award GCP19980904. We
would like to thank Byron Boots for initial discussion, Jim Fan, De-An Huang,
Christopher B. Choy, and NVIDIA AI Algorithms team for their insightful
feedback, and the USC CLVR lab members for constructive feedback.

## References

  * Levine et al. [2016] S. Levine, C. Finn, T. Darrell, and P. Abbeel.  End-to-end training of deep visuomotor policies.  _The Journal of Machine Learning Research_ , 17(1):1334–1373, 2016.
  * Suárez-Ruiz and Pham [2016] F. Suárez-Ruiz and Q.-C. Pham.  A framework for fine robotic assembly.  In _IEEE International Conference on Robotics and Automation_ , pages 421–426, 2016.
  * Rajeswaran et al. [2018] A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine.  Learning complex dexterous manipulation with deep reinforcement learning and demonstrations.  In _Robotics: Science and Systems_ , 2018.
  * Jain et al. [2019] D. Jain, A. Li, S. Singhal, A. Rajeswaran, V. Kumar, and E. Todorov.  Learning deep visuomotor policies for dexterous hand manipulation.  In _IEEE International Conference on Robotics and Automation_ , pages 3636–3643. IEEE, 2019.
  * Andrychowicz et al. [2021] M. Andrychowicz, A. Raichuk, P. Stańczyk, M. Orsini, S. Girgin, R. Marinier, L. Hussenot, M. Geist, O. Pietquin, M. Michalski, S. Gelly, and O. Bachem.  What matters for on-policy deep actor-critic methods? a large-scale study.  In _International Conference on Learning Representations_ , 2021.  URL https://openreview.net/forum?id=nIAxjsniDzg.
  * Clegg et al. [2018] A. Clegg, W. Yu, J. Tan, C. K. Liu, and G. Turk.  Learning to dress: Synthesizing human dressing motion via deep reinforcement learning.  _ACM Transactions on Graphics (TOG)_ , 37(6):1–10, 2018.
  * Lee et al. [2019] Y. Lee, S.-H. Sun, S. Somasundaram, E. S. Hu, and J. J. Lim.  Composing complex skills by learning transition policies.  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=rygrBhC5tQ.
  * Peng et al. [2019] X. B. Peng, M. Chang, G. Zhang, P. Abbeel, and S. Levine.  Mcp: Learning composable hierarchical control with multiplicative compositional policies.  In _Neural Information Processing Systems_ , 2019.
  * Lee et al. [2020] Y. Lee, J. Yang, and J. J. Lim.  Learning to coordinate manipulation skills via skill behavior diversification.  In _International Conference on Learning Representations_ , 2020.
  * Ghosh et al. [2018] D. Ghosh, A. Singh, A. Rajeswaran, V. Kumar, and S. Levine.  Divide-and-conquer reinforcement learning.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=rJwelMbR-.
  * Konidaris and Barto [2009] G. Konidaris and A. Barto.  Skill discovery in continuous reinforcement learning domains using skill chaining.  In _Advances in Neural Information Processing Systems_ , volume 22, pages 1015–1023, 2009.
  * Bagaria and Konidaris [2020] A. Bagaria and G. Konidaris.  Option discovery using deep skill chaining.  In _International Conference on Learning Representations_ , 2020.
  * Pastor et al. [2009] P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal.  Learning and generalization of motor skills by learning from demonstration.  In _IEEE International Conference on Robotics and Automation_ , pages 763–768, 2009.
  * Kober et al. [2010] J. Kober, K. Mülling, O. Krömer, C. H. Lampert, B. Schölkopf, and J. Peters.  Movement templates for learning of hitting and batting.  In _IEEE International Conference on Robotics and Automation_ , pages 853–858, 2010.
  * Mülling et al. [2013] K. Mülling, J. Kober, O. Kroemer, and J. Peters.  Learning to select and generalize striking movements in robot table tennis.  _The International Journal of Robotics Research_ , 32(3):263–279, 2013.
  * Hausman et al. [2018] K. Hausman, J. T. Springenberg, Z. Wang, N. Heess, and M. Riedmiller.  Learning an embedding space for transferable robot skills.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=rk07ZXZRb.
  * Lee et al. [2021] Y. Lee, E. S. Hu, and J. J. Lim.  IKEA furniture assembly environment for long-horizon complex manipulation tasks.  In _IEEE International Conference on Robotics and Automation_ , 2021\.  URL https://clvrai.com/furniture.
  * Lillicrap et al. [2015] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra.  Continuous control with deep reinforcement learning.  _arXiv preprint arXiv:1509.02971_ , 2015.
  * Schulman et al. [2015] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz.  Trust region policy optimization.  In _International Conference on Machine Learning_ , pages 1889–1897, 2015.
  * Haarnoja et al. [2018] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine.  Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor.  In _International Conference on Machine Learning_ , pages 1856–1865, 2018.
  * Riedmiller et al. [2018] M. Riedmiller, R. Hafner, T. Lampe, M. Neunert, J. Degrave, T. Van de Wiele, V. Mnih, N. Heess, and J. T. Springenberg.  Learning by playing-solving sparse reward tasks from scratch.  _arXiv preprint arXiv:1802.10567_ , 2018.
  * Pomerleau [1989] D. A. Pomerleau.  Alvinn: An autonomous land vehicle in a neural network.  In _Advances in Neural Information Processing Systems_ , pages 305–313, 1989.
  * Schaal [1997] S. Schaal.  Learning from demonstration.  In _Advances in Neural Information Processing Systems_ , pages 1040–1046, 1997.
  * Finn et al. [2016] C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel.  Deep spatial autoencoders for visuomotor learning.  In _IEEE International Conference on Robotics and Automation_ , pages 512–519. IEEE, 2016.
  * Pathak* et al. [2018] D. Pathak*, P. Mahmoudieh*, M. Luo*, P. Agrawal*, D. Chen, F. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell.  Zero-shot visual imitation.  In _International Conference on Learning Representations_ , 2018.
  * Ng and Russell [2000] A. Y. Ng and S. J. Russell.  Algorithms for inverse reinforcement learning.  In _International Conference on Machine Learning_ , volume 1, page 2, 2000.
  * Abbeel and Ng [2004] P. Abbeel and A. Y. Ng.  Apprenticeship learning via inverse reinforcement learning.  In _International Conference on Machine Learning_ , 2004.
  * Ziebart et al. [2008] B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey.  Maximum entropy inverse reinforcement learning.  In _Association for the Advancement of Artificial Intelligence_ , volume 8, pages 1433–1438, 2008.
  * Ho and Ermon [2016] J. Ho and S. Ermon.  Generative adversarial imitation learning.  In _Advances in Neural Information Processing Systems_ , pages 4565–4573, 2016.
  * Fu et al. [2018] J. Fu, K. Luo, and S. Levine.  Learning robust rewards with adversarial inverse reinforcement learning.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=rkHywl-A-.
  * Kostrikov et al. [2019] I. Kostrikov, K. K. Agrawal, D. Dwibedi, S. Levine, and J. Tompson.  Discriminator-actor-critic: Addressing sample inefficiency and reward bias in adversarial imitation learning.  In _International Conference on Learning Representations_ , 2019.
  * Zolna et al. [2020] K. Zolna, S. Reed, A. Novikov, S. G. Colmenarej, D. Budden, S. Cabi, M. Denil, N. de Freitas, and Z. Wang.  Task-relevant adversarial imitation learning.  In _Conference on Robot Learning_ , 2020.
  * Sutton et al. [1999] R. S. Sutton, D. Precup, and S. Singh.  Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning.  _Artificial intelligence_ , 112(1-2):181–211, 1999.
  * Schmidhuber [1990] J. Schmidhuber.  _Towards compositional learning with dynamic neural networks_.  Inst. für Informatik, 1990.
  * Bacon et al. [2017] P.-L. Bacon, J. Harb, and D. Precup.  The option-critic architecture.  In _Association for the Advancement of Artificial Intelligence_ , pages 1726–1734, 2017.
  * Nachum et al. [2018] O. Nachum, S. S. Gu, H. Lee, and S. Levine.  Data-efficient hierarchical reinforcement learning.  In _Advances in Neural Information Processing Systems_ , pages 3303–3313, 2018.
  * Levy et al. [2019] A. Levy, G. Konidaris, R. Platt, and K. Saenko.  Learning multi-level hierarchies with hindsight.  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=ryzECoAcY7.
  * Konidaris et al. [2012] G. Konidaris, S. Kuindersma, R. Grupen, and A. Barto.  Robot learning from demonstration by constructing skill trees.  _The International Journal of Robotics Research_ , 31(3):360–375, 2012.
  * Kipf et al. [2019] T. Kipf, Y. Li, H. Dai, V. Zambaldi, A. Sanchez-Gonzalez, E. Grefenstette, P. Kohli, and P. Battaglia.  Compile: Compositional imitation learning and execution.  In _International Conference on Machine Learning_ , pages 3418–3428. PMLR, 2019.
  * Lu et al. [2021] Y. Lu, Y. Shen, S. Zhou, A. Courville, J. B. Tenenbaum, and C. Gan.  Learning task decomposition with ordered memory policy network.  In _International Conference on Learning Representations_ , 2021.
  * Frans et al. [2018] K. Frans, J. Ho, X. Chen, P. Abbeel, and J. Schulman.  Meta learning shared hierarchies.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=SyX0IeWAW.
  * Kulkarni et al. [2016] T. D. Kulkarni, K. Narasimhan, A. Saeedi, and J. Tenenbaum.  Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation.  In _Advances in Neural Information Processing Systems_ , pages 3675–3683, 2016.
  * Oh et al. [2017] J. Oh, S. Singh, H. Lee, and P. Kohli.  Zero-shot task generalization with multi-task deep reinforcement learning.  In _International Conference on Learning Representations_ , pages 2661–2670, 2017.
  * Merel et al. [2019] J. Merel, A. Ahuja, V. Pham, S. Tunyasuvunakool, S. Liu, D. Tirumala, N. Heess, and G. Wayne.  Hierarchical visuomotor control of humanoids.  In _International Conference on Learning Representations_ , 2019.  URL https://openreview.net/forum?id=BJfYvo09Y7.
  * Andreas et al. [2017] J. Andreas, D. Klein, and S. Levine.  Modular multitask reinforcement learning with policy sketches.  In _International Conference on Machine Learning_ , pages 166–175, 2017.
  * Kase et al. [2020] K. Kase, C. Paxton, H. Mazhar, T. Ogata, and D. Fox.  Transferable task execution from pixels through deep planning domain learning.  In _IEEE International Conference on Robotics and Automation_ , pages 10459–10465, 2020.
  * Driess et al. [2021] D. Driess, J.-S. Ha, R. Tedrake, and M. Toussaint.  Learning geometric reasoning and control for long-horizon tasks from visual input.  In _IEEE International Conference on Robotics and Automation_ , 2021\.
  * Niekum et al. [2013] S. Niekum, S. Chitta, A. G. Barto, B. Marthi, and S. Osentoski.  Incremental semantically grounded learning from demonstration.  In _Robotics: Science and Systems_ , 2013.
  * Knepper et al. [2013] R. A. Knepper, T. Layton, J. Romanishin, and D. Rus.  Ikeabot: An autonomous multi-robot coordinated furniture assembly system.  In _IEEE International Conference on Robotics and Automation_ , pages 855–862, 2013.
  * Suárez-Ruiz et al. [2018] F. Suárez-Ruiz, X. Zhou, and Q.-C. Pham.  Can robots assemble an ikea chair?  _Science Robotics_ , 3(17):eaat6385, 2018.
  * Sutton [1984] R. S. Sutton.  _Temporal credit assignment in reinforcement learning_.  PhD thesis, University of Massachusetts Amherst, 1984.
  * Zhu et al. [2018] Y. Zhu, Z. Wang, J. Merel, A. Rusu, T. Erez, S. Cabi, S. Tunyasuvunakool, J. Kramár, R. Hadsell, N. de Freitas, and N. Heess.  Reinforcement and imitation learning for diverse visuomotor skills.  In _Robotics: Science and Systems_ , 2018.
  * Peng et al. [2021] X. B. Peng, Z. Ma, P. Abbeel, S. Levine, and A. Kanazawa.  Amp: Adversarial motion priors for stylized physics-based character control.  _arXiv preprint arXiv:2104.02180_ , 2021.
  * Schulman et al. [2017] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ , 2017.
  * Pertsch et al. [2020] K. Pertsch, Y. Lee, and J. J. Lim.  Accelerating reinforcement learning with learned skill priors.  In _Conference on Robot Learning_ , 2020.

##  Appendix A Environment Details

We choose two tasks, table_lack and chair_ingolf, from the IKEA furniture
assembly environment [17] for our experiments. For reinforcement learning, we
use a heavily shaped multi-phase dense reward from [17]. For the robotic
agent, we use the 7-DoF Rethink Sawyer robot operated via joint velocity
control. For imitation learning, we collected 200 demonstrations for each
furniture part assembly with a programmatic assembly policy. Each
demonstration for single-part assembly consists of around $200$-$900$ steps
long due to the long-horizon nature of the task.

The observation space includes robot observations ($29$ dim), object
observations ($35$ dim), and task phase information ($8$ dim). The robot
observations consist of robot joint angles ($7$ dim), joint velocities ($7$
dim), gripper state ($2$ dim), gripper position ($3$ dim), gripper quaternion
($4$ dim), gripper velocity ($3$ dim), and gripper angular velocity ($3$ dim).
The object observations contain the positions ($3$ dim) and quaternions ($4$
dim) of all five furniture pieces in the scene. In addition, the
$8$-dimensional task information is an one-hot embedding representing the
phase of the state (e.g. reaching, grasping, lifting, moving, aligning).

In our experiments, we define a subtask as assembling one part to another;
thus, we have four subtasks for each task. Subtasks are independently trained
on the initial states sampled from the environment with random noise ranging
from [$-2$cm, $2$cm] and [$-3$°, $3$°] in the $(x,y)$-plane. Moreover, this
subtask decomposition is given, i.e., the environment can be initialized for
each subtask and the agent is informed when the subtask is completed. Once two
parts are attached, the corresponding subtask is completed and the robot arm
moves back to its initial pose in the center of the workplace.

##  Appendix B Network Architectures

For fair comparison, we use the same network architecture for our method and
the baseline methods. The policy and critic networks consist of two fully
connected layers of 128 and 256 hidden units with ReLU nonlinearities,
respectively. The output layer of the actor network outputs an action
distribution, which consists of the mean and standard deviation of a Gaussian
distribution. The critic network outputs only a single critic value. The
discriminator for GAIL [29] and initiation set discriminator for our proposed
method use a two-layer fully connected network with 256 hidden units. $\tanh$
is used as a nonlinear activation function. These discriminators’ outputs are
clipped between $[0,1]$ following Peng et al. [53].

##  Appendix C Training Details

During policy learning, we sample a batch of 128 elements from the expert (for
GAIL) and agent experience buffers. To acquire subtask polices for the policy
sequencing baseline [6] and our method, we run from 20M to 25M steps depending
on the difficulty of each subtask. We train all methods except BC for 48 hours
which collect 200M steps for PPO, GAIL, GAIL + PPO; and 5M steps for SPiRL. We
train the policy sequencing baseline and our method for additional 100M steps;
thus, in total 200M steps. For the final evaluation, we evaluate a trained
policy every 50 updates and report the best performing checkpoints due to
unstable adversarial training. Most of the runs are converged within these
training steps.

Table 2: PPO hyperparameters. Hyperparameter | Value
---|---
# Workers | 16
Rollout Size | 1,024
Learning Rate | 0.0003
Learning Rate Decay | Linear decay
Mini-batch Size | 128
# Epochs per Update | 10
Discount Factor | 0.99
Entropy Coefficient | 0.003
Reward Scale | 0.05
State Normalization | True

##### BC [22]:

The policy is trained with batch size 256 for 1000 epochs using the Adam
optimizer with learning rate 1e-4 and momentum (0.9, 0.999).

##### PPO [54]:

Any reinforcement learning algorithm can be used for policy optimization, but
we choose to use Proximal Policy Optimization (PPO) [54] and we use the
default hyperparameters of PPO (see Table 2).

##### GAIL [29]:

We tested different variants of GAIL implementation and found the AMP [53]
formulation is most stable. Please refer to the paper [53] for implementation
details about GAIL training and GAIL reward. In this paper, we specifically
use an agent states $s$ to discriminate agent and expert trajectories, instead
of state-action pairs $(s,a)$.

##### GAIL + PPO:

We use the same setup with the GAIL baseline, but with the environment reward.
We use $\lambda_{1}=\lambda_{2}=0.5$.

##### SPiRL [55]:

We use the official implementation of the original paper.

##### Policy Sequencing [6]:

We use the same implementation and hyperparameters as ours, but without the
terminal state regularization. To prevent catastrophic forgetting, we sample
an initial state from the environment for 20% and sample from the Gaussian
distribution for 80%.

#####  T-STAR (Ours):

We use $\lambda_{3}=10000$ for the terminal state regularization. Similar to
PS, we sample an initial state from the environment for 20% and sample from
the terminal state buffer of the previous skill for 80%.


# Weak Electronic Correlations Observed in Magnetic Weyl Semimetal Mn3Ge

Susmita Changdar1,∗, Susanta Ghosh1,∗, Anumita Bose2, Indrani Kar1, Achintya
Low1, Patrick Le Fevre3, François Bertran3, Awadhesh Narayan2, Setti
Thirupathaiah1,† 1Condensed Matter and Materials Physics Department, S. N.
Bose National Centre for Basic Sciences, Kolkata, West Bengal-700106, India
2Solid State and Structural Chemistry Unit, Indian Institute of Science,
Bangalore 560012, India  3SOLEIL Synchrotron, L’Orme des Merisiers,
Départementale 128, F-91190 Saint-Aubin, France  ∗ These authors contributed
equally  † Corresponding author<EMAIL_ADDRESS>
###### Abstract

Using angle-resolved photoemission spectroscopy (ARPES) and density functional
theory (DFT) calculations, we systematically studied the electronic band
structure of Mn3Ge in the vicinity of the Fermi level. We observe several
bands crossing the Fermi level, confirming the metallic nature of the studied
system. We further observe several flat bands along various high symmetry
directions, consistent with the DFT calculations. The calculated partial
density of states (PDOS) suggests a dominant Mn $3d$ orbital contribution to
the total valence band DOS. With the help of orbital-resolved band structure
calculations, we qualitatively identify the orbital information of the
experimentally obtained band dispersions. Out-of-plane electronic band
dispersions are explored by measuring the ARPES data at various photon
energies. Importantly, our study suggests relatively weaker electronic
correlations in Mn3Ge compared to Mn3Sn.

  *

##  1 Introduction

The coexistence of magnetism and topology in magnetic semimetals manifests
several intriguing quantum materials in the condensed matter such as the Weyl
semimetals [1, 2, 3, 4]. Mn3Ge is believed to be one such magnetic Weyl
semimetal from the family of Mn3X (X= Sn & Ge) kagome systems, with
noncollinear antiferromagnetic (AFM) order, demonstrating exotic electronic
properties like giant anomalous Hall effect (AHE) [5, 6, 7, 8], anomalous
Nernst effect [9, 10, 11, 12, 13], Kerr effect [14, 15, 16], and flat bands
along with Weyl nodes near the Fermi level [17, 18], leading to compelling
quantum technological application [19, 20]. Usually, the Weyl points are
manifested either by the broken time-reversal symmetry (TRS) or by the broken
inversion symmetry. Out of these two categories, the TRS broken systems are in
principle magnetic WSMs which host strong electronic correlations [21, 22,
23], exhibiting several fascinating bulk electronic and magnetic properties.
On the other hand, in the case of the Mn3X family of materials, it is the
translational plus time-reversal symmetry (TRS) breaking that gives rise to
the Weyl physics [24, 18, 12, 25]. An inversion symmetry broken (under the
time-reversal symmetry) Weyl system can host a minimum of two pairs of Weyl
nodes (four Weyl points), while a time-reversal symmetry broken (under the
inversion symmetry) Weyl system can host a minimum of one pair of Weyl nodes
(two Weyl points). Each pair of Weyl nodes with opposite chirality acts as the
source and sink of the Berry curvature, leading to large AHE [26, 5, 6, 27].
Thus, a systematic study of the low-energy electronic band structure of these
systems is crucial for understanding the observed large AHE.

Figure 1: Hexagonal (a) crystal structure and (b) Brillouin zone of Mn3Ge. (c)
In-plane Fermi surface map. (d) Constant energy contour map taken at 0.1 eV
below the Fermi level. Energy distribution maps (EDMs) taken along (e)
$\Gamma-K$, (f) $\Gamma-M$, and (g) $M-K$ high symmetry directions are shown
on the top-panel. The bottom panels of (e)-(g) demonstrate the second
derivatives of the corresponding EDMs. The overlapped black-solid curves in
(e)–(g) are the DFT band structure.

Although there exist many density functional theory (DFT) calculations
exploring the topological properties of Mn3X (X = Sn and Ge ) systems by
studying their electronic band structure near the Fermi level [17, 28, 29, 30,
31], so far only one ARPES report is available experimentally demonstrating
the electronic band structure of Mn3Sn [32]. The DFT calculations predict
several Weyl points near the Fermi level from both Mn3Sn and Mn3Ge. A recent
theoretical report even suggested that the Weyl points in these systems are
accidental [33]. However, experimentally, it is hard to detect these Weyl
points due to difficulties in resolving the valence band electronic structure.
Because of this complexity, not much excitement is generated in the research
community for experimentally studying the electronic band structure of these
very fascinating magnetic topological Weyl semimetals.

In this manuscript, we present the low-energy electronic band structure of
Mn3Ge studied by using the ARPES technique and compared it with the calculated
band structure using DFT. We find a qualitative agreement between the ARPES
data and DFT calculations. However, detection of the theoretically predicted
Weyl points from our ARPES data has been quite challenging as they are
predicted well above the Fermi level [17, 18]. Since Mn3X compounds are
suggested to show strong electronic correlations [32, 34], instead of
searching for the Weyl points, we focused on the details of low-energy
electronic band structure along the in-plane and the out-of-plane ($k_{z}$)
orientations. The partial density of state (PDOS) calculations confirm the
presence of dominant Mn $3d$ orbital characters near the Fermi level. We
further carried out the orbital-resolved DFT calculations and compared them
with the ARPES data to understand the orbital contributions to the
experimental electronic band structure. In addition, as the Mn3Ge is
constructed from layered kagome lattice, these are predicted to host flat
bands [35, 36, 37]. Our ARPES data demonstrate the presence of such flat bands
along different high-symmetry directions of the Brillouin zone.

##  2 Experimental Details

Mn3Ge single crystals were grown by the melt-growth method [5]. To grow the
crystals, stoichiometric amounts of high quality Mn (99.95%) and Ge (99.999%)
powders were mixed thoroughly under an argon environment and sealed in a
quartz tube at 10-4 mbar vacuum. The quartz tube was kept at 1050oC for 24
hours before cooling down to 740oC at a rate of 2o/hr. After prolonged
annealing for 120 hrs, the tube was quenched in ice water. In this way, we
could grow the single crystals of Mn3Ge with a typical size of 2$\times$3 mm2.
As grown single crystals were studied structurally using the X-ray diffraction
(XRD) technique to confirm the hexagonal crystal structure with a space group
of P63/mmc (194). The energy dispersive X-ray spectroscopy (EDS) study
suggests an actual chemical composition of Mn2.94±0.03Ge for the single
crystals which were used for the ARPES measurements. The ARPES measurements
were performed at CASSIOPÉE beamline in the SOLEIL synchrotron radiation
centre, France, equipped with a SCIENTA R4000 analyzer. The samples were
cleaved $in-situ$ under a chamber vacuum of $\approx 5\times 10^{-11}$ mbar.
The data were collected using photon energies between 83 and 143 eV at a
sample temperature of 35 K. The total energy resolution was set to 15 meV.

##  3 Computational Methods

Band structure calculations were performed on Mn3Ge using density functional
theory (DFT) within the generalized gradient approximation (GGA) of Perdew,
Burke, and Ernzerhof (PBE) exchange and correlation potential [38] as
implemented in the Quantum Espresso simulation package [39, 40]. Ultrasoft
non-relativistic and fully relativistic pseudopotentials were used to perform
the calculations without spin-orbit coupling (SOC) and with SOC, respectively.
The electronic wave function is expanded using plane waves up to a cutoff
energy of 50 Ry. Brillouin zone sampling is done over a $\Gamma$-centred
7$\times$7$\times$7 Monkhorst-Pack k-grid. The internal coordinates of the
system are relaxed before producing the band structure. The ground state of
this system is in the noncollinear AFM order with each Mn magnetic moment
aligned at an angle of 120o with its neighbouring moment vector as shown in
Fig. 1(a). Each Mn atom in Mn3Ge carries an average magnetic moment of 2.638
$\mu_{B}$. Also, a slight tilt in Mn moments causes an in-plane net magnetic
moment of $\approx$ 0.008 $\mu_{B}$/cell. We observe that overall the
experimental band structure matches better with DFT calculations for
$U=0~{}eV$, consistent with previous studies [17, 32].

##  4 Results and Discussion

Figure 2: (a) Partial density of states (PDOS) of Mn3Ge. ARPES spectra along
the $\Gamma-K$ direction measured with (b) $p-$polarized and (c) $s-$polarized
lights overlapped with the DFT band structure (with SOC). Arrows in panel (b)
show the ($E,k$) location of various flat bands. The top panel in (d) shows
schematic ARPES measuring geometry with $p-$ and $s-$polarized lights defined
with respect to the scattering plane (sp). The bottom panel in (d) shows
various Mn $3d$ orbitals projected onto the hexagonal Brillouin zone and
oriented w.r.t. the scattering plane. In this measuring geometry, the
$p-$polarized light has the even parity and the $s-$polarized light has the
odd parity. Similarly, w.r.t. to sp the orbital characters dxz,
d${}_{x^{2}-y^{2}}$, and d${}_{z^{2}}$ have the even parity, while dxy and dyz
have the odd parity. (e) Orbital-resolved DFT band structure (without SOC)
calculated for Mn $3d$ orbitals, $d_{z^{2}}$, $d_{xz}$, $d_{x^{2}-y^{2}}$,
$d_{xy}$, and $d_{yz}$.

Mn3Ge crystallizes in the hexagonal structure as shown in Fig. 1(a) with a
nonsymmorphic space group of $P6_{3}/mmc$ (No. 194). One unit cell consists of
two layers of Mn3Ge, while each layer consists of three Mn and one Ge atoms.
The two layers of Mn3Ge stacked along the $\it{c}$-axis are connected by
inversion symmetry. The three Mn atoms in a layer form a kagome lattice
consisting of triangles and hexagons with the Ge atom positioned at the center
of the hexagon [28, 18]. Fig. 1(b) shows the hexagonal Brillouin zone (BZ)
along with various high symmetry points located at their respective positions.
Fig. 1(c) shows an in-plane Fermi surface (FS) map measured using
$p-$polarized light with a photon energy of $h\nu$=120 eV. Later, we will show
that the 120 eV photon energy extracts the electronic bands from
$k_{z}$$\approx$4.0 $\pi/c$ plane, i.e., the $\Gamma KM$ plane of the
Brillouin zone.

From Fig. 1(c), we can notice that the FS map has a hexagonal symmetry that is
consistent with the hexagonal crystal structure. In the figure, the Fermi
pockets on the FS map are shown by the white-colored elliptical shapes. It is
to be noted that since multiple bands cross the Fermi level at $K$ and $M$
points, we observe spectral intensities at both $K$ and $M$ points surrounding
the $\Gamma$ point as shown in Fig. 1(c). The large number of bands at both
$K$ and $M$ points makes it challenging to resolve distinct energy pockets at
the Fermi level. Therefore, what may appear as energy pockets in Fig. 1(c) are
largely influenced by the matrix element effects, leading to asymmetric
spectral intensities. Fig. 1(d) shows a constant energy map taken at 100 meV
below the Fermi level (EF). The constant energy map taken at EF-100 meV is
consistent with the Fermi surface of Mn3Sn [32, 34], where we observe six
high-intensity Fermi pockets at six $M$ points of the BZ. Moreover, the size
of the Fermi pockets decreases as we go from $E_{F}$ to EF-100 meV, suggesting
that the Fermi pockets enclosed in the elliptical shape are of the electron-
type. This interpretation is consistent with the EDM shown in Fig. 1(g) from
which we can notice four electron-like band dispersions crossing the Fermi
level.

For a better understanding on the nature of Fermi pockets, energy distribution
maps (EDMs) and their second derivative with respect to momentum along the
$\Gamma-K$, $\Gamma-M$, and $M-K$ high symmetry directions are shown in Figs.
1(e), 1(f), and 1(g), respectively. From these EDMs, we notice several well-
dispersive electronic bands crossing $E_{F}$ at high symmetry points $M$ and
$K$, while we could not find any band crossing the $E_{F}$ at the high
symmetry point $\Gamma$.

Figure 3: (a) Out-of-plane ($k_{z}$) Fermi surface map of Mn3Ge. (b) Stacked
momentum dispersive curves (MDCs) taken at the Fermi level from the EDMs
measured using the photon energies between 83 and 143 eV. (c), (d), and (e)
EDMs taken along the $A-H$, $K-H$, and $\Gamma-A$ high symmetry directions,
respectively. The EDMs shown in (d) and (e) are taken along the cuts shown by
the down-arrows in (a). The overlapped black-solid curves in (c)–(e) are the
DFT band structure. The arrows in (c)-(e) show the ($E,k$) location of several
flat bands. The right panel of (d)-(e) represents the second derivative of the
EDMs shown in the left. The green boxes mark the energy range of the EDMs,
over which the second derivative was performed.

The electronic band structure calculated using the DFT is overlapped on the
EDMs as shown in Figs. 1(e)-(g). It is to be noted that the Fermi level of the
DFT band structure is shifted by 180 meV towards higher binding energy to
match the experimental band dispersions, as the Mn deficiency gives a net
effect of hole doping into the system. Nevertheless, we find a qualitative
agreement between the DFT band structure and the ARPES data for all high
symmetry directions, as can be seen from Figs. 1(e)-(g). An earlier report on
Mn3Sn showed the presence of strong electronic correlations, as they had to
perform a band renormalization by a factor of five to the DFT band structure
in order to obtain a good overlap with ARPES data [32]. In our case, we
renormalized the DFT band structure by a factor of 1.18 to match our ARPES
data, suggesting that the electronic correlations in Mn3Ge are relatively
weaker compared to Mn3Sn. This observation is consistent with an earlier
theoretical study, which says that Mn3Ge has a wider bandwidth leading to a
smaller band renormalization of a factor of 2 [18].

Fig. 2(a) shows the partial density of states (PDOS) calculated using the DFT.
From Fig. 2(a) it is evident that the Mn $3d$ orbital characters dominate the
valence band structure, particularly near $E_{F}$ only the Mn $3d$ orbitals
are present. ARPES spectra collected along the $\Gamma-K$ direction with $p-$
and $s-$polarized lights are displayed in Figs. 2(b) and 2(c), respectively.
As per the measuring geometry shown Fig. 2(d), the $p-$polarized light
extracts the even parity Mn $3d$ orbital characters like dxz,
d${}_{x^{2}-y^{2}}$, and d${}_{z^{2}}$ and the $s-$polarized light extracts
the odd parity orbital characters like dxy and dyz in the vicinity of Fermi
level. Orbital-resolved band structures for all Mn $3d$ characters are shown
in Fig. 2(e). By closely comparing the ARPES data shown in Figs. 2(b) and 2(c)
with DFT band structure shown in Fig. 2(e), we can qualitatively identify that
the $p-$polarized light predominantly extracts the d${}_{z^{2}}$ and
d${}_{x^{2}-y^{2}}$ orbitals, while the $s-$polarized light predominantly
extracts the dyz orbital character. Moreover, Mn3Ge is known to possess flat
bands due to the localization of the Mn $3d$ electrons within the kagome
lattice [41, 32]. In agreement with that, we also observe flat bands around
the $K$-point (shown by the arrows) from the DFT calculations. A hint of the
flat band feature is also visible near the Fermi level in the EDM taken along
the $\Gamma-K$ direction measured with $p-$polarized [see Fig. 2(b)].

Fig. 3(a) shows out-of-plane ($k_{z}$) Fermi surface map of Mn3Ge measured by
varying the photon energy between 83 and 143 eV with an energy interval of 3
eV using $p-$polarized light. Using an inner potential of $V_{0}$=12.5$\pm$0.5
eV, we identified that the photon energy 122 eV extracts the electronic states
from $k_{z}$=4.0 $\pi/c$ (5.84 $\AA^{-1}$) plane and the photon energy 92 eV
extracts the states from $k_{z}$=3.5 $\pi/c$ (5.12 $\AA^{-1}$) plane,
following the relation
$k_{z}=\sqrt{\frac{2m_{e}}{\hbar^{2}}[E_{kin}cos^{2}\theta+V_{0}]}$. Here,
$m_{e}$ is the electron mass and the lattice constant $\it{c}$=4.305 $\AA$.
Therefore, the ARPES data shown in Fig. 1 measured with 120 eV of photon
energy are nearly extracted from $k_{z}$=4.0 $\pi/c$ ($\Gamma KM$) plane. Fig.
3(b) depicts stacked momentum dispersive curves (MDC) taken at $E_{F}$ from
the EDMs measured using the photon energies between 83 and 143 eV. From the
MDCs, we can identity two bands $\alpha_{1}$ and $\alpha_{2}$ at around $K$
point which are hardly dispersing in going from $K$ to $H$ except that the
$\alpha_{1}$ and $\alpha_{2}$ band intensities vary with photon energy due to
the matrix element effects.

Next, the EDM shown in Fig. 3(c) is measured with a photon energy of 92 eV
corresponding to the band dispersion along the $A-H$ high symmetry direction.
As can be seen from Fig. 3(c), the overlapped DFT band structure of the $A-H$
direction quantitatively matches the experimental band dispersions, supporting
the interpretation that the 92 eV photon energy extracts the electronic states
from the $AHL$ plane. Figs. 3(d) and 3(e) show the EDMs taken along the $H-K$
and $A-\Gamma$ directions, respectively. Again, the calculated band structure
is qualitatively overlapped on these $k_{z}$ dependent EDMs. Particularly, we
noticed a flat band at around -0.23 eV below the Fermi level from the EDM of
the $A-\Gamma$ direction [see Fig. 3(e)] and several other flat bands are also
noticed far below the Fermi level (-2.5 to -3 eV) from the EDMs of the $H-K$
and $A-H$ directions. These experimental observations are consistent with our
DFT calculations. It is important to note that despite Mn3Ge displays
relatively weaker electronic correlations it also features flat bands. This
intriguing observation can be attributed to the kagome lattice network in
Mn3Ge rather than to the strong electronic correlations. In kagome systems the
electrons confine locally through the destructive quantum interference,
resulting the formation of flat bands. This has been observed in the other
kagome systems such as in CoSn [42] and GdV6Sn6 [43] which also hosts weaker
electronic correlations.

Overall, we find a qualitative agreement between the experimental and DFT band
structures of Mn3Ge. However, we were unable to detect the Weyl points and the
surface Fermi arcs from our ARPES measurements as they are predicted above the
Fermi level [17, 18]. Our sample has a 2% of Mn (Mn2.94Ge) deficiency compared
to the stoichiometric Mn3Ge, leading to a further shift of the experimental
Fermi level towards the higher binding energy, making it difficult to capture
the predicted dispersive surface Fermi arcs. To experimentally visualize the
Weyl points in Mn3Ge using ARPES, we need Mn3Ge single crystals without
disorder. It is worth noting that, so far, the Mn3Ge single crystals were
synthesized either with excess or deficient Mn. As a result, the ARPES data
suffers from bad energy and momentum resolutions due to the disorder.
Additionally, it is essential to note that the Weyl points appear above the
Fermi level, which falls outside the detection range of ARPES. To overcome
this limitation, the system needs to be doped with electrons in order to shift
the Fermi level towards higher kinetic energy to reach the Weyl points.
Further, chiral anomaly is a crucial phenomenon that serves as compelling
evidence for the Weyl semimetallic phase. This phenomenon can be detected
through the angle-dependent magnetotransport measurements [44]. Thus, the
investigation of chiral anomaly in Mn3Ge can offer robust confirmation of the
existence of Weyl nodes in this system. In addition, unlike Mn3Sn which shows
band renormalization by a factor of five, we do not observe significant band
renormalization in the experimental band structure of Mn3Ge when compared with
the DFT calculations. Further, the experimental band structure of this study
is also broadened similar to Mn3Sn despite weaker electronic correlations
[18]. We think, the defects in the case of Mn deficient Mn3Ge (our study) or
the Mn impurities in the case of Mn excess Mn3Sn are responsible for the
broadened ARPES data rather than the electronic correlations.

##  5 Conclusions

In conclusion, we have systematically investigated the in-plane and out-of-
plane electronic band structure of the non-collinear antiferromagnetic Weyl
semimetal Mn3Ge using ARPES and DFT calculations. Overall, the ARPES data
match qualitatively with the DFT band structure near the Fermi level.
Consistent with DFT predictions, we could observe flat bands near and well
below the Fermi level. Calculated PDOS suggests that the Mn $3d$ orbital
characters dominate the valence band structure, particularly near the Fermi
level. With the help of orbital-resolved band structure calculations, we
identify various Mn $3d$ orbital contributions to the experimentally obtained
band structure. We suggest relatively weaker electronic correlations in Mn3Ge
compared to Mn3Sn.

##  6 Acknowledgement

S.C. and S. G. acknowledge the University Grants Commission (UGC), India, for
the Ph.D. fellowship. This research has made use of the Technical Research
Centre (TRC) instrument facilities of the S. N. Bose National Centre for Basic
Sciences, established under the TRC project of the Department of Science and
Technology (DST), Govt. of India. A.B. is supported by the Prime Minister’s
Research Fellowship (PMRF). A. N. thanks the Indian Institute of Science for a
start-up grant. S.T. thanks the financial support provided by UGC-DAE CSR
through grant no. CRS/2021-22/01/373. S.T. thanks the Science and Engineering
Research Board (SERB), Department of Science and Technology (DST), India, for
the financial support through grant no. SRG/2020/000393.

## References

## References

  * [1] Burkov A A and Balents L 2011 Phys. Rev. Lett. 107(12) 127205 URL https://link.aps.org/doi/10.1103/PhysRevLett.107.127205
  * [2] Wan X, Turner A M, Vishwanath A and Savrasov S Y 2011 Phys. Rev. B 83(20) 205101 URL https://link.aps.org/doi/10.1103/PhysRevB.83.205101
  * [3] Yan B and Felser C 2017 Annu. Rev. Condens. Matter Phys. 8 337–354 URL https://doi.org/10.1146/annurev-conmatphys-031016-025458
  * [4] Armitage N P, Mele E J and Vishwanath A 2018 Rev. Mod. Phys. 90(1) 015001 URL https://link.aps.org/doi/10.1103/RevModPhys.90.015001
  * [5] Nayak A K, Fischer J E, Sun Y, Yan B, Karel J, Komarek A C, Shekhar C, Kumar N, Schnelle W, Kübler J, Felser C and Parkin S S P 2016 Sci. Adv. 2 e1501870 URL https://www.science.org/doi/abs/10.1126/sciadv.1501870
  * [6] Kiyohara N, Tomita T and Nakatsuji S 2016 Phys. Rev. Appl. 5(6) 064009 URL https://link.aps.org/doi/10.1103/PhysRevApplied.5.064009
  * [7] Nakatsuji S, Kiyohara N and Higo T 2015 Nature 527 212–215 URL https://doi.org/10.1038/nature15723
  * [8] Taylor J M, Markou A, Lesne E, Sivakumar P K, Luo C, Radu F, Werner P, Felser C and Parkin S S P 2020 Phys. Rev. B 101(9) 094404 URL https://link.aps.org/doi/10.1103/PhysRevB.101.094404
  * [9] Narita H, Higo T, Ikhlas M, Nakatsuji S and Otani Y 2020 Appl. Phys. Lett. 116 072404 URL https://doi.org/10.1063/1.5126615
  * [10] Narita H, Ikhlas M, Kimata M, Nugroho A A, Nakatsuji S and Otani Y 2017 Appl. Phys. Lett. 111 202404 URL https://doi.org/10.1063/1.5000815
  * [11] Wuttke C, Caglieris F, Sykora S, Scaravaggi F, Wolter A U B, Manna K, Süss V, Shekhar C, Felser C, Büchner B and Hess C 2019 Phys. Rev. B 100(8) 085111 URL https://link.aps.org/doi/10.1103/PhysRevB.100.085111
  * [12] Guo G Y and Wang T C 2017 Phys. Rev. B 96(22) 224415 URL https://link.aps.org/doi/10.1103/PhysRevB.96.224415
  * [13] Li X, Xu L, Ding L, Wang J, Shen M, Lu X, Zhu Z and Behnia K 2017 Phys. Rev. Lett. 119(5) 056601 URL https://link.aps.org/doi/10.1103/PhysRevLett.119.056601
  * [14] Wu M, Isshiki H, Chen T, Higo T, Nakatsuji S and Otani Y 2020 Applied Physics Letters 116 132408 URL https://doi.org/10.1063/1.5143959
  * [15] Higo T, Man H, Gopman D B, Wu L, Koretsune T, van’t Erve O M, Kabanov Y P, Rees D, Li Y, Suzuki M T et al. 2018 Nat. Photonics 12 73–78 URL https://doi.org/10.1038/s41566-017-0086-z
  * [16] Kurt H, Baadji N, Rode K, Venkatesan M, Stamenov P, Sanvito S and Coey J M D 2012 Appl. Phys. Lett. 101 132410 URL https://doi.org/10.1063/1.4754123
  * [17] Yang H, Sun Y, Zhang Y, Shi W J, Parkin S S and Yan B 2017 New J. Phys. 19 015008 URL https://iopscience.iop.org/article/10.1088/1367-2630/aa5487
  * [18] Chen T, Tomita T, Minami S, Fu M, Koretsune T, Kitatani M, Muhammad I, Nishio-Hamane D, Ishii R, Ishii F et al. 2021 Nat. Commun. 12 572 URL https://doi.org/10.1038/s41467-020-20838-1
  * [19] Popović R S 2004 Hall Effect Devices (CRC Press)
  * [20] Ramsden E 2006 Hall-Effect Sensors: Theory and Application (Elsevier) URL https://doi.org/10.1201%2Fnoe0750308557-8
  * [21] Pan Y, Le C, He B, Watzman S J, Yao M, Gooth J, Heremans J P, Sun Y and Felser C 2022 Nat. Mater. 21 203–209 URL https://doi.org/10.1038/s41563-021-01149-2
  * [22] Destraz D, Das L, Tsirkin S S, Xu Y, Neupert T, Chang J, Schilling A, Grushin A G, Kohlbrecher J, Keller L et al. 2020 npj Quantum Materials 5 5 URL https://doi.org/10.1038/s41535-019-0207-7
  * [23] Ikhlas M, Dasgupta S, Theuss F, Higo T, Kittaka S, Ramshaw B, Tchernyshyov O, Hicks C and Nakatsuji S 2022 Nat. Phys. 18 1086–1093 URL https://doi.org/10.1038/s41567-022-01645-5
  * [24] Liu Z, Zhang Y, Liu G, Ding B, Liu E, Jafri H M, Hou Z, Wang W, Ma X and Wu G 2017 Sci. Rep. 7 515 URL https://doi.org/10.1038/s41598-017-00621-x
  * [25] Rai V, Jana S, Meven M, Dutta R, Perßon J and Nandi S 2022 Phys. Rev. B 106(19) 195114 URL https://link.aps.org/doi/10.1103/PhysRevB.106.195114
  * [26] Kübler J and Felser C 2014 Europhys. Lett. 108 67001 URL https://dx.doi.org/10.1209/0295-5075/108/67001
  * [27] Wang X, Pan D, Zeng Q, Chen X, Wang H, Zhao D, Xu Z, Yang Q, Deng J, Zhai T et al. 2021 Nanoscale 13 2601–2608 URL https://doi.org/10.1039/D0NR07946D
  * [28] Kübler J and Felser C 2018 Europhys. Lett. 120 47002 URL https://dx.doi.org/10.1209/0295-5075/120/47002
  * [29] Huang C Y, Aramberri H, Lin H and Kioussis N 2020 Phys. Rev. B 102(9) 094403 URL https://link.aps.org/doi/10.1103/PhysRevB.102.094403
  * [30] Xu Y, Elcoro L, Song Z D, Wieder B J, Vergniory M, Regnault N, Chen Y, Felser C and Bernevig B A 2020 Nature 586 702–707 URL https://doi.org/10.1038/s41586-020-2837-0
  * [31] Elcoro L, Wieder B J, Song Z, Xu Y, Bradlyn B and Bernevig B A 2021 Nat. Commun. 12 5965
  * [32] Kuroda K, Tomita T, Suzuki M T, Bareille C, Nugroho A, Goswami P, Ochi M, Ikhlas M, Nakayama M, Akebi S et al. 2017 Nat. Mater. 16 1090–1095 URL https://doi.org/10.1038/nmat4987
  * [33] Bernevig B A, Felser C and Beidenkopf H 2022 Nature 603 41–51 URL https://doi.org/10.1038%2Fs41586-021-04105-x
  * [34] Yu T, Liu R, Peng Y, Zheng P, Wang G, Ma X, Yuan Z and Yin Z 2022 Phys. Rev. B 106(20) 205103 URL https://link.aps.org/doi/10.1103/PhysRevB.106.205103
  * [35] Li M, Wang Q, Wang G, Yuan Z, Song W, Lou R, Liu Z, Huang Y, Liu Z, Lei H et al. 2021 Nat. Commun. 12 3129 URL https://doi.org/10.1038/s41467-021-23536-8
  * [36] Kang M, Fang S, Ye L, Po H C, Denlinger J, Jozwiak C, Bostwick A, Rotenberg E, Kaxiras E, Checkelsky J G et al. 2020 Nat. Commun. 11 4004 URL https://doi.org/10.1038/s41467-020-17465-1
  * [37] Kang M, Ye L, Fang S, You J S, Levitan A, Han M, Facio J I, Jozwiak C, Bostwick A, Rotenberg E et al. 2020 Nat. Mater. 19 163–169 URL https://doi.org/10.1038/s41563-019-0531-0
  * [38] Perdew J P, Burke K and Ernzerhof M 1996 Phys. Rev. Lett. 77(18) 3865–3868 URL https://link.aps.org/doi/10.1103/PhysRevLett.77.3865
  * [39] Giannozzi P, Baroni S, Bonini N, Calandra M, Car R, Cavazzoni C, Ceresoli D, Chiarotti G L, Cococcioni M, Dabo I et al. 2009 J. Phys. Condens. Matter 21 395502 URL https://doi.org/10.1088/0953-8984/21/39/395502
  * [40] Giannozzi P, Andreussi O, Brumme T, Bunau O, Nardelli M B, Calandra M, Car R, Cavazzoni C, Ceresoli D, Cococcioni M et al. 2017 J. Phys. Condens. Matter 29 465901 URL https://doi.org/10.1088/1361-648X/aa8f79
  * [41] Vicencio R A, Cantillano C, Morales-Inostroza L, Real B, Mejía-Cortés C, Weimann S, Szameit A and Molina M I 2015 Phys. Rev. Lett. 114(24) 245503 URL https://link.aps.org/doi/10.1103/PhysRevLett.114.245503
  * [42] Liu Z, Li M, Wang Q, Wang G, Wen C, Jiang K, Lu X, Yan S, Huang Y, Shen D et al. 2020 Nat. Commun. 11 4002 URL https://doi.org/10.1038/s41467-020-17462-4
  * [43] Ding J, Zhao N, Tao Z, Huang Z, Jiang Z, Yang Y, Cho S, Liu Z, Liu J, Guo Y et al. 2023 Journal of Physics: Condensed Matter 35 405502 URL https://iopscience.iop.org/article/10.1088/1361-648X/ace2a2
  * [44] Zhang C L, Xu S, Belopolski I, Yuan Z, Lin Z, Tong B, Bian G, Alidoust N, Lee C C, Huang S M, Chang T R, Chang G, Hsu C, Jeng H T, Neupane M, Sanchez D S, Zheng H, Wang J, Lin H, Zhang C, Lu H Z, Shen S Q, Neupert T, Hasan M Z and Jia S 2016 Nat. Commun. 7 10735 URL https://doi.org/10.1038/ncomms10735