# Demonstration-Efficient Guided Policy Search via
Imitation of Robust Tube MPC

Andrea Tagliabue, Dong-Ki Kim, Michael Everett, Jonathan P. How All the
authors are with the MIT Department of Aeronautics and Astronautics.
{atagliab, dkkim93, mfe<EMAIL_ADDRESS>
###### Abstract

We propose a demonstration-efficient strategy to compress a computationally
expensive Model Predictive Controller (MPC) into a more computationally
efficient representation based on a deep neural network and Imitation Learning
(IL). By generating a Robust Tube variant (RTMPC) of the MPC and leveraging
properties from the tube, we introduce a data augmentation method that enables
high demonstration-efficiency, being capable to compensate the distribution
shifts typically encountered in IL. Our approach opens the possibility of
zero-shot transfer from a single demonstration collected in a nominal domain,
such as a simulation or a robot in a lab/controlled environment, to a domain
with bounded model errors/perturbations. Numerical and experimental
evaluations performed on a trajectory tracking MPC for a quadrotor show that
our method outperforms strategies commonly employed in IL, such as DAgger and
Domain Randomization, in terms of demonstration-efficiency and robustness to
perturbations unseen during training.

##  I Introduction

Model Predictive Control (MPC) [1] enables impressive performance on complex,
agile robots [2, 3, 4, 5, 6, 7]. However, its computational cost often limits
the opportunities for onboard, real-time deployment [8], or takes away
critical computational resources needed by other components governing the
autonomous system. Recent works have mitigated MPC’s computational
requirements by relying on computationally efficient deep neural networks,
which are leveraged to imitate task-relevant demonstrations generated by MPC.
Such demonstrations are generally collected via Guided Policy Search (GPS) [9,
10, 11, 12] and Imitation Learning (IL) [13, 14, 15] and are then used to
train a DNN via supervised learning.

A common issue in existing IL methods (e.g., Behavior Cloning (BC) [16, 17,
18], Dataset-Aggregation (DAgger) [19]) is that they require collecting a
relatively large number of MPC demonstrations, preventing training directly on
a real robot and requiring a simulation environment that accurately represents
the deployment domain. One of the causes for such demonstration inefficiency
is the need to take into account and correct for the compounding of errors in
the learned policy [19], which may otherwise create shifts (covariate shifts)
from the training distribution, with catastrophic consequences [16]. These
errors can be caused by:  a) approximation errors in the learned policy;  b)
mismatches in the simulation dynamics due to modelling errors (i.e., sim2real
gap); or  c) model changes or disturbances that may not be present in a
controlled training environment (e.g., simulation, or lab/factory when
training on a real robot), but do appear during deployment in the real-world
(i.e., lab2real gap).  Approaches employed to compensate for such gaps, such
as Domain Randomization (DR) [20, 21], introduce further challenges, for
example, requiring to apply disturbances/model changes during training.

Figure 1: Overview of the proposed approach. To compress a computationally
expensive MPC controller in a DNN-based policy $\pi_{\theta}$ in a
demonstration efficient way, we generate a Robust Tube MPC controller using a
model of the disturbances encountered in the deployment domain. We use
properties of the tube to derive a data augmentation strategy which generates
extra state-action pairs $(x^{+},u^{+})$, obtaining $\pi_{\hat{\theta}^{*}}$
via Imitation Learning. Our approach enables zero-shot transfer from a single
demonstration collected in simulation (sim2real) or a controlled environment
(lab, factory, lab2real).

In this work, we address the problem of generating a compressed MPC policy in
a demonstration-efficient manner by providing a data augmentation strategy
that systematically compensates for the effects of covariate shifts that might
be encountered during real-world deployment. Specifically, our approach relies
on a model of the perturbations/uncertainties encountered in a deployment
domain, which is used to generate a Robust Tube version of the given MPC
(RTMPC) to guide the collection of demonstrations and the data augmentation.
An overview of the strategy is provided in Figure 1. Our approach is related
to the recent LAG-ROS framework [22], which provides a learning-based method
to compress a planner in a DNN by extracting relevant information from the
robust tube. LAG-ROS emphasizes the importance of nonlinear contraction-based
controllers (e.g., CV-STEM [23]) to obtain robustness and stability
guarantees. In a complementary way, our work emphasizes minimal requirements -
namely a tube and a data augmentation strategy - to achieve demonstration
efficiency and robustness to real-world conditions. By decoupling these
aspects from the need for complex nonlinear models and control strategies, we
greatly simplify the controller design and reduce the computational
complexity, which enables lab2real transfers.

Contributions. Via numerical comparison with previous IL methods and
experimental validations, we show that demonstration efficiency can be
achieved in MPC-compression by generating a corresponding Robust Tube MPC
(RTMPC) and using the tube to guide the data augmentation strategy for IL
methods (DAgger, BC). To this end, we propose a data-sparse, computationally
efficient (i.e., scales linearly in state size) adversarial Sampling
Augmentation (SA) strategy for data augmentation. We highlight that the
proposed approach, for example, can be used to train the robot in a low-
fidelity simulation environment while achieving robustness to real-world
perturbations unseen during the training phase. We validate the proposed
approach by providing the first experimental (hardware) demonstration of zero-
shot transfer of a DNN-based trajectory tracking controller for an aerial
robot, learned from a single demonstration, in an environment (low-fidelity
simulation) without disturbances, and transferred to an environment with wind-
like disturbances.

##  II Related Work

MPC-like policy compression for mobile robots via IL and GPS. IL methods have
found application in multiple robotics tasks. The works in [14, 24] use DAgger
[19] to control aerial and ground robots, while [13] uses a combination of
DAgger and DR to learn to perform acrobatic maneuvers with a quadrotor.
Similarly, GPS methods have been demonstrated in simulation for navigation of
a multirotor [11, 10], and to control a legged robot [12, 15]. These methods
achieve impressive performance, but at the cost of requiring multiple
demonstrations to execute a single trajectory, and do not explicitly take into
account the effects of disturbances encountered in the deployment domain.

Sample efficiency and robustness in IL. Robustness in IL, required to deal
with distribution shifts caused by the sim2real (model errors) or lab2real
transfer (model changes, external disturbances), has been achieved  a) by
modifying the training domain so that its dynamics match the deployment
domain, as done in DR [20, 21, 25, 26], or  b) by modifying the actions of the
expert, so that the state distribution during training matches the one
encountered at deployment, as proposed in [27, 28, 29, 30].  Although
effective, these approaches do not leverage extra information available in the
RTMPC, thus requiring a larger number of demonstrations.

Data augmentation in IL. Data augmentation is a commonly employed
robustification strategy in IL. Most approaches focus on reducing overfitting
in the high-dimensional policy input space (e.g., images), by applying noise
[31], transformations [32] or adversarially-generated perturbations [33, 34],
while maintaining the corresponding action label unchanged. Data augmentation
is also employed to reduce covariate shift in self-driving by generating
transformed observations [35, 36, 18] with the corresponding action label
computed via a feedback controller. These approaches do not directly apply to
our context, as they do not rely on RTMPC and we assume available state
estimate. Aligned to our findings, [9, 12] observe that adding extra samples
from the tube of an existing Iterative- Linear Quadratic Regulator (LQR) can
achieve increased demonstration efficiency in GPS. Compared to these, thanks
to RTMPC, we can additionally consider the effects of disturbances encountered
in the sim2real or lab2real transfer, providing additional robustness.

##  III Method

This section explains the given MPC expert and its Robust-Tube variant, which
we leverage to design a data augmentation strategy. We additionally cast the
demonstration-efficiency challenge in IL as a robust IL problem in the context
of transferring a policy between two different domains, and we present the SA
strategy to improve demonstration efficiency and robustness of MPC-guided
policies learned via IL.

###  III-A MPC and Robust Tube MPC demonstrator

Model predictive trajectory tracking controller. We assume a trajectory
tracking linear MPC [1] is given that controls a system subject to bounded
uncertainty. The linearized, discrete-time model of the system is:

$\check{x}_{t+1}=A\check{x}_{t}+B\check{u}_{t}+w_{t}$ (1)

where $x\in\mathbb{X}\subseteq\mathbb{R}^{n_{x}}$ represents the state (size
$n_{x}$), $u\in\mathbb{U}\subseteq\mathbb{R}^{n_{u}}$ are the commanded
actions (size $n_{u}$), and $w\in\mathbb{W}\subseteq\mathbb{R}^{n_{x}}$ is an
additive perturbation/uncertainty. $A\in\mathbb{R}^{n_{x}\times n_{x}}$ and
$B\in\mathbb{R}^{n_{x}\times n_{u}}$ represent the nominal dynamics;
$\check{\cdot}$ denotes internal variables of the optimization. At every
discrete timestep $t$ is given an estimate of the state of the actual system
$x_{t}$ and a reference trajectory
$x^{\text{ref}}_{0,t},\dots,x^{\text{ref}}_{N,t}$. The controller computes a
sequence of $N$ actions $\check{u}_{0}(x_{t}),\dots,\check{u}_{N-1}(x_{t})$,
subject to state and input constraints $\mathbb{X},\mathbb{U}$, and executes
the first computed optimal action
$u_{t}={\check{u}^{\text{MPC,*}}_{0}}(x_{t})$ from the sequence. The optimal
sequence of actions is computed by minimizing the value function (dropping
$x_{t}$ in the notation):

$V_{t}(\check{u}_{0},...,\check{u}_{N-1},\check{x}_{0}\\!)\\!=\\!\\!\sum_{n=0}^{N-1}\\!(e_{n}^{T}Qe_{n}+\check{u}_{n}^{T}R\check{u}_{n})+e_{N}^{T}Pe_{N}\\!,$
(2)

where $e_{n}=(\check{x}_{n}-x^{\text{ref}}_{n,t})$. Matrices $Q$ (size
$n_{x}\times n_{x}$) and $R$ (size $n_{u}\times n_{u}$) are user-selected,
positive definite weights that define the stage cost, while $P$ (size
$n_{x}\times n_{x}$, positive definite) represents the terminal cost. The
prediction horizon is an integer $N\geq 1$. The system is additionally subject
to constraints $\check{x}_{0}=x_{t}$; the predicted states
$\check{x}_{1},\dots,\check{x}_{N}$ are obtained from the model in Eq. 1
assuming the disturbance $w=0$. The optimization problem is solved again at
every timestep, executing the newly recomputed optimal action.

Robust Tube MPC. Given the MPC expert, we generate a Robust Tube variant using
[37]. At every discrete timestep $t$, the RTMPC operates in a similar way as
MPC, but it additionally accounts for the effects of $w$ by introducing a
feedback policy, called ancillary controller

$u_{t}=\check{u}_{0}^{*}(x_{t})+K(x_{t}-\check{x}_{0}^{*}(x_{t})),$ (3)

where $u_{t}$ represents the executed action. The quantities
$\check{u}_{0}^{*}(x_{t})$ represents an optimal, feedforward action, and
$\check{x}_{0}^{*}(x_{t})$ is an optimal reference, and are computed by the
RTMPC given the current state estimate $x_{t}$. The ancillary controller
ensures that the state $x$ of the controlled system remains inside a set
(tube) $\mathbb{Z}\in\mathbb{R}^{n_{x}}$, centered around
$\check{x}_{0}^{*}(x_{t})$, for every possible realization of
$w\in\mathbb{W}$. The quantities $\check{u}_{0}^{*}(x_{t})$ and
$\check{x}_{0}^{*}(x_{t})$ are obtained by solving (dropping the dependence on
$x_{t}$)

$\check{x}_{0}^{*},\check{u}_{0}^{*},\dots={\arg\min}_{\check{x}_{0},\check{u}_{0},\dots,\check{u}_{N-1}}V_{t}(\check{u}_{0},\dots,\check{u}_{N-1},\check{x}_{0})$
(4)

under the constraint that $x_{t}\in\check{x}_{0}+\mathbb{Z}$. As in the
original MPC formulation, the optimization problem is additionally subject to
the given input and actuation constraints, tightened by an amount that takes
into account the effects of the disturbances. The gain matrix
$K\in\mathbb{R}^{n_{u}\times n_{x}}$ in Eq. 3 is computed such that
$A_{K}:=A+BK$ is stable, for example by solving the infinite-horizon,
discrete-time LQR problem using ($A$, $B$, $Q$, $R$). The set $\mathbb{Z}$ has
constant size, and it determines the shape/width of the tube. It is defined as
disturbance invariant set for the closed-loop system $A_{K}$, and satisfies
the property that $\forall x_{t}\in\mathbb{Z}$, $\forall w_{t}\in\mathbb{W}$,
$\forall t\in\mathbb{N}^{+}$, $x_{t+1}=A_{K}x_{t}+w_{t}\in\mathbb{Z}$. In
practice, $\mathbb{Z}$ can be computed offline using $A_{K}$ and the model of
the disturbance $\mathbb{W}$ via ad-hoc algorithms [1, 37], or can be learned
from data [38]. The set $\mathbb{Z}$ and the ancillary controller in Eq. 3
ensure (see [37]) that, given a state
$x_{t}\in\check{x}_{0}^{*}(x_{t})+\mathbb{Z}$, the perturbed system in Eq. 1
will remain in the tube centered around the trajectory of $\check{x}_{0}^{*}$,
no matter the disturbance realization $w\in\mathbb{W}$, as shown in Figure 2.
This additionally implies that the tube represents a model of the states that
the system may visit when subject to the disturbances in $\mathbb{W}$. The
ancillary controller provides a computationally-effective way to generate a
control action to counteract such perturbations.

Figure 2: Illustration of the robust control invariant tube $\mathbb{Z}$
centered around the optimal reference $\check{x}_{0}^{*}(x_{t})$ computed by
RTMPC for each state of the system $x_{t}$.

###  III-B Covariate shift in sim2real and lab2real transfer

This part describes the demonstration-efficiency issue in IL as the ability to
efficiently predict and compensate for the effects of covariate shifts during
real-world deployment. We assume that the causes of such distribution shifts
can be modeled as additive state perturbations/uncertainties $w\in\mathbb{W}$
encountered in the deployment domains.

Policies and state densities. We model the dynamics of the real system as
Markovian and stochastic [39]. The stochasticity with respect to state
transitions is introduced by unknown perturbations, assumed to be additive (as
in Eq. 1) and belonging to the bounded set $\mathbb{W}$, sampled under a
(possibly unknown) probability distribution. These perturbations capture the
effects of noise, approximation errors in the learned policy, model changes
and other disturbances acting on the system. Two different domains
$\mathcal{D}$ are considered: a training domain $\mathcal{S}$ (source) and a
deployment domain $\mathcal{T}$ (target). The two domains differ in their
transition probabilities, effectively representing the sim2real or lab2real
settings. We additionally assume that the considered system is controlled by a
deterministic policy
$\pi_{\theta}:\mathbb{X}\times\mathbb{X}^{\text{ref}}_{0,\dots,N}\rightarrow\mathbb{U}$,
where $\mathbb{X}^{\text{ref}}_{0,\dots,N}$ represents the reference
trajectory. Given
$(x^{\text{ref}}_{0,t},\dots,x^{\text{ref}}_{N,t})\in\mathbb{X}^{\text{ref}}_{0,\dots,N}$,
the resulting transition probability is
$p(x_{t+1}|x_{t},u_{t}=\pi_{\theta}(x_{t},x^{\text{ref}}_{0,t},\dots,x^{\text{ref}}_{N,t}),\mathcal{D})$,
denoted $p_{\pi_{\theta},\mathcal{D}}(x_{t+1}|x_{t})$ to simplify the
notation. The probability of collecting a $T$-step trajectory
$\xi=(x_{0},u_{0},x_{1},u_{1},\dots,x_{T-1})$ given a generic policy
$\pi_{\theta}$ in $\mathcal{D}$ is
$p(\xi|\pi_{\theta},\mathcal{D})=p(x_{0})\prod_{t=0}^{T-1}p_{\pi_{\theta},\mathcal{D}}(x_{t+1}|x_{t})$,
where $p(x_{0})$ represents the initial state distribution.

Robust IL objective. Following [27], given an expert RTMPC policy
$\pi_{\theta^{*}}$, the objective of IL is to find parameters $\theta$ of
$\pi_{\theta}$ that minimize a distance metric $J(\theta,\theta^{*}|\xi)$.
This metric captures the differences between the actions generated by the
expert $\pi_{\theta^{*}}$ and the action produced by the learner
$\pi_{\theta}$ across the distribution of trajectories induced by the learned
policy $\pi_{\theta}$, in the perturbed target domain $\mathcal{T}$:

$\hat{\theta}^{*}=\text{arg}\min_{\theta}\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)$
(5)

A choice of distance metric that we consider in this paper is the MSE loss:
$J(\theta,\theta^{*}|\xi)=\frac{1}{T}\sum_{t=0}^{T-1}\|\pi_{\theta}(x_{t})-\pi_{\theta^{*}}(x_{t})\|_{2}^{2}$.

Covariate shift due to sim2real and lab2real transfer. Since in practice we do
not have access to the target environment, our goal is to try to solve Eq. 5
by finding an approximation of the optimal policy parameters
$\hat{\theta}^{*}$ in the source environment:

$\hat{\theta}^{*}=\text{arg}\min_{\theta}\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}J(\theta,\theta^{*}|\xi).$
(6)

The way this minimization is solved depends on the chosen IL algorithm. The
performance of the learned policy in target and source domain can be related
via:

$\begin{multlined}\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)=\\\
\underbrace{\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)-\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}J(\theta,\theta^{*}|\xi)}_{\text{covariate
shift due to transfer}}\\\
+\underbrace{\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}J(\theta,\theta^{*}|\xi)}_{\text{\acs{IL}
objective}},\end{multlined}\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)=\\\
\underbrace{\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)-\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}J(\theta,\theta^{*}|\xi)}_{\text{covariate
shift due to transfer}}\\\
+\underbrace{\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}J(\theta,\theta^{*}|\xi)}_{\text{\acs{IL}
objective}},$ (7)

which clearly shows the presence of a covariate shift induced by the transfer.
The last term corresponds to the objective minimized by performing IL in
$\mathcal{S}$. Attempting to solve Eq. 5 by directly optimizing Eq. 6 (e.g.,
via BC [16]) provides no guarantees of finding a policy with good performance
in $\mathcal{T}$.

Compensating transfer covariate shift via Domain Randomization. A well known
strategy to compensate for the effects of covariate shifts between source and
target domain is Domain Randomization (DR) [20], which modifies the transition
probabilities of the source $\mathcal{S}$ by trying to ensure that the
trajectory distribution in the modified training domain
$\mathcal{S}_{\text{DR}}$ matches the one encountered in the target domain:
$p(\xi|\pi_{\theta},\mathcal{S}_{\text{DR}})\approx
p(\xi|\pi_{\theta},\mathcal{T})$. This is in practice done by sampling
perturbations $w\in\mathbb{W}$ according to some knowledge/hypotheses on their
distribution $p_{\mathcal{T}}(w)$ in the target domain [20], obtaining the
perturbed trajectory distribution $p(\xi|\pi_{\theta},\mathcal{S},w)$. The
minimization of Eq. 5 can then be approximately performed by minimizing
instead:

$\mathbb{E}_{p_{\mathcal{T}}(w)}[\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S},w)}J(\theta,\theta^{*}|\xi)].$
(8)

This approach, however, requires the ability to apply disturbances/model
changes to the system, which may be unpractical e.g., in the lab2real setting,
and may require a large number of demonstrations due to the need to sample
enough state perturbations $w$.

###  III-C Covariate shift compensation via Sampling Augmentation

We propose to mitigate the covariate shift introduced by the compression
procedure not only by collecting demonstrations from the RTMPC, but by using
additional information computed in the controller. Unlike DR, the proposed
approach does not require to explicitly apply disturbances in the training
phase. During the collection of a trajectory $\xi$ in the source domain
$\mathcal{S}$, we utilize instead the tube computed by the RTMPC demonstrator
to obtain knowledge of the states that the system may visit when subjected to
perturbations. Given this information, we propose a state sampling strategy,
called Sampling Augmentation (SA), to extract relevant states from the tube.
The corresponding actions are provided at low computational cost by the
demonstrator. The collected state-actions pairs are then included in the set
of demonstrations used to learn a policy via IL. The following paragraphs
frame the tube sampling problem in the context of covariate shift reduction in
IL, and present two tube sampling strategies.

RTMPC tube as model of state distribution under perturbations. The key
intuition of the proposed approach is the following. We observe that, although
the density $p(\xi|\pi_{\theta},\mathcal{T})$ is unknown, an approximation of
its support $\mathfrak{R}$, given a demonstration $\xi$ collected in the
source domain $\mathcal{S}$, is known. Such support corresponds to the tube
computed by the RTMPC when collecting $\xi$:

$\mathfrak{R}_{\xi^{+}|\pi_{\theta^{*}},\xi}=\bigcup_{t=0}^{T-1}\\{\check{x}_{0}^{*}(x_{t})+\mathbb{Z}\\}.$
(9)

where $\xi^{+}$ is a trajectory in the tube of $\xi$. This is true thanks to
the ancillary controller in Eq. 3, which guarantees that the system remains
inside Eq. 9 for every possible realization of $w\in\mathbb{W}$. The ancillary
controller additionally provides a way to easily compute the actions to apply
for every state inside the tube. Let $x^{+}$ be a state inside the tube
computed when the system is at $x_{t}$ (formally
$x^{+}\in\check{x}_{0}^{*}(x_{t})+\mathbb{Z}$), then the corresponding robust
control action $u^{+}$ is simply:

$u^{+}=\check{u}_{0}^{*}(x_{t})+K(x^{+}-\check{x}_{0}^{*}(x_{t})).$ (10)

For every timestep in $\xi$, extra state-action samples $(x^{+},u^{+})$
collected from within the tube can be used to augment the dataset employed for
empirical risk minimization, obtaining a way to approximate the expected risk
in the domain $\mathcal{T}$ by only having access to demonstrations collected
in $\mathcal{S}$:

$\begin{multlined}\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)\approx\\\
\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}[J(\theta,\theta^{*}|\xi)+\mathbb{E}_{p(\xi^{+}|\pi_{\theta^{*}},\xi)}J(\theta,\theta^{*}|\xi^{+})].\end{multlined}\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{T})}J(\theta,\theta^{*}|\xi)\approx\\\
\mathbb{E}_{p(\xi|\pi_{\theta},\mathcal{S})}[J(\theta,\theta^{*}|\xi)+\mathbb{E}_{p(\xi^{+}|\pi_{\theta^{*}},\xi)}J(\theta,\theta^{*}|\xi^{+})].$
(11)

The demonstrations in the source domain $\mathcal{S}$ can be collected using
existing techniques, such as BC and DAgger.

Tube approximation and sampling strategies.

Figure 3: The two alternative strategies evaluated to sample extra
state/actions pairs from an approximation of the tube of the RTMPC expert:
dense (left) and sparse (right).

In practice, the set $\mathbb{Z}$ may have arbitrary shape (not necessarily
politopic), and the density $p(\xi^{+}|\xi,\pi_{\theta^{*}})$ may not be
available, making difficult to establish where/which states to sample in order
to derive a data augmentation strategy. We proceed by approximating
$\mathbb{Z}$ as an hyper-rectangle $\hat{\mathbb{Z}}$, outer approximation of
the tube. We consider an adversarial approach to the problem by sampling from
the states visited under worst-case perturbations. We investigate two
strategies, shown in Figure 3, to obtains state samples $x^{+}$ at every state
$x_{t}$ in $\xi$:  i) dense sampling: sample extra states from the vertices of
$\check{x}_{0}^{*}(x_{t})+\hat{\mathbb{Z}}$. The approach produces $2^{n_{x}}$
extra state-action samples. It is more conservative, as it produces more
samples, but more computationally expensive.  ii) sparse sampling: sample one
extra state from the center of each facet of
$\check{x}_{0}^{*}(x_{t})+\hat{\mathbb{Z}}$, producing $2n_{x}$ additional
state-action pairs. It is less conservative and more computationally
efficient.

##  IV Results

###  IV-A Evaluation approach

MPC for trajectory tracking on a multirotor. We evaluate the proposed approach
in the context of trajectory tracking control for a multirotor, using the
controller proposed in [5], modified to obtain a RTMPC. We model $\mathbb{W}$
under the assumption that the system is subject to force-like perturbations up
to $30\%$ of the weight of the robot (approximately the safe physical limit of
the robot). The tube $\mathbb{Z}$ is approximated via Monte-Carlo sampling of
the disturbances in $\mathbb{W}$, evaluating the state deviations of the
closed loop system $A_{K}$. The derived controller generates tilt (roll,
pitch) and thrust commands ($n_{u}=3$) given the state of the robot ($n_{x}=8$
consisting of position, velocity and tilt) and the reference trajectory. The
reference is a sequence of desired positions and velocities for the next $3$s,
discretized with sampling time of $0.1$s (corresponding to a planning horizon
of $N=30$, and $180$-dim. vector). The controller takes into account position
constraints (e.g., available 3D flight space), actuation limits, and
velocity/tilt limits.

Policy architecture. The compressed policy is a $2$-hidden layers, fully
connected DNN, with $32$ neurons per layer, and ReLU activation function. The
total input dimension of the DNN is $188$ (position, velocity, current tilt
expressed in an inertial frame, and the desired reference trajectory). The
output dimension is $3$ (desired thrust and tilt expressed in an inertial
frame). We rotate the tilt output of the DNN in body frame to avoid taking
into account yaw, which is not part of the optimization problem [5], not
causing any relevant computational cost. We additional apply the non-linear
attitude compensation scheme as in [5].

Training environment and training details. Training is performed in a custom-
built non-linear quadrotor simulation environment, where the robot follows
desired trajectories, starting from randomly generated initial states,
centered around the origin. Demonstrations are collected with a sampling time
of $0.1$s and training is performed for $50$ epochs via the ADAM [40]
optimizer, with a learning rate of $0.001$.

Evaluation details and metrics. We apply the proposed SA strategies to DAgger
and BC, and compare their performance against the two without SA, and the two
combined with DR. Target and source domain differs due to perturbations
sampled from $\mathbb{W}$ in target. During training with DR we sample
disturbances from the entire $\mathbb{W}$. In all the comparisons, we set the
probability of using actions of the expert $\beta$, hyperparameter of DAgger
[19], to be $1$ at the first demonstration, and $0$ otherwise (as this was
found to be the best performing setup). We monitor:  i) robustness (Success
Rate), as the percentage of episodes where the robot never violates any state
constraint;  ii) performance (MPC Stage Cost), as
$\sum_{t}e_{t}^{T}Qe_{t}+u_{t}^{T}Ru_{t}$ along the trajectory.

###  IV-B Numerical evaluation of demonstration-efficiency, robustness and
performance for tracking a single trajectory

Figure 4: Robustness (Success Rate) in the task of flying along an eight-shaped, $7$s long-trajectory, subject to wind-like disturbances (left) and without (right), starting from different initial states (Task T1). Evaluation is repeated across $8$ random seeds, $20$ times per demonstration per seed. We additionally show the $95\%$ confidence interval. Method | Training |  Robustness succ. rate (%) |  Performance expert gap (%) |  Demonstration Efficiency
---|---|---|---|---
Robustif. |  Imitation |  Easy |  Safe |  T1 |  T2 |  T1 |  T2 |  T1 |  T2
- |  BC |  Yes |  Yes |  < 1 |  100 |  24.15 |  29.47 |  - |  6
DAgger |  Yes |  No |  98 |  100 |  15.79 |  1.34 |  7 |  6
DR |  BC |  No |  Yes |  95 |  100 |  10.04 |  1.27 |  14 |  9
DAgger |  No |  No |  100 |  100 |  4.09 |  1.45 |  10 |  6
SA-Dense |  BC |  Yes |  Yes |  100 |  100 |  25.64 |  1.34 |  1 |  1
DAgger |  Yes |  Yes∗ |  100 |  100 |  10.21 |  1.66 |  1 |  1
SA-Sparse |  BC |  Yes |  Yes |  100 |  100 |  4.23 |  1.13 |  1 |  1
DAgger |  Yes |  Yes∗ |  100 |  100 |  3.75 |  1.07 |  1 |  1
TABLE I: Comparison of the IL methods considered for RT-MPC compression. T1
refers to the trajectory tracking task under wind-like disturbances, T2 under
model errors (drag coefficient mismatch). At convergence (iteration 20-30) we
evaluate, in the target domain, robustness (success rate) and performance
(relative percent error between actions of the expert and of the compressed
policy). Demonstration-Efficiency represents the number of demonstrations
required to achieve for the first time full success rate. An approach is
considered easy if it does not require to apply disturbances/perturbations
during training (e.g., in lab2real transfer); an approach is considered safe
if does not execute actions that may cause state constraints violations
(crashes) during training. *Safe in our numerical evaluation, but not
guaranteed.

Tasks description. Our objective is to compress an RTMPC policy capable to
track a $7$s long, eight-shaped trajectory. We evaluate the considered
approaches in two different target domains, with wind-like disturbances (T1)
or model errors (T2). Disturbances in T1 are sampled adversarially from
$\mathbb{W}$ ($\approx$25–30% of the UAV weight), while model errors in T2 are
applied via mismatches in the drag coefficients used between training and
testing.

Results. We start by evaluating the robustness in T1 as a function of the
number of demonstrations collected in the source domain. The results are shown
in Fig 4, highlighting that:  i) while all the approaches achieve robustness
(full success rate) in the source domain, SA achieves full success rate after
only a single demonstration, being 5-6 times more sample efficient than the
baseline methods;  ii) SAis able to achieve full robustness in the target
domain, while baseline methods do not fully succeed, and converge at much
lower rate.  These results remark the presence of a distribution shift between
the source and target, which is not fully compensated by baselines methods
such as BC, due to lack of exploration and robustness. The performance
evaluation and additional results are summarized in Table I. We highlight that
in the target domain, sparse SA combined with DAgger achieves closest
performance to the expert. Dense SA suffers from performance drops,
potentially due to the limited capacity of the considered DNN or challenges in
training introduced by this data augmentation. Because of its effectiveness
and greater computational efficiency, we use sparse SA for the rest of the
work. Table I additionally presents the results for task T2. Although this
task is less challenging (i.e., all the approaches achieve full robustness),
the proposed method (sparse SA) achieves highest demonstration efficiency and
lowest expert gap, with similar trends as T1.

Computation. The average latency (on i7-8750H laptop with NVIDIA GTX1060 GPU)
for the expert (MATLAB) is $55.3$ms, while for the compressed policy (PyTorch)
is $0.25$ms, achieving a two-orders of magnitude improvement. The average
latency for the compressed policy on an Nvidia TX2 CPU (PyTorch) is $1.66$ms.

Figure 5: Experimental evaluation performed by hovering with and without wind
disturbances produced by a leaf blower. The employed compressed RTMPC policy
is trained in simulation from a single demonstration of the desired
trajectory. The wind-like disturbances produce a large position error, but do
not destabilize the system. The thrust decreases due to the robot being pushed
up by the disturbances. The state estimate (shown in the plot) is provided by
onboard VIO. The $y$-axis points approximately in the same direction as the
wind.

###  IV-C Hardware evaluation for tracking a single trajectory

(a) Reference and actual trajectory

(b) Experimental setup showing the trajectory executed by the robot, and the
leaf blowers used to generate disturbances

(c) Effects of wind

Figure 6: Experimental evaluation of the proposed approach (Figure 6(b)) under
wind-like disturbances. The robot learns to track an eight-shaped reference
trajectory (Figure 6(a)) from a single demonstration collected in simulation,
achieving zero-shot transfer. It is additionally able to withstand
disturbances produced by an array of leaf-blowers, unseen during the training
phase, and whose effects are clearly visible in the altitude errors (and
change in commanded thrust) in Figure 6(c).

We validate the demonstration efficiency, robustness and performance of the
proposed approach by experimentally testing policies trained after a single
demonstration collected in simulation using DAgger/BC (which operate
identically since we use DAgger with $\beta=1$ for the first demonstration).
The data augmentation strategy is based on the sparse SA. We use the MIT/ACL
open-source snap-stack [41] for controlling the attitude of the MAV, while the
compressed RTMPC runs at $100$Hz on the onboard Nvidia TX2 (on its CPU), with
the reference trajectory provided at $100$Hz. State estimation is obtained via
a motion capture system or onboard VIO. The first task considered is to hover
under wind disturbances produced by a leaf blower. The results are shown in
Figure 5, and highlight the ability of the system to remain stable despite the
large position error caused by the wind. The second task is to track an eight-
shaped trajectory, with velocities up to $3$m/s. We evaluate the robustness of
the system by applying a wind-like disturbance produced by an array of $3$
leaf blowers (Figure 6). The given position reference and the corresponding
trajectory are shown in Figure 6(a). The effects of the wind disturbances are
clearly visible in the altitude errors and changes in commanded thrust in
Figure 6(a) (at $t=11$s and $t=23$s). These experiments show that the
controller can robustly track the desired reference, withstanding challenging
perturbations unseen during the training phase. The video submission presents
more experiments, including lab2real transfer.

Figure 7: Robustness (Success Rate, top row) and performance (MPC Stage Cost,
bottom row) of the proposed approach (with $95\%$ confidence interval), as a
function of the number of demonstrations used for training, for the task of
learning to track previously unseen circular, eight-shaped and constant
position reference trajectories, sampled from the same training distribution.
The left column presents the results in the training domain (no disturbance),
while the right column under wind-like perturbations (with disturbance). The
proposed RTMPC driven sparse SA strategy learns to track multiple trajectory
and generalize to unseen ones requiring less demonstrations. At convergence
(from demonstration $20$ to $30$), DAgger+SA achieves the closes performance
to the expert ($2.7\%$ expert gap), followed by BC+SA ($3.0\%$ expert gap).
Evaluation performed using $20$ randomly sampled trajectories per
demonstration, repeated across 6 random seeds, with prediction horizon of
$N=20$ to speed-up demonstration collection, and the DNN input size is
adjusted accordingly. Figure 8: Examples of different, arbitrary chosen
trajectories from the training distribution, tested in hardware experiments
with and without strong wind-like disturbances produced by leaf blowers. The
employed policy is trained with $10$ demonstrations (when other baseline
methods have not fully converged yet, see Figure 7) using DAgger+SA (sparse).
This highlights that sparse SA can learn multiple trajectories in a more
sample-efficient way than other IL methods, retaining RTMPC’s robustness and
performance. Prediction horizon used is $N=20$, and DNN input size is adjusted
accordingly.

###  IV-D Numerical and hardware evaluation for learning and generalizing to
multiple trajectories

We evaluate the ability of the proposed approach to track multiple
trajectories while generalizing to unseen ones. To do so, we define a training
distribution of reference trajectories (circle, position step, eight-shape)
and a distribution for these trajectory parameters (radius, velocity,
position). During training, we sample at random a desired, $7$s long ($70$
steps) reference with randomly sampled parameters, generating a demonstration
and updating the proposed policy, while testing on a set of $20$, $7$s long
trajectories randomly sampled from the defined distributions. We monitor the
robustness and performance of the different methods, with force disturbances
applied in the target domain. The results of the numerical evaluation, shown
in Figure 7, confirm that sparse SA  i) achieves robustness and performance
comparable to the one of the expert in a sample efficient way, requiring less
than half the demonstrations than baseline approaches;  ii) simultaneously
learns to generalize to multiple trajectories randomly sampled from the
training distribution.  The hardware evaluation, performed with DAgger
augmented via sparse SA, is shown in Figure 8. It confirms that the proposed
approach is experimentally capable of tracking multiple trajectories under
real-world disturbances/model errors.

##  V Conclusion and Future Work

This work has presented a demonstration-efficient strategy to compress a MPC
in a computationally efficient representation, based on a DNN, via IL. We
showed that greater sample efficiency and robustness than existing IL methods
(DAgger, BC and their combination with DR) can be achieved by designing a
Robust Tube variant of the given MPC, using properties of the tube to guide a
sparse data augmentation strategy. Experimental results – showing trajectory
tracking control for a multirotor after a single demonstration under wind-like
disturbances – confirmed our numerical findings. Future work will focus on
designing an adaptation strategy.

## ACKNOWLEDGMENT

This work was funded by the Air Force Office of Scientific Research MURI
FA9550-19-1-0386.

## References

  * [1] F. Borrelli, A. Bemporad, and M. Morari, _Predictive control for linear and hybrid systems_. Cambridge University Press, 2017.
  * [2] B. T. Lopez, J.-J. E. Slotine, and J. P. How, “Dynamic tube mpc for nonlinear systems,” in _2019 American Control Conference (ACC)_. IEEE, 2019, pp. 1655–1662.
  * [3] B. T. Lopez, “Adaptive robust model predictive control for nonlinear systems,” Ph.D. dissertation, Massachusetts Institute of Technology, 2019.
  * [4] W. Li and E. Todorov, “Iterative linear quadratic regulator design for nonlinear biological movement systems.” in _ICINCO (1)_. Citeseer, 2004, pp. 222–229.
  * [5] M. Kamel, M. Burri, and R. Siegwart, “Linear vs nonlinear mpc for trajectory tracking applied to rotary wing micro aerial vehicles,” _IFAC-PapersOnLine_ , vol. 50, no. 1, pp. 3463–3469, 2017.
  * [6] M. V. Minniti, F. Farshidian, R. Grandia, and M. Hutter, “Whole-body mpc for a dynamically stable mobile manipulator,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 4, pp. 3687–3694, 2019.
  * [7] G. Williams, P. Drews, B. Goldfain, J. M. Rehg, and E. A. Theodorou, “Aggressive driving with model predictive path integral control,” in _2016 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2016, pp. 1433–1440.
  * [8] J. B. Rawling and P. Kumar, “Industrial, large-scale model predictive control with deep neural networks,” http://helper.ipam.ucla.edu/publications/lco2020/lco2020_16406.pdf, (Accessed on 05/18/2021).
  * [9] S. Levine and V. Koltun, “Guided policy search,” in _International conference on machine learning_. PMLR, 2013, pp. 1–9.
  * [10] G. Kahn, T. Zhang, S. Levine, and P. Abbeel, “Plato: Policy learning using adaptive trajectory optimization,” in _2017 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2017, pp. 3342–3349.
  * [11] T. Zhang, G. Kahn, S. Levine, and P. Abbeel, “Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search,” in _2016 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2016, pp. 528–535.
  * [12] J. Carius, F. Farshidian, and M. Hutter, “Mpc-net: A first principles guided policy search,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 2897–2904, 2020.
  * [13] E. Kaufmann, A. Loquercio, R. Ranftl, M. Müller, V. Koltun, and D. Scaramuzza, “Deep drone acrobatics,” _arXiv preprint arXiv:2006.05768_ , 2020.
  * [14] S. Ross, N. Melik-Barkhudarov, K. S. Shankar, A. Wendel, D. Dey, J. A. Bagnell, and M. Hebert, “Learning monocular reactive uav control in cluttered natural environments,” in _2013 IEEE international conference on robotics and automation_. IEEE, 2013, pp. 1765–1772.
  * [15] A. Reske, J. Carius, Y. Ma, F. Farshidian, and M. Hutter, “Imitation learning from mpc for quadrupedal multi-gait control,” _arXiv preprint arXiv:2103.14331_ , 2021.
  * [16] D. A. Pomerleau, “Alvinn: An autonomous land vehicle in a neural network,” Carnegie-Mellon Univ Pittsburgh PA Artificial Intelligence and Psychology, Tech. Rep., 1989.
  * [17] T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and J. Peters, “An algorithmic perspective on imitation learning,” _arXiv preprint arXiv:1811.06711_ , 2018.
  * [18] M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang _et al._ , “End to end learning for self-driving cars,” _arXiv preprint arXiv:1604.07316_ , 2016.
  * [19] S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in _Proceedings of the fourteenth international conference on artificial intelligence and statistics_. JMLR Workshop and Conference Proceedings, 2011, pp. 627–635.
  * [20] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization,” in _2018 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2018, pp. 3803–3810.
  * [21] A. Loquercio, E. Kaufmann, R. Ranftl, A. Dosovitskiy, V. Koltun, and D. Scaramuzza, “Deep drone racing: From simulation to reality with domain randomization,” _IEEE Transactions on Robotics_ , vol. 36, no. 1, pp. 1–14, 2019.
  * [22] H. Tsukamoto and S.-J. Chung, “Learning-based robust motion planning with guaranteed stability: A contraction theory approach,” _IEEE Robotics and Automation Letters_ , 2021.
  * [23] ——, “Neural contraction metrics for robust estimation and control: A convex optimization approach,” _IEEE Control Systems Letters_ , vol. 5, no. 1, pp. 211–216, 2020.
  * [24] Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. Theodorou, and B. Boots, “Agile autonomous driving using end-to-end deep imitation learning,” _arXiv preprint arXiv:1709.07174_ , 2017.
  * [25] A. Farchy, S. Barrett, P. MacAlpine, and P. Stone, “Humanoid robots learning to walk faster: From the real world to simulation and back,” in _Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems_ , 2013, pp. 39–46.
  * [26] Y. Chebotar, A. Handa, V. Makoviychuk, M. Macklin, J. Issac, N. Ratliff, and D. Fox, “Closing the sim-to-real loop: Adapting simulation randomization with real world experience,” in _2019 International Conference on Robotics and Automation (ICRA)_. IEEE, 2019, pp. 8973–8979.
  * [27] M. Laskey, J. Lee, R. Fox, A. Dragan, and K. Goldberg, “Dart: Noise injection for robust imitation learning,” in _Conference on robot learning_. PMLR, 2017, pp. 143–156.
  * [28] M. D. Laskey, “On and off-policy deep imitation learning for robotics,” Ph.D. dissertation, UC Berkeley, 2018.
  * [29] J. Hanna and P. Stone, “Grounded action transformation for robot learning in simulation,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 31, no. 1, 2017.
  * [30] S. Desai, I. Durugkar, H. Karnan, G. Warnell, J. Hanna, P. Stone, and A. Sony, “An imitation from observation approach to transfer learning with dynamics mismatch,” _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [31] P. Florence, L. Manuelli, and R. Tedrake, “Self-supervised correspondence in visuomotor policy learning,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 492–499, 2019.
  * [32] D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Lakshminarayanan, “Augmix: A simple data processing method to improve robustness and uncertainty,” _arXiv preprint arXiv:1912.02781_ , 2019.
  * [33] M. Shu, Y. Shen, M. C. Lin, and T. Goldstein, “Adversarial differentiable data augmentation for autonomous systems,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021.
  * [34] D. Antotsiou, C. Ciliberto, and T.-K. Kim, “Adversarial imitation learning with trajectorial augmentation and correction,” _arXiv preprint arXiv:2103.13887_ , 2021.
  * [35] M. Toromanoff, E. Wirbel, F. Wilhelm, C. Vejarano, X. Perrotton, and F. Moutarde, “End to end vehicle lateral control using a single fisheye camera,” in _2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2018, pp. 3613–3619.
  * [36] A. Amini, I. Gilitschenski, J. Phillips, J. Moseyko, R. Banerjee, S. Karaman, and D. Rus, “Learning robust control policies for end-to-end autonomous driving from data-driven simulation,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 1143–1150, 2020.
  * [37] D. Q. Mayne, M. M. Seron, and S. Raković, “Robust model predictive control of constrained linear systems with bounded disturbances,” _Automatica_ , vol. 41, no. 2, pp. 219–224, 2005.
  * [38] D. D. Fan, A.-a. Agha-mohammadi, and E. A. Theodorou, “Deep learning tubes for tube mpc,” _arXiv preprint arXiv:2002.01587_ , 2020.
  * [39] R. S. Sutton and A. G. Barto, _Reinforcement learning: An introduction_. MIT press, 2018.
  * [40] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [41] Aerospace Controls Laboratory, “snap-stack: Autopilot code and host tools for flying snapdragon flight-based vehicles,” https://gitlab.com/mit-acl/fsw/snap-stack, (Accessed on 02/23/2020).

OC

    Optimal Control
LQR

    Linear Quadratic Regulator
MAV

    Micro Aerial Vehicle
GPS

    Guided Policy Search
UAV

    Unmanned Aerial Vehicle
MPC

    Model Predictive Control
RTMPC

    Robust Tube MPC
DNN

    deep neural network
BC

    Behavior Cloning
DR

    Domain Randomization
SA

    Sampling Augmentation
IL

    Imitation Learning
DAgger

    Dataset-Aggregation
MDP

    Markov Decision Process
VIO

    Visual-Inertial Odometry
  *[MPC]: Model Predictive Control
  *[GPS]: Guided Policy Search
  *[IL]: Imitation Learning
  *[DNN]: deep neural network
  *[BC]: Behavior Cloning
  *[DAgger]: Dataset-Aggregation
  *[DR]: Domain Randomization
  *[RTMPC]: Robust Tube MPC
  *[SA]: Sampling Augmentation
  *[LQR]: Linear Quadratic Regulator
  *[VIO]: Visual-Inertial Odometry


# Chirality Enables Thermal Magnon Transistors

Tao Yu<EMAIL_ADDRESS>School of Physics, Huazhong University of Science
and Technology, Wuhan 430074, China  Chengyuan Cai  School of Physics,
Huazhong University of Science and Technology, Wuhan 430074, China  Gerrit E.
W. Bauer<EMAIL_ADDRESS>WPI-AIMR and Institute for Materials
Research and CSRN, Tohoku University, Sendai 980-8577, Japan  Kavli Institute
for Theoretical Sciences, University of the Chinese Academy of Sciences,
Beijing 100190, China

###### Abstract

We report a theory of thermal spin pumping into proximity magnets under a
transverse-bias-driven heat flow of magnons in magnetic films when the dipolar
coupling to the magnetic gate is tuned to be “chiral”. While there is no
rectification of the magnon current in the film, we predict that chirality
diverts a large percentage (50$\%$ for perfect chirality) of it into the gate.
This transverse thermal spin pumping effect can be controlled by rotating the
film magnetization and may help manage the heat flow in future magnonic
circuits.

###### pacs:

75.30.Ds, 05.60.Gg, 75.60.Ej, 33.55.Ad

Introduction.—Thermal rectification implies the possibility to transport heat
against a temperature gradient, a potentially important tool to modulate and
manipulate the temperature in nanoscale structures [2, 3, 1]. Rectification or
diode effects in electronics and phononics [2, 2, 3] are phenomena beyond
linear response theory. Another phenomenon is the non-reciprocity or chirality
in the excitation and propagation of coherent quasi-particles, i.e. the
coupling between their propagation direction and internal degrees of freedom,
such as spin or orbital angular momentum [4, 5, 6]. The chiral nature of
Damon-Eshbach spin waves on the surface of a ferromagnet [7], for example,
enables “conveyor belt” transport of magnons in one direction [8, 9, 10]
irrespective of a temperature gradient. Non-reciprocity was also observed in
non-local thermal transport experiments in thin yttrium iron garnet (YIG)
films with Pt spin injection and detection as ascribed to an interfacial
Dzyaloshinskii-Moriya interaction [11, 12] or an attached floating permalloy
gate [13]. Similar non-reciprocal magnon transport was reported in
antiferromagnets as well [14]. The blocking of one in favor of the other
current direction can then be confused with rectification or diode effects.

In this work, we address thermal transport in the linear regime for a generic
magnon transistor [15, 16, 17] composed of a YIG film that is partly covered
by an array of magnetic nanowires, as sketched in Fig. 1. Baumgaertl and
Grundler report that such a structure may serve as a non-volatile memory in
which a magnon current in the YIG film switches the nanowire magnetization
[18], which urgently calls for a microscopic model. The dipolar interaction
between quasiparticles in the film and wires is “chiral”, i.e., for a fixed
magnetization it depends on the propagation direction [4, 5], as widely
predicted and observed for coherent waves in the form of, e.g., chiral pumping
[19, 20, 23, 21, 22], chiral spin transport [24], non-reciprocal transmission
of microwaves [26, 27, 25, 28], and the “phonon diode” [29, 30, 31, 32]. These
systems represent the general problem characterized by Hamiltonians with non-
reciprocal coupling constant $|g_{k}|\neq|g_{-k}|$ between two different
fields or particles [5]. The thermodynamic consequences of chirality have yet
often been overlooked, as reflected by the frequent but incorrect use of the
word “diode effect” or “rectification” when the chirality is meant. The former
is a relation $J(\Delta T)\neq-J(-\Delta T)$ for a current $J$ that when
driven in linear response by a temperature bias $\Delta T$ would imply a (non-
existent) autonomous Maxwell Demon or “trapdoor”. Thermal fluctuations in the
temperature gradients and currents that average out to zero over time cause,
for instance, the Johnson-Nyquist noise in electronic circuits [33]. A
relation $J(\Delta T)\neq-J(-\Delta T)$ would rectify the equilibrium thermal
noise, in violation of the Second Law of Thermodynamics.

Figure 1: The exchange of magnons between a magnetic film that carries a heat
current and a proximity magnetic nanowires can be chiral, i.e., depending on
the propagation direction of the magnons in the film. $a_{1,2}$ are the input
amplitudes of (spin) waves and $b_{1,2}$ are their output. The gate (the blue-
striped region) can be tuned to modulate only the transmission amplitude
$a_{1}$.

In the linear response regime, the chiral coupling cannot cause rectification,
so our emphasis lies on the elucidation of its observable physical
consequences. To this end, we treat the (floating) magnetic nanowire gate as a
third thermal bath that exchanges magnons with the film in a chiral manner,
e.g. by causing damping of the waves in the film that propagate into one
direction, but not those into the other. We formulate the problem in terms of
self-consistent non-equilibrium Green functions and confirm that in the
absence of a temperature difference between the source and drain contacts, all
currents vanish on average. There is no rectification of the equilibrium
currents $J$ under a temperature bias $\Delta T$, viz. $J(\Delta T)=-J(-\Delta
T)$. Nevertheless, we predict that chirality causes a surprising and contra-
intuitive effect: In the absence of chirality and a symmetric configuration,
the floating gate is inert but switching on the chirality diverts a large
percentage (one-half for perfect chirality) of the magnon heat and spin
currents into the gate. Since the chirality depends strongly on the magnetic
configuration we can control the effect by a weak in-plane magnetic field,
which might be useful for thermal management in magnonic circuits.

Model for a non-unitary scattering matrix.—We illustrate the physics and our
formalism at the hand of ballistic magnons in thin YIG films of thickness
${\cal O}(10~{}{\rm nm})$ that couple to metallic magnetic nanowires [19, 20,
21, 23, 34] or magnetic bi- and multilayers [35, 36, 37, 38] that when
strongly damped act as the reservoirs in the scattering theory of transport
[39, 40]. More generally, they form a non-Hermitian dynamic “scattering
potential”, in the adiabatic limit giving rise to spin pumping and current-
induced magnetization dynamics. Here we address the effects of chiral coupling
on dissipative and non-adiabatic transport.

For computational convenience, we adopt a generic scattering region that
consists of a regular array of floating metallic magnetic nanowires on top of
the YIG film [21, 34] and along the $\hat{\bf z}$-direction with inter-wire
distance $L_{0}$, counted by the index $l=\\{1,\cdots,N\\}$ as in Fig. 1. A
thin insulating layer such as ${\rm Al_{2}O_{3}}$ suppresses any interlayer
exchange interaction, such that the magnetizations are only coupled by the
magneto-dipolar interaction as observed in many recent experiments [21, 41,
22, 18]. A strong external field along the wires first aligns the
magnetizations. For a YIG film, a weak ($\gtrsim$5 mT) in-plane applied
magnetic field ${\bf H}_{0}$ freely rotates the substrate magnetization into
any direction, while the shape anisotropy strongly pins the wire
magnetization, as substantiated by model calculation in the Supplemental
Material (SM).

Magnons in thin YIG films thermally populate the lower perpendicular standing
spin waves (PSSWs), which is effectively a single one when the film is
sufficiently thin and/or the temperature is sufficiently low [42]. The lowest
PSSW also couples most efficiently with adjacent magnetic nanowires. We defer
the standard calculation of the magnon modes to the SM. Hence, we focus on
this two-dimensional limit with Hamiltonian for the film

$\displaystyle\hat{H}_{m}/\hbar=\sum_{\kappa}\sum_{k_{z}}\omega_{\kappa}(k_{z})\hat{m}^{\dagger}_{{\kappa}}(k_{z})\hat{m}_{{\kappa}}(k_{z}),$
(1)

where $\omega_{\kappa}(k_{z})$ is the magnon dispersion relation and
$\hat{m}_{\kappa}(k_{z})$ annihilates a magnon with wave vector
${\kappa}\hat{\bf y}+k_{z}\hat{\bf z}$. $\hat{\bf y}$ is the transport
direction. The dipolar interaction between wires for inter-wire distances
$L_{0}$ several times larger than their widths $w$ can safely be disregarded.
The nanowires are thin and narrow, so their Hamiltonian reads

$\displaystyle\hat{H}_{s}/\hbar=\sum_{l}\sum_{k_{z}}\Omega({k_{z}})\hat{\beta}_{l}^{\dagger}(k_{z})\hat{\beta}_{l}(k_{z}),$
(2)

where $\Omega({k_{z}})$ is the frequency of a magnon $\hat{\beta}_{l}(k_{z})$
in the $l$-th wire that propagates freely along the wire $\hat{\bf
z}$-direction. The coupling between the magnons in film and wires

$\displaystyle\hat{H}_{c}/\hbar=\sum_{l,k_{z}}\sum_{\kappa}g_{\kappa,l}({k_{z}})\hat{\beta}_{l}^{\dagger}(k_{z})\hat{m}_{{\kappa}}(k_{z})+{\rm
H.c.},$ (3)

where the dipolar interaction $g_{\kappa,l}(k_{z})$ is non-reciprocal, viz.
$|g_{\kappa,l}(k_{z})|\neq|g_{-\kappa,l}(k_{z})|$ [20], referring for details
to the SM.

For sufficiently large damping of the magnetization dynamics in the wires, any
non-equilibrium state is dissipated quickly and cannot cause a back action on
the film. The energy transfer from the film to the wires then breaks the
unitarity of the scattering matrix of magnons in the film. Calling the
conserved wave number along the long wires $k_{z}$, the scattering amplitude
of film magnons from $\kappa\hat{\bf y}$ to $\kappa^{\prime}\hat{\bf y}$ may
be expressed by the $T$-matrix [44, 43, 45]

$\displaystyle T_{{\kappa}^{\prime}{\kappa}}(k_{z})$
$\displaystyle=\delta_{{\kappa}^{\prime}{\kappa}}+\frac{{\cal
M}_{{\kappa}^{\prime}}^{\dagger}(k_{z}){\cal
G}^{r}_{k_{z}}(\omega_{\kappa,k_{z}}){\cal
M}_{\kappa}(k_{z})}{\omega_{\kappa}(k_{z})-\omega_{{\kappa}^{\prime}}(k_{z})+i0_{+}},$
(4)

where ${\cal
M}_{\kappa}(k_{z})=\begin{pmatrix}g_{\kappa,1}(k_{z}),\cdots,g_{\kappa,N}(k_{z})\end{pmatrix}^{T}$
and the matrix ${\cal G}^{r}_{k_{z}}(\omega)=\left(\omega-{\cal
H}_{k_{z}}(\omega)\right)^{-1}$ is the (retarded) Green function of the non-
Hermitian wire Hamiltonian [45, 46]

${\cal
H}_{k_{z}}(\omega)|_{ll^{\prime}}=\Omega({k_{z}})(1-i\alpha_{G})\delta_{ll^{\prime}}+\Sigma_{k_{z}}(\omega)|_{ll^{\prime}}.$
(5)

Its damping Gilbert constant $\alpha_{G}$ can easily be engineered to be large
${\cal O}(0.01-0.1)$. $\Sigma$ is the self-energy matrix of the wire magnons

$\displaystyle\Sigma_{k_{z}}(\omega)|_{ll^{\prime}}=\sum_{\kappa^{\prime}}\frac{g_{\kappa^{\prime},l}(k_{z})g^{*}_{\kappa^{\prime},l^{\prime}}(k_{z})}{\omega-\omega_{\kappa^{\prime}}(k_{z})+i0_{+}}.$
(6)

We disregard the damping of the film magnons, which is allowed for a material
such as YIG with Gilbert constant much smaller than the wire $\alpha_{G}$.

The transmission and reflection amplitudes that make up the magnon scattering
matrix $S$ for given $\kappa>0$ reads according to Eq. (4) [44, 43, 45]

$\displaystyle S_{21}(\kappa,k_{z})=1-\frac{iL_{y}}{|v_{\kappa}(k_{z})|}{\cal
M}_{\kappa}^{\dagger}(k_{z}){\cal G}^{r}_{k_{z}}(\omega_{\kappa,k_{z}}){\cal
M}_{\kappa}(k_{z}),$ $\displaystyle
S_{11}(\kappa,k_{z})=-\frac{iL_{y}}{|v_{\kappa}(k_{z})|}{\cal
M}_{-\kappa}^{\dagger}{\cal G}^{r}_{k_{z}}(\omega_{\kappa,k_{z}}){\cal
M}_{\kappa}(k_{z}),$ (7)

where $v_{\kappa}(k_{z})=\partial\omega_{\kappa}(k_{z})/\partial\kappa$ is the
group velocity along the transport $\hat{\bf y}$-direction and $L_{y}$ is the
sample length along $\hat{\bf y}$. We disregard a constant propagation phase
factor in $S_{21}(\kappa,k_{z})$. $S_{12}(\kappa,k_{z})$ and
$S_{22}(\kappa,k_{z})$ are obtained by replacing
$|\kappa|\rightarrow-|\kappa|$ in Eq. (7). In the chiral limit, the
reflections vanish with either $g_{\kappa,l}(k_{z})=0$ or
$g_{-\kappa,l}(k_{z})=0$. This scattering matrix is unitary
$|S_{21}(\kappa,k_{z})|^{2}+|S_{11}(\kappa,k_{z})|^{2}=1$ only when the magnon
number is conserved, i.e., when $\alpha_{G}\rightarrow 0$. In the limit of a
single narrow wire, we can treat the problem analytically with

$\displaystyle S_{21}(\kappa,k_{z})$
$\displaystyle=1-\frac{i\Gamma_{R}(\kappa,k_{z})}{\omega_{\kappa,k_{z}}-\Omega_{k_{z}}+i\left[\alpha_{G}\Omega_{k_{z}}+(\Gamma_{L}+\Gamma_{R})/{2}\right]},$
$\displaystyle S_{12}(\kappa,k_{z})$
$\displaystyle=1-\frac{i\Gamma_{L}(\kappa,k_{z})}{\omega_{\kappa,k_{z}}-\Omega_{k_{z}}+i\left[\alpha_{G}\Omega_{k_{z}}+(\Gamma_{L}+\Gamma_{R})/{2}\right]},$

where
$\Gamma_{L}(\kappa,k_{z})={L_{y}}g^{2}_{-|\kappa|}(k_{z})/{v_{\kappa}(k_{z})}$
and
$\Gamma_{R}(\kappa,k_{z})={L_{y}}g^{2}_{|\kappa|}(k_{z})/{v_{\kappa}(k_{z})}$.
In the chiral limit $\Gamma_{L}=0$, the transmission $S_{12}$ for a single
wire is unity, while the reflection $S_{22}$ vanishes, as illustrated by Fig.
1.

In the following, we present numerical results for an array of $N=10$
nanowires with dimensions used in experiments Refs. [21, 34]. We calculate the
retarded Green function ${\cal G}^{r}_{k_{z}}$ and scattering matrix Eq. (7)
by inverting the non-Hermitian Hamiltonian (5), referring to the SM for
details. We choose Co nanowires with width $w=100$ nm, thickness $d=30$ nm,
magnetization $\mu_{0}\tilde{M}_{s}=1.1$ T, and exchange stiffness
$\tilde{\alpha}_{\rm ex}=3.1\times 10^{-17}$ ${\rm m}^{2}$. The YIG film is
$s=10$ nm thick, with magnetization $\mu_{0}{M}_{s}=0.2$ T, and exchange
stiffness $\alpha_{\rm ex}=3.0\times 10^{-16}$ ${\rm m}^{2}$ [21, 34].
Moreover, $\mu_{0}H_{0}=0.05$ T, $L_{0}=300$ nm between the neighboring wires,
the Co damping coefficient $\alpha_{G}=0.05$, $\mu_{0}=4\pi\times 10^{-7}$
$\rm H/m$, and $\gamma=1.82\times 10^{11}$ $\rm s^{-1}T^{-1}$.

Figures 2(a)-(c) show the moduli of the transmission amplitudes $|S_{12}({\bf
k})|$ ($|S_{21}({\bf k})|$) of spin waves in the YIG film with wave vectors
${\bf k}$ that are launched from the left (right) to interact with the Co
nanowires that form an angle $\varphi$ of the film magnetization with the wire
$\hat{\bf z}$-direction. Figure 2(d) express the non-unitarity of the
scattering matrix when the magnetizations are antiparallel. Figures 2(a)-(c)
reveal strong inter-magnet interactions over a wide range of wavelengths of up
to $\sim 50$ nm and deviations from the ideal chiral limit. The non-
reciprocity with respect to $k_{y}$ is pronounced in the antiparallel
$\varphi=\pi$ configuration, i.e., when the dipolar fields of the spin waves
in the film match those of the wire dynamics, as sketched in the SM. The
damping in the Co wires breaks the unitarity of the scattering matrix, as
emphasized by Fig. 2(d). The non-unitarity is also chiral, almost perfectly so
for $k_{z}=0$.

The non-reciprocity of the scattering matrix can be measured by spatially
resolved Brillouin light scattering [47] and NV center magnetometry [48, 49]
of the spin wave stray fields. It also uniquely affects thermal magnon
transport as demonstrated in the following.

Figure 2: Transmission coefficients $\\{|S_{12}({\bf k})|,|S_{21}({\bf
k})|\\}$ and non-unitarity of the scattering matrix of spin waves in a YIG
film as a function of wave vector ${\bf k}$ launched to the gate formed by a
Co nanowire array for different magnetic configurations. (a)-(c) shows nearly
total reflections when the film magnetization is parallel, normal, and
antiparallel to that of the wires. (d) shows the deviation of ${|S_{11}({\bf
k})|^{2}+|S_{21}({\bf k})|^{2}}$ from unity, which is caused by the Gilbert
damping in the wires. The associated non-unitarity is chiral with respect to
$k_{y}$.

Chirality-enabled thermal spin pumping.—Here we address the nontrivial role of
chirality in the thermal spin pumping into the magnetic nanowire. A
temperature difference $T_{S/D}=T_{0}\pm\Delta T/2$ between the source “S” and
drain “D” reservoirs as shown in Fig. 1 drives heat and magnon currents in the
YIG film. We formulate the magnon transport in our dissipative system in terms
of self-consistent non-equilibrium Green functions. The loss rate of magnons
in the film equals the difference in the rates of change of the ensemble
averages $\langle\cdots\rangle$ in the presence and absence of the interaction
Eq. (3).

The density operator
$\hat{\rho}_{\kappa}(k_{z})=\hat{m}^{\dagger}_{\kappa}(k_{z})\hat{m}_{\kappa}(k_{z})$
obeys the Heisenberg equation of motion

$\displaystyle\left\langle\frac{d\hat{\rho}_{\kappa}(k_{z})}{dt}\right\rangle$
$\displaystyle=\frac{i}{\hbar}\left\langle\left[\hat{H}_{m}+\hat{H}_{c},\hat{\rho}_{\kappa}(k_{z})\right]\right\rangle$
$\displaystyle=2{\rm
Re}\left(\sum_{l}g_{\kappa,l}^{*}({k}_{z})G_{\kappa,l}^{<}(k_{z};t,t)\right).$
(8)

The transfer of magnons into the wires is therefore proportional to the
coupling strength and the “lesser” Green function
$G_{{\kappa,}l}^{<}(k_{z};t,t^{\prime})\equiv-i\left\langle\hat{\beta}^{\dagger}_{l}(k_{z},t^{\prime})\hat{m}_{{\kappa}}(k_{z},t)\right\rangle$.
In frequency space [50, 51],
$G_{\kappa,l}^{<}(k_{z},\omega)=\sum_{l^{\prime}}g_{\kappa,l^{\prime}}(k_{z})[{\cal
G}^{r}_{k_{z},ll^{\prime}}(\omega){G}^{(0)}_{\kappa,<}(k_{z},\omega)+{\cal
G}_{k_{z},ll^{\prime}}^{<}(\omega){G}_{\kappa,a}^{(0)}(k_{z},\omega)]$, where
for magnons in the unperturbed film ${G}^{(0)}_{\kappa,<}(k_{z},\omega)=2\pi
if(\omega,T)\delta\left(\omega-\omega_{\kappa,k_{z}}\right)$ and the
“advanced” Green function
${G}^{(0)}_{{\kappa},a}(k_{z},\omega)={1}/({\omega-\omega_{\kappa,k_{z}}-i0_{+}})$
with Planck distribution $f(\omega,T)=1/\\{\exp[\hbar\omega/(k_{B}T)]-1\\}$ at
temperature $T$. On the other hand, when the Gilbert damping and magnon-magnon
interaction in the wires is strong, their magnon distribution is at
equilibrium with the ambient temperature $T_{0}$, allowing for exploiting the
equilibrium Green function ${\cal
G}_{k_{z}}^{<}(\omega)=-f(\omega,T_{0})[{\cal G}^{r}_{k_{z}}(\omega)-{\cal
G}^{r\dagger}_{k_{z}}(\omega)]$ [50, 51]. Substituting this into Eq. (8), we
arrive at

$\displaystyle\left\langle{d\hat{\rho}_{\kappa}(k_{z})}/{dt}\right\rangle$
$\displaystyle={\rm Re}\big{[}i{\cal M}^{\dagger}_{\kappa}(k_{z}){\cal
G}_{k_{z}}^{r}(\omega_{\kappa,k_{z}}){\cal M}_{\kappa}(k_{z})\big{]}$
$\displaystyle\times\left(f(\omega_{\kappa,k_{z}},T)-f(\omega_{\kappa,k_{z}},T_{0})\right)$
$\displaystyle+{\rm Re}\big{[}i{\cal M}^{\dagger}_{\kappa}(k_{z}){\cal
G}_{k_{z}}^{r}(\omega_{\kappa,k_{z}}){\cal
M}_{\kappa}(k_{z})\big{]}f(\omega_{\kappa,k_{z}},T)$ $\displaystyle+{\rm
Re}\big{[}i{\cal M}^{\dagger}_{\kappa}(k_{z}){\cal
G}_{k_{z}}^{r\dagger}(\omega_{\kappa,k_{z}}){\cal
M}_{\kappa}(k_{z})\big{]}f(\omega_{\kappa,k_{z}},T_{0}).$ (9)

The terms associated with the distribution $f(T)$ account for the spin
injection from films to wires, which is balanced by the inverse processes
proportional to $f(T_{0})$. When $T=T_{0}$,
$\left\langle{d\hat{\rho}_{\kappa}(k_{z})}/{dt}\right\rangle={\rm Re}[i{\cal
M}^{\dagger}_{\kappa}(k_{z})({\cal G}_{k_{z}}^{r}(\omega_{\kappa,k_{z}})+{\cal
G}_{k_{z}}^{r\dagger}(\omega_{\kappa,k_{z}})){\cal
M}_{\kappa}(k_{z})]f(\omega_{\kappa,k_{z}},T_{0})=0$, so the scattering
vanishes as required by the principle of detailed balance, as do equilibrium
magnon currents, in spite of the chirality of the interaction.

Next, we relate the magnon transmission Eq. (7) to the spin-injection rates of
magnons with $\kappa\hat{\bf y}+k_{z}\hat{\bf z}$. The magnons in the film are
weakly perturbed such that $\delta
f_{\kappa}(k_{z})=f(\omega_{\kappa,k_{z}},T)-f(\omega_{\kappa,k_{z}},T_{0})$
is small. Here we distinguish the wave numbers $\kappa_{L}$ and $\kappa_{R}$
in the left “L” and right “R” reservoirs. Under the temperature bias

$\displaystyle\delta f_{\kappa}(k_{z})=\frac{\partial
f(\omega_{\kappa,k_{z}},T)}{\partial T_{0}}\left\\{\begin{array}[]{c}\Delta
T/2,~{}~{}~{}~{}\kappa\in L\\\ -\Delta T/2,~{}~{}~{}\kappa\in
R\end{array}\right.,$ (12)

magnons with $\kappa_{L}>0$ are absorbed by the nanowires and after
thermalization re-emitted to travel into the right reservoir at the same rate.
The spin-injection rate from different reservoirs relates to the magnon
transmission (7) as $\left\langle
d\hat{\rho}_{\kappa,k_{z}}/dt\right\rangle={\delta
f_{\kappa}(k_{z})}{\Gamma_{\kappa}(k_{z})}$, viz., the wires dissipate magnon
energy at the rates

$\Gamma_{\kappa}(k_{z})=\frac{|v_{\kappa}(k_{z})|}{L_{y}}\left\\{\begin{array}[]{c}1-{\rm
Re}S_{21}(\kappa,k_{z}),~{}~{}~{}\kappa_{L,R}>0\\\ 1-{\rm
Re}S_{12}(\kappa,k_{z}),~{}~{}~{}\kappa_{L,R}<0\end{array}\right.,$

which vanish when the transmission is high but become significant
$\Gamma_{\kappa}(k_{z})\rightarrow|v_{\kappa}(k_{z})|/L_{y}$ otherwise. In the
chiral limits ${\rm Re}S_{21}\rightarrow-1$ or ${\rm Re}S_{12}\rightarrow-1$,
the injection rate $\Gamma_{\kappa}(k_{z})$ is enhanced by a factor 2 because
the source injects magnons into the wires while the drain contributes magnon
“holes”.

Exchange magnons carry an angular momentum $\hbar$ and energy
$\hbar\omega_{\kappa}(k_{z}).$ The spin [heat] current density flowing from
the film into the gate then read

$\displaystyle J_{s[E]}$
$\displaystyle=\frac{\hbar}{L_{z}}\sum_{\kappa}\sum_{k_{z}}{\delta
f_{\kappa}(k_{z})}\big{[}\omega_{\kappa}(k_{z})\big{]}{\Gamma_{\kappa}(k_{z})}$
$\displaystyle=\frac{\hbar}{L_{y}L_{z}}\frac{\Delta
T}{2}\sum_{\kappa>0,k_{z}}\frac{\partial f(\omega_{\kappa,k_{z}})}{\partial
T_{0}}\big{[}\omega_{\kappa}(k_{z})\big{]}$
$\displaystyle\times|v_{\kappa,k_{z}}|\big{(}{\rm
Re}S_{12}(-\kappa,k_{z})-{\rm Re}S_{21}(\kappa,k_{z})\big{)}.$ (13)

These expressions do not depend on a sufficiently large width $L_{z}$. The
right- and left-moving spin waves are distributed according to the temperature
of their source. The hot magnons from the source are annihilated when injected
into the nanowires. Without chirality ${\rm Re}\,S_{12}(-|\kappa|,k_{z})={\rm
Re}\,S_{21}(|\kappa|,k_{z})$, the in- and out-going magnon currents cancel to
$J_{s[E]}=0$. However, a finite chirality Eq. (13) leads to heating vs.
cooling of the wire, depending on the coupling parameter that can be tuned by
the in-plane magnetic field and the sign of the applied temperature bias.

The transistor can divert maximally $1/2$ of the spin current into the
nanowires when the chiral damping is complete such that all magnons from one
direction are fully dissipated, while those flowing in the other direction are
not affected at all, i.e., $[{\rm Re}S_{12}(\kappa,k_{z})-{\rm
Re}S_{21}(-\kappa,k_{z})]=1$ for all $k_{z}$, leading to the spin current
injected into wires $J_{s}=\hbar({\Delta T}/2)\int_{0}^{\infty}d\kappa\int
dk_{z}|v_{\kappa,k_{z}}|{\partial_{T_{0}}}f(\omega_{\kappa,k_{z}})$. On the
other hand, the flow in the substrate contains the contribution from the right
reservoir that is the same as $J_{s}$, so the current flowing in the substrate
$J_{L}=2J_{s}$.

Figure 3 plots the spin current dissipated by the nanowires as a function of
magnetization direction $\varphi$ of the film as well as magnetic wire number
$N$ when $T_{0}=10$ K and $\Delta T=2$ K. The magnetization direction
modulates the chirality and thereby the thermal spin pumping into the
nanowires by two orders when rotated from a parallel $(\varphi=0)$ to an
antiparallel $(\varphi=\pi)$ configuration (Fig. 2). Increasing the number of
wires from 1 to 20 at a constant inter-wire distance enhances the effect by an
order of magnitude. In the YIG/Co system, $J_{L}/\hbar$ is typically
$1.5\times 10^{19}$ ${\rm m}^{-1}{\rm s}^{-1}$ for $\Delta T=2$ K and
$T_{0}=10$ K. We compute a modulation efficiency $J_{s}/J_{L}$ of $\sim 13\%$
($\sim 27\%$) for $N=10$ magnetic wires of width $w=100$ nm (50 nm) and
$\mu_{0}H_{0}=5$ mT with magnetizations opposite to that of the substrate,
which is not far from the ideal maximal efficiency [Fig. 3(c) and (d)].

Figure 3: Spin current $J_{s}$ channeled off into the gate from the source to
drain [(a) and (b)] and its modulation efficiency $J_{s}/J_{L}$ [(c) and (d)]
as a function of magnetization direction $\varphi$, the external magnetic
field, and the number of magnetic wires $N$.

The spin injection from the substrate into nanowires enabled by the chirality
provides an anti-damping torque on the nanowire magnetization, which is the
largest in the antiparallel configuration. The effect might allow switching by
temperature gradient and explain the observed magnetization reversal [18], but
more work is needed to understand the magnitude of the effect.

Diverting heat and spin flows by chirality.—The above result corresponds to a
magnon transistor action [15, 16, 17]. Even though the gate is not biased, it
controls the thermal magnon current from the high-temperature contact into the
gate by the angle $\varphi$. This can be understood in terms of the Landauer-
Büttiker formula [39, 40] for a three-terminal device, in which the gate is
modeled as a thermodynamic reservoir (refer to the SM for a derivation). In
this formalism, all dissipation occurs only in the reservoir, while the
Hamiltonian of the scattering region is Hermitian. The spin current leaving
the source then reads

$\displaystyle J_{L}$
$\displaystyle=\frac{\hbar}{L_{y}L_{z}}\sum_{\kappa>0,k_{z}}v_{\kappa,k_{z}}\left(1-\sum_{\kappa^{\prime}>0}\left|T_{-\kappa\kappa^{\prime}}(k_{z})\right|^{2}\right)\delta
f_{L}(\omega_{\kappa,k_{z}})$
$\displaystyle-\frac{\hbar}{L_{y}L_{z}}\sum_{\kappa>0,k_{z}}v_{\kappa,k_{z}}\left(\sum_{\kappa^{\prime}<0}\left|T_{-\kappa\kappa^{\prime}}(k_{z})\right|^{2}\right)\delta
f_{R}(\omega_{\kappa,k_{z}}),$

which vanishes at equilibrium $f_{L}=f_{R}=f(T_{0})$, even when chirality
fully suppresses the magnon propagation in one direction. Moreover, under a
temperature bias $\delta f_{L/R}(\omega_{\kappa,k_{z}})=({\partial
f(\omega_{\kappa,k_{z}},T)}/{\partial T_{0}})(+/-){\Delta T}/{2}$,
$J_{L}(\Delta T)=-J_{L}(-\Delta T)$, so there is no rectification of the
thermal current, either. In the chiral limit, these results confirm that
optimally $J_{L}=\hbar{\Delta T}\int_{0}^{\infty}d\kappa\int
dk_{z}|v_{\kappa,k_{z}}|{\partial_{T_{0}}}f(\omega_{\kappa,k_{z}})=2J_{s}$.
This result cannot explain the non-reciprocity observed in non-local thermal
transport experiments in thin YIG films with Pt spin injection and detection
under a floating permalloy gate [13].

Conclusion.—We predict a non-local thermal spin pumping or spin Seebeck effect
that only appears in the presence of chiral damping in gated magnetic
nanostructures. Via tuning the direction of the magnetization in the film, a
large percentage (50$\%$ for perfect chirality) of the heat and spin currents
in the film can be diverted into the magnetic nanowires, a functionality that
may help manage heat in magnonic circuits. The spin injection may help explain
the observed magnetization reversal by magnon currents. Our results may be
relevant for other chirally coupled non-Hermitian systems, such as chiral
optics [4], phononics, and plasmonics [5].

###### Acknowledgements.

This work is financially supported by the National Key Research and
Development Program of China under Grant No. 2023YFA1406600, the National
Natural Science Foundation of China under Grant No. 12374109, the startup
grant of Huazhong University of Science and Technology, as well as JSPS
KAKENHI Grants No. 19H00645 and 22H04965. We thank Prof. Jing-Tao Lü for
valuable discussions.

## References

  * [1] N. A. Roberts and D. G. Walker, A review of thermal rectification observations and models in solid materials, Int. J. Thermal Sci. 50, 648 (2011).
  * [2] D. Segal, Single Mode Heat Rectifier: Controlling Energy Flow Between Electronic Conductors, Phys. Rev. Lett. 100, 105901 (2008).
  * [3] A. Ott, R. Messina, P. Ben-Abdallah, and S.-A. Biehs, Radiative thermal diode driven by nonreciprocal surface waves, Appl. Phys. Lett. 114, 163105 (2019).
  * [4] P. Lodahl, S. Mahmoodian, S. Stobbe, A. Rauschenbeutel, P. Schneeweiss, J. Volz, H. Pichler, and P. Zoller, Chiral quantum optics, Nature (London) 541, 473 (2017).
  * [5] T. Yu, Z. C. Luo, and G. E. W. Bauer, Chirality as Generalized Spin-Orbit Interaction in Spintronics, Phys. Rep. 1009, 1 (2023).
  * [6] S.-H. Yang, R. Naaman, Y. Paltiel, and S. S. P. Parkin, Chiral spintronics, Nat. Rev. Phys. 3, 328 (2021).
  * [7] R. W. Damon and J. R. Eshbach, Magnetostatic modes of a ferromagnet slab, J. Phys. Chem. Solids 19, 308 (1961).
  * [8] T. An, V. I. Vasyuchka, K. Uchida, A. V. Chumak, K. Yamaguchi, K. Harii, J. Ohe, M. B. Jungfleisch, Y. Kajiwara, H. Adachi, B. Hillebrands, S. Maekawa, and E. Saitoh, Unidirectional spin-wave heat conveyer, Nat. Mater. 12, 549 (2013).
  * [9] E. Shigematsu, Y. Ando, S. Dushenko, T. Shinjo, and M. Shiraishi, Spin-wave-induced lateral temperature gradient in a YIG thin film/GGG system excited in an ESR cavity, Appl. Phys. Lett. 112, 212401 (2018).
  * [10] P. Wang, L. F. Zhou, S. W. Jiang, Z. Z. Luan, D. J. Shu, H. F. Ding, and D. Wu, Unidirectional Spin-Wave-Propagation-Induced Seebeck Voltage in a PE-DOT:PSS/YIG Bilayer, Phys. Rev. Lett. 120, 047201(2018).
  * [11] R. Schlitz, S. Vélez, A. Kamra, C.-H. Lambert, M. Lammel, S. T. B. Goennenwein, and P. Gambardella, Control of Nonlocal Magnon Spin Transport via Magnon Drift Currents, Phys. Rev. Lett. 126, 257201 (2021).
  * [12] S. de-la-Peña, R. Schlitz, S. Vélez, J. C. Cuevas, and A. Kamra, Theory of drift-enabled control in nonlocal magnon transport, J. Phys.: Condens. Matter 34, 295801 (2022).
  * [13] J. H. Han, Y. B. Fan, B. C. McGoldrick, J. Finley, J. T. Hou, P. X. Zhang, and L. Q. Liu, Nonreciprocal Transmission of Incoherent Magnons with Asymmetric Diffusion Length, Nano Lett. 21, 7037 (2021).
  * [14] J. Gückelhorn, S. de-la-Peña, M. Grammer, M. Scheufele, M. Opel, S. Geprägs, J. C. Cuevas, R. Gross, H. Huebl, A. Kamra, and M. Althammer, Observation of nonreciprocal magnon Hanle effect, Phys. Rev. Lett. 130, 216703 (2023).
  * [15] L. J. Cornelissen, J. Liu, B. J. van Wees, and R. A. Duine, Spin-Current-Controlled Modulation of the Magnon Spin Conductance in a Three-Terminal Magnon Transistor, Phys. Rev. Lett. 120, 097702 (2018).
  * [16] T. Wimmer, M. Althammer, L. Liensberger, N. Vlietstra, S. Geprägs, M. Weiler, R. Gross, and H. Huebl, Spin Transport in a Magnetic Insulator with Zero Effective Damping, Phys. Rev. Lett. 123, 257201 (2019).
  * [17] J. Liu, X.-Y. Wei, G. E. W. Bauer, J. Ben Youssef, and B. J. van Wees, Electrically induced strong modulation of magnon transport in ultrathin magnetic insulator films, Phys. Rev. B 103, 214425 (2021).
  * [18] K. Baumgaertl and D. Grundler, Reversal of nanomagnets by propagating magnons in ferrimagnetic yttrium iron garnet enabling nonvolatile magnon memory, Nat. Commun. 14, 1490 (2023).
  * [19] Y. Au, E. Ahmad, O. Dmytriiev, M. Dvornik, T. Davison, and V. V. Kruglyak, Resonant microwave-to-spin-wave transducer, Appl. Phys. Lett. 100, 182404 (2012).
  * [20] T. Yu, C. P. Liu, H. M. Yu, Y. M. Blanter, and G. E. W. Bauer, Chiral excitation of spin waves by magnetic nanowire gratings, Phys. Rev. B 99, 134424 (2019).
  * [21] J. L. Chen, T. Yu, C. P. Liu, T. Liu, M. Madami, K. Shen, J. Y. Zhang, S. Tu, M. S. Alam, K. Xia, M. Z. Wu, G. Gubbiotti, Y. M. Blanter, G. E. W. Bauer, and H. M. Yu, Excitation of unidirectional exchange spin waves by a nanoscale magnetic grating, Phys. Rev. B 100, 104427 (2019).
  * [22] L. T. Sheng, M. Elyasi, J. L. Chen, W. Q. He, Y. Z. Wang, H. C. Wang, H. M. Feng, Y. Zhang, I. Medlej, S. Liu, W. J. Jiang, X. F. Han, D. P. Yu, J.-P. Ansermet, G. E. W. Bauer, and H. M. Yu, Nonlocal detection of interlayer three-magnon coupling, Phys. Rev. Lett. 130, 046701 (2023).
  * [23] T. Yu, Y. M. Blanter, and G. E. W. Bauer, Chiral pumping of spin waves, Phys. Rev. Lett. 123, 247202 (2019).
  * [24] X. Huang, X. Chen, Y. Li, J. Mangeri, H. Zhang, M. Ramesh, H. Taghinejad, P. Meisenheimer, L. Caretta, S. Susarla et al, Manipulating chiral-spin transport with ferroelectric polarization, arXiv:2306.02185.
  * [25] A. G. Gurevich, Radiotekh. Elektron. (Moscow) 8, 780 (1963).
  * [26] T. Yu, X. Zhang, S. Sharma, Y. M. Blanter, and G. E. W. Bauer, Chiral coupling of magnons in waveguides, Phys. Rev. B 101, 094414 (2020).
  * [27] L. H. Zhong, C. Zhang, and B. M. Yao, Controlling the dynamics of cavity magnon polariton via microwave polarization, AIP Advances 12, 085323 (2022).
  * [28] H. P. Zhan, L. H. Sun, and H. T. Tan, Chirality-induced one-way quantum steering between two waveguide-mediated ferrimagnetic microspheres, Phys. Rev. B 106, 104432 (2022).
  * [29] M. R. Xu, K. Yamamoto, J. Puebla, K. Baumgaertl, B. Rana, K. Miura, H. Takahashi, D. Grundler, S. Maekawa, and Y. Otani, Nonreciprocal surface acoustic wave propagation via magneto-rotation coupling, Sci. Adv. 6, abb1724 (2020).
  * [30] T. Yu, Nonreciprocal surface magnetoelastic dynamics, Phys. Rev. B 102, 134417 (2020).
  * [31] M. Küß, M. Heigl, L. Flacke, A. Hörner, M. Weiler, M. Albrecht, and A. Wixforth, Nonreciprocal Dzyaloshinskii–Moriya Magnetoacoustic Waves, Phys. Rev. Lett. 125, 217203 (2020).
  * [32] P. J. Shah, D. A. Bas, I. Lisenkov, A. Matyushov, N. Sun, and M. R. Page, Giant nonreciprocity of surface acoustic waves enabled by the magnetoelastic interaction, Sci. Adv. 6, eabc5648 (2020).
  * [33] H. Nyquist, Thermal Agitation of Electric Charge in Conductors, Phys. Rev. 23, 110 (1928).
  * [34] J. L. Chen, C. P. Liu, T. Liu, Y. Xiao, K. Xia, G. E. W. Bauer, M. Z. Wu, and H. M. Yu, Strong Interlayer Magnon-Magnon Coupling in Magnetic Metal-Insulator Hybrid Nanostructures, Phys. Rev. Lett. 120, 217202 (2018).
  * [35] M. Ishibashi, Y. Shiota, T. Li, S. Funada, T. Moriyama, and T. Ono, Switchable giant nonreciprocal frequency shift of propagating spin waves in synthetic antiferromagnets, Sci. Adv. 6, eaaz6931 (2020).
  * [36] Y. Shiota, T. Taniguchi, M. Ishibashi, T. Moriyama, and T. Ono, Tunable Magnon-Magnon Coupling Mediated by Dynamic Dipolar Interaction in Synthetic Antiferromagnets, Phys. Rev. Lett. 125, 017203 (2020).
  * [37] R. A. Gallardo, T. Schneider, A. K. Chaurasiya, A. Oelschlägel, S. S. P. K. Arekapudi, A. Roldán-Molina, R. Hübner, K. Lenz, A. Barman, J. Fassbender, J. Lindner, O. Hellwig, and P. Landeros, Reconfigurable Spin-Wave Nonreciprocity Induced by Dipolar Interaction in a Coupled Ferromagnetic Bilayer, Phys. Rev. Appl. 12, 034012 (2019).
  * [38] M. Grassi, M. Geilen, D. Louis, M. Mohseni, T. Brächer, M. Hehn, D. Stoeffler, M. Bailleul, P. Pirro, and Y. Henry, Slow-Wave-Based Nanomagnonic Diode, Phys. Rev. Appl. 14, 024047 (2020).
  * [39] S. Datta, Electronic Transport in Mesoscopic Systems (Cambridge University Press, New York, 1995).
  * [40] Y. Imry and R. Landauer, Conductance viewed as transmission, Rev. Mod. Phys. 71, S306 (1999).
  * [41] Wang et al, Nonreciprocal coherent coupling of nanomagnets by exchange spin waves, Nano Research 14, 2133 (2021).
  * [42] X.-Y. Wei, O. Alves Santos, C. H. Sumba Lusero, G. E. W. Bauer, J. Ben Youssef, and B. J. van Wees, Giant magnon spin conductivity in ultrathin yttrium iron garnet films, Nat. Mat. 21, 1352 (2022).
  * [43] Y. Xu, Y. Li, R. K. Lee, and A. Yariv, Scattering-theory analysis of waveguide-resonator coupling, Phys. Rev. E 62, 7389 (2000).
  * [44] S. Fan, P. R. Villeneuve, and J. D. Jaonnopoulos, Theoretical analysis of channel drop tunneling processes, Phys. Rev. B 59, 15882 (1999).
  * [45] G. D. Mahan, Many Particle Physics (Plenum, New York, 1990).
  * [46] A. L. Fetter and J. D. Walecka, Quantum Theory of Many Particle Systems (McGraw-Hill, New York, 1971).
  * [47] S. O. Demokritov, B. Hillebrands, and A. N. Slavin, Brillouin light scattering studies of confined spin waves: linear and nonlinear confinement, Phys. Rep. 348, 441 (2001).
  * [48] T. van der Sar, F. Casola, R. L. Walsworth, and A. Yacoby, Nanometre-scale probing of spin waves using single electron spins, Nat. Commun. 6, 7886 (2015).
  * [49] I. Bertelli, B. G. Simon, T. Yu, J. Aarts, G. E. W. Bauer,Y. M. Blanter, and T. van der Sar, Imaging Spin-Wave Damping Underneath Metals Using Electron Spins in Diamond, Adv. Quantum Technol. 4, 2100094 (2021).
  * [50] H. Haug and A. P. Jauho, Quantum Kinetics in Transport and Optics of Semiconductors (Springer, Berlin, 1996).
  * [51] A. A. Abrikosov, L. P. Gorkov, and I. E. Dzyaloshinski, Methods of Quantum Field Theory in Statistical Physics (Prentice Hall, Englewood Cliffs, N. J., 1963).