# TinyM2Net: A Flexible System Algorithm Co-designed Multimodal Learning
Framework for Tiny Devices

Hasib-Al Rashid<EMAIL_ADDRESS>University of Maryland, Baltimore
CountyBaltimoreMarylandUSA21250 ,  Pretom Roy Ovi<EMAIL_ADDRESS>University
of Maryland, Baltimore County1000 Hilltop CircleBaltimoreMarylandUSA21250 ,
Carl Busart  U.S. Army Research LaboratoryAdelphiMarylandUSA ,  Aryya
Gangopadhyay  University of Maryland, Baltimore County1000 Hilltop
CircleBaltimoreMarylandUSA21250 and  Tinoosh Mohsenin  University of Maryland,
Baltimore County1000 Hilltop CircleBaltimoreMarylandUSA21250

(2022)

###### Abstract.

With the emergence of Artificial Intelligence (AI), new attention has been
given to implement AI algorithms on resource constrained tiny devices to
expand the application domain of IoT. Multimodal Learning has recently become
very popular with the classification task due to its impressive performance
for both image and audio event classification. This paper presents _TinyM
2Net_ \- a flexible system algorithm co-designed multimodal learning framework
for resource constrained tiny devices. The framework was designed to be
evaluated on two different case-studies: COVID-19 detection from multimodal
audio recordings and battle field object detection from multimodal images and
audios. In order to compress the model to implement on tiny devices,
substantial network architecture optimization and mixed precision quantization
were performed (mixed 8-bit and 4-bit). _TinyM 2Net_ shows that even a tiny
multimodal learning model can improve the classification performance than that
of any unimodal frameworks. The most compressed _TinyM 2Net_ achieves 88.4%
COVID-19 detection accuracy (14.5% improvement from unimodal base model) and
96.8% battle field object detection accuracy (3.9% improvement from unimodal
base model). Finally, we test our _TinyM 2Net_ models on a Raspberry Pi 4 to
see how they perform when deployed to a resource constrained tiny device.

Multimodal Learning, TinyML, Micro AI, COVID-19 Detection, Battle Field Object
Detection.

††copyright: rightsretained††journalyear: 2022

##  1\. Introduction

Artificial Intelligence (AI) has a huge impact on our daily lives now-a-days.
In our daily lives, AI has brought convenience and ease of use to the table.
The AI devices are now able to perform computationally intensive tasks and
eliminate human error from the system to a large extent, making this
convenience possible. We see AI techniques and devices being used in domains
such as medical diagnosis, security and combat fields, robotics, vision
analytics, knowledge reasoning, navigation, etc. today. To integrate AI in our
day-to-day life, it is being implemented on resource constrained mobile and
edge platforms. With the exponential growth of resource constrained micro-
controller (MCU) and micro-processor (MPU) powered devices, a new generation
of neural networks has emerged, one that is smaller in size and more concerned
with model efficiency than model accuracy. These low-cost, low-energy MCUs and
MPUs open up a whole new world of tiny machine learning (TinyML)
possibilities. We can directly do data analytics near the sensor by running
deep learning models on very tiny devices, greatly expanding the field of AI
applications.

Modern IoT and wearable devices, such as activity trackers, environmental
sensors, images, and audio sensors can generate large volumes of data on a
regular basis. Modern AI is increasingly reliant on data from numerous sources
in order to produce more accurate findings. Learning process for human is
multimodal. We human can take our decision by processing different modalities
of data. To mimic human-like behavior, AI algorithms should integrate
multimodal data as well. Multimodal learning combines disparate, heterogeneous
data from a variety of sensors and data sources into a single model. In
contrast to standard unimodal learning systems, multimodal systems can convey
complimentary information about one another, which becomes apparent only when
both are integrated into the learning process. Thus, learning-based systems
that incorporate data from many modalities can generate more robust inference
or even novel insights, which would be unachievable in a unimodal system.
Multimodal learning has two key advantages. To begin, several sensors
observing the same data can produce more robust predictions, as recognizing
changes in it may need the presence of both modalities. Second, the
integration of many sensors enables the capture of complementing data or
trends that individual modalities may miss. However, increased model
parameters and computations limit multimodal learning to be adopted for
resource constrainded edge and tiny ML applications.

Commodity MCUs and MPUs have a very limited resource in terms of memory (SRAM)
and storage (Flash) budget. A typical MCU has an SRAM of less than 512kB,
which is insufficient for installing the majority of off-the-shelf deep
learning networks. Even on more capable hardware such as the Raspberry Pi 4,
configuring inference to run in the L2 cache (1MB) can dramatically increase
energy efficiency. These new issues add to the difficulty of performing
efficient multimodal learning inference with a low peak memory consumption. In
this paper, we address this challenge and implement multimodal learning on
tiny hardware. We take advantages of state-of-the-art compression techniques
and combined them with computationally relaxed layers to implement energy
efficient multimodal learning on tiny processing hardware. We proposed a
flexible system algorithm co-designed framework _TinyM 2Net_ which is re-
configurable in terms of input data modality and data shapes, number of
layers, filter sizes etc. hyper-parameters for the sake of application
requirements. We evaluated _TinyM 2Net_ with two different case-studies: audio
processing with multimodal audios and object detection with multimodal images
and audios. _TinyM 2Net_ is then implemented on commodity tiny MPU, Raspberry
Pi 4 to measure real-time performance on tiny hardware. The main contributions
of this paper are as follows:

  * •

Propose _TinyM 2Net_, a novel flexible system-algorithm co-designed multimodal
learning framework for resource constrained devices. _TinyM 2Net_ that can
take multimodal inputs (images and audios) and be re-configured for
application specific requirements. _TinyM 2Net_ allows the system and
algorithms to quickly integrate new sensors data that are customized to
various types of scenarios.

  * •

Perform network architecture optimization, and mixed-precision quantization
with the purpose of decreasing computation complexity and memory size for
resource constrained hardware implementation while maintaining accuracy.

  * •

Evaluated proposed _TinyM 2Net_ for two different case-studies. _Case-study 1_
includes Covid-19 detection from multimodal cough and speech audio recordings.
_Case-study 2_ includes battlefield object detection using multimodal images
and audios.

  * •

Implement _TinyM 2Net_ on commodity microprocessor unit, Raspberry Pi 4. We
measured inference time while it was in use, as well as providing the
appropriate power profiling to ensure that our system is adaptable. To be
called a real-time implementable tinyml system, _TinyM 2Net_ meets all of the
requirements.

##  2\. Related Works

Authors in (mazumder2021survey, ) presented a high level overview on the
optimization techniques of deep neural networks (DNNs) for tinyML on device
inferences. TinyML model optimization includes different algorithms of
Parameter Search, Sparsification and Quantization techniques. Element-wise
pruning(meng2021fixyfpga, ), Structured Pruning (anwar2017structured, ;
lemaire2019structured, ) these techniques reduces the unimportant weights and
compress the models to be implemented on tinyML devices. Extreme low precision
quantization (liang2018fp, ; alemdar2017ternary, ) and mixed precision
quantization (yao2021hawq, ; hubara2020improving, ; wang2019haq, ;
wu2018mixed, ; hosseini2021qs, ) is adopted by the researchers to decrease the
memory requirements of the DNN models. MCUNet (lin2020mcunet, ), MicroNets
(banbury2021micronets, ) are proposed to deploy DNN models on micro-controller
units (MCU). Recently, multimodal learning attracts researchers to improve the
classification accuracy of the models integrating different modalities of data
fusion (bouchey2021multimodal, ; abdullah2021multimodal, ; ma2021smil, ;
9556252, ). However, implementation of multimodal learning into resource
constrained tiny hardware is very limited due to its large model sizes. We
present a novel multimodal learning framework _TinyM 2Net_ which is system
algorithm co-designed and different model compression techniques were
implemented to compress the large multimodal models to be implemented on tiny
devices.

##  3\. TinyM2Net Framework

Figure 1 shows the proposed _TinyM 2Net_ framework along with its detailed
architecture. Based on the case studies we mention in section 5, _TinyM 2Net_
is able to integrate two different modalities of data and classify them.
Proposed _TinyM 2Net_ is designed based on mainly convolutional neural
networks (CNN). CNN performed very promising with images and audio data
classification previously which is the reason behind choosing CNN as our base
model.

We will evaluate _TinyM 2Net_ on two different case studies described in
details in section 5. Case-study 1 is detecting COVID-19 signatures using two
different modalities of audio recordings, cough sound and speech sound. Case-
study 2 is based on detecting battle field object using images and audios.
While processing audio data, we have divided the whole audio into shorted
window frames, the size of those frames are variable based on application
requirements. Then the window frames are converted into Mel-Frequency Cepstral
Coefficients (MFCCs) spectrograms. In the next step, two different data
modalities, whether be images or audios, are sent to the CNN layers for
feature extraction. The number of layers of our CNN layers can be adjusted to
suit application specific requirements. Maxpooling layers is used to reduce
the size of the feature map. A number of fully interconnected layers are
applied once the output is flattened to achieve the needed tiny feature map
size in order to isolate enough information with linkages between nodes. The
outputs from two parallel feature extraction layers from two data modalities
are then concatenated and processed through fully connected layers to produce
the final label. The activation function for each layer is a rectified linear
unit (ReLU). Softmax activation function is used to generate a probability
distribution for the final layer.

Figure 1. The proposed _TinyM 2Net_ framework for multimodal learning on tiny
hardwares. _TinyM 2Net_ is flexible in terms of number of input, number of
layers and hyper-parameters based on application specific requirement. New
data modality suited for various settings can be readily included into the
device. Some of the input information is images, other input data can be
auditory.

##  4\. Model Compression for Tiny Devices

Traditional CNN models are very notorious for being bulky in terms of memory
and computation requirements. To implement CNN on low powered embedded tiny
devices, researchers proposed various compression techniques which results in
highly optimized CNN models. Our proposed _TinyM 2Net_ adopts different model
compression techniques which optimizes the network architectures and memory
requirements. _TinyM 2Net_ adopts Depthwise Separable CNN (DS-CNN) to reduce
the computation from traditional CNN layers. To have improvement on memory
requirements, _TinyM 2Net_ adopts low precision and mixed-precision (MP)
quantization. We emphasize on MP quantization as uniform low precision
quantization degrades model accuracy.

###  4.1. Network Architecture Optimization with DS-CNN

Figure 3 presents the conventions and the techniques that is done in
traditional CNN and DS-CNN . In traditional CNN, if the input is of size
${D_{f}\times D_{f}\times M}$ and $N$ is the number of filters having a size
${D_{k}\times D_{k}\times M}$ then output of this layer without zero padding
applied is of size $D_{p}\times D_{p}\times M$. If the stride for the
convolution is $S$ then $D_{p}$ is determined by the following equation:

(1) $D_{p}=\frac{D_{f}-D_{k}}{S}+1$

In this layer, the filter convolves over the input by performing element wise
multiplication and summing all the values. A very important note is that depth
of the filter is always same as depth of the input given to this layer. The
computational cost for traditional convolution layer is $M\times
D_{k}^{2}\times D_{p}^{2}\times N$ (howard2017mobilenets, ).

Figure 2. Detailed operations inside traditional CNN and DS-CNN. Table 1.
Number of parameter and required computations equations for different types of
convolution layers.

CNN types | No. of Parameters | No. of Computations
---|---|---
Traditional | $M\times D_{k}^{2}\times N$ | $M\times D_{k}^{2}\times D_{p}^{2}\times N$
Depthwise Separable | $M\times D_{k}^{2}+M\times N$ | $M\times D_{p}^{2}\times D_{k}^{2}+M\times D_{p}^{2}\times N$

Depthwise Separable convolution is a combination of depthwise and pointwise
convolution (kaiser2017depthwise, ). In contrast to traditional CNNs, which
apply convolution to all $M$ channels at once, depthwise operations only apply
convolution to a single channel at a time. So here the filters/kernels will be
of size $D_{k}\times D_{k}\times 1$. As there are $M$ channels at the input,
$M$ numbers of such filters are needed. This will produce a output of size
$D_{p}\times D_{p}\times M$. A single convolution operation require
$D_{k}\times D_{k}$ multiplications. Since the filter are slided by
$D_{p}\times D_{p}$ times across all the $M$ channels. The total number of
computation for one depthwise convolution comes to be $M\times D_{p}^{2}\times
D_{k}^{2}$. In point-wise operation, a $1\times 1$ convolution is applied on
the $M$ channels. The filter shape for this operation will be $1\times 1\times
M$. If we use $N$ such filters, the output shape becomes $D_{p}\times
D_{p}\times N$. One convolution operation in this needs $1\times M$
multiplications. The total number of operations for one pointwise convolution
is $M\times D_{p}^{2}\times N$. Therefore, total computational cost of one
depthwise separable convolution is $M\times D_{p}^{2}\times D_{k}^{2}+M\times
D_{p}^{2}\times N$ (howard2017mobilenets, ). Table 1 shows the equations
required for calculating the number of parameters and number of computations
for each of the traditional CNN and DS-CNN layers. Here $D_{k}\times D_{k}$ is
the size of the filter, $D_{p}\times D_{p}$ is the size of the output, $M$ is
number the of input channels and $N$ is the number of output channels.

###  4.2. Model Quantization

To have lesser memory requirements, model quantization is now attracting to
the researchers to design tinyML models. The accuracy of a model can be
significantly degraded if it is uniformly quantized to low bit precision. It
is possible to address this with mixed-precision quantization in which each
layer is quantized with different bit precision. The main idea behind mixed
precision quantization is to keep sensitive layers at higher precision and
insensitive layers at lower precision. However, the search space for this bit
setting grows exponentially as the number of layers increase, which is
challenging. Various methods have been offered to deal with this enormous
search area. Reinforcement learning (RL) and Neural Architecture Search (NAS)
have recently been presented as efficient approaches for searching the search
space. As a result, these methods (wang2019haq, ; wu2018mixed, ;
hosseini2021qs, ) often require a huge amount of computational resources and
their performance is highly dependent on hyperparameters and even
initialization. An algorithm known as Integer Linear Programming (ILP) is
employed in (yao2021hawq, ; hubara2020improving, ). ILP is very light weight
and gives result within second. We adopted ILP and formulated our problem
following the methodology described in (yao2021hawq, ), simplifying some
constraints to get the mixed precision settings for our _TinyM 2Net_. ILP
equations were solved using a python module called Pulp.

To tackle the accuracy degradation with extreme low bit precision
quantization, we chose two different bit precision settings ($X=2$), INT4 and
INT8 for our _TinyM 2Net_ framework. As our _TinyM 2Net_ is flexible in terms
of number of layers, for a model with $Y$ layers, the search space for ILP
becomes $X^{Y}$. ILP will find the best bit precision choices from $X^{Y}$
search spaces to have optimal trade-off between model perturbation $\Omega$
and user specific constraints i.e. model size and Bit Operations, BOPS. Each
of these bit-precision options has the potential to produce different model
perturbation. We assumed the perturbation for each layer are independent to
each other (yao2021hawq, ) (i.e., $\Omega$ =
$\sum_{i=1}^{Y}\Omega_{i}^{x_{i}}$, where $\Omega_{i}^{x_{i}}$ is the
perturbation of i-th layer with $x_{i}$ bit). This enables us to pre-calculate
the sensitivity of each layer independently, with only $XY$ computations
required. Hessian based perturbation, presented in (yao2021hawq, ) is used as
sensitivity metric. Minimizing this sensitivity, ILP tries to find the right
bit precision settings. The ILP equations would be as follows: Objective:

(2) $\small\min{\left\\{x_{i}\right\\}_{i=1}^{Y}}\hskip
2.84526pt\sum_{i=1}^{Y}\Omega_{i}^{\left(x_{i}\right)},$

Subject to:

(3) $\small\sum_{i=1}^{Y}\mu_{i}^{\left(x_{i}\right)}\leq Model\hskip
2.84526ptSize,$

(4) $\small\sum_{i=1}^{Y}\beta_{i}^{\left(x_{i}\right)}\leq BOPS\hskip
2.84526ptLimit,$

Here, $\mu_{i}^{\left(x_{i}\right)}$ denotes the size of the $i$-th layer with
$x_{i}$ bit quantization and $\beta_{i}^{\left(x_{i}\right)}$ is the
corresponding BOPS required for computing that layer. All the equations are
adopted from (yao2021hawq, ). We have considered the bit precision for both
weights and activations to be same so that the mathematical operations become
efficient. The overall MP quantization process is summarized in figure
[somethin].

Figure 3. Finding Mixed Precision Bit Setting Using Integer Linear Programming
(ILP)

##  5\. TinyM2Net Evaluation

We evaluated proposed _TinyM 2Net_ with two very important real-world case
studies: COVID-19 detection from multimodal audios and Battlefield Object
Detection from multimodal images and audios. Both of the case studies proves
to be important sectors where tinyML implementations is much required.

###  5.1. Case Study 1: Covid Detection from Multimodal Audio Recordings

Combining numerous data sources has always been a high priority topic, but
with the advent of new AI-based learning algorithms, it has become critical to
combine the complementary capabilities of distinct data sources for effective
diagnosis, treatment, prognosis, and planning in a variety of medical
applications. With the onset of COVID-19 pandemic, patient pre-screening from
passively recorded audios has become an active area of research. Therefore, a
bunch of unimodal and multimodal COVID-19 audio dataset have been presented
(bagad2020cough, ; brown2020exploring, ; orlandic2021coughvid, ;
sharma2020coswara, ). The ultimate goal in this research is to provide
COVID-19 pre-screening mobile or tiny devices. Figure 4 shows the highlevel
overview of the evaluation of _TinyM 2Net_ in terms of COVID-19 detection.

Figure 4. The proposed _TinyM 2Net_ framework to detect COVID-19 from two
different modalities of audios, cough audios and speech audios.

####  5.1.1. Dataset Description

We have used the dataset from the COVID-19 cough sub-challenge (CCS) and the
COVID-19 speech sub-challenge (CSS) from the Inter Speech 2021 ComParE
challenge (schuller2021interspeech, ). This dataset is a subset of the bigger
dataset (brown2020exploring, ) collected by University of Cambridge. In this
dataset there are 929 cough audios from 397 participants and 893 speech
recordings from 366 participents. Each recording included a COVID-19 test
result that was self-reported by the participant. To build the two-class
classification task, the original COVID-19 test results were mapped to
positive (designated as ‘P’) or negative (designated as ‘N’) categories.

####  5.1.2. Experimental Setups, Results and Analysis

To create a balanced multimodal dataset we have taken 893 cough recordings
from 366 participants matching their IDs mentioned in the metadata so that we
have both cough and speech recordings contributing from a same person. Then we
have divided the audios into 2 sec audio chunks and produces 6000 random
samples out of that. Then we converted them into MFCC spectrogram and passed
them to _TinyM 2Net_. _TinyM 2Net_ process two different modalities of audios
with its parallel CNN layers, extracts features, combines them and classify at
the end as binary classification. We have used the 1st layer as traditional
CNN and the later layers as DS-CNN. The detailed network architecture is
mentioned in table 2.

Table 2. Details of the network architecture for multimodal COVID-19 detection

Layers | Description | Output
---|---|---
Input Layer | Cough Audio MFCC Vector | 203$\times$20 $\times$ 1
Input Layer | Speech Audio MFCC Vector | 333$\times$13$\times$1
Conv2D | Kernels = 16 $\times$(3$\times$3) - BN - ReLU | 201$\times$18$\times$16
Conv2D | Kernels = 64 $\times$(3$\times$3) - BN- ReLU | 333$\times$13$\times$64
Separable Conv2D | Kernels = 32 $\times$(3$\times$3) - ReLU | 199x16x32
Separable Conv2D | Kernels = 32 $\times$(3$\times$3) - ReLU | 329$\times$9$\times$32
MaxPooling2D | Pool size = (3$\times$3), 20% Dropout | 66$\times$5$\times$32
MaxPooling2D | Pool size = (2$\times$2), 20% Dropout | 164$\times$4$\times$32
Separable Conv2D | Kernels = 32$\times$(3$\times$3) - ReLU | 64$\times$3$\times$32
Separable Conv2D | Kernels = 16$\times$(3$\times$3) - ReLU | 162 $\times$ 2 $\times$16
MaxPooling2D | Pool size = ($\times$3), 20% Dropout | 21$\times$1$\times$32
MaxPooling2D | Pool size = (2$\times$2), 20% Dropout | 81$\times$1$\times$16
Flatten | 21$\times$1$\times$32 | 672
Flatten | 81$\times$1$\times$16 | 1296
Dense | Neurons = 32 - ReLU - 20% Dropout | 32
Dense | Neurons = 32 - ReLU - 20% Dropout | 32
Concatenate | 32 + 32 | 64
Dense | Neurons = 256 - ReLU - 20% Dropout | 256
Dense | Neurons = 128 - ReLU - 20% Dropout | 128
Dense | Neurons = 2 - Softmax | 2

We trained our model with categorical cross-entropy loss and Adam optimizer.
We achieved 90.4% classification accuracy with FP32 bit precision. We then
quantize our model to uniform 8-bit and 4-bit precision and achieved 89.6% and
83.6% classification accuracy. Our MP quantization technique improves the
classification accuracy to 88.4%. The best baseline accuracies for unimodal
COVID-19 detection from cough audio and speech audio were 73.9% and 72.1%
(Schuller21-TI2). Our _TinyM 2Net_ outperforms the baseline results by 14.5%.

###  5.2. Case Study 2: Battlefield Component Detection from Multimodal Images
and Audios

Research in the field of computer vision has always focused on the detection
of specific targets in an image. Research on the identification of armored
vehicles in the battlefield environment, as well as the deployment dynamics,
recognition and tracking, precise strike, and so on, are critical military
objectives. There are still many difficulties in detecting armored vehicles on
the battlefield because of the complication of the environment
(xiaozhu2017object, ; zheng2019object, ). Authors in (mees2016choosing, ;
sindagi2019mvx, ) proposed multimodal learning approach in object detection
task. We have presented a novel multimodal learning approach in battlefield
object detection based on image and audio modality. Figure 5 shows the
highlevel overview of the evaluation of _TinyM 2Net_ in terms of battlefield
object detection.

Figure 5. The proposed _TinyM 2Net_ framework to detect four different
battlefield objects, Helicopter, Tank, Bomb, Gun from two different
modalities, images and audios.

####  5.2.1. Dataset Description

As open-sourced dataset for research on battlefield environment is very
limited, we have created our own dataset for this case-study. This also
contributed to the novelty of our work. We have created a dataset for
multiclass classification problem with 4 classes as Helicopter, Bomb, Gun, and
Tank. We have selected 4 publicly available YouTube videos (gshawnadams_2020,
; xsmasher4ya_2019, ; youtube_2017, ; youtube_2020, ) from where we extracted
the images and corresponding audios of Helicopter, Bomb, Gun, and Tank.
Sampling rate of the image extraction was one frame per second. We have
collected the images in .jpg format. We extracted in total 2745 images for all
the 4 classes. On the contrary, sampling frequency was 22050Hz for 1 sec
audio. Length of our audio was 1 sec. We then converted the audio signal into
MFCCs. We extracted in total 2745 MFCC spectrogram for 4 classes. The total
number of extracted images and corresponding audios were as follows:
Helicopter-1066, Gun - 1008, Bomb - 161, Tank- 510.

####  5.2.2. Experimental Setups, Results and Analysis

We passed both the input and corresponding audio MFCCs to _TinyM 2Net_. _TinyM
2Net_ process two different modalities with its parallel CNN layers, extracts
features, combines them and classify at the end as multiclass classification.
We have used the 1st layer as traditional CNN and the later layers as DS-CNN.
The detailed network architecture is mentioned in table 3.

We trained our model with categorical cross-entropy loss and Adam optimizer.
We achieved 98.5% classification accuracy with FP32 bit precision. We then
quantize our model to uniform 8-bit and 4-bit precision and achieved 97.9% and
88.7% classification accuracy. Our MP quantization technique improves the
classification accuracy to 97.5% which is very comparable to both 8-bit and
32-bit quantized models. We achieved 93.6% classification accuracy with
unimodal (only image data) implementation. Our multimodal approach improved
the object detection accuracy to 3.9%.

Table 3. Details of the network architecture for multimodal battle field
object detection

Layers | Description | Output
---|---|---
Input Layer | Audio MFCC Vector | 44$\times$13$\times$1
Input Layer | Image Vector | 32$\times$32$\times$3
Conv2D | Kernels = 64$\times$(3 $\times$3) - BN - ReLU | 44$\times$13$\times$64
Conv2D | Kernels = 64 $\times$(3$\times$3)- BN - ReLU | 32$\times$32$\times$64
Separable Conv2D | Kernels = 32 $\times$(3$\times$3) - ReLU | 44$\times$13$\times$32
Separable Conv2D | Kernels = 32 $\times$(3$\times$3) - ReLU | 32$\times$32$\times$64
MaxPooling2D | Pool size = (2$\times$2), 20% Dropout | 22$\times$6$\times$32
MaxPooling2D | Pool size = (2$\times$2), 20% Dropout | 16$\times$16$\times$32
Separable Conv2D | Kernels = 64 $\times$(3$\times$3) - ReLU | 22 $\times$ 6 $\times$64
Separable Conv2D | Kernels = 64 $\times$(3$\times$3) - ReLU | 16 $\times$ 16 $\times$64
MaxPooling2D | Pool size = (2$\times$2), 20% Dropout | 11$\times$3$\times$64
MaxPooling2D | Pool size = (2$\times$2), 20% Dropout | 8$\times$8$\times$64
Flatten | 11$\times$3$\times$64 | 2112
Flatten | 8$\times$8$\times$64 | 4096
Dense | Neurons = 64 - ReLU - 20% Dropout | 64
Dense | Neurons = 64 - ReLU - 20% Dropout | 64
Concatenate | 64 + 64 | 128
Dense | Neurons = 64 - ReLU - 20% Dropout | 64
Dense | Neurons = 4 - Softmax | 4

Table 4. Summary of the _TinyM 2Net_ Framework Evaluation Results

| Case
---
Studies
Quantization |  | Accuracy
---
(%)
| Model Size
---
(KB)
1 | Floating Points | 90.4 | 845
1 | W8A8 (uniform 8) | 89.6 | 216
1 | W4/8A4/8 (MP) | 88.4 | 145
1 | W4A4 (uniform 4) | 83.6 | 107
2 | Floating Points | 98.5 | 1605
2 | W8A8 (uniform 8) | 97.9 | 407
2 | W4/8A4/8 (MP) | 96.8 | 269
2 | W4A4 (uniform 4) | 91.3 | 205

##  6\. TinyM2Net Running on Tiny Devices

The inference stage must be implemented on resource constrained tiny devices
in order to make the _TinyM 2Net_ system real-time. We implemented _TinyM
2Net_ on Rasberry Pi 4 which has quad-core Cortex-A72 (ARM v8) and 2GB LPDDR4
memory. Performance evaluation of the _TinyM 2Net_ on resource constrained
Raspberry Pi 4 was based on two metrics: inference time and power consumption
during inference. The capacity of a framework to run in real time is
determined by its inference time or running time. Data loading, model loading,
and visual display of the final result all contribute to this inference time’s
length. The inference time was measured with the help of Raspbian OS’s ‘time’
function. We used a batch size of 1, which is the time it takes to process a
single data point, to calculate the inference time. We also need to consider
the model’s power profile when deploying it in the actual world. The running
power of any deep model should be well within the device’s sustainable range.
Power consumption is calculated by deducting idle power from the peak power
indicated during inference operation and reporting the result. For reporting,
we use the metric unit milliwatt (mW), and a USB power meter has been
employed. The inference time and required power during inference with the most
compressed models are mentioned in table 5 for both of the case studies.

Table 5. Implementation of the TinyM2Net framework to resource constrained
Raspberry Pi 4 device

Case Studies |  | Inference Time
---
(S)
| Power
---
(mW)
1 | 1.2 | 798
2 | 1.7 | 959

##  7\. Conclusion

This paper presents _TinyM 2Net_, a flexible system algorithm co-designed
multimodal learning framework which employs as much as correlated information
a multimodal dataset provides in an attempt to exploit deep learning
algorithms evaluating two important tinyML evaluation case-studies: to detect
the signature of COVID-19 into participents’ cough and speech sound and battle
field object detection from multimodal images and audios. To implement into
tiny hardware, extensive model compression was done in terms of networks
architecture optimization and MP quantization (mixed 8-bit and 4-bit). The
most compressed _TinyM 2Net_ achieves 88.4% COVID-19 detection accuracy and
96.8% battle field object detection accuracy. Finally, we test our _TinyM
2Net_ model on a Raspberry Pi 4 to see how they perform when deployed to a
resource constrained tiny device.

##  8\. ACKNOWLEDGMENT

We acknowledge the support of the U.S. Army Grant No. W911NF21-20076.

## References

  * [1] Sharmeen M Saleem Abdullah Abdullah, Siddeeq Y Ameen Ameen, Mohammed AM Sadeeq, and Subhi Zeebaree.  Multimodal emotion recognition using deep learning.  Journal of Applied Science and Technology Trends, 2(02):52–58, 2021\.
  * [2] Shawn Adams.  Best helicopter sounds. top sounds that helicopters make, url: https://www.youtube.com/watch?v=e8backzbqzc.
  * [3] Hande Alemdar, Vincent Leroy, Adrien Prost-Boucle, and Frédéric Pétrot.  Ternary neural networks for resource-efficient ai applications.  In 2017 international joint conference on neural networks (IJCNN), pages 2547–2554. IEEE, 2017.
  * [4] Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung.  Structured pruning of deep convolutional neural networks.  ACM Journal on Emerging Technologies in Computing Systems (JETC), 13(3):1–18, 2017.
  * [5] Piyush Bagad et al.  Cough against covid: Evidence of covid-19 signature in cough sounds.  arXiv preprint arXiv:2009.08790, 2020.
  * [6] Colby Banbury, Chuteng Zhou, Igor Fedorov, Ramon Matas, Urmish Thakker, Dibakar Gope, Vijay Janapa Reddi, Matthew Mattina, and Paul Whatmough.  Micronets: Neural network architectures for deploying tinyml applications on commodity microcontrollers.  Proceedings of Machine Learning and Systems, 3, 2021.
  * [7] Bettyjo Bouchey, Jill Castek, and John Thygeson.  Multimodal learning.  Innovative Learning Environments in STEM Higher Education: Opportunities, Challenges, and Looking Forward, pages 35–54, 2021.
  * [8] Chloë Brown et al.  Exploring automatic diagnosis of covid-19 from crowdsourced respiratory sound data.  arXiv preprint arXiv:2006.05919, 2020.
  * [9] Morteza Hosseini and Tinoosh Mohsenin.  Qs-nas: Optimally quantized scaled architecture search to enable efficient on-device micro-ai.  IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 11(4):597–610, 2021.
  * [10] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  arXiv preprint arXiv:1704.04861, 2017.
  * [11] Itay Hubara, Yury Nahshan, Yair Hanani, Ron Banner, and Daniel Soudry.  Improving post training neural quantization: Layer-wise calibration and integer programming.  arXiv preprint arXiv:2006.10518, 2020.
  * [12] Lukasz Kaiser, Aidan N Gomez, and Francois Chollet.  Depthwise separable convolutions for neural machine translation.  arXiv preprint arXiv:1706.03059, 2017.
  * [13] Carl Lemaire, Andrew Achkar, and Pierre-Marc Jodoin.  Structured pruning of neural networks with budget-aware regularization.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9108–9116, 2019.
  * [14] Shuang Liang, Shouyi Yin, Leibo Liu, Wayne Luk, and Shaojun Wei.  Fp-bnn: Binarized neural network on fpga.  Neurocomputing, 275:1072–1086, 2018.
  * [15] Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, and Song Han.  Mcunet: Tiny deep learning on iot devices.  arXiv preprint arXiv:2007.10319, 2020.
  * [16] Mengmeng Ma, Jian Ren, Long Zhao, Sergey Tulyakov, Cathy Wu, and Xi Peng.  Smil: Multimodal learning with severely missing modality.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2302–2310, 2021.
  * [17] Arnab Neelim Mazumder, Jian Meng, Hasib-Al Rashid, Utteja Kallakuri, Xin Zhang, Jae-sun Seo, and Tinoosh Mohsenin.  A survey on the optimization of neural network accelerators for micro-ai on-device inference.  IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2021.
  * [18] Oier Mees, Andreas Eitel, and Wolfram Burgard.  Choosing smartly: Adaptive multimodal fusion for object detection in changing environments.  In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 151–156. IEEE, 2016.
  * [19] Jian Meng, Shreyas Kolala Venkataramanaiah, Chuteng Zhou, Patrick Hansen, Paul Whatmough, and Jae-sun Seo.  Fixyfpga: Efficient fpga accelerator for deep neural networks with high element-wise sparsity and without external memory access.  In 2021 31st International Conference on Field-Programmable Logic and Applications (FPL), pages 9–16. IEEE, 2021.
  * [20] Lara Orlandic et al.  The coughvid crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms.  Scientific Data, 8(1):1–10, 2021.
  * [21] Pretom Roy Ovi et al.  Aris: A real time edge computed accident risk inference system.  In 2021 IEEE International Conference on Smart Computing (SMARTCOMP), pages 47–54, 2021.
  * [22] Sintonizar Productions.  Real explosion sound effects asmr, url:https://www.youtube.com/watch?v=lhyhy5uioji&t=64s.
  * [23] Björn W Schuller, Anton Batliner, Christian Bergler, Cecilia Mascolo, Jing Han, Iulia Lefter, Heysem Kaya, Shahin Amiriparian, Alice Baird, Lukas Stappen, et al.  The interspeech 2021 computational paralinguistics challenge: Covid-19 cough, covid-19 speech, escalation & primates.  arXiv preprint arXiv:2102.13468, 2021.
  * [24] Neeraj Sharma et al.  Coswara–a database of breathing, cough, and voice sounds for covid-19 diagnosis.  2020\.
  * [25] Vishwanath A Sindagi, Yin Zhou, and Oncel Tuzel.  Mvx-net: Multimodal voxelnet for 3d object detection.  In 2019 International Conference on Robotics and Automation (ICRA), pages 7276–7282. IEEE, 2019.
  * [26] Car News TV.  The best tanks of world war ii start sound and ride, url:https://www.youtube.com/watch?v=kpvg5r7niso.
  * [27] PC Gaming Videos.  Battlefield 5 gun sounds of all weapons, url:https://www.youtube.com/watch?v=tmwagle6pr8.
  * [28] Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han.  Haq: Hardware-aware automated quantization with mixed precision.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8612–8620, 2019.
  * [29] Bichen Wu, Yanghan Wang, Peizhao Zhang, Yuandong Tian, Peter Vajda, and Kurt Keutzer.  Mixed precision quantization of convnets via differentiable neural architecture search.  arXiv preprint arXiv:1812.00090, 2018.
  * [30] Xie Xiaozhu and He Cheng.  Object detection of armored vehicles based on deep learning in battlefield environment.  In 2017 4th International Conference on Information Science and Control Engineering (ICISCE), pages 1568–1570. IEEE, 2017.
  * [31] Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang, Qijing Huang, Yida Wang, Michael Mahoney, et al.  Hawq-v3: Dyadic neural network quantization.  In International Conference on Machine Learning, pages 11875–11886. PMLR, 2021.
  * [32] Guangdi Zheng, Xinjian Wu, Yuanyuan Hu, and Xiaofei Liu.  Object detection for low-resolution infrared image in land battlefield based on deep learning.  In 2019 Chinese Control Conference (CCC), pages 8649–8652. IEEE, 2019.


[1]Peter Meisenheimer These authors contributed equally to this work.

These authors contributed equally to this work.

These authors contributed equally to this work.

[1,3,4,9]Ramamoorthy Ramesh

[1]Department of Materials Science and Engineering, University of California,
Berkeley, CA, USA

2]Department of Materials Science and Engineering, Cornell University, Ithaca,
NY, USA

3] Materials Science Division, Lawrence Berkeley National Laboratory,
Berkeley, CA, USA

4]Department of Physics, University of California, Berkeley, CA, USA

5]Department of Physics, Korea Advanced Institute of Science and Technology
(KAIST), Daejeon, South Korea

6]Department of Physics, Brown University, Providence, RI, USA

7]School of Engineering, Brown University, Providence, RI, USA

8]Department of Physics, Northeastern University, Boston, MA, USA

9]Departments of Physics and Astronomy and Materials Science and
NanoEngineering and Rice Advanced Materials Institute, Rice University,
Houston, TX, USA

10]Department of Chemistry, Rice University, Houston, TX, USA

11]Materials Research and Technology Department, Luxembourg Institute of
Science and Technology (LIST), Esch-sur-Alzette, Luxembourg

12]Department of Physics and Materials Science, University of Luxembourg,
Belvaux, Luxembourg

13]Kavli Institue for Nanoscale Science, Cornell University, Ithaca, NY USA

14]Leibniz-Institut für Kristallzüchtung, Berlin, Germany

15]Applied Mathematics and Computational Research Division, Lawrence Berkeley
National Laboratory, Berkeley, CA, USA

# Designed spin-texture-lattice to control anisotropic magnon transport in
antiferromagnets

<EMAIL_ADDRESS>Maya Ramesh<EMAIL_ADDRESS>Sajid Husain
<EMAIL_ADDRESS>Isaac Harris<EMAIL_ADDRESS>Hyeon Woo Park
<EMAIL_ADDRESS>Shiyu Zhou<EMAIL_ADDRESS>Hossein Taghinejad
<EMAIL_ADDRESS>Hongrui Zhang<EMAIL_ADDRESS>Lane W.
Martin<EMAIL_ADDRESS>James Analytis<EMAIL_ADDRESS>Paul Stevenson
<EMAIL_ADDRESS>Jorge Íñiguez-González<EMAIL_ADDRESS>Se
Kwon Kim<EMAIL_ADDRESS>Darrell G. Schlom<EMAIL_ADDRESS>Lucas
Caretta<EMAIL_ADDRESS>Zhi Yao<EMAIL_ADDRESS><EMAIL_ADDRESS>*  [  [  [  [  [  [  [  [  [  [  [  [  [  [

###### Abstract

Spin waves in magnetic materials are promising information carriers for future
computing technologies due to their ultra-low energy dissipation and long
coherence length. Antiferromagnets are strong candidate materials due, in
part, to their stability to external fields and larger group velocities.
Multiferroic aniferromagnets, such as BiFeO3 (BFO), have an additional degree
of freedom stemming from magnetoelectric coupling, allowing for control of the
magnetic structure, and thus spin waves, with electric field. Unfortunately,
spin-wave propagation in BFO is not well understood due to the complexity of
the magnetic structure. In this work, we explore long-range spin transport
within an epitaxially engineered, electrically tunable, one-dimensional (1D)
magnonic crystal. We discover a striking anisotropy in the spin transport
parallel and perpendicular to the 1D crystal axis. Multiscale theory and
simulation suggests that this preferential magnon conduction emerges from a
combination of a population imbalance in its dispersion, as well as
anisotropic structural scattering. This work provides a pathway to
electrically-reconfigurable magnonic crystals in antiferromagnets.

###### keywords:

Multiferroics, antiferromagnet, magnonics

## Introduction

In the push to design next-generation memory and logic technologies that
utilize spin, antiferromagnets are particularly attractive due to their
stability against external magnetic fields and large magnon propagation group
velocities [1, 2, 3]. One pathway to control magnon transport is by
lithographically patterning periodic structures to create a so-called magnonic
crystal [4, 5, 6, 7, 8, 9]. A disadvantage of these extrinsic structures,
however, is that the periodic structure is fixed. Their utility would be
enhanced if they were tunable or reconfigurable [10]. To this end, the
interaction of spin waves with magnetic domain walls has been studied in
ferromagnets [11, 12, 13, 14], since domain walls can be written and moved
using current-induced effects [15, 16, 17]. In order to design and create a
magnonic crystal, however, periodic arrays of domain walls would need to be
created. Here, we investigate magnon-transport behavior in a room temperature
magnetoelectric with an intrinsically periodic, one-dimensional-cycloidal
magnetic structure. Because this spin-texture-lattice is intrinsic to a
multiferroic material and governed by the directionality of the polarization,
we demonstrate that electric fields can be used to manipulate the magnonic
crystal and control transport behavior.

For this study, BiFeO${}_{\textrm{3}}$ (BFO) is used as a model system. BFO is
a well-studied antiferromagnetic oxide due to the strong coupling between
polarization (ferroelectricity) and magnetic (antiferromagnetism) order at
room temperature, enabling all-electric-field control of magnetization [18,
19, 20, 21]. Due to this interaction, BFO has garnered interest in the field
of spintronics, especially with recent observations showing that electric-
field control of spin transport is possible. [22]. The fundamental magnetic
order in BFO consists of G-type antiferromagnetism, which is modulated by
broken symmetries stemming from the polarization and from antiferrodistortive
FeO6 octahedral rotations These phenomena give rise to a long-period
($\lambda\approx$ 65 nm) spin cycloid [23]. In its equilibrium structure, this
antiferromagnetic cycloid propagates along three symmetry-equivalent
directions: $\bm{k}_{1}$ ($[\bar{1}10]$), $\bm{k}_{2}$ ($[\bar{1}01]$), and
$\bm{k}_{3}$ ($[0\bar{1}1]$), when polarization, $\bm{P}$, is parallel to
$[111]$ [24], with Fe atomic moments rotating in the $(11\bar{2})$,
$(1\bar{2}1)$, and $(\bar{2}11)$, for $\bm{k}_{1}$, $\bm{k}_{2}$,
$\bm{k}_{3}$, respectively [25]. The antiferromagnetic cycloid is then further
perturbed by the Dyzaloshinskii-Moriya interaction (DMI), canting the moments
slightly out of the $\bm{P}-\bm{k}$ plane and resulting in a spin-density wave
($\bm{M}$) with the same period as the antiferromagnetic cycloid [26, 27].
This intrinsic, periodic magnetic structure in BFO can be modulated by an
electric field, a capability attributed to its strong magnetoelectric coupling
[28, 29]. Despite its fundamental nature, the behavior of spin propagation
within such a complex spin structure remains not fully understood. This gap in
knowledge suggests that the cycloid’s periodicity could have significant, yet
unexplored, effects on magnon dispersion.

The intimate coupling between magnetic order and ferroelectric polarization in
BFO allows for the use of its ferroelectric structure as a tool to design
periodic magnetic lattices, facilitating spin-wave transport. Previous studies
have shown that the ferroelectric domain structure of thin-film BFO can be
controlled using electrostatics and the epitaxial strain arising from the
lattice mismatch to the substrate [30], leading to quasi-ordered arrays of
either 71$\degree$ or 109$\degree$ domains [31]. This directly influences the
magnetic structure [32, 33]. Our current study demonstrates that by
engineering these phenomena through the boundary conditions imposed by
electrostatics and lattice-mismatch strain, we can effectively guide the
ferroelectric domain structure to modulate the spin cycloid. This manipulation
results in the creation of a long-range, intrinsic, quasi-1D-ordered lattice.
Such a lattice structure creates a gap in the magnon dispersion and introduces
a significant spin-transport anisotropy, which is further tunable with an
electric field.

Figure 1: 1D ordered lattice. In the two BFO variants, the ferroelectric
domains are arranged into ground states containing 71$\degree$ (a) and
109$\degree$ (b) ferroelastic domain walls. Insets depict the two types of
domain walls and their corresponding polarizations in the BFO unit cell. In
films deposited on DSO substrates, the cycloid prefers the $\bm{k}_{1}$
variant, which results in a chevron-like pattern across the ferroelectric
domains (c). For films on TSO, however, the domain structure allows selection
of the other symmetry allowed axis, in this case $\bm{k}_{2}$ (d).
Corresponding Fourier transforms are shown as insets revealing the period and
symmetry of the magnetic structures. (e) Zoom of the area shown in green,
where the cycloid is unperturbed by the fine, 10-30 nm ferroelastic domain
walls. (f) This occurs due to the symmetry of $\bm{k}_{2}$ across the
109$\degree$ domain wall, where the directionality of the cycloid is preserved
with the fixed net polarization $P_{net}$ imposed by the ferroelectric
domains.

## Designing the multiferroic structure

To control the ferroelectric and magnetic structure of BFO, we synthesize
epitaxial thin films on both DyScO3 (DSO, $\sim$0.3% compressive strain) and
TbScO3 (TSO, $\sim$0% strain) substrates (Methods). From X-ray diffraction
studies (Extended data Figure 1), films are confirmed to be $\sim$100 nm thick
and of excellent crystalline quality. In addition to slightly different
epitaxial strain states from the choice of substrate, these systems are chosen
to engineer films with different ferroelectric domain structures. These films
host characteristic stripe-like 71$\degree$ ferroelastic domains on DSO and
109$\degree$ ferroelastic domains on TSO [34]. Piezoresponse force microscopy
(PFM) images (Figure 1a,b) demonstrate the two-variant stripe patterns along
$[100]$. The nanoscale magnetic structures of these different strain and
domain configurations are then investigated in real space using scanning
nitrogen-vacancy (NV) magnetometry (Methods) [29, 33, 24, 35].

In films deposited on DSO (Figure 1c), NV magnetometry shows a chevron-like
magnetic texture of the stray magnetic field, revealing a
$\bm{k}_{1}$-oriented cycloid that switches direction with the change of the
$\bm{P}$ directions across the 71$\degree$ ferroelastic domain walls. This is
consistent with previous reports [29] and is a result of the magnetoelastic
anisotropy from the substrate lattice mismatch [32, 35]. Strikingly, in the
sample deposited on TSO (Figure 1d), only a single $\bm{k}$ variant of the
cycloid is observed, projected along $[100]$. The longer period of these
oscillations indicates that this corresponds to a $\bm{k}_{2}$, $[101]$ axis
of the cycloid, which is now preferred due to the lower magnetoelastic energy
from the smaller epitaxial strain of TSO. Importantly, this variant of the
cycloid even persists across the approximately 20-30 nm wide 109$\degree$
ferroelectric domains and extends over long distances (up to at least 20
$\mu$m). This results in the formation of a quasi-long-range, 1D magnonic
crystal, as illustrated in (Figure 1e).

This propagation of the $\bm{k}_{2}$ cycloid variant across ferroelectric
domains can be understood through the symmetry of the 109$\degree$
ferroelastic domain walls. Starting with the 71$\degree$ domains in films on
DSO substrates, the in-plane $\bm{k}_{1}$ directions are strongly
magnetoelastically preferred. At the ferroelectric domain walls, where
polarization P rotates from $[\bar{1}11]$ to $[111]$, the cycloid wave vector
directions must be shifted, for example, from $[110]$ to $[\bar{1}10]$, to
ensure the perpendicular relation $\bm{k}_{1}\perp\bm{P}$. This shift leads to
a chevron-like pattern being favored as an equilibrium state. In contrast, in
films on TSO substrates, the magnetoelastic energy is lowered, making the
$\bm{k}_{2}$ and $\bm{k}_{3}$ variants closer in energy. Furthermore,
considering the 109$\degree$ domain walls in the films on TSO substrates,
assuming $\bm{k}_{2}$ is now an allowed direction, the propagation axis would
not need to alter with the polarization change. This is because $[101]$ is
perpendicular to both $[\bar{1}11]$ and $[11\bar{1}]$ (Figure 1f), making
$\bm{k}_{2}$ also a symmetry-favored equilibrium in this configuration.
Consequently, $\bm{k_{2}}$ emerges as the globally preferred equilibrium
state, leading to a single cycloid variant that persists across the
109$\degree$ ferroelastic domain walls, despite their $\sim$20-30 nm period
[30].

Figure 2: Spin wave anisotropy. (a) Schematic of the sample geometry of the
test structures $\parallel$ ($\theta=0\degree$) and $\perp$
($\theta=90\degree$) to $\bm{k}$. These structures give rise to different
magnon conductivities due to the directionality of the cycloid. In the
quasi-1-D lattice of 109$\degree$ domains, (b), there is high conductivity
along the ferromagnetically coupled direction, [010], and low conductivity
along $\bm{k}$, [100]. Conversely, for the 71$\degree$ domain scenario, (c),
the cycloid, and thus the conductivity, display no distinct anisotropy between
the $\theta=0\degree$ and $90\degree$ directions. This reflects the magnetic
cycloid’s interaction with both the 109$\degree$ and 71$\degree$ ferroelastic
domain walls. In the case of 109$\degree$ cycloid, viewed along the directions
parallel to $[010]$ ($\theta=0\degree$), d), or $[100]$ ($\theta=90\degree$,
e), the magnetic moments only deviate slightly across the ferroelectric domain
walls, but cycle by $2\pi$ in the projected direction of $\bm{k}_{2}$. In e,
the $109\degree$ ferroelastic domain walls are shown along with the deviation
of $\bm{M}$ across the wall. In e, the antiferromagnetic Fe moments rotate in
the $\bm{k}-\bm{P}$ plane, along with the magnitude of $\bm{M}$ traced by the
green line. Unit cells are shown in an approximately 15:1 scale. The
orientation of schematics d and e is viewed along the orthogonal directions
shown in a.

## Magnon transport

To test spin transport with respect to the directions defined by the quasi-1D
spin-spiral lattice, we fabricated test structures that are schematically
shown in Figure 2a. These structures consist of source and detector wires
aligned both parallel and perpendicular to $\bm{k}$. The magnon-induced
voltage signal in these structures is compared along multiple directions with
respect to $\bm{k}$ to show anisotropy, as well as in both the quasi-1D and
chevron-like cycloid structures (Figure 1c). First, considering thermally
excited magnons in a two-wire scheme, a current in the source wire is used to
create a temperature gradient ($\nabla T$) in the device, causing thermally
excited spin waves to propagate $\bm{q}\parallel-\nabla T$, where $\bm{q}$ is
the magnon propagation direction, via the spin Seebeck effect [22]. These
magnons are then sensed in the detector wire through the inverse spin Hall
effect (ISHE), which converts the spin current to a DC voltage, $\Delta
V_{NL,2\omega}$ [36, 37]. To preserve the ferroelectric domains, we performed
this measurement again by reversing the directionality of the thermal
transport with respect to the in-plane polarization (details in Supp. Note 1).
Confirmation that the signal is magnetic in origin comes from testing with
different detector materials. For example, changing the detector wire from Pt
to W (with a larger, opposite-signed spin-Hall angle compared to Pt) reverses
the sign of $\Delta V_{NL,2\omega}$. Similarly, moving to a material with a
much larger spin-Hall angle, such as SrIrO3 (SIO) [38], proportionally
increases the ISHE voltage.

In films featuring the quasi-1D ordered cycloid lattice, a strong anisotropy
is observed with respect to directions $\bm{q}\parallel\bm{k}$ and
$\bm{q}\perp\bm{k}$. We define $\theta$ as the angle between $\bm{q}$ and
$\bm{k}$. Notably, the ISHE voltage signal, indicative of spin wave
conductivity, is approximately three times larger when $\theta=90\degree$,
aligning with the cycloid’s uniform direction. In contrast, samples with the
alternating chevron-like structure, characterized by a net
$\bm{k}_{net}=\sum\bm{k}=0$, exhibit minimal spin transport anisotropy (Figure
2c). Additionally, in the chevron-like samples, where locally
$\theta=45\degree$, $\Delta V_{NL,2\omega}$ lies between the values at
$\theta=90\degree$ and $0\degree$ seen in the 1D ordered system. This behavior
can be initially understood by examining the nanoscopic magnetic spin
structure within the cycloid as described schematically in Figure 2d,e. In a
line cut $\perp\bm{k}$, the spins are modulated by the presence of the
109$\degree$ ferroelastic domain walls, but possess a persistent net
magnetization component, illustrated in Figure 2e. In contrast, oriented
parallel to $\bm{k}$, the spins rotate by a full $2\pi$ over the period of the
cycloid (65 nm, Figure 2d). To further evaluate the physics associated with
transport in the quasi-1D structure, we turn to both mesoscale phase-field
simulations and model Hamiltonian calculations of spin transport $\parallel$
and $\perp\bm{k}$.

Figure 3: Simulation of magnon conductivity. Simulations of frequency-
dependent magnon power when the spin wave is driven along the direction of
$\bm{q}$, which is (a) perpendicular, and (b) parallel to, $\bm{k}$,
respectively, in the quasi-1D lattice. Energy carried by the magnons is
extracted as $\Omega\times f$, i.e., the product of the conical angle,
$\Omega$, of the time-dependent procession of magnetization in Fourier space,
and the frequency $f$. Input and output spectra are measured at the
initialization location and far side of the simulated cell. The integral of
this spectra is then reported as a relative conductivity, (c), simulated as a
function of the 1D anisotropy, $K$, of the cycloid texture. The shaded region
represents a range of energies reported in the literature, as well as values
matched to the experimental and analytical results presented here. The
horizontal dashed line corresponds to the measured efficiency, which is
related to the plot by the intercept shown. (d) Calculated magnon dispersion,
illustrating the emergence of a gap along the $\bm{q}\parallel\bm{k}$
direction which leads to the reduced spin wave conductivity. $X$ and $Y$ are
the high symmetry points corresponding to the [100] ($\parallel\bm{k}$) and
[010] ($\perp\bm{k}$), respectively.

From phase-field simulations, we observe an anisotropy characterized by
reduced spin-wave conductivity along the $\bm{q}\parallel\bm{k}$ direction,
compared to $\bm{q}\perp\bm{k}$. Simulations are carried out by imposing a
cycloidal magnetic ground state comparable to those observed experimentally
and by initiating spin waves within a specified frequency range at one edge of
the sample (Methods). The efficiency of these waves was quantified by
comparing the magnetization precession at the point of origin and a defined
distance from the source. In scenarios where $\bm{q}\perp\bm{k}$, the initial
waveform exhibits negligible damping even at a distance equivalent to
$10\lambda$ (Figure 3a). When $\bm{q}\parallel\bm{k}$, however, the spin wave
exhibits significant attenuation over the same distance. This attenuation is
evident in both the spectrum of the magnetization precession angle and its
integrated energy across the spectrum, which is shown to be influenced by the
magnitude of the anisotropy within the cycloid. As illustrated in Figure 3b, a
scenario with an anisotropy of $K$ = 100 kJ m-3 leads to approximately 50%
damping in spin-wave-energy magnitude over the same length scale. This
transport efficiency can then be studied as a function of $K$ to compare with
previous calculations and experimental results, defining a reasonable window
for the value of $K$. The precession angles corresponding to other values of
$K$ are shown in Extended data Figure 5.

At the microscopic scale, analytical calculations of the momentum-dependent
magnon dispersion in the quasi-1D lattice illustrate the difference in the
magnon conduction $\perp\bm{k}$ and $\parallel\bm{k}$. Analytical solutions
for the magnon bands of BFO are based on a continuum Hamiltonian whose density
is given by [39]:

$\mathcal{H}=\mathcal{A}(\nabla\bm{\mathrm{n}})^{2}-\alpha\bm{\mathrm{P}}\cdot[\bm{\mathrm{n}}(\nabla\cdot\bm{\mathrm{n}})+\bm{\mathrm{n}}\times(\nabla\times\bm{\mathrm{n}})]-2\beta
M_{0}\bm{\mathrm{P}}\cdot(\bm{\mathrm{m}}\times\bm{\mathrm{n}})-K_{u}n^{2}_{c}+\lambda\bm{\mathrm{m}}^{2}\,,$
(1)

where the first term is the exchange energy, the second and third terms are
DMI terms, the fourth term is the anisotropy, and the last term represents the
suppression of the net magnetization due to strong antiferromagnetic coupling.
The ground-state configuration can then be obtained (Supp. Note 3), showing
the well-known anharmonic spin-cycloid structure [39, 40, 41]. Solving the
time-dependent Euler-Lagrange equation for the small fluctuations, two spin-
wave modes result: 1) An in-plane mode with spin fluctuations within the spin-
cycloid plane and 2) an out-of-plane mode whose spin fluctuations are
orthogonal to the spin-cycloid plane. By considering the quasi-2D geometry of
the plane-wave-like spin texture of BFO and neglecting the Hamiltonian term
$\propto\beta$ that is known to be much weaker than the rest of the terms in
the Hamiltonian [42], the band structures of in-plane (IP) magnons and out-of-
plane (OOP) magnon modes are obtained analytically (Figure 3d). This clearly
shows the formation of a gap in the dispersion along the
$\bm{q}\parallel\bm{k}$ direction, due to the periodicity of the magnetic
lattice. This gap indicates a lower density of states along this direction,
resulting in an anisotropic, lower magnon conductivity.

In parallel to measurements of thermally excited magnons, non-local transport
measurements have been performed using injected spin currents via the spin-
Hall effect (SHE) of the source wire [36] (Figure 4a). In this case, the sign
of the output signal will correspond directly to the sign of the input signal
(i.e., a positive voltage will result in a positively polarized spin current,
which then results in a positive voltage at the detector), rather than only
the amplitude of the input (so-called first-harmonic and second-harmonic
measurements). While the spin Hall angles of platinum and tungsten are too low
to generate an appreciable ISHE signal at the 2 $\mu$m electrode spacing, the
efficient spin-charge transduction of SIO allows detection of the SHE excited
magnon signal [38]. In the device geometry where $\bm{q}$ is perpendicular to
the domain walls, where electric fields allow the ferroelastic domain walls to
remain largely stationary [22, 35, 19] (Supp. Figure 2), combining the
engineered spin texture with an epitaxial, large SHE oxide can further
optimize functionality. This should allow for deterministic switching of the
magnetic structure under a continuous rotation of $\bm{P}$ [35].

Figure 4: Polarization control of magnon propagation. (a) Schematic showing
the non-local device geometry with magnons generated by injection of spin from
the source wire. (b) We hypothesize that the non-reciprocity of the transport
is due to a Rashba-like asymmetry of the magnon dispersion emerges from the
crystal asymmetry, which is coupled directly to the ferroelectric polarization
and DMI. (c,d) Illustration showing the sense of the cycloid with respect to
the device geometry for films deposited on the two different substrates. In
this orientation, the ferroelectric domain structure remains stationary with
cycling electric field. (e,f) Switchable non-local signal overlaid with the
ferroelectric hysteresis for both the single variant and chevron-like cases.
(g) Combined with the above measurements, we hypothesize that the anisotropy
can be explained by a split and anisotropic magnon dispersion, where the
magnon conductivity maps to a surface with directions defined by $\bm{q}$ and
$\bm{k}$. The distance from the origin is representative of the conductivity,
and thus the population.

In test structures using SIO electrodes, we observe a similar anisotropic spin
transport, dependent on the underlying magnetic structure. Here, magnons are
excited by the SHE and the ferroelectric polarization (noted $\bm{P}\uparrow$
and $\bm{P}\downarrow$, where the direction of magnon propagation,
$\bm{q}\uparrow$, is fixed), is switched using an in-plane electric field.
Upon switching the in-plane component of the polarization, where the direction
of magnon propagation, $\bm{q}$, is fixed, the detected first-harmonic non-
local voltage, $V_{NL,1\omega}$, switches hysteretically on the scale of
$\sim$1 $\mu$V for a device spacing of 2 $\mu$m. The repeatable difference
between the $\bm{P}\uparrow\bm{q}\uparrow$ and
$\bm{P}\downarrow\bm{q}\uparrow$ states is indicative of a non-reciprocity in
the magnon dispersion tied to the direction of $\bm{P}$. This can be
qualitatively rationalized in the form of a Rashba-like, DMI-induced splitting
of the spin-dependent magnon bands (sketched in Figure 4b) [43, 44, 45, 46,
47]. The result is a $V_{NL,1\omega}$ that is both nonreciprocal and
switchable. Since the samples under investigation exhibit no net magnetic
moment, the inference is that magnon non-reciprocity in BFO should stem from a
chiral component of the magnetic cycloid, which interacts with $\bm{P}$.

Under an applied electric field, this hysteresis in $V_{NL,1\omega}$ follows
the switching of $\bm{P}$, where $\bm{P}$ and $\Delta V_{NL,1\omega}$ both
flip signs at the same coercive voltage (Figure 4d,e) [48]. As the
ferroelastic domain walls remain stationary during this event (Supp. Figure
2), a comparison can be made between samples with quasi-1D and chevron-like
configurations (Figure 4c,d). In quasi-1D samples with a single $\bm{k}$
state, the magnitude of $V_{NL,1\omega}\perp\bm{k}$ is two-times higher than
that of the samples showing a chevron-like magnetic cycloid. The magnitude of
this scaling directly parallels the observations in Figure 2, where the
chevron-like texture shows a two-times lower conductivity than in the single
$\bm{k}$ samples when $\bm{q}\perp\bm{k}$. Taken together, these measurements
allow us to sketch the approximate shape of the magnon dispersion projected
into the plane of the film (Figure 4g), where $\theta=90\degree$
($\bm{P}\perp\bm{k}$) is high conductivity, $\theta=0\degree$
($\bm{P}\parallel\bm{k}$) is low, $\theta=45\degree$ is intermediate, and
$\theta=90\degree$ and $\theta=-90\degree$ carry an opposite sign angular
momentum.

These results demonstrate control of the magnetic structure of BFO and a
method for effectively decoupling the magnetic structure from the
ferroelectric domains. This has significant implications for magnonic devices
using BFO, in that fundamental anisotropies of the cycloid structure can be
used to optimize transport. The formation of a quasi-1D ordered magnonic
lattice comprised of the spin cycloid should be of fundamental interest in
terms of phase transitions in a 1D lattice and the possibility of obtaining
long-range order in them. In this sense, future studies could be directed
towards controlling the ferroelectric/ferroelastic domain structure through
the use of vicinally cut substrates [30].

## Conclusion

Using heteroepitaxy as a building block and imposing a combination of
electrostatic and elastic constraints, we can strategically design both the
ferroelectric and magnetic domain structure of multiferroic BFO. By doing so,
a quasi-1D crystal of the antiferromagnetic spin cycloid can be chosen as the
ground state. This anisotropic structure results in preferential spin-wave
conduction orthogonal to its axis, due to the opening of a gap in the magnon
dispersion created by the periodicity of the magnetic lattice. Leveraging
magnetoelectric coupling, this magnonic crystal can be dynamically tuned using
an electric field, allowing us to empirically map the magnon dispersion in the
film plane with respect to the direction of ferroelectric polarization. This
opens the door for the design of magnonic devices utilizing the anisotropy of
the spin texture, as well as providing fundamental insights into the
conduction of spin waves in complex magnetic systems.

## Methods

Sample Fabrication 100 nm thick BFO samples were deposited on DSO and TSO
substrates using both molecular-beam epitaxy (MBE) and pulsed laser deposition
(PLD). MBE films were grown by reactive MBE in a VEECO GEN10 system using a
mixture of 80 % ozone (distilled) and 20 % oxygen. Elemental sources of
bismuth and iron were used at fluxes of $1.5\text{\times}{10}^{14}\text{\,}$
and
$2\text{\times}{10}^{13}\text{\,}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{s}\mathrm{/}\mathrm{c}\mathrm{m}\mathrm{{}^{2}}\mathrm{s}$
respectively. All films were grown at a substrate temperature of
$675\text{\,}\mathrm{\SIUnitSymbolCelsius}$ and chamber pressure of 5 x 10-6
Torr. PLD samples were deposited using a 248 nm KrF laser with $\sim$1.5 J
cm-2 at 700 $\degree$C and 90 mTorr O2. Pt (15 nm) and W (15 nm) layers were
deposited using magnetron sputtering at room temperature. SIO layers were
deposited using MBE using a mixture of 80 $\%$ ozone and 20 $\%$ O2 along with
elemental sources of strontium and iridium. Due to the high temperatures
needed to generate an iridium flux, an electron beam was used to evaporate
iridium. The samples were grown at a substrate temperature of 695 $\degree$C
and a background pressure of 1 x 10-6 Torr. Devices were fabricated using
standard optical lithography followed by ion milling.

Sample Characterization X-ray diffraction was performed on a PANalytical
Xpert3 diffractometer. Piezoresponse force microscopy was performed on an
Asylum MFP3D. NV magnetometry was performed on a Qnami ProteusQ system with a
parabolic Quantilever MX+ diamond cantilever.

Phase Field Simulations We employed MagneX, a substantial GPU-enabled
micromagnetic simulation code package (available on  GitHub), to
phenomenologically explore the directionality of magnon transport in the
cycloid. The time-domain Landau-Lifshitz-Gilbert equation is solved,
incorporating exchange coupling, DMI coupling, anisotropy coupling, and
demagnetization coupling. A set of abstract ferromagnetic properties is
utilized to phenomenologically represent the antiferromagnetic material. The
magnetization is pinned by an external applied magnetic field that mimics the
cycloid texture. The Gilbert damping constant is set to be 0.0005, the
exchange coupling constant is $3.0\times 10^{-12}$ J m-1, and the DMI coupling
constant is $1.0935\times 10^{-3}$ J m-2. The anisotropic hard axis is set to
the [111] direction, while the anisotropy constant is swept in the range of
$1.0\times 10^{4}$ J m-3 and $2.0\times 10^{6}$ J m-3. The total magnetization
is normalized to be one. On one side of the sample, a modified Gaussian pulse
in the effective magnetic field is applied in both the $x$ and $y$ direction,
mimicking injected spins from the top SHE electrode. The pulse is in the
mathematical form of $e^{-(t-t_{0})^{2}/2T_{0}^{2}}\cos(2\pi f_{0}t)$, where
$f_{0}$ is 700 GHz. The output port is on the opposite side of the input port
in the in-plane direction. The simulated time-varying magnetization components
are converted into the frequency domain with the Fourier transform, from which
we calculate the cone angle of the magnetization. For more details, see Supp.
Note 2.

Analytical Calculations See Supp. Note 3.

Transport Measurements Transport measurements were performed using 3-terminal
devices where two were used to inject current and two for output. Two cross-
terminals were used to apply electric field for polarization control. All
operations were controlled through the in-house developed code. $V_{NL}$ was
measured using an SR830 lock-in amplifier and locking to the second (in the
case of thermally excited magnons) or first (in the case of spin Hall excited
magnons) harmonic of the excitation voltage. An AC signal at 7 Hz was used for
both first- and second-harmonic measurements. In the first-harmonic
measurement, the magnitude of the injected current is chosen to be small to
avoid the thermal drift in the output voltage.

## Author Contributions

PM designed experiments. MR and SH deposited samples. PM and SH performed PFM.
PM, SZ, and PS performed NV microscopy. ZY performed phase field simulations.
HWP and SKK performed analytical calculations. IH and HT fabricated devices.
SH and IH performed spin transport. PM, MR, and SH wrote the manuscript. All
authors have contributed to manuscript edits.

This work was primarily supported by the U.S. Department of Energy, Office of
Science, Office of Basic Energy Sciences, Materials Sciences and Engineering
Division under Contract No. DE-AC02-05-CH11231 (Codesign of Ultra-Low-Voltage
Beyond CMOS Microelectronics (MicroelecLBLRamesh)) for the development of
materials for low-power microelectronics. P.M., R.R., M.R. and D.G.S.
additionally acknowledge funding from the Army Research Office under the ETHOS
MURI via cooperative agreement W911NF-21-2-0162. S.Z. and L.C. acknowledge
funding from Brown School of Engineering and Office of the Provost. P.S.
acknowledges support from the Massachusetts Technology Collaborative, Award
number 22032. Computations in this paper were performed using resources of the
National Energy Research Scientific Computing Center (NERSC), a U.S.
Department of Energy Office of Science User Facility operated under contract
no. DE-AC02-05CH11231. The work performed at the Molecular Foundry was
supported by the Office of Science, Office of Basic Energy Sciences, of the
U.S. Department of Energy under Contract No. DEAC02-05CH11231. S.K.K. and
H.W.P. acknowledge support from the Brain Pool Plus Program through the
National Research Foundation of Korea funded by the Ministry of Science and
ICT (2020H1D3A2A03099291).

The authors declare no competing interests

Data relevant to this manuscript have been uploaded to zenodo under accession
code XYZ. Other data is available from the corresponding author upon
reasonable request.

## References

  *   * Jungwirth et al. [2016] Jungwirth, T., Marti, X., Wadley, P., Wunderlich, J.: Antiferromagnetic spintronics. Nature Nanotechnology 11(3), 231–241 (2016) https://doi.org/10.1038/nnano.2016.18 . Number: 3 Publisher: Nature Publishing Group. Accessed 2023-06-29
  * Han et al. [2023] Han, J., Cheng, R., Liu, L., Ohno, H., Fukami, S.: Coherent antiferromagnetic spintronics. Nature Materials 22(6), 684–695 (2023) https://doi.org/10.1038/s41563-023-01492-6 . Number: 6 Publisher: Nature Publishing Group. Accessed 2023-06-29
  * Jungwirth et al. [2018] Jungwirth, T., Sinova, J., Manchon, A., Marti, X., Wunderlich, J., Felser, C.: The multiple directions of antiferromagnetic spintronics. Nature Physics 14(3), 200–203 (2018) https://doi.org/10.1038/s41567-018-0063-6 . Number: 3 Publisher: Nature Publishing Group. Accessed 2023-10-02
  * Qiu et al. [2014] Qiu, R.-k., Huang, T., Zhang, Z.-d.: Magnon band structure and magnon density in one-dimensional magnonic crystals. Journal of Magnetism and Magnetic Materials 368, 180–190 (2014) https://doi.org/10.1016/j.jmmm.2014.05.033 . Accessed 2023-12-18
  * Chumak et al. [2017] Chumak, A.V., Serga, A.A., Hillebrands, B.: Magnonic crystals for data processing. Journal of Physics D: Applied Physics 50(24), 244001 (2017) https://doi.org/10.1088/1361-6463/aa6a65 . Publisher: IOP Publishing. Accessed 2023-12-19
  * Deng et al. [2002] Deng, D.S., Jin, X.F., Tao, R.: Magnon energy gap in a periodic anisotropic magnetic superlattice. Physical Review B 66(10), 104435 (2002) https://doi.org/10.1103/PhysRevB.66.104435 . Publisher: American Physical Society. Accessed 2023-12-18
  * Zivieri et al. [2011] Zivieri, R., Montoncello, F., Giovannini, L., Nizzoli, F., Tacchi, S., Madami, M., Gubbiotti, G., Carlotti, G., Adeyeye, A.O.: Collective spin modes in chains of dipolarly interacting rectangular magnetic dots. Physical Review B 83(5), 054431 (2011) https://doi.org/10.1103/PhysRevB.83.054431 . Publisher: American Physical Society. Accessed 2023-12-18
  * Chumak et al. [2009] Chumak, A.V., Pirro, P., Serga, A.A., Kostylev, M.P., Stamps, R.L., Schultheiss, H., Vogt, K., Hermsdoerfer, S.J., Laegel, B., Beck, P.A., Hillebrands, B.: Spin-wave propagation in a microstructured magnonic crystal. Applied Physics Letters 95(26), 262508 (2009) https://doi.org/10.1063/1.3279138 . Accessed 2023-12-18
  * Chumak et al. [2010] Chumak, A.V., Tiberkevich, V.S., Karenowska, A.D., Serga, A.A., Gregg, J.F., Slavin, A.N., Hillebrands, B.: All-linear time reversal by a dynamic artificial crystal. Nature Communications 1(1), 141 (2010) https://doi.org/10.1038/ncomms1142 . Number: 1 Publisher: Nature Publishing Group. Accessed 2023-12-18
  * Wessler et al. [2022] Wessler, C., Roessli, B., Krämer, K.W., Stuhr, U., Wildes, A., Braun, H.B., Kenzelmann, M.: Dipolar spin-waves and tunable band gap at the Dirac points in the 2D magnet ErBr3. Communications Physics 5(1), 1–7 (2022) https://doi.org/10.1038/s42005-022-00965-5 . Number: 1 Publisher: Nature Publishing Group. Accessed 2023-12-18
  * Hertel et al. [2004] Hertel, R., Wulfhekel, W., Kirschner, J.: Domain-Wall Induced Phase Shifts in Spin Waves. Physical Review Letters 93(25), 257202 (2004) https://doi.org/10.1103/PhysRevLett.93.257202 . Publisher: American Physical Society. Accessed 2023-12-19
  * Li et al. [2023] Li, R., Riddiford, L.J., Chai, Y., Dai, M., Zhong, H., Li, B., Li, P., Yi, D., Zhang, Y., Broadway, D.A., Dubois, A.E.E., Maletinsky, P., Hu, J., Suzuki, Y., Ralph, D.C., Nan, T.: A puzzling insensitivity of magnon spin diffusion to the presence of 180-degree domain walls. Nature Communications 14(1), 2393 (2023) https://doi.org/10.1038/s41467-023-38095-3 . Number: 1 Publisher: Nature Publishing Group. Accessed 2023-12-19
  * Pirro et al. [2021] Pirro, P., Vasyuchka, V.I., Serga, A.A., Hillebrands, B.: Advances in coherent magnonics. Nature Reviews Materials 6(12), 1114–1135 (2021) https://doi.org/10.1038/s41578-021-00332-w . Number: 12 Publisher: Nature Publishing Group. Accessed 2023-12-19
  * Hämäläinen et al. [2018] Hämäläinen, S.J., Madami, M., Qin, H., Gubbiotti, G., Dijken, S.: Control of spin-wave transmission by a programmable domain wall. Nature Communications 9(1), 4853 (2018) https://doi.org/10.1038/s41467-018-07372-x . Number: 1 Publisher: Nature Publishing Group. Accessed 2023-12-19
  * Caretta et al. [2020] Caretta, L., Oh, S.-H., Fakhrul, T., Lee, D.-K., Lee, B.H., Kim, S.K., Ross, C.A., Lee, K.-J., Beach, G.S.D.: Relativistic kinematics of a magnetic soliton. Science 370(6523), 1438–1442 (2020) https://doi.org/10.1126/science.aba5555 . Publisher: American Association for the Advancement of Science. Accessed 2023-12-19
  * Avci et al. [2019] Avci, C.O., Rosenberg, E., Caretta, L., Büttner, F., Mann, M., Marcus, C., Bono, D., Ross, C.A., Beach, G.S.D.: Interface-driven chiral magnetism and current-driven domain walls in insulating magnetic garnets. Nature Nanotechnology 14(6), 561–566 (2019) https://doi.org/10.1038/s41565-019-0421-2 . Number: 6 Publisher: Nature Publishing Group. Accessed 2023-12-19
  * Thomas et al. [2006] Thomas, L., Hayashi, M., Jiang, X., Moriya, R., Rettner, C., Parkin, S.S.P.: Oscillatory dependence of current-driven magnetic domain wall motion on current pulse length. Nature 443(7108), 197–200 (2006) https://doi.org/10.1038/nature05093 . Number: 7108 Publisher: Nature Publishing Group. Accessed 2023-12-19
  * Bibes and Barthélémy [2008] Bibes, M., Barthélémy, A.: Multiferroics: Towards a magnetoelectric memory. Nature Materials 7(6), 425–426 (2008) https://doi.org/10.1038/nmat2189
  * Heron et al. [2014] Heron, J.T., Bosse, J.L., He, Q., Gao, Y., Trassin, M., Ye, L., Clarkson, J.D., Wang, C., Liu, J., Salahuddin, S., Ralph, D.C., Schlom, D.G., Íñiguez, J., Huey, B.D., Ramesh, R.: Deterministic switching of ferromagnetism at room temperature using an electric field. Nature 516(7531), 370–373 (2014) https://doi.org/10.1038/nature14004 . Accessed 2017-10-12
  * Spaldin and Ramesh [2019] Spaldin, N.A., Ramesh, R.: Advances in magnetoelectric multiferroics. Nature Materials 18(3), 203–212 (2019) https://doi.org/10.1038/s41563-018-0275-2 . Number: 3 Publisher: Nature Publishing Group. Accessed 2023-12-13
  * Rovillain et al. [2010] Rovillain, P., Sousa, R., Gallais, Y., Sacuto, A., Méasson, M.A., Colson, D., Forget, A., Bibes, M., Barthélémy, A., Cazayous, M.: Electric-field control of spin waves at room temperature in multiferroic BiFeO3. Nature Materials 9(12), 975–979 (2010) https://doi.org/10.1038/nmat2899 . Number: 12 Publisher: Nature Publishing Group. Accessed 2023-12-18
  * Parsonnet et al. [2022] Parsonnet, E., Caretta, L., Nagarajan, V., Zhang, H., Taghinejad, H., Behera, P., Huang, X., Kavle, P., Fernandez, A., Nikonov, D., Li, H., Young, I., Analytis, J., Ramesh, R.: Nonvolatile Electric Field Control of Thermal Magnons in the Absence of an Applied Magnetic Field. Physical Review Letters 129(8), 087601 (2022) https://doi.org/10.1103/PhysRevLett.129.087601 . Publisher: American Physical Society. Accessed 2023-05-15
  * Burns et al. [2020] Burns, S.R., Paull, O., Juraszek, J., Nagarajan, V., Sando, D.: The Experimentalist’s Guide to the Cycloid, or Noncollinear Antiferromagnetism in Epitaxial BiFeO3. Advanced Materials 32(45), 2003711 (2020) https://doi.org/10.1002/adma.202003711 . _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.202003711. Accessed 2023-04-25
  * Zhong et al. [2022] Zhong, H., Finco, A., Fischer, J., Haykal, A., Bouzehouane, K., Carrétéro, C., Godel, F., Maletinsky, P., Munsch, M., Fusil, S., Jacques, V., Garcia, V.: Quantitative Imaging of Exotic Antiferromagnetic Spin Cycloids in BiFeO3 Thin Films. Physical Review Applied 17(4), 044051 (2022) https://doi.org/10.1103/PhysRevApplied.17.044051 . Publisher: American Physical Society. Accessed 2023-04-20
  * Sosnowska et al. [1982] Sosnowska, I., Neumaier, T.P., Steichele, E.: Spiral magnetic ordering in bismuth ferrite. Journal of Physics C: Solid State Physics 15(23), 4835 (1982) https://doi.org/10.1088/0022-3719/15/23/020 . Accessed 2023-04-26
  * Ramazanoglu et al. [2011] Ramazanoglu, M., Laver, M., Ratcliff, W., Watson, S.M., Chen, W.C., Jackson, A., Kothapalli, K., Lee, S., Cheong, S.-W., Kiryukhin, V.: Local Weak Ferromagnetism in Single-Crystalline Ferroelectric BiFeO3. Physical Review Letters 107(20), 207206 (2011) https://doi.org/10.1103/PhysRevLett.107.207206 . Publisher: American Physical Society. Accessed 2023-04-21
  * Rahmedov et al. [2012] Rahmedov, D., Wang, D., Íñiguez, J., Bellaiche, L.: Magnetic Cycloid of BiFeO3 from Atomistic Simulations. Physical Review Letters 109(3), 037207 (2012) https://doi.org/10.1103/PhysRevLett.109.037207 . Publisher: American Physical Society. Accessed 2023-07-09
  * Lebeugle et al. [2008] Lebeugle, D., Colson, D., Forget, A., Viret, M., Bataille, A.M., Gukasov, A.: Electric-Field-Induced Spin Flop in BiFeO3 Single Crystals at Room Temperature. Physical Review Letters 100(22), 227602 (2008) https://doi.org/10.1103/PhysRevLett.100.227602 . Publisher: American Physical Society. Accessed 2023-04-21
  * Gross et al. [2017] Gross, I., Akhtar, W., Garcia, V., Martínez, L.J., Chouaieb, S., Garcia, K., Carrétéro, C., Barthélémy, A., Appel, P., Maletinsky, P., Kim, J.-V., Chauleau, J.Y., Jaouen, N., Viret, M., Bibes, M., Fusil, S., Jacques, V.: Real-space imaging of non-collinear antiferromagnetic order with a single-spin magnetometer. Nature 549(7671), 252 (2017) https://doi.org/%****␣Designed_Magnon_forsubmission.tex␣Line␣875␣****10.1038/nature23656 . Accessed 2017-12-04
  * Chu et al. [2006] Chu, Y.-H., Zhan, Q., Martin, L.W., Cruz, M.P., Yang, P.-L., Pabst, G.W., Zavaliche, F., Yang, S.-Y., Zhang, J.-X., Chen, L.-Q., Schlom, D.G., Lin, I.-N., Wu, T.-B., Ramesh, R.: Nanoscale Domain Control in Multiferroic BiFeO3 Thin Films. Advanced Materials 18(17), 2307–2311 (2006) https://doi.org/10.1002/adma.200601098 . _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.200601098. Accessed 2023-07-14
  * Martin et al. [2008] Martin, L.W., Chu, Y.-H., Holcomb, M.B., Huijben, M., Yu, P., Han, S.-J., Lee, D., Wang, S.X., Ramesh, R.: Nanoscale Control of Exchange Bias with BiFeO${}_{\textrm{3}}$ Thin Films. Nano Letters 8(7), 2050–2055 (2008) https://doi.org/%****␣Designed_Magnon_forsubmission.tex␣Line␣925␣****10.1021/nl801391m
  * Sando et al. [2013] Sando, D., Agbelele, A., Rahmedov, D., Liu, J., Rovillain, P., Toulouse, C., Infante, I.C., Pyatakov, A.P., Fusil, S., Jacquet, E., Carrétéro, C., Deranlot, C., Lisenkov, S., Wang, D., Le Breton, J.-M., Cazayous, M., Sacuto, A., Juraszek, J., Zvezdin, A.K., Bellaiche, L., Dkhil, B., Barthélémy, A., Bibes, M.: Crafting the magnonic and spintronic response of BiFeO${}_{\textrm{3}}$ films by epitaxial strain. Nature Materials 12(7), 641–646 (2013) https://doi.org/10.1038/nmat3629 . Accessed 2018-03-25
  * Haykal et al. [2020] Haykal, A., Fischer, J., Akhtar, W., Chauleau, J.-Y., Sando, D., Finco, A., Godel, F., Birkhölzer, Y.A., Carrétéro, C., Jaouen, N., Bibes, M., Viret, M., Fusil, S., Jacques, V., Garcia, V.: Antiferromagnetic textures in BiFeO3 controlled by strain and electric field. Nature Communications 11(1), 1704 (2020) https://doi.org/10.1038/s41467-020-15501-8 . Number: 1 Publisher: Nature Publishing Group. Accessed 2023-06-15
  * Chu et al. [2007] Chu, Y.-H., Cruz, M.P., Yang, C.-H., Martin, L.W., Yang, P.-L., Zhang, J.-X., Lee, K., Yu, P., Chen, L.-Q., Ramesh, R.: Domain Control in Multiferroic BiFeO3 through Substrate Vicinality. Advanced Materials 19(18), 2662–2666 (2007) https://doi.org/10.1002/adma.200602972 . _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.200602972. Accessed 2023-10-02
  * [35] Meisenheimer, P., Moore, G.C., Zhou, S., Zhang, H., Huang, X., Husain, S., Chen, X., Martin, L.W., Persson, K.A., Griffin, S.M., Caretta, L., Stevenson, P., Ramesh, R.: Persistent anisotropy of the spin cycloid in BiFeO3 through ferroelectric switching. Science Advances under review
  * Cornelissen et al. [2015] Cornelissen, L.J., Liu, J., Duine, R.A., Youssef, J.B., Wees, B.J.: Long-distance transport of magnon spin information in a magnetic insulator at room temperature. Nature Physics 11(12), 1022–1026 (2015) https://doi.org/10.1038/nphys3465 . Number: 12 Publisher: Nature Publishing Group. Accessed 2023-09-12
  * Wesenberg et al. [2017] Wesenberg, D., Liu, T., Balzar, D., Wu, M., Zink, B.L.: Long-distance spin transport in a disordered magnetic insulator. Nature Physics 13(10), 987–993 (2017) https://doi.org/10.1038/nphys4175 . Number: 10 Publisher: Nature Publishing Group. Accessed 2023-10-04
  * Huang et al. [2021] Huang, X., Sayed, S., Mittelstaedt, J., Susarla, S., Karimeddiny, S., Caretta, L., Zhang, H., Stoica, V.A., Gosavi, T., Mahfouzi, F., Sun, Q., Ercius, P., Kioussis, N., Salahuddin, S., Ralph, D.C., Ramesh, R.: Novel Spin–Orbit Torque Generation at Room Temperature in an All-Oxide Epitaxial La0.7Sr0.3MnO3/SrIrO3 System. Advanced Materials 33(24), 2008269 (2021) https://doi.org/10.1002/adma.202008269 . _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.202008269. Accessed 2023-01-08
  * Park et al. [2014] Park, J.-G., Le, M.D., Jeong, J., Lee, S.: Structure and spin dynamics of multiferroic BiFeO3. Journal of Physics: Condensed Matter 26(43), 433202 (2014) https://doi.org/10.1088/0953-8984/26/43/433202 . Publisher: IOP Publishing. Accessed 2023-07-07
  * Sosnowska and Zvezdin [1995] Sosnowska, I., Zvezdin, A.K.: Origin of the long period magnetic ordering in BiFeO3. Journal of Magnetism and Magnetic Materials 140-144, 167–168 (1995) https://doi.org/10.1016/0304-8853(94)01120-6 . Accessed 2023-12-03
  * Kadomtseva et al. [2006] Kadomtseva, A.M., Popov, Y.F., Pyatakov, A.P., Vorob’ev, G.P., Zvezdin, A.K., Viehland, D.: Phase transitions in multiferroic BiFeO3 crystals, thin-layers, and ceramics: enduring potential for a single phase, room-temperature magnetoelectric ‘holy grail’. Phase Transitions 79(12), 1019–1042 (2006) https://doi.org/10.1080/01411590601067235 . Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/01411590601067235. Accessed 2023-12-03
  * Park et al. [2014] Park, H.S., Yu, X., Aizawa, S., Tanigaki, T., Akashi, T., Takahashi, Y., Matsuda, T., Kanazawa, N., Onose, Y., Shindo, D., Tonomura, A., Tokura, Y.: Observation of the magnetic flux and three-dimensional structure of skyrmion lattices by electron holography. Nature Nanotechnology 9(5), 337–342 (2014) https://doi.org/10.1038/nnano.2014.52 . Number: 5 Publisher: Nature Publishing Group. Accessed 2023-05-11
  * Gitgeatpong et al. [2017] Gitgeatpong, G., Zhao, Y., Piyawongwatthana, P., Qiu, Y., Harriger, L., Butch, N., Sato, T., Matan, K.: Nonreciprocal Magnons and Symmetry-Breaking in the Noncentrosymmetric Antiferromagnet. Physical Review Letters 119(4), 047201 (2017) https://doi.org/10.1103/PhysRevLett.119.047201 . Publisher: American Physical Society. Accessed 2023-11-07
  * dos Santos et al. [2020] Santos, F.J., Santos Dias, M., Lounis, S.: Nonreciprocity of spin waves in noncollinear magnets due to the Dzyaloshinskii-Moriya interaction. Physical Review B 102(10), 104401 (2020) https://doi.org/10.1103/PhysRevB.102.104401 . Publisher: American Physical Society. Accessed 2023-11-07
  * Kawano et al. [2019] Kawano, M., Onose, Y., Hotta, C.: Designing Rashba–Dresselhaus effect in magnetic insulators. Communications Physics 2(1), 1–8 (2019) https://doi.org/10.1038/s42005-019-0128-6 . Number: 1 Publisher: Nature Publishing Group. Accessed 2023-11-07
  * Kawano and Hotta [2019] Kawano, M., Hotta, C.: Discovering momentum-dependent magnon spin texture in insulating antiferromagnets: Role of the Kitaev interaction. Physical Review B 100(17), 174402 (2019) https://doi.org/10.1103/PhysRevB.100.174402 . Publisher: American Physical Society. Accessed 2023-11-07
  * Shiomi et al. [2017] Shiomi, Y., Takashima, R., Okuyama, D., Gitgeatpong, G., Piyawongwatthana, P., Matan, K., Sato, T.J., Saitoh, E.: Spin Seebeck effect in the polar antiferromagnet $\ensuremath{\alpha}\text{\ensuremath{-}}{\mathrm{Cu}}_{2}{\mathrm{V}}_{2}{\mathrm{O}}_{7}$. Physical Review B 96(18), 180414 (2017) https://doi.org/%****␣Designed_Magnon_forsubmission.tex␣Line␣1275␣****10.1103/PhysRevB.96.180414 . Publisher: American Physical Society. Accessed 2023-11-07
  * Bhattacharjee et al. [2014] Bhattacharjee, S., Rahmedov, D., Wang, D., Íñiguez, J., Bellaiche, L.: Ultrafast Switching of the Electric Polarization and Magnetic Chirality in BiFeO${}_{\textrm{3}}$ by an Electric Field. Physical Review Letters 112(14), 147601 (2014) https://doi.org/10.1103/PhysRevLett.112.147601

## Extended Data

Extended data for "Designed spin-texture-lattice to control anisotropic magnon
transport in antiferromagnets"

Extended Data Figure 1: X-ray diffraction a $2\theta-\omega$ line scans of the
two samples with Pendellösung fringes around the film peaks, showing the
crystalline quality. b Reciprocal space maps of the $332_{O}$ diffraction
peak. Differences are due to the different configrations of the rhombohedral
BFO unit cell in the different domain configurations [30]. Extended Data
Figure 2: Strain and domain wall competition a,b Schematics showing the
symmetry allowed $\bm{k}$ directions at 109$\degree$ and 71$\degree$ domain
walls. c NV images showing the magnetic structure at different permutations of
strain and domain wall symmetry. When strain is high, as in DSO, $\bm{k}_{1}$
is preferred, but becomes frustrated in 109$\degree$ domain walls, leading to
competition between the type-i and type-ii cycloids. In samples with
71$\degree$ domains, it appears that $\bm{k}_{1}$ is also preferentially
chosen as the ground state. It is only through the combination of low strain
and allowed symmetry where $\bm{k}_{2}$, and the single variant state, is
preferred. Extended Data Figure 3: Antiphase boundaries The 109$\degree$
domain samples exhibit a hierarchical domain pattern characteristic of BFO.
Superimposed on the fine-scale 109$\degree$ domain ensemble with a spacing of
20-30nm, families of domains with net antiparallel in-plane polarization
components are also observed. Comparing the ferroelectric polarization from
PFM with the NV data, we observe discontinuities in the 1-D cycloid lattice
that correlate with the family domain walls (a,b). Though $\bm{k}_{2}$ does
not change across the fine-scale (20-30nm) domain walls, it does change (e.g.
$\bm{k}_{2}[101]$ to $\bm{k}_{2}^{\prime}[\bar{1}01]$) across these large
domains with a different in-plane polarization (c). Though $\bm{k}_{2}$ and
$\bm{k}_{2}^{\prime}$ in these two domains are orthogonal to each other, the
in-plane component of the cycloid does not change, giving the appearance of
discontinuities (akin to an antiphase boundary in ordered alloys) when
projected in-plane. When the material is electrically poled, these macro
domain walls are annealed out of the sample. Extended Data Figure 4: Material
and power dependence a The thermally-driven non-local conductivity changes
with the sign and magnitude of the spin Hall angle of the detector wire
material (Pt, W, and SIO), demonstrating its magnetic origin. Symbols are the
raw data and solid lines represent the time average. $P$ and $q$ are the
polarization and the magnon-propagation direction, respectively. b The output
voltage also scales with the magnitude of the spin Hall angle of the detector
material. The dotted lines correspond to orders of magnitude. Extended Data
Figure 5: Spectral dependence of magnetocrystalline anisotropy Resonant
spectra of excited spin waves extracted from phase field simulations in both
the $\bm{q}\perp\bm{k}$ and $\bm{q}\parallel\bm{k}$ cases, used to calculate
the efficiency scaling in Figure 3c. Frequency dependent cone angles,
$\Omega$, are calculated as the Fourier transforms of the time dependent
magnetization procession. Anisotropy ($K$) values are intentionally chosen
both above and below the expected range.  Extended Data Figure 6: 2D magnon
dispersion visualizations of the of the lowest IP (a) and OOP (b) bands,
showing both $\bm{q}_{x}\perp\bm{k}$ (red) and $\bm{q}_{y}\parallel\bm{k}$
(blue). The in-plane and out-of-plane mode corresponds to the spin
fluctuations within the spin-cycloid plane and perpendicular to the spin-
cycloid plane, respectively. The striking difference between these energy
bands demonstrates the anisotropic magnon-dispersion within the spin cycloid
1D lattice.  Extended Data Figure 7: Ferroelectric switching Ferroelectric
hysteresis loops (a,e) and corresponding PFM (c,d,g,h) of the poled
ferroelectric states. In both cases, devices can be electrically poled with
$E\perp$ the domain walls and conserve the domain structure (b). When
$E\parallel$ to to the domain walls, (h) however, the domains reorient into
the striped 71$\degree$ structure such that $E\perp$ the domain walls,
regardless of the starting configuration (f). This can be observed in the
hysteresis loop in d, where, starting from the as-deposited state, parallel
109$\degree$ domains are progressively transformed into perpendicular
71$\degree$ domains as the device is cycled 3-4 times.