11institutetext: BioMedIA Group, Department of Computing, Imperial College
London, UK 22institutetext: HeartFlow, USA 33institutetext: Data Science
Institute, Imperial College London, UK 44institutetext: Department of Brain
Sciences, Imperial College London, UK 55institutetext: Klinikum rechts der
Isar, Technical University of Munich, Germany 55email:
<EMAIL_ADDRESS>
# MaxStyle: Adversarial Style Composition for Robust Medical Image
Segmentation

Chen Chen(✉) 11 Zeju Li 11 Cheng Ouyang 11 Matt Sinclair 1122 Wenjia Bai
113344 Daniel Rueckert 1155

###### Abstract

Convolutional neural networks (CNNs) have achieved remarkable segmentation
accuracy on benchmark datasets where training and test sets are from the same
domain, yet their performance can degrade significantly on unseen domains,
which hinders the deployment of CNNs in many clinical scenarios. Most existing
works improve model out-of-domain (OOD) robustness by collecting multi-domain
datasets for training, which is expensive and may not always be feasible due
to privacy and logistical issues. In this work, we focus on improving model
robustness using a _single-domain_ dataset only. We propose a novel data
augmentation framework called MaxStyle, which maximizes the effectiveness of
style augmentation for model OOD performance. It attaches an auxiliary style-
augmented image decoder to a segmentation network for robust feature learning
and data augmentation. Importantly, MaxStyle augments data with improved image
style diversity and hardness, by expanding the style space with noise and
searching for the worst-case style composition of latent features via
adversarial training. With extensive experiments on multiple public cardiac
and prostate MR datasets, we demonstrate that MaxStyle leads to significantly
improved out-of-distribution robustness against unseen corruptions as well as
common distribution shifts across _multiple, different, unseen_ sites and
_unknown_ image sequences under both low- and high-training data settings. The
code can be found at https://github.com/cherise215/MaxStyle.

##  1 Introduction

Convolutional neural networks have demonstrated remarkable segmentation
accuracy on images that come from the same domain (e.g. images from the same
scanner at the same site), yet they often generalize poorly to unseen out-of-
domain (OOD) datasets, which hinders the deployment of trained models in real-
world medical applications. The performance degradation is mainly caused by
the distribution mismatch between training and test data, often originating
from the change of vendors, imaging protocols across different hospitals as
well as imperfect acquisition processes which affect image quality. To
alleviate this problem, a group of works focus on learning domain-invariant
features with multi-domain datasets [1, 2, 3, 4]. However, collecting and
labeling such large datasets can be extraordinarily time-consuming, expensive,
and not always feasible due to data privacy and ethical issues. In this work,
we consider a more realistic yet more challenging setting: domain
generalization from a single source only.

A straightforward solution to improve domain robustness is data augmentation,
which transforms and/or perturbs source domain data to resemble unseen data
shifts [5, 6, 7, 8]. Existing works mainly consider directly perturbing the
input space, which in general requires domain knowledge and expertise to
design the perturbation function [7]. Very few approaches investigate feature
augmentation possibly due to the risk of perturbing semantic content of
images. Perturbing features might change semantic attributes such as
anatomical structures, which are essential for precise segmentation. An
exception is MixStyle [9], which conducts content-preserving feature
augmentation by randomly interpolating two samples’ feature style statistics.
However, there are several limitations with MixStyle: 1) difficulty in
visualizing and interpreting augmented features in the high-dimensional space,
2) limited diversity of augmented styles due to the linear interpolation
mechanism, 3) sub-optimal effectiveness for model robustness as it does not
take the network’s vulnerability into account at training.

In this work, we propose a novel data augmentation architecture, where an
auxiliary image decoder is attached to a segmentation network to perform self-
supervised image reconstruction and style augmentation. Such a design not only
improves the interpretability of style augmented examples but also forces the
network to learn auxiliary reconstructive features for improved out-of-domain
robustness (Sec. 3.2). We further propose an enhanced style augmentation
method: MaxStyle, which maximizes the effectiveness of style augmentation for
model OOD performance (Sec. 3.3). Different from MixStyle, we expand the style
space with additional noise and search for the worst-case style composition
for the segmentor via adversarial training. Through extensive experiments, we
show that MaxStyle achieves competitive robustness against corruptions and
improves robustness against common distribution shifts across different sites
and different image sequences (Sec. 4.3).

##  2 Related work

Style transfer: Style transfer modifies the visual style of an image while
preserving its semantic content. Previous works on style transfer mainly focus
on designing advanced generative adversarial networks (GAN) for improved image
diversity and fidelity under the assumption that a large-scale dataset with
diverse image styles is available [10, 11]. Recent findings show that domain
shift is closely related to image style changes across different domains [9,
12] and can be alleviated by increasing the diversity of training image styles
[13, 9, 14, 15, 16, 17]. One such successful example is MixStyle [9], which
generates ‘novel’ styles via simply linearly mixing style statistics from two
arbitrary training instances from the same domain at feature level. DSU [15]
augments feature styles with random noise to account for potential style
shifts. Compared with MixStyle and DSU, our proposed method, MaxStyle does not
only span a _larger_ area in the feature style space but also covers _harder_
cases.
Adversarial data augmentation: Adversarial data augmentation focuses on
generating data samples that fool the network to predict the wrong class [18,
19]. Such an approach can improve robustness against adversarial and natural
corruptions [18, 20], and has shown promise for improved model generalization
[21, 22, 23, 7, 8, 17]. Most methods apply adversarial perturbation to the
input space to improve unseen domain performance, e.g., adversarial noise [24,
22, 23], and adversarial bias field [7]. Fewer explore feature space
perturbation. Huang et al. proposed to adversarially drop dominant features
(RSC) during training as a way to improve cross-domain performance [25]. Chen
et al. proposed to project masked latent features back to input space in an
adversarial manner for improved interpretability and effectiveness [8]. Our
method follows this feature-to-input space data augmentation mechanism, but
restricts the adversarial perturbation at the style level, generating _style-
varying, shape-preserving_ hard examples.

##  3 Method

###  3.1 Preliminaries: MixStyle

As mentioned before, MixStyle [9] is a feature augmentation method for
improving model generalization and robustness, which perturbs feature style
statistics to regularize network training. Formally, let
$\mathbf{f}_{i},\mathbf{f}_{j}\in\mathbb{R}^{c\times h\times w}$ be
$c$-dimensional feature maps extracted at a certain CNN layer for image
$\boldsymbol{x}_{i}$ and image $\boldsymbol{x}_{j}$, respectively. MixStyle
performs style augmentation for $\mathbf{f}_{i}$ by first normalizing it with
its channel-wise means and standard deviations
$\mu(\mathbf{f}_{i}),\sigma(\mathbf{f}_{i})\in\mathbb{R}^{c}$ :
$\overline{\mathbf{f}}_{i}=\frac{\mathbf{f}_{i}-\mu(\mathbf{f}_{i})}{\sigma(\mathbf{f}_{i})}$
and then transforming the feature with a linear combination of style
statistics $i:\\{\sigma(\mathbf{f}_{i}),\mu(\mathbf{f}_{i})\\}$ and
$j:\\{\sigma(\mathbf{f}_{j}),\mu(\mathbf{f}_{j})\\}$. This process can be
defined as:

$\operatorname{MixStyle}(\mathbf{f}_{i})=\boldsymbol{\gamma}_{mix}\odot\overline{\mathbf{f}}_{i}+{\boldsymbol{\beta}_{mix}},\\\
\boldsymbol{\gamma}_{mix}=\lambda_{mix}\sigma(\mathbf{f}_{i})+(1-\lambda_{mix})\sigma(\mathbf{f}_{j}),\,\boldsymbol{\beta}_{mix}=\lambda_{mix}\mu(\mathbf{f}_{i})+(1-\lambda_{mix})\mu(\mathbf{f}_{j}),$
(1)

where $\odot$ denotes element-wise multiplication; $\lambda_{mix}$ is a
coefficient controlling the level of interpolation, randomly sampled from
$[0,1]$. Acting as a plug-and-play module, MixStyle can be inserted between
any CNN layers. It is originally designed as an explicit regularization method
for a standard encoder-decoder structure, which perturbs shallow features
(e.g., features from the first three convolutional blocks) in an image encoder
$E_{\theta}$ parametersied by $\theta$, see Fig. 1. A segmentation decoder
$D_{\phi_{s}}$ then performs prediction with the perturbed high-level
representation $\hat{\boldsymbol{z}}$. The whole network is optimized to
minimize the segmentation loss
$\mathcal{L}_{seg}(D_{\phi_{s}}(\hat{\boldsymbol{z}}),\mathbf{y})$, supervised
by the ground-truth $\mathbf{y}$.

Figure 1: A) the original use of MixStyle as feature augmentation-based
regularization method with a standard encoder-decoder structure. B) MixStyle
applied to regularize a dual-branch network with an auxiliary image decoder
$D_{\phi_{i}}$ attached for image reconstruction. C) We propose to apply
MixStyle in the auxiliary decoder $D_{\phi_{i}}$ to generate stylized images
for feature-to-input space data augmentation instead (MixStyle-DA), which
leads to improved model robustness across different OOD test data, compared to
A, B (sebottom). GT: ground-truth.

###  3.2 Robust feature learning and improved interpretability with auxiliary
image decoder

In this work, for improved OOD robustness and better interpretability of style
augmentation, we propose to adapt the standard encoder-decoder to a dual-
branch network with an auxiliary image decoder $D_{\phi_{i}}$ attached, see
Fig. 1B,C. The network is supervised with an additional image reconstruction
loss $\mathcal{L}_{rec}(\hat{\boldsymbol{x}},\boldsymbol{x})$, allowing itself
to exploit both complementary image content features and task-specific shape
features for the segmentation task [8]. We further propose to insert the style
augmentation layers in the image decoder $D_{\phi_{i}}$ (Fig. 1C) rather than
in the encoder $E_{\theta}$ (Fig. 1B), allowing to generate diverse stylized
images with the same high-level semantic features $\boldsymbol{z}$ for
_direct_ data augmentation. Such a design also improves the interpretability
of feature style augmentation. More importantly, our experimental results show
that $C>B>A$ in terms of OOD robustness, see Fig. 1 (bottom) and Fig. S1
(supplementary). The segmentation network trained w/ approach C is more robust
against corruptions and unseen domain shifts across different sites and
different image sequences. We name this new method as ‘MixStyle-DA’ to
differentiate it from the original one. We believe this data augmentation-
based approach is preferred for model robustness as it tackles model over-
fitting from the root, limited training data.

###  3.3 MaxStyle

Figure 2: MaxStyle overview. a) MaxStyle reconstructs $\boldsymbol{x}_{i}$
with augmented feature styles via style mixing and noise perturbation in the
image decoder $D_{\phi_{i}}$. Adversarial training is applied, in order to
search for ‘harder’ style composition to fool the segmentation network
($E_{\theta}\circ D_{\phi_{s}}$). b) MaxStyle generates a style-optimized
image $\hat{\boldsymbol{x}}^{*}$, which fools the network to under-segment
($\hat{\mathbf{p}}^{*}$). The anatomical structures remain almost unchanged
with _high_ correlation (Corr) between two images’ gradient fields:
$\nabla{\boldsymbol{x}}$, $\nabla\hat{\boldsymbol{x}}^{*}$.

On top of the dual-branch architecture with the data augmentation strategy
presented above (Fig. 1C), we propose MaxStyle, a novel method that consists
of both style mixing and noise perturbation, improving the diversity and
sample hardness. As visualized in Fig. 2(a), we introduce additional style
noise to expand the style space and apply adversarial training in order to
find optimal linear coefficients as well as to generate adversarial style
noise for effective data augmentation. Specifically, given feature
$\mathbf{f}_{i}$ extracted at a certain CNN layer in the image decoder
$D_{\phi_{i}}$ with $\boldsymbol{x}_{i}$ as input, MaxStyle augments
$\mathbf{f}_{i}$ via:

$\displaystyle\operatorname{MaxStyle}(\mathbf{f}_{i})=(\boldsymbol{\gamma}_{mix}+\Sigma_{\gamma}\cdot\boldsymbol{\epsilon}_{\gamma})\odot\overline{\mathbf{f}}_{i}+(\boldsymbol{\beta}_{mix}+\Sigma_{\beta}\cdot\boldsymbol{\epsilon}_{\beta}),\;$
(2)

where the normalized feature $\overline{\mathbf{f}_{i}}$ is transformed with
mixed styles $\boldsymbol{\gamma}_{mix},\boldsymbol{\beta}_{mix}$ (Eq. 1) plus
additional style noise
$\Sigma_{\gamma}\cdot\boldsymbol{\epsilon}_{\gamma},\Sigma_{\beta}\cdot\boldsymbol{\epsilon}_{\beta}\in\mathbb{R}^{c}$
to better explore the space of possible unknown domain shifts. In order to
bound the style noise within a reasonable range, similar to [15], we sample
the style noises from a re-scaled Gaussian distribution with variance
$\Sigma_{\gamma},\Sigma_{\beta}\in\mathbb{R}^{c}$ estimated from a batch of
$B$ instances’ style statistics (including $\boldsymbol{x}_{i}$):
$\Sigma_{\gamma}=\sigma^{2}(\\{\sigma(\mathbf{f}_{j})\\}_{j={1...i,...B}})\;,\Sigma_{\beta}=\sigma^{2}(\\{\mu(\mathbf{f}_{j})\\}_{j={1...i,...B}}),\boldsymbol{\epsilon}_{\gamma},\boldsymbol{\epsilon}_{\beta}\sim\mathcal{N}(\mathbf{0},\mathbf{1})$
111The re-parameterization trick is applied here for ease of follow-up
optimization..
Style optimization via adversarial training: The generated noise
$\boldsymbol{\epsilon}_{\gamma},\boldsymbol{\epsilon}_{\beta}$ and the style
mixing coefficient $\lambda_{mix}$ are then updated by maximizing the
segmentation loss $\mathcal{L}_{seg}$ so that the synthesized image
$\hat{\boldsymbol{x}}=D_{\phi_{i}}(E_{\theta}(\boldsymbol{x});\boldsymbol{\epsilon}_{\gamma},\boldsymbol{\epsilon}_{\beta},\lambda_{mix})$222For
simplicity, we omit non-learnable parameters such as the sampling operator to
choose instance $\boldsymbol{x}_{j}$ from a batch for style mixing. could fool
the network to produce an incorrect prediction
$\hat{\mathbf{p}}=D_{\phi_{s}}(E_{\theta}(\hat{\boldsymbol{x}}))$. Gradient
ascent is employed to update the underlying style-related parameters:

$\displaystyle\boldsymbol{\epsilon}_{\gamma}\leftarrow\boldsymbol{\epsilon}_{\gamma}+\alpha\nabla_{\boldsymbol{\epsilon}_{\gamma}}{\mathcal{L}_{seg}(\hat{\mathbf{p}},\mathbf{y})},\;\;\boldsymbol{\epsilon}_{\beta}\leftarrow\boldsymbol{\epsilon}_{\beta}+\alpha\nabla_{\boldsymbol{\epsilon}_{\beta}}\mathcal{L}_{seg}(\hat{\mathbf{p}},\mathbf{y}),$
(3)
$\displaystyle\boldsymbol{\lambda}_{mix}\leftarrow\operatorname{Clip}_{[0,1]}(\boldsymbol{\lambda}_{mix}+\alpha\nabla_{\boldsymbol{\lambda}_{mix}}{\mathcal{L}_{seg}(\hat{\mathbf{p}},\mathbf{y})}).$
(4)

Here $\alpha$ denotes the step size. We clip the value of $\lambda_{mix}$ to
ensure it lies in $[0,1]$. To summarize, training with MaxStyle augmentation
can be then written as minimax optimization using the training set
$\mathcal{D}$:

$\displaystyle\min_{\theta,\phi_{i},\phi_{s}}$
$\displaystyle\operatorname{\mathbb{E}}_{\boldsymbol{x},\mathbf{y}\sim\mathcal{D}}\underbrace{\mathcal{L}_{seg}(D_{\phi_{s}}(E_{\theta}({\boldsymbol{x}})),\mathbf{y})+\mathcal{L}_{rec}(D_{\phi_{i}}(E_{\theta}(\boldsymbol{x})),\boldsymbol{x})}_{{\color[rgb]{.5,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,.5,.5}\pgfsys@color@gray@stroke{.5}\pgfsys@color@gray@fill{.5}\boldsymbol{x}\;\small{\textit{as
input for network optimization}}}}$ (5)
$\displaystyle+\underbrace{\mathcal{L}_{seg}(D_{\phi_{s}}(E_{\theta}(\hat{\boldsymbol{x}}^{*})),\mathbf{y})+\mathcal{L}_{rec}(D_{\phi_{i}}(E_{\theta}(\hat{\boldsymbol{x}}^{*})),\boldsymbol{x})}_{{\color[rgb]{.5,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,.5,.5}\pgfsys@color@gray@stroke{.5}\pgfsys@color@gray@fill{.5}\hat{\boldsymbol{x}}^{*}\;\small{\textit{as
input for network optimization}}}}$ $\displaystyle s.t.$
$\displaystyle\lambda_{mix}^{*},\boldsymbol{\epsilon}_{\gamma}^{*},{\boldsymbol{\epsilon}_{\beta}}^{*}=\operatorname*{arg\,max}\mathcal{L}_{seg}(\hat{\mathbf{p}},\mathbf{y}){{\color[rgb]{.5,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{.5,.5,.5}\pgfsys@color@gray@stroke{.5}\pgfsys@color@gray@fill{.5}\scriptsize{\textit{\;\;(adversarial
style optimization)}}}}.$

The whole network is optimized using both input image $\boldsymbol{x}$ and its
style augmented image $\hat{\boldsymbol{x}}^{*}$, minimizing the multi-task
loss: segmentation loss $\mathcal{L}_{seg}$ and image reconstruction loss
$\mathcal{L}_{rec}$. Here
$\hat{\boldsymbol{x}}^{*}=D_{\phi_{i}}(E_{\theta}(\boldsymbol{x});\lambda_{mix}^{*},\boldsymbol{\epsilon}_{\gamma}^{*},\boldsymbol{\epsilon}_{\beta}^{*})$
is generated using optimized style parameters. An example of MaxStyle
generated images is shown in Fig. 2b. The network is fooled to under-segment
the style-augmented image $\hat{\boldsymbol{x}}^{*}$ although the underlying
target structures remain almost the same. More examples are provided in the
supplementary. Of note, MaxStyle is suitable for general segmentation networks
thanks to its plug-and-play nature and the attached decoder can be removed at
test time.

##  4 Experiments and results

###  4.1 Data: Cardiac MR segmentation datasets

a) Single source domain: The public ACDC dataset [26] is used for network
training and intra-domain test, which contains 100 subjects (bSSFP sequences)
collected from a single site. We randomly split the dataset into 70/10/20 for
training/validation/intra-domain test.To simulate common clinical scenarios
where the training data is far less than testing data, we randomly selected 10
subjects (10 out of 70) for training and repeated experiments three times. We
also validated our method using all 70 training subjects following [8].
b) Multiple unseen test domains: We collated several _public_ datasets where
each capture different types of distribution shift or quality degradation for
comprehensive OOD performance evaluation. They are 1) ACDC-C for robustness
against imaging artefacts evaluation. Following [8], we synthesized a
corrupted version of the ACDC test set where each test image has been randomly
corrupted by one of four, different types of MR artefacts, three times for
each artefact type, using TorchIO toolkit [27]; ACDC-C contains
$240\;(20\times 3\times 4)$ subjects in total; 2) a large _cross-site_ test
set with real-world distribution variations. It consists of 195 subjects
(bSSFP sequences) across four, different sites, which are from two public
challenge datasets: M&Ms [28], MSCMRSeg [29]; 3) cross-sequence test set,
which consists of 45 LGE sequences from MSCMRSeg [29]. Detailed information
can be found in the Supplementary. We employed the image pre-processing and
standard data augmentation (incl. common photo-metric and geometric image
transformations) described in [8] as our default setting.

###  4.2 Implementation and experiment set-up

We adopted the dual-branch network presented in [8] as our backbone, which has
demonstrated state-of-the-art robustness on the cardiac segmentation task. We
applied MaxStyle layers to shallow features in $D_{\phi_{i}}$ for optimal
performance, i.e. inserted after each of the last three convolutional blocks.
Each layer is activated at a probability of 0.5 and batch features at each
layer are randomly shuffled for style mixing to trade-off diversity and
strength as suggested by [9]. For the adversarial style optimization, we
randomly sample $\lambda_{mix}$ from the uniform distribution
$\mathcal{U}_{[0,1]}$ and $\boldsymbol{\epsilon}_{\gamma,\beta}$ from
$\mathcal{N}(\mathbf{0},\mathbf{1})$. Adam optimizer was adopted for both
style optimization and network optimization. We empirically set $\alpha=0.1$
with 5 iterations, yielding sufficient improvement. For network optimization,
we set learning rate $=1e^{-4}$ and batch size $=20$, following [8]. For
cardiac small training set, we trained the network for 1,500 epochs to ensure
convergence. For larger training data, 600 epochs were sufficient. We employed
mean squared error loss for $\mathcal{L}_{rec}$ and cross-entropy loss for
$\mathcal{L}_{seg}$. We compare our method with the _baseline_ (w/o MaxStyle
augmentation) and competitive random and adversarial augmentation methods
using the same network and the same multi-task loss, including: a) input space
augmentation: RandConv [6], adversarial noise (Adv Noise) [24], and
adversarial bias field (Adv Bias) [7]; b) feature augmentation: RSC [25],
MixStyle [9], DSU [15]; c) feature-to-input space data augmentation via
decoder $D_{\phi_{i}}$: latent space masking (LSM) [8]. All methods were
implemented in PyTorch with their recommended set-ups.

###  4.3 Results

Results of models trained with cardiac low data regime and high data regime
are provided Table 1. We also plot the segmentation results in Fig. 3 for
visual comparison. From the results, we see that MaxStyle outperforms _all_
baseline methods across _all_ OOD test sets with different types of domain
shifts, providing the largest improvements in both low- and high-data training
regimes (+25%, +11%). In particular, MaxStyle significantly improves the
segmentation performance on the most challenging cross-sequence dataset with
the largest domain shift. By generating images with challenging styles during
training, MaxStyle forces the model to be more shape-biased rather than
texture/style-biased, which is beneficial for model robustness [30]. To
provide additional insights of MaxStyle, we also provide t-SNE visualization
of latent feature embeddings in the supplementary. We also found that the
model trained w/ MaxStyle using 10 ACDC subjects even achieves higher OOD
performance than the _baseline_ model trained w/ 70 subjects (0.7420 vs
0.7287), suggesting MaxStyle’s superior data efficiency. In addition to
cardiac segmentation, we also validate our method on public prostate
segmentation datasets [2, 31, 3, 32, 33, 34] to test the generality of our
method for medical image segmentation. MaxStyle also achieves the top OOD
performance against the competitive methods across _six, different, unseen_
test sites. Datasets details and results are shown in the supplementary.

Table 1: Cardiac segmentation performance across multiple unseen test sets.
IID: _ACDC_ test set performance. OOD: average performance across unseen OOD
test sets: ACDC-C, cross-site, and cross-sequence. Reported values are average
Dice scores across different test sets.

Method | Low-data regime (10 training subjects) | High-data regime (70 training subjects)
---|---|---
IID | OOD | ACDC-C | Cross-site | Cross-sequence | IID | OOD | ACDC-C | Cross-site | Cross-sequence
_baseline_ | 0.8108 | 0.5925 (-) | 0.6788 | 0.6741 | 0.4244 | 0.8820 | 0.7287 (-) | 0.7788 | 0.8099 | 0.5974
+RandConv | 0.8027 | 0.6033 (+2%) | 0.6872 | 0.6717 | 0.4510 | 0.8794 | 0.7371 (+1%) | 0.7967 | 0.7747 | 0.6399
+Adv Noise | 0.8080 | 0.6104 ($+3\%$) | 0.6868 | 0.6804 | 0.4641 | 0.8852 | 0.7291 (+0%) | 0.7808 | 0.8095 | 0.5970
+Adv Bias | 0.8114 | 0.6594 (+11%) | 0.6937 | 0.7517 | 0.5327 | 0.8865 | 0.7450 (+2%) | 0.7839 | 0.8250 | 0.6262
+RSC | 0.8169 | 0.6252 (+6%) | 0.7010 | 0.7015 | 0.4729 | 0.8844 | 0.7379 (+1%) | 0.7757 | 0.8115 | 0.6267
+MixStyle | 0.8024 | 0.6263 (+6%) | 0.7002 | 0.6984 | 0.4804 | 0.8788 | 0.7651 (+5%) | 0.7961 | 0.8130 | 0.6863
+DSU | 0.8127 | 0.6088 (+3%) | 0.6972 | 0.7008 | 0.4285 | 0.8784 | 0.7553 (+4%) | 0.8031 | 0.8126 | 0.6503
+LSM | 0.7899 | 0.6462 (+9%) | 0.7221 | 0.7015 | 0.5149 | 0.8754 | 0.7579 (+4%) | 0.8213 | 0.8025 | 0.6500
+MaxStyle | $0.8104^{\sim}$ | 0.7420 (+25%)¶ | 0.7745 | 0.7645 | 0.6869 | 0.8727§ | 0.8125 (+11%)¶ | 0.8408 | 0.8294 | 0.7673

  * $\sim$

:p-value$>0.5$

  * §

p-value$>1e^{-4}$

  * ¶

p-value$\ll 1e^{-4}$ (compared to Baseline results)

Figure 3: Qualitative results under the cardiac low-data regime. GT: ground-
truth. Table 2: The effect of expanding style search space (style noise &
style mixing) and applying adversarial style optimization (AdvOpt) for domain
generalization. Reported values are mean (std) of Dice scores across all _ten_
cardiac test (sub)sets. Experiments were performed under the cardiac high-data
training setting.

w/o Style noise | w/o Style mixing | w/o AdvOpt | w/o AdvOpt & Style mixing | w/o AdvOpt & Style noise (MixStyle-DA) | MaxStyle
---|---|---|---|---|---
0.8058 (0.0467) | 0.8295 (0.0395) | 0.8188 (0.0478) | 0.8121 (0.0516) | 0.7981 (0.0652) | 0.8321(0.0339)

Ablation study: We highlight the importance of increased style diversity via
composing style mixing and style noise as well as improved style hardness with
adversarial training in the ablation study. Results are shown in Table 2. It
can be observed that removing each component leads to performance degradation.

##  5 Discussion and conclusion

We introduced MaxStyle, which is a powerful plug-in data augmentation module
for single domain generalization. It attaches an auxiliary style-augmented
image decoder to a segmentation network for image reconstruction and style
augmentation. MaxStyle is capable of generating stylized images with improved
diversity and hardness, forcing the network to learn robust features for
domain generalization. From the causal perspective [35], the image decoder
with MaxStyle can be viewed as a latent data generation process with multi-
scale style interventions, conditioned on the extracted high-level causal
variables (e.g., content and style). By explicitly feeding the network with
images of diverse ‘hard’ styles, the spurious correlation between image style
and labels are weakened. The network thus gains the ability to resist unseen
distributional shifts. We also highlighted the importance of combining both
reconstructive features and discriminant features for robust segmentation
(Sec. 3.2), which is in line with classical robust subspace methods [36]. We
validated MaxStyle’s efficacy on both cardiac datasets and prostate datasets
across various unseen test sets, demonstrating its superiority over
competitive methods under both low- and high-data training regimes. It is
worth mentioning here that our method does not necessarily sacrifice the IID
performance. On the prostate segmentation task, our method can significantly
improve the IID performance compared to the baseline method (0.8597 vs 0.8277,
see Table S3). For the cardiac segmentation task, the IID performance slightly
drops (e.g, 0.8104 vs 0.8108). We hypothesize that the IID performance
degradation is task and dataset-dependent. Like other adversarial data
augmentation methods [18, 24], the main limitation of MaxStyle is that the
improved robustness comes at the cost of longer training time due to the
adversarial optimization procedure, e.g. increased by a factor of $\sim 1.8$
in our case. We hope that MaxStyle will enable more data-efficient, robust and
reliable deep models. Integrating MaxStyle into composite data augmentation
frameworks [37] would be an interesting direction.

####  5.0.1 Acknowledgment:

This work was supported by two EPSRC Programme Grants (EP/P001009/1,
EP/W01842X/1) and the UKRI Innovate UK Grant (No.104691).

## References

  * [1] Qian Tao et al.  “Deep Learning–based Method for Fully Automatic Quantification of Left Ventricle Function from Cine MR Images: A Multivendor, Multicenter Study”  In _Radiology_ 290.1, 2019, pp. 180513  DOI: 10.1148/radiol.2018180513
  * [2] Quande Liu, Qi Dou, Lequan Yu and Pheng Ann Heng  “Ms-net: Multi-site network for improving prostate segmentation with heterogeneous mri data”  In _TMI_ IEEE, 2020
  * [3] Quande Liu, Qi Dou and Pheng Ann Heng  “Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains”  In _MICCAI_ , 2020
  * [4] Qi Dou, Daniel C Castro, Konstantinos Kamnitsas and Ben Glocker  “Domain Generalization via Model-Agnostic Learning of Semantic Features”  In _NeurIPS 2019_ , 2019, pp. 6447–6458  URL: http://arxiv.org/abs/1910.13580
  * [5] Jindong Wang et al.  “Generalizing to Unseen Domains: A Survey on Domain Generalization”  In _IJCAI 2021_ ijcai.org, 2021, pp. 4627–4635  DOI: 10.24963/ijcai.2021/628
  * [6] Zhenlin Xu et al.  “Robust and Generalizable Visual Representation Learning via Random Convolutions”  In _ICLR 2021_ OpenReview.net, 2021
  * [7] Chen Chen et al.  “Realistic Adversarial Data Augmentation for MR Image Segmentation”  In _MICCAI 2020_ Springer International Publishing, 2020, pp. 667–677  DOI: 10.1007/978-3-030-59710-8˙65
  * [8] Chen Chen et al.  “Cooperative Training and Latent Space Data Augmentation for Robust Segmentation”  In _MICCAI 2021_ Springer International Publishing, 2021
  * [9] Kaiyang Zhou, Yongxin Yang, Yu Qiao and Tao Xiang  “Domain Generalization with MixStyle”  In _ICLR 2021_ OpenReview.net, 2021  URL: https://openreview.net/forum?id=6xHJ37MVxxp
  * [10] Tero Karras, Samuli Laine and Timo Aila  “A Style-Based Generator Architecture for Generative Adversarial Networks”  In _TPAMI_ 43.12, 2021, pp. 4217–4228  DOI: 10.1109/TPAMI.2020.2970919
  * [11] Xun Huang, Ming-Yu Liu, Serge J. Belongie and Jan Kautz  “Multimodal Unsupervised Image-to-Image Translation”  In _ECCV 2018_ 11207, Lecture Notes in Computer Science  Springer, 2018, pp. 179–196  DOI: 10.1007/978-3-030-01219-9“˙11
  * [12] Yanghao Li et al.  “Adaptive Batch Normalization for practical domain adaptation”  In _Pattern recognition_ 80 Elsevier Ltd, 2018, pp. 109–117
  * [13] Philip T.. Jackson et al.  “Style Augmentation: Data Augmentation via Style Randomization”  In _CVPR Workshops 2019_ Computer Vision Foundation / IEEE, 2019, pp. 83–92  URL: http://openaccess.thecvf.com/content%5C_CVPRW%5C_2019/html/Deep%5C_Vision%5C_Workshop/Jackson%5C_Style%5C_Augmentation%5C_Data%5C_Augmentation%5C_via%5C_Style%5C_Randomization%5C_CVPRW%5C_2019%5C_paper.html
  * [14] Rikiya Yamashita et al.  “Learning domain-agnostic visual representation for computational pathology using medically-irrelevant style transfer augmentation”  In _IEEE transactions on medical imaging_ PP, 2021
  * [15] Xiaotong Li et al.  “Uncertainty Modeling for Out-of-Distribution Generalization”  In _International Conference on Learning Representations_ , 2022  URL: https://openreview.net/forum?id=6HN7LHyzGgC
  * [16] Sophia J. Wagner et al.  “Structure-Preserving Multi-domain Stain Color Augmentation Using Style-Transfer with Disentangled Representations”  In _MICCAI 2021_ 12908 Springer, 2021, pp. 257–266  DOI: 10.1007/978-3-030-87237-3“˙25
  * [17] Zhun Zhong, Yuyang Zhao, Gim Hee Lee and Nicu Sebe  “Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation”  In _Under Review_ , 2021
  * [18] Aleksander Madry et al.  “Towards Deep Learning Models Resistant to Adversarial Attacks”  In _International Conference on Learning Representations_ , 2017  arXiv: http://arxiv.org/abs/1706.06083
  * [19] Ian J Goodfellow, Jonathon Shlens and Christian Szegedy  “Explaining and Harnessing Adversarial Examples”  In _ICLR 2015_ , 2015  URL: http://arxiv.org/abs/1412.6572
  * [20] Justin Gilmer, Nicolas Ford, Nicholas Carlini and Ekin D. Cubuk  “Adversarial Examples Are a Natural Consequence of Test Error in Noise”  In _ICML 2019_ 97 PMLR, 2019, pp. 2280–2289
  * [21] Cihang Xie et al.  “Adversarial Examples Improve Image Recognition”  In _CVPR 2020_ Computer Vision Foundation / IEEE, 2020, pp. 816–825  DOI: 10.1109/CVPR42600.2020.00090
  * [22] Riccardo Volpi et al.  “Generalizing to Unseen Domains via Adversarial Data Augmentation”  In _NeurIPS 2018_ , 2018, pp. 5339–5349  URL: http://papers.nips.cc/paper/7779-generalizing-to-unseen-domains-via-adversarial-data-augmentation
  * [23] Fengchun Qiao, Long Zhao and Xi Peng  “Learning to learn single domain generalization”  In _CVPR 2020_ , 2020, pp. 12556–12565
  * [24] Takeru Miyato, Shin-Ichi Maeda, Masanori Koyama and Shin Ishii  “Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning”  In _TPAMI_ , 2018  URL: http://arxiv.org/abs/1704.03976
  * [25] Zeyi Huang, Haohan Wang, Eric P Xing and Dong Huang  “Self-challenging Improves Cross-Domain Generalization”  In _ECCV 2020_ Springer International Publishing, 2020, pp. 124–140
  * [26] Olivier Bernard et al.  “Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?”  In _TMI_ 37.11, 2018, pp. 2514–2525  DOI: 10.1109/TMI.2018.2837502
  * [27] Fernando Pérez-García, Rachel Sparks and Sébastien Ourselin  “TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning”  In _Computer Methods and Programs in Biomedicine_ , 2021, pp. 106236  DOI: https://doi.org/10.1016/j.cmpb.2021.106236
  * [28] Victor M Campello et al.  “Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge”  In _IEEE Transactions on Medical Imaging_ , 2021  DOI: 10.1109/TMI.2021.3090082
  * [29] Xiahai Zhuang et al.  “Cardiac Segmentation on Late Gadolinium Enhancement MRI: A Benchmark Study from Multi-Sequence Cardiac MR Segmentation Challenge”  In _arXiv Preprint_ , 2020  arXiv:2006.12434 [eess.IV]
  * [30] Robert Geirhos et al.  “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness”  In _International Conference on Learning Representations_ , 2018, pp. 1–20
  * [31] Michela Antonelli et al.  “The Medical Segmentation Decathlon”  In _Arxiv_ , 2021  arXiv:2106.05735
  * [32] Bloch N., Madabhushi A., Huisman H. and Freymann J.  “NCI-ISBI 2013 Challenge: Automated Segmentation of Prostate Structures” https://www.cancerimagingarchive.net, 2015
  * [33] Guillaume Lemaıtre et al.  “Computer-Aided Detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: a review”  In _Computers in biology and medicine_ 60, 2015, pp. 8–31
  * [34] Geert Litjens et al.  “Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge”  In _Medical image analysis_ 18.2, 2014, pp. 359–373
  * [35] Daniel C Castro, Ian Walker and Ben Glocker  “Causality matters in medical imaging”  In _Nature Communications_ 11.1, 2020, pp. 3673  DOI: 10.1038/s41467-020-17478-w
  * [36] Sanja Fidler, Danijel Skocaj and Ales Leonardis  “Combining reconstructive and discriminative subspace methods for robust classification and regression by subsampling”  In _TPAMI_ 28.3, 2006, pp. 337–350
  * [37] Dan Hendrycks et al.  “AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty”  In _ICLR 2020_ URL: https://openreview.net/forum?id=S1gmrxHFvB

Table S1: Cardiac segmentation datasets. bSSFP: balanced steady-state free
precession; LGE:late gadolinium-enhanced.

| Single source domain | Multiple unseen test domains
---|---|---
| ACDC | ACDC w/ corruptions (ACDC-C) | Cross-site datasets | Cross-sequence
Dataset | Train/val/intra-domain test | Bias field | Ghosting | Spiking | Motion | Site A | Site B | Site C | Site D | LGE
# subjects |  | Low data regime: 10/10/20
---
High data regime: 70/10/20
60 | 60 | 60 | 60 | 75 | 50 | 25 | 45 | 45
Sequence | bSSFP | bSSFP | bSSFP | LGE
Data source | ACDC [26] | ACDC [26]+TorchIO [27] | M&Ms [28] | M&Ms [28] | M&Ms [28] | MSCMRSeg [29] | MSCMRSeg [29]

Figure S1: Cardiac intra-domain and out-of-domain segmentation performance of
networks trained with the three approaches presented in Fig.1. The feature-to-
input space data augmentation-based (C) clearly outperforms feature
augmentation based ones (A,B) on the out-of-domain test data, especially on
the most challenging _cross-sequence_ test set. Models were trained under the
cardiac low training data regime.  Figure S2: MaxStyle generates images with
diverse appearances (brightness, textures) and different image quality, e.g.,
w/o or w/ blurring, fabric patterns to fool the segmentor to produce incorrect
predictions. Figure S3: t-SNE feature visualization on the _most challenging_
unseen cardiac test set (cross-sequence). MaxStyle leads to well separated
compact latent embeddings for different classes (in different colors),
suggesting its improved robustness against unseen domain shifts. Features
analyzed are from the last hidden layers of models trained under the cardiac
low training data setting. Table S2: Prostate segmentation datasets. The
_public_ prostate segmentation dataset from the Medical Decathlon challenge
[31] is used for training and intra-domain test, where images are all from a
_single_ site. A prostate multi-site dataset [2, 3] collated from three
independent public challenge datasets is used for cross-site robustness test.
All images have been preprocessed to have the same spatial resolution:
$0.625\times 0.625\times 3.6\leavevmode\nobreak\ \textit{mm}^{3}$, the same
image size (via padding/cropping): $224\times 224$, and the same intensity
range: $[0,1]$.

| Single source domain | Multi-site unseen test domains
---|---|---
Dataset | Train/val/intra-domain test | Site A | Site B | Site C | Site D | Site E | Site F
# subjects | 22/3/7 | 30 | 30 | 19 | 13 | 12 | 12
Data source | Medical Decathlon [31] | NCI-ISBI-2013 [32] | NCI-ISBI-2013 [32] | I2CVB [33] | PROMISE12 [34]

Table S3: Evaluation results on the prostate intra-domain and unseen cross-
site test sets. Models were all trained using the same setting for fair
comparison, e.g., the same backbone dual-branch network, the same multi-task
loss with an Adam optimizer (batch size$=20$ learning rate$=1e^{-4}$, 600
epochs).

Method | IID | OOD | A | B | C | D | E | F
---|---|---|---|---|---|---|---|---
Baseline | 0.8277 | 0.7017 (-) | 0.8644 | 0.5517 | 0.8240 | 0.8269 | 0.5640 | 0.5789
RandConv | 0.8469 | 0.7476 (+7%) | 0.8828 | 0.6621 | 0.8554 | 0.8789 | 0.5992 | 0.6074
Adv Noise | 0.8633 | 0.7325 (+4%) | 0.8939 | 0.5887 | 0.8480 | 0.8698 | 0.5877 | 0.6071
Adv Bias | 0.8462 | 0.8069 (+15%) | 0.9026 | 0.7363 | 0.8195 | 0.9198 | 0.6679 | 0.7953
RSC | 0.8582 | 0.7449 (+6%) | 0.8864 | 0.6224 | 0.8502 | 0.8646 | 0.6112 | 0.6343
MixStyle | 0.8403 | 0.7260 (+3%) | 0.8898 | 0.5438 | 0.8154 | 0.8405 | 0.6369 | 0.6295
DSU | 0.8311 | 0.7256 (+3%) | 0.8821 | 0.5924 | 0.8224 | 0.8304 | 0.5856 | 0.6403
LSM | 0.8439 | 0.8209 (+17%) | 0.8941 | 0.7065 | 0.8763 | 0.9021 | 0.7827 | 0.7635
MaxStyle | 0.8597 |  0.8439 (+20%) | 0.9054 | 0.8362 | 0.8864 | 0.9212 | 0.7160 | 0.7983

Table S4: The effect of the number of iterations for style optimization
($n_{iter}$). Reported values are mean (std) Dice scores over ten test (sub)
sets using models trained under the high-data training setting. MaxStyle is
_stable_ with respect to different iterations. The segmentation performance
reaches the best results when performing $5$ iterations (our default setting)
for style optimization.

|  $n_{iter}$=0 |  $n_{iter}$=1 |  $n_{iter}$=3 |  $n_{iter}$=5 |  $n_{iter}$=7
---|---|---|---|---|---
Mean (std) Dice scores | 0.8188 (0.0478) | 0.8256 (0.0449) | 0.8297 (0.0365) | 0.8321 (0.0339) | 0.8317 (0.0364)


# Resonant phonon-magnon interactions in free-standing metal-ferromagnet
multilayer structures

Urban Vernik 1<EMAIL_ADDRESS>Alexey M. Lomonosov2 Vladimir S.
Vlasov1 Leonid N. Kotov1 Dmitry A. Kuzmin1 Igor V. Bychkov1 Paolo Vavassori3,4
Vasily V. Temnov1<EMAIL_ADDRESS>1LSI, Ecole Polytechnique,
CEA/DRF/IRAMIS, CNRS, Institut Polytechnique de Paris, F-91128, Palaiseau,
France  2University of Applied Sciences, 77723 Gengenbach, Germany  3CIC
nanoGUNE, E-20018 Donostia-San Sebastian, Spain  4IKERBASQUE, Basque
Foundation for Science, E-48013 Bilbao , Spain

###### Abstract

We analyze resonant magneto-elastic interactions between standing
perpendicular spin wave modes (exchange magnons) and longitudinal acoustic
phonon modes in free-standing hybrid metal-ferromagnet bilayer and trilayer
structures. Whereas the ferromagnetic layer acts as a magnetic cavity, all
metal layers control the frequencies and eigenmodes of acoustic vibrations.
The here proposed design allows for achieving and tuning the spectral and
spatial modes overlap between phonons and magnons that results in their strong
resonant interaction. Realistic simulations for gold-nickel multilayers show
that sweeping the external magnetic field should allow for observing
resonantly enhanced interactions between individual magnon and phonon modes in
a broad range of frequencies spanning from tens of GHz up to several hundreds
of GHz, which can be finely tuned through the multilayer design. Our results
would enable the systematic study and the deep understanding of resonantly
enhanced magneto-elastic coupling between individual phonon and magnon modes
up to frequencies of great contemporary fundamental and applied interest.

ultrafast magnetization dynamics, magneto-acoustics, exchange magnons,
nanomagnetism

###### pacs:

Valid PACS appear here

##  I Introduction

Over the last decade studies of ultrafast magneto-elastic interactions in
ferromagnetic thin films have attracted significant attention motivated by
possibility of control of the magnetization dynamics at the nano-scale
Scherbakov  _et al._ (2010); Kim  _et al._ (2012); Thevenard  _et al._ (2010);
Janušonis  _et al._ (2016); Chang  _et al._ (2017); Vlasov  _et al._ (2022),
towards magnetization switching Thevenard  _et al._ (2013); Vlasov  _et al._
(2020). Being intrinsically weak, magneto-elastic interactions can be enhanced
by quasi-monochromatic elastic excitations tuned in resonance with acoustic
vibrations Janušonis  _et al._ (2016); Chang  _et al._ (2017); Berk  _et al._
(2019); Dianov  _et al._ (2013); Vlasov  _et al._ (2014). Magneto-elastic
interactions can excite not only the spatially homogeneous mode of
ferromagnetic resonance (FMR) precession, but also spin waves (magnons). In
case of ferromagnetic thin films of nanometer thickness perpendicular
(exchange) spin wave resonances across the film Van Kampen  _et al._ (2002)
can be excited by ultrashort picosecond pulses of longitudinal acoustic
phonons propagating through the film Bombeck  _et al._ (2012); Besse  _et al._
(2020). In this case, the resonant enhancement of phonon-magnon interactions
requires creating a magneto-acoustic cavity across the film, namely a cavity
enabling the frequency/phase matching between the magnetic and acoustic modes.
Such a cavity would enable the systematic study and the deep understanding of
magneto-elastic interactions between individual phonon and magnon modes. A way
to create a magneto-acoustic cavity is to use freestanding ferromagnetic thin
films Kim and Bigot (2017). However, the difference between dispersion
relations of phonons and magnons, in experimentally accessible nm-thick films,
does not allow achieving the frequency/phase matching between magnetic and
acoustic modes, apart from the trivial case of the magnetic uniform FMR
precession mode. For example, a recently reported experiment in a freestanding
YIG/GGG/YIG trilayer demonstrated fingerprints of the magneto-elastic coupling
between two FMR-resonances in spatially separated YIG thin films, mediated by
long-lived standing phonon modes An  _et al._ (2020, 2022).

In this article we propose a pathway to design magneto-acoustic cavities using
free-standing metal-ferromagnet multilayers that can sustain exchange magnons
and acoustic modes, and can be adjusted to achieve the frequency/phase
matching between the modes to ensure their strong coupling. The proposed
design allows the independent tuning of the acoustic and magnetic cavities
thereby enabling the observation of magneto-elastic interactions with higher-
order (exchange) magnon modes with nanometer wavelengths and high frequencies
exceeding 100 GHz. Indeed, in such structures the magnon spectrum is
determined just by the thickness of the ferromagnetic layer while the spectrum
of acoustic vibrations is determined by the total thickness of the multilayer.
Thus, an independent control and frequency/phase matching between individual
phonon and magnon modes becomes possible. The present manuscript presents a
detailed theoretical study, notably from the acoustic perspective, of phonon-
magnon interactions in free-standing bi- and trilayers targeting the
fundamental understanding of magneto-elastics with high-frequency exchange
magnons towards their experimental observation in ultrafast magneto-optical
pump-probe experiments. The dominant role of elastic boundary conditions is
inspected for both types of structures and advantages of trilayer structures
are discussed within the framework of modes symmetry.

The paper is organized as follows. Section II discusses basics of exchange
magnons, generation of ultrashort laser-induced acoustic transients, their
decomposition in acoustic eigenmodes and equations between magneto-elastic
interactions induced by in free-standing metal/ferromagntet multilayers.
Section III discusses the details of acoustic mode decomposition in bilayers
and symmetric trilayers and Section IV describes the numerical procedure to
decompose an arbitrary initial strain distribution in (non-orthogonal)
acoustic strain eigenmodes. Section V presents the numerical results comparing
resonant magneto-elastic interactions in Ni/Au bilayers and Au/Ni/Au trilayers
and their interpretation. The discussion (Section VI) discusses the obtained
results within the context of recent experimental works.

##  II Physical model

Figure 1:  (a) In a free-standing Ni/Au bilayer the magnetic excitations are
assumed to be localized in Nickel. Standing modes of exchange magnons (with
magnon index $m=0,1,2$ etc.) precess around the equilibrium magnetization
direction ${\bf m_{0}}$ under the presence of a tilted external magnetic field
${\bf H}$. (b) Magnon frequencies $\omega_{m}$ depend on the magnon order
$m=k_{m}L_{1}/\pi$ and the amplitude of an external magnetic field (see text
for details); $L_{1}=30~{}$nm.

An example of a simple and experimentally feasible model system for studying
resonant phonon-magnon interactions is a free-standing metal-ferromagnet
(gold-nickel) bilayer Assuming that spin transport across metal-ferromagnetic
interface can be neglected in this study, we are going to discuss their
magnetic and acoustic properties separately, see Fig. 1 for standing spin-wave
modes localized in Nickel and Fig. 2 for acoustic eigenmodes in the entire
bilayer and their excitation.

In this article we focus on the analysis of the acoustically driven
precessional motion of magnetisation vector of constant, time-independent unit
length $|{\bf m}(t)|=1$ around the stationary magnetization direction
$\bf{m_{0}}$, see Fig. 1(a). The latter is parallel to the effective magnetic
field $\mathbf{H}_{eff}={\bf H}+{\bf H_{d}}$, where ${\bf
H}=(H\sin\xi,0,H\cos\xi)$ is the external magnetic field and ${\bf
H_{d}}=(0,0,-M_{0}\cos\theta)$ is the demagnetizing field for a thin
ferromagnetic film characterized by the saturation magnetization $M_{0}$. It
is instructive to note that in this non-collinear geometry the magnetization
angle $\theta$, which minimizes the magneto-static free energy density,
consisting of a thin-film shape anisotropy and the Zeeman term, leads to the
following relation

$\sin(\theta-\xi)=\frac{M_{0}}{2H}\sin 2\theta\,,$ (1)

which is solved numerically to obtain $\theta=\theta(H,\xi)$.

The dispersion relation for spin waves has been obtained by the linearization
of the Landau-Lifshitz-Gilbert equations with an additional exchange term in
the vicinity of the equilibrium magnetization direction $\theta(\xi)$ Besse
_et al._ (2020). In a thin film of thickness $L_{1}$ the general form of the
spatially inhomogeneous magnetization precession ${\bf m}(z,t)={\bf
m_{0}}+{\bf s}(z,t)$ can be represented as a superposition in magnetic
eigenmodes, i.e. the perpendicular standing spin wave resonances

${\bf s}(z,t)=\sum_{m=0}^{\infty}{\bf s}^{(m)}(t)\cos(k_{m}z)\,,$ (2)

for $m=0,1,2,...$ across the film with quantized wave vectors $k_{m}=\pi
m/L_{1}$ and frequencies

$\omega_{m}=\gamma\mu_{0}\sqrt{(H\cos\xi-(M_{0}-{\tilde{D}}k^{2}_{m})\cos\theta)^{2}+(H\sin\xi+{\tilde{D}}k^{2}_{m}\sin\theta)(H\sin\xi+(M_{0}+{\tilde{D}}k^{2}_{m})\sin\theta)}\,,$
(3)

where ${\tilde{D}}=D^{*}/(\hbar\gamma\mu_{0})$ is the exchange stiffness (the
numerical value $D^{*}$=430 [meV Å2] for Ni is taken from Ref. Van Kampen  _et
al._ (2002)), $\gamma$ stands for the gyromagnetic ratio, $\mu_{0}$ is vacuum
permeability and $h$ is Planck’s constant. Fig. 1(b) shows that the magnon
frequencies depend on the magnon order $m$ and can be tuned on the amplitude
$H$ and/or the angle $\xi$ of an external magnetic field.

Now we turn to the discussion of acoustic properties. Optical excitation of a
free-standing absorbing thin film (we have chosen a 30 nm Ni/100 nm Au bilayer
for illustration purposes) by an ultrashort, typically femtosecond, laser
pulse results in ultrafast optical heating of nickel thin film Saito  _et al._
(2003); Kim and Bigot (2017). The subsequent thermoelastic expansion results
in the generation of an ultrashort picosecond acoustic pulse
$\varepsilon_{zz}(z,t)$ bouncing back and forth between nickel-air and gold-
air interfaces, see Fig. 2(a). This propagating pulse can be represented as a
superposition of acoustic eigenmodes:

$\varepsilon_{zz}(z,t)=\sum_{n=1}^{\infty}a_{n}e^{i\omega_{n}t}\varepsilon_{zz}^{(n)}(z)\,,$
(4)

oscillating at frequencies $\omega_{n}$ and characterized by time-independent
expansion coefficients $a_{n}$. Spatial profiles of the first five acoustic
eigenmodes are presented in Fig. 2(b). Strain profiles in Fig. 2(a) have been
obtained using Eq. (4); the details of acoustic mode calculations will be
provided in the next section.

After having provided this unified description of magnetic and acoustic
eigenmodes we discuss their coupling. The long-lived acoustic transients, to
be viewed either as propagating acoustic pulses or a superposition of
individual acoustic modes, can excite the magnetization dynamics in a thin
ferromagnetic layer via the magneto-strictive mechanism Bombeck  _et al._
(2012); Besse  _et al._ (2020).

Figure 2:  (a) Ultrafast optical excitation of a free-standing Ni/Au bilayer
generates ultrashort, picosecond strain pulses $\varepsilon_{zz}(z,t)$
bouncing back and forth in the layer. (b) The strain $\varepsilon_{zz}(z,t)$
in (a) can be represented as a superposition of acoustic eigenmodes numbered
by their mode index $n=1,2,3$ etc.; $L_{1}=30~{}$nm, $L_{2}=70~{}$nm.

It has been shown in the previous work Besse  _et al._ (2020) that in the
linear, small-amplitude approximation, the dynamics of an individual magnon
mode $s^{(m)}(t)$ can be approximated by an equation of a magneto-elastically
driven harmonic oscillator:

$\frac{d^{2}s_{z}^{(m)}}{dt^{2}}+2\alpha\omega_{m}\frac{ds_{z}^{(m)}}{dt}+\omega_{m}^{2}s^{(m)}_{z}=f_{m}(t)\,,$
(5)

where the damping is determined by the dimensionless Gilbert damping parameter
$\alpha$ (we have assumed $\alpha=0.02$ in this work Salikhov  _et al._
(2019)) and the parametric magneto-elastic modulation of magnon frequency
Chang  _et al._ (2017) is neglected. The magneto-elastic driving force is
determined by a rather complex expression, a product of a prefactor and an
integral term:

$f_{m}(t)=P_{m}({\bf
H})\int_{-L_{1}}^{0}\varepsilon_{zz}(z,t)\cos(k_{m}z)\mathrm{d}z\,,$ (6)

where the prefactor

$P_{m}({\bf
H})=\frac{\mu_{0}\gamma^{2}b_{1}\sin(2\theta)\left({\tilde{D}}k_{m}^{2}\sin\theta+H\sin\xi\right)}{M_{0}L_{1}}$
(7)

depends on the nickel’s exchange stiffness ${\tilde{D}}$ and magnetostrictive
coefficient $b_{1}$. It turns to zero both for in-plane ($\theta=0$) or out-
of-plane ($\theta=\pi/2$) magnetization directions. This restriction imposes
the use of an experimental configuration with with the magnetization vector
tilted with respect to the surface normal. Using the strain decomposition in
eigenmodes (see Eq. (4)), the integral term

$\int_{-L_{1}}^{0}\varepsilon_{zz}(z,t)\cos(k_{m}z)\mathrm{d}z=\sum_{n=1}^{\infty}a_{n}I_{mn}{\rm
e}^{i\omega_{n}t}$ (8)

is determined by the acoustic spectrum $a_{n}$ and spatial overlap integrals
$I_{mn}$ between individual magnetic acoustic modes:

$I_{mn}=\int_{-L_{1}}^{0}\varepsilon_{zz}^{(n)}(z)\cos(k_{m}z)\mathrm{d}z\,.$
(9)

The driven solution of Eq. (5) can be written in a compact analytical form

$s_{z}({\bf H},\omega)=\sum_{m=0}^{\infty}\sum_{n=1}^{\infty}\frac{P_{m}({\bf
H})a_{n}I_{nm}}{\omega_{m}^{2}-\omega_{n}^{2}+2i\alpha\omega_{m}\omega_{n}}\,,$
(10)

where resonant interactions between individual magnetic and acoustic modes are
described by the Lorentz functions. According to Eq. (10) the magnetization
oscillates at discrete frequencies $\omega=\omega_{n}$ of the acoustic driving
force. In order to take into account for finite lifetimes of acoustic modes it
is convenient to assume that acoustic lifetimes
$\tau_{n}^{(ac)}=1/(\alpha_{ac}\omega_{n})$ for individual modes are
characterized by a single phenomenological damping parameter $\alpha_{ac}$,
($\alpha_{ac}=0.005$ throughout of the manuscript, the order-of-magnitude
taken from Hatanaka  _et al._ (2022)). Then Eq. (10) is transforms in the
final expression

$s_{z}({\bf H},\omega)=\sum_{m=0}^{\infty}\sum_{n=1}^{\infty}\frac{P_{m}({\bf
H})a_{n}I_{nm}}{\omega_{m}^{2}-\omega_{n}^{2}+2i\alpha\omega_{m}\omega_{n}}L_{n}^{(ac)}(\omega,\omega_{n})\,,$
(11)

where each acoustic mode becomes broadened by the the acoustic Lorentzian
$L_{n}^{(ac)}(\omega,\omega_{n})\propto
1/(\omega^{2}-\omega_{n}^{2}+i\alpha_{ac}\omega\omega_{n})$. The rest of the
manuscript will be devoted to the numerical analysis of this solution as well
as individual terms in this equation for model bi-layer and trilayer
structures. Equation (11) is implemented numerically to generate smooth two-
dimensional maps as a function of the continuous frequency $\omega$ and the
magnetic field strength $H$ in figures 4-6.

##  III Acoustic eigenmodes in freestanding multilayers

To calculate longitudinal acoustic eigenmodes displayed in Fig. 2(b), we
represent the elastic displacement $u_{z}$ in the $z$-direction in each layer
as the sum of right and left propagating plane waves, respectively:

$\displaystyle\begin{split}u_{z}^{(1)}&=A{\rm e}^{i(-k^{(1)}z+\omega t)}+B{\rm
e}^{i(k^{(1)}z+\omega t)}\\\ u_{z}^{(2)}&=C{\rm e}^{i(-k^{(2)}z+\omega
t)}+D{\rm e}^{i(k^{(2)}z+\omega t)}\,\end{split}$ (12)

The associated strains are given by:

$\displaystyle\begin{split}\epsilon_{zz}^{(1)}=\frac{\partial
u_{z}^{(1)}}{\partial z}\,,\,\epsilon_{zz}^{(2)}=\frac{\partial
u_{z}^{(2)}}{\partial z}\,.\end{split}$ (13)

The expressions (12) and (13) satisfy the following boundary conditions:

$\displaystyle\begin{split}\epsilon_{zz}^{(1)}(-L_{1})&=0\\\
\epsilon_{zz}^{(2)}(L_{2})&=0\\\ u_{z}^{(1)}(0)&=u_{z}^{(2)}(0)\\\
v_{1}Z_{1}\epsilon_{zz}^{(1)}(0)&=v_{2}Z_{2}\epsilon_{zz}^{(2)}(0)\,,\end{split}$
(14)

where the acoustic impedance $Z=v\rho$ is determined by the acoustic velocity
$v$ and the material density $\rho$. This linear system of four equations is
quanitified by a matrix

$\begin{pmatrix}{\rm e}^{i\omega\frac{L_{1}}{v_{1}}}&-{\rm
e}^{-i\omega\frac{L_{1}}{v_{1}}}&0&0\\\ 1&1&-1&-1\\\
-Z_{1}&Z_{1}&Z_{2}&-Z_{2}\\\ 0&0&{\rm e}^{-i\omega\frac{L_{2}}{v_{2}}}&-{\rm
e}^{i\omega\frac{L_{2}}{v_{2}})}\end{pmatrix}$ (15)

leading to the following secular equation:

$Z_{1}\sin\left(\omega\frac{L_{1}}{v_{1}}\right)\cos\left(\omega\frac{L_{2}}{v_{2}}\right)+Z_{2}\sin\left(\omega\frac{L_{2}}{v_{2}}\right)\cos\left(\omega\frac{L_{1}}{v_{1}}\right)=0$
(16)

which roots $\omega_{n}$ are obtained numerically. The corresponding
eigenvectors $\\{A_{n},B_{n},C_{n},D_{n}\\}$ read:

$\displaystyle\begin{split}B_{n}&=A_{n}{\rm
e}^{2i\omega_{n}\frac{L_{1}}{v_{1}}}\\\
C_{n}&=\frac{Z_{1}\left(1-B_{n}\right)+Z_{2}\left(1+B_{n}\right)}{2Z_{2}}\\\
D_{n}&=A_{n}+B_{n}-C_{n}\,,\end{split}$ (17)

where $A_{n}$ is chosen to normalize to equal acoustic energies for all modes,
with the elastic energy on the $n$-th acoustic mode being:

$\frac{1}{2}\left(\int_{-L_{1}}^{0}v_{1}Z_{1}(\varepsilon^{(n)}_{zz}(z))^{2}\mathrm{d}z+\int_{0}^{L_{2}}v_{2}Z_{2}(\varepsilon^{(n)}_{zz}(z))^{2}\mathrm{d}z\right)$
(18)

The results of numerical calculations for bilayers can be generalized to
calculate acoustic eigenmodes in symmetric metal/ferromagnet/metal trilayers
in a rather straight-forward manner. Using the system of coordinates in Fig.
1, a symmetric trilayer can be built by reflecting the bilayer shown in Fig. 1
symmetrically about its left boundary plane $z=-L_{1}$, so that the structure
now comprises two gold layers at both sides with thickness $L_{2}$ and the
nickel layer with thickness $2L_{1}$ in the center. It is convenient and
reader friendly to denote the half thickness of nickel by $l_{1}$ and the gold
layer thickness as $l_{2}$ for trilayers. In this new notation the plane of
symmetry of the trilayer lies at $z=-l_{1}$, gold-nickel interface lies at
$z=0$, and the right-hand free boundary of gold lies at $z=l_{2}$. In that
symmetric trilayer there exist two types of eigenmodes of elastic vibrations,
namely the symmetric ones with displacements on opposites sides of the plane
of symmetry $z=-l_{1}$ related by $u_{+}=-u_{-}$, and antisymmetric ones with
$u_{+}=u_{-}$. Due to the continuity of displacements and stresses, at the
symmetry plane, the displacement $u(-l_{1})$ vanishes for the symmetric modes,
and its spatial derivative (and hence the stress $\varepsilon_{zz}(-l_{1})$ \-
for the antisymmetric modes. Therefore, the set of antisymmetric eigenmodes
coincides with those calculated above for freestanding bilayers if replace
thicknesses $l_{i}$ with $L_{i}$. Due to the symmetry of the structure we can
consider only the half of it with the boundary conditions defined at the free
boundary, at the interface plane, and at the plane of symmetry. The total set
ob boundary conditions for symmetric modes is different and takes the
following form:

$\displaystyle\begin{split}u^{(1)}_{z}(-l_{1})&=0\\\
\varepsilon^{(2)}_{zz}(l_{2})&=0\\\ u^{(1)}_{z}(0)&=u^{(2)}(0)\\\
v_{1}Z_{1}\varepsilon^{(1)}_{zz}(0)&=v_{2}Z_{2}\varepsilon^{(2)}_{zz}(0)\,,\end{split}$
(19)

for the symmetric modes, and for the antisymmetric ones the first condition in
(19) should be replaced with $u^{(1)}_{z}(-l_{1})~{}=0$. Note that the
antisymmetric modes in trilayer are equivalent to the bilayer modes ((14)),
hence (15) remains valid for the antisymmetric modes with substitution
$L_{1}=2l_{1}$. For the symmetric modes the matrix of boundary conditions
differs in only signs of two elements:

$\begin{pmatrix}{\rm e}^{-i\omega\frac{l_{1}}{v_{1}}}&{\rm
e}^{i\omega\frac{l_{1}}{v_{1}}}&0&0\\\ 0&0&{\rm
e}^{i\omega\frac{l_{2}}{v_{2}}}&-{\rm e}^{-i\omega\frac{l_{2}}{v_{2}}}\\\
1&1&-1&-1\\\ -Z_{1}&Z_{1}&Z_{2}&-Z_{2}\end{pmatrix}$ (20)

Eigenfrequencies of the symmetric trilayer system are the solutions to the
secular equation which make the determinant of the boundary condition matrix
vanish. It can be written in a concise form for both types of modes:

$\tan\left(\omega\frac{l_{1}}{v_{1}}+\alpha\right)\tan\left(\omega\frac{l_{2}}{v_{2}}\right)=\frac{Z_{1}}{Z_{2}}$
(21)

with phase parameter $\alpha=0$ for the symmetric modes and $\alpha=\pi/2$ for
the antisymmetric ones.

##  IV Decomposition of an arbitrary input strain into acoustic modes

Using an example of a bilayer here we show how to decompose any spatio-
temporal strain distribution $\varepsilon_{zz}(z,t)$ in acoustic eigenmodes:

$\varepsilon_{zz}(z,t)=\sum_{n=1}^{\infty}a_{n}e^{i\omega_{n}t}\varepsilon_{zz}^{(n)}(z),$
(22)

where every $a_{n}$ satisfy the following algebraic matrix equation:

$\int_{-L_{1}}^{L_{2}}\varepsilon(z,t=0)\varepsilon_{zz}^{(n)}(z)\mathrm{d}z=\sum_{n^{\prime}}a_{n^{\prime}}\int_{-L_{1}}^{L_{2}}\varepsilon_{zz}^{(n^{\prime})}(z)\varepsilon_{zz}^{(n)}\mathrm{d}z\,.$
(23)

Since the acoustic eigenmodes in multilayer structures above are in general
not orthogonal the coefficients $a_{n}$s should obey the following matrix
equation:

$\hat{M}\vec{a}=\vec{b},$ (24)

where

$(\hat{M})_{ij}=\int_{-L_{1}}^{L_{2}}\varepsilon_{zz}^{(i)}(z)\varepsilon_{zz}^{(j)}(z)\mathrm{d}z$
(25)

and

$\displaystyle\vec{a}=\begin{pmatrix}a_{1}\\\ \vdots\\\
a_{cutoff}\end{pmatrix}$
$\displaystyle\vec{b}=\begin{pmatrix}\int_{-L_{1}}^{L_{2}}\varepsilon(z,t=0)\varepsilon_{zz}^{(1)}(z)\mathrm{d}z\\\
\vdots\\\
\int_{-L_{1}}^{L_{2}}\varepsilon(z,t=0)\varepsilon_{zz}^{(cutoff)}(z)\mathrm{d}z\end{pmatrix}\,.$
(26)

By numerical inversion of a truncated matrix $\hat{M})$, with the cutoff-index
being chosen such that the results do not depend on it, delivers the desired
expansion coefficients (or the acoustic spectrum) $a_{n}$.

The performance of this decomposition method is illustrated in Fig. 3, where
the initial strain profile in Ni (exponential with a characteristic decay
length of approximately 20 nm Saito  _et al._ (2003)) is accurately reproduced
as a superposition of the first 220 acoustic eigenmodes. This initial strain
distribution, i.e. exponential in Ni and zero in Au, will be used in all
numerical calculations in the forthcoming sections.

Figure 3:  (a) The acoustic spectrum $a_{n}$ for a 30nm Ni/70nm Au bilayer
obtained by using the matrix inversion protocol. (b) The reconstructed pulse
with $n_{cutoff}$=220 allows for accurate reconstruction of the initial
exponential strain profile.

##  V Resonant magneto-elastic coupling in multilayers

Figure 4:  (a) Profiles of the first eight acoustic strain eigenmodes. (b) The
magnetic field dependence of the elastically induced magnetization dynamics
according to Eq. (10). (c) The overlap integral $I_{mn}$ for the first four
magnon modes.

In order to illustrate the peculiarities of phonon-magnon interactions we
decided to compare two different freestanding multilayer structures: a 30 nm
Ni/70 nm Au bilayer and a symmetric 35 nm Au/30 nm Ni/35 nm Au trilayer of the
same total thickness of 100 nm. The results for both structures are presented
in Fig. 4(a-c) and Fig. 4(d-f), respectively. Whereas the acoustic spectra
$\omega_{n}$ for both spectra are nearly identical, their eigenmodes are
fundamentally different. All eigenmodes for the bilayer (Fig. 4(a)) are
naturally asymmetric. In contrast, eigenmodes in the trilayer in Fig.4 (d)
split in two sub-families: symmetric (for $n=1,3,5,...$) and antisymmetric
(for $n=2,4,6,...$) modes with respect to the symmetry plane in the center of
the Ni-layer. This intrinsic difference in the symmetry between the two cases
has important consequences for phonon-magnon interactions given that magnon
modes have their own and well defined symmetry and are also split in two sub-
families: symmetric (for $m=0,2,4,...$) and antisymmetric (for $m=1,3,5,...$)
. Panels (b) and (e) compare the Fourier spectra of magnetization dynamics
obtained with Eq. (10), generalized with acoustic damping. Figure 4(b) clearly
demonstrates that the resonant enhancement of magnetic signal when magnon
dispersions (white continuous lines) cross acoustic resonances (blue dashed
lines). This indicates that prefactors $P_{m}(H)a_{n}I_{nm}$ are nonzero.
Moreover, due to the non-zero Gilbert damping, individual magnetic resonances
excited by the same acoustic mode, overlap: see the continuous blue profiles
in Fig. 4(b). The situation becomes fundamentally different for the trilayer
in Fig. 4(e): only a few resonances are excited and they do not overlap. This
indicates that some of the prefactors $P_{m}(H)a_{n}I_{nm}$ must be zero. This
can be understood based on the symmetry considerations reported above. Indeed,
since in this case both modes, magnonic and acoustic, have well defined
symmetries split into two families. Thereby, as a detailed analysis confirms,
when the magnonic and acoustic modes belong to different symmetry sub-families
the overlap integral $I_{mn}$ becomes negligibly small. As a consequence,
whereas for the bilayer the $I_{mn}$ is nonzero for most of the modes (Fig.
4(c)), for the trilayer it displays many zeros (Fig. 4(f)). Applied to the
plot in Fig. 4(e), these symmetry considerations result in that the symmmetric
$m=0,2,..$ (antisymmertic $m=1,3,..$) magnon modes should only interact with
symmetric $n=1,3,...$ (antisymmetric $n=2,4,...$) acoustic modes. In other
words the most closely spaced $m=0$ and $m=1$ magnon modes will never interact
with the same acoustic modes and their resonances will not overlap. As such,
the freestanding metal/ferromagnet/metal trilayers structures can serve as an
excellent playground to investigate individual phonon-magnon interactions.

Figure 5:  The magnetic field dependence of the elastically induced
magnetization dynamics for three symmetric Au(35 nm)/Ni (X nm)/Au(35
nm)-trilayer structures: (a) $2l_{1}$=70 nm, (b) $2l_{1}$=130 nm and (c)
$2l_{1}$=200 nm.

In order to illustrate the capability of symmetric freestanding trilayers to
separate individual phonon-magnon resonances, we have performed the very same
simulation for a series of structures with varying Ni thickness and the same
thickness of the gold layer of 35 nm. Figure 5 compares the magnetization
response for Ni thickness of 70, 130 and 200 nm, respectively. With increasing
Ni thickness the spectral density of acoustic and magnetic modes increases.
Nevertheless the magneto-elastic resonances driven by the two lowest order
acoustic modes $n$=1 and $n$=2 remain clearly resolved for all structures. The
situation becomes very different for $n$=3-driven resonances: for $2l_{1}$=70
nm is excites partially overlapping $m$=0 and $m$=2 magnons with a significant
amplitude. For $2l_{1}$=130 nm this resonance almost disappears and for
$2l_{1}$=200 nm becomes larger again. In the latter case, however, the
$n$=4-driven resonance disappears. For higher $n$ resonances with multiple $m$
overlap giving rise to broad complex-shaped peaks of large amplitude. It
should be noticed that the $n$=2-driven magneto-elastic resonance remains
well-defined and its amplitude is growing with the nickel thickness, becoming
larger that $n$=1-driven FMR-resonance. The above qualitative analysis shows
that metal/ferromagnet/metal trilayers allow allow for an unambiguous
discrimination between magneto-elastic excitation of ($m=0$) FMR and ($m\leq
1$) exchange magnon modes due to the symmetry of acoustic eigenmodes, for a
wide range of structures with the thickness from 100 to 270 nm.

This situation corresponds very well to the capabilities to produce
macroscopically large samples of suspended membranes: either using chemical
means Kim and Bigot (2017) or nondestructive nanostructuring of thin films
using single ultrashort laser pulses Domke  _et al._ (2014); Temnov  _et al._
(2020).

It appears to be instructive, at least from the theoretical perspective, to
analyse phonon-magnon interactions for ultrathin multilayers. Figure 6
displays the results for a Au(3 nm)/Ni(10 nm)/Au(3 nm) trilayer. It is
remarkable that magnons with frequencies of several hundreds GHz be excited:
see $m=2,n=3$ and $m$=2,$n$=5 resonances at 434 GHz and 707 GHz, respectively.
Moreover, the $m$=2,$n$=3 magneto-elastic resonance at 434 GHz should be
observable at relatively low, experimentally easily accessible external
magnetic fields with amplitude of approximately 0.8 Tesla.

Figure 6:  The magnetic field dependence of the elastically induced
magnetization dynamics for a symmetric Au(3 nm)/Ni(10 nm)/Au(3 nm)-trilayer
structure demonstrates the possibility of magneto-elastic excitation of
exchange magnons in the Thz-frequency range.

##  VI Discussion

The theoretical analysis in this paper can be particularly useful in the
context of recent magneto-elastic studies in freestanding
ferromagnet/dielectric/ferromagnet trilayers discussing the strong
acoustically mediated coupling between FMR excitations in two identical
ferromagnetic (YIG) layers An  _et al._ (2020, 2022). To be more specific, it
has been shown that the magneto-elastic coupling between FMR-oscillators in a
YIG1(200 nm)—GGG(0.5 mm)—YIG2(200 nm) results in the formation of the bandgap
An  _et al._ (2020) as well as bright and dark eigenmodes of two coupled
oscillators An  _et al._ (2022). It would be very interesting to investigate
if the same mechanism could work for exchange magnons with $n\geq 1$ in YIG
layers. The detailed understanding of acoustic modes in the present paper
could be quite helpful.

Another recent work Jaris  _et al._ (2020); Yang and Schmidt (2021) monitors
the excitation of disc nanomagnets with diameter $D$ using monochromatic
surface acoustic waves with wavelength $\lambda_{SAW}$. It reports on the
increase of the effective Gilbert damping in the regime $D>\lambda_{SAW}$,
which is associated to the effect of inhomogeneous broadening in ensemble
measurements. At the same time, it follows from our analysis that magneto-
elastic excitation of (in-plane) magnons can modify the shape of magnetic
response though overlapping of FMR and magnon resonances. Spatially
inhomogeneous magnon modes could contribute to the time-resolved MOKE-
microscopy signals due to the enhanced sensitivity reported on discs of
similar dimensions Berger  _et al._ (2015).

From the theoretical perspective the simplified description of magnon
excitation based on Eq. (5) can be generalized model the time-dependent
solutions in ultrafast magneto-elastic interactions Besse  _et al._ (2020);
Jaris  _et al._ (2020); Yang and Schmidt (2021). Among different non-
stationary processes the magneto-elastic parametric effects Chang  _et al._
(2017) play the particular role since they are sensitive both to the Gilbert
damping and the amplitude of the parametric modulation.

##  VII Summary and conclusions

The main result of this paper is the relatively simple approach to describe
ultrafast (GHz-to-THz frequency) magneto-acoustic interactions in free-
standing ferromagnetic multilayers on the basis of eigenmode decomposition,
both for for magnons and phonons. Semi-analytical calculations of acoustic
eigenmodes in freestanding bilayers are generalized to describe symmetric
trilayer structures. The numerically calculated acoustic eigenmodes are used
to quantify resonantly enhanced interactions between individual phonon and
magnon modes. Whereas in asymmetric bilayers nearly all phonon and magnon
modes are coupled, the symmetry of trilayers results in the suppression of
certain magnon-phonon interactions due to the zero overlap integrals of
eigenmodes. This fact should allow for unambiguous discrimination between
phonon-($m=0$)FMR and phonon-($m\geq 1$)magnon interactions in ultrafast
magneto-acoustic experiments. Several forward-looking statements have been
made with respect to the existing experimental configurations.

###### Acknowledgements.

Financial support by the ANR-21-CE15-0048-01 MRSEI ”IRON-MAG” is gratefully
acknowledged.

## References

  * Scherbakov  _et al._ (2010) A. V. Scherbakov, A. S. Salasyuk, A. V. Akimov, X. Liu, M. Bombeck, C. Brüggemann, D. R. Yakovlev, V. F. Sapega, J. K. Furdyna, and M. Bayer, Phys. Rev. Lett. 105, 117204 (2010).
  * Kim  _et al._ (2012) J.-W. Kim, M. Vomir, and J.-Y. Bigot, Phys. Rev. Lett. 109, 166601 (2012).
  * Thevenard  _et al._ (2010) L. Thevenard, E. Peronne, C. Gourdon, C. Testelin, M. Cubukcu, E. Charron, S. Vincent, A. Lemaître, and B. Perrin, Phys. Rev. B 82, 104422 (2010).
  * Janušonis  _et al._ (2016) J. Janušonis, C. L. Chang, T. Jansma, A. Gatilova, V. S. Vlasov, A. M. Lomonosov, V. V. Temnov, and R. I. Tobey, Phys. Rev. B 94, 024415 (2016).
  * Chang  _et al._ (2017) C. L. Chang, A. M. Lomonosov, J. Janusonis, V. S. Vlasov, V. V. Temnov, and R. I. Tobey, Phys. Rev. B 95, 060409(R) (2017).
  * Vlasov  _et al._ (2022) V. S. Vlasov, A. V. Golov, L. N. Kotov, V. I. Shcheglov, A. M. Lomonosov, and V. V. Temnov, Acoustical Physics 68, 18 (2022).
  * Thevenard  _et al._ (2013) L. Thevenard, J.-Y. Duquesne, E. Peronne, H. J. Von Bardeleben, H. Jaffres, S. Ruttala, J.-M. George, A. Lemaitre, and C. Gourdon, Physical Review B 87, 144402 (2013).
  * Vlasov  _et al._ (2020) V. S. Vlasov, A. M. Lomonosov, A. V. Golov, L. N. Kotov, V. Besse, A. Alekhin, D. A. Kuzmin, I. V. Bychkov, and V. V. Temnov, Phys. Rev. B 101, 024425 (2020).
  * Berk  _et al._ (2019) C. Berk, M. Jaris, W. Yang, S. Dhuey, S. Cabrini, and H. Schmidt, Nature communications 10, 1 (2019).
  * Dianov  _et al._ (2013) M. Y. Dianov, V. S. Vlasov, D. S. Beznosikov, L. N. Kotov, V. G. Shavrov, and V. I. Shcheglov, Bulletin of the Russian Academy of Sciences. Physics 77, 1258 (2013).
  * Vlasov  _et al._ (2014) V. S. Vlasov, V. G. Shavrov, and V. I. Shcheglov, Journal of Communications Technology and Electronics 59, 441 (2014).
  * Van Kampen  _et al._ (2002) M. Van Kampen, C. Jozsa, J. Kohlhepp, P. LeClair, L. Lagae, W. De Jonge, and B. Koopmans, Phys. Rev. Lett. 88, 227201 (2002).
  * Bombeck  _et al._ (2012) M. Bombeck, A. Salasyuk, B. Glavin, A. Scherbakov, C. Brüggemann, D. Yakovlev, V. Sapega, X. Liu, J. Furdyna, A. Akimov, _et al._ , Phys. Rev. B 85, 195324 (2012).
  * Besse  _et al._ (2020) V. Besse, A. V. Golov, V. S. Vlasov, A. Alekhin, D. Kuzmin, I. V. Bychkov, L. N. Kotov, and V. V. Temnov, J. Magn. Magn. Mater. 502, 166320 (2020).
  * Kim and Bigot (2017) J.-W. Kim and J.-Y. Bigot, Phys. Rev. B 95, 144422 (2017).
  * An  _et al._ (2020) K. An, A. N. Litvinenko, R. Kohno, A. A. Fuad, V. V. Naletov, L. Vila, U. Ebels, G. de Loubens, H. Hurdequint, N. Beaulieu, _et al._ , Physical Review B 101, 060407 (2020).
  * An  _et al._ (2022) K. An, R. Kohno, A. N. Litvinenko, R. L. Seeger, V. V. Naletov, L. Vila, G. de Loubens, J. B. Youssef, N. Vukadinovic, G. E. Bauer, _et al._ , Physical Review X 12, 011060 (2022).
  * Saito  _et al._ (2003) T. Saito, O. Matsuda, and O. Wright, Physical Review B 67, 205421 (2003).
  * Salikhov  _et al._ (2019) R. Salikhov, A. Alekhin, T. Parpiiev, T. Pezeril, D. Makarov, R. Abrudan, R. Meckenstock, F. Radu, M. Farle, H. Zabel, and V. V. Temnov, Phys. Rev. B 99, 104412 (2019).
  * Hatanaka  _et al._ (2022) D. Hatanaka, M. Asano, H. Okamoto, Y. Kunihashi, H. Sanada, and H. Yamaguchi, Physical Review Applied 17, 034024 (2022).
  * Domke  _et al._ (2014) M. Domke, L. Nobile, S. Rapp, S. Eiselen, J. Sotrop, H. P. Huber, and M. Schmidt, Physics Procedia 56, 1007 (2014).
  * Temnov  _et al._ (2020) V. V. Temnov, A. Alekhin, A. Samokhvalov, D. S. Ivanov, A. Lomonosov, P. Vavassori, E. Modin, and V. P. Veiko, Nano Letters 20, 7912 (2020).
  * Jaris  _et al._ (2020) M. Jaris, W. Yang, C. Berk, and H. Schmidt, Physical Review B 101, 214421 (2020).
  * Yang and Schmidt (2021) W.-G. Yang and H. Schmidt, Applied Physics Reviews 8, 021304 (2021).
  * Berger  _et al._ (2015) A. Berger, R. A. de la Osa, A. K. Suszka, M. Pancaldi, J. M. Sáiz, F. Moreno, H. P. Oepen, and P. Vavassori, Physical review letters 115, 187403 (2015).